Administration – Random IT Utensils

Availability Anywhere Part 26 — Working remotely like a pro

afish — Wed, 24 Apr 2024 19:59:55 +0000

This is the twentieth sixth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Over the years I improved my remote work setup significantly. This blog post summarizes what various approaches I took and how I configured my setup over the years. One note before moving on – I assume you don’t have any “hardware” but the computer (so no programmable boards, weird mobile devices, confidential hardware etc.).

Level 1

First approach is what most of us consider “remote work”. You just take your corporate laptop with you and you work remotely. You probably have a corporate VPN that lets you connect to the company’s resources from wherever you are.

Pros:

You can do everything the same way you did while in the office

Cons:

You can’t work from devices that are not onboarded with your corporate infrastructure – mobile phones, tablets, private laptops, VR goggles, etc.
You need to take the corporate laptop with you anywhere you go
You work from a device that is probably hot and loud working crazy on your office stuff (we all know the bloatware they install on the machines…)
You can rarely pick your hardware, so your laptop is probably big and bulky (and yes, Macbook Air is big and bulky)
You may not be able to connect to the VPN in some places due to weird networking conditions

While this is enough to actually do the work, I don’t like this approach. First, I always want to be able to be online (that doesn’t mean that I’m online all the time) no matter where I am and what I have on me. Specifically, I want to be able to access the infrastructure and do all my stuff using my mobile phone, private laptop, etc. I mean, I want to be able to do coding or video processing from my mobile phone while on a train (yes, I did that).

Second, I don’t like taking my corporate machine with me. This is especially troublesome when I need to travel for conferences for which I need to take more hardware with me (like 2 laptops and many mobile phones). Taking another laptop on the plane is just cumbersome. This is sometimes even dangerous depending on where you go and when they want to control your corporate machines. Not to mention, that your corporate policy may prohibit you from taking the corporate laptop to some places.

Third, I don’t like when I can’t pick my hardware. My laptop must be light and quiet, especially when I sit in a deckchair or lie down.

Level 1.5

Sometimes you cannot connect to the VPN while working remotely. This is due to weird networking, policy, or simply geolocation block.

You can fix that by getting a networking device that will connect to some other VPN and expose a tunnel for you. Typical solution is to get a router with built-in VPN. You can see this tutorial to learn how to do that.

However, I don’t recommend taking a router for that. Just get yourself a decent Windows x86 tablet, for instance Dell Venue 8 Pro and install SoftEther VPN. First, such a tablet is a “full Windows machine” so you can do whatever is needed to get connected (think about captive portals or weird passwords). Second, since this is a tablet, you can take it with you wherever you need.

Level 2

Let’s not bring the corporate machine with us at all. Get yourself a corporate virtual machine (typically called virtual desktop or VDI) and do all the work from it. Next, to work remotely, you just need to connect to the VDI from your device (which we’ll call road runner) using RDP + SSH. To do that, you may need to onboard your road runner with your corporate VPN. However, there are many solutions to work that around, so from now on we assume your road runner is not onboarded with the corporate bloatware.

Pros:

You can do almost everything the same way you did while in the office
You can work from your laptop
You can take your hardware anywhere you wish

Cons:

You may still be unable to work from mobile phones, tablets, VR goggles, etc.
You may need to run video conferencing software on your road runner (which may not be possible)
Your road runner may still get hot and loud when taking crazy video call with greenscreen effects
You may not be able to do some corporate tasks – think about GPU-heavy things like video processing

This is much better now since we can connect from our road runner, so we can get a light laptop like Acer Swift 5 which is 1kg/2.2lb (your Macbook Air is at least 25% heavier).

When it comes to video conferences, they may be tricky. First, your webcam may not work over RDP. Second, RDP introduces noticeable audio delay. To fix that, you may need to run the conference software locally (I recommend doing that in browser). While it solves the problem, it also makes your road runner loud and hot.

Level 3

Let’s finally address the problem that we can’t work from any device. Let’s add a jump host. Get yourself a decent Windows machine and put it somewhere on the Internet. I generally recommend Hetzner (but whatever you choose, just get yourself a dedicated host). And BTW, choose Intel CPU instead of AMD (your virtual machines will thank you for that). Next, connect to that machine with plain RDP. This way, you can connect to it using any road runner – be it laptop, mobile phone, tablet, or whatever else. And since it can be a mobile phone, just get yourself a decent foldable. Yes, that really makes a difference when you’re working remotely.

Pros:

You can do almost everything the same way you did while in the office
You can work from nearly any device
You can take your hardware anywhere you wish

Cons:

You may need to run video conferencing software on your road runner (which may not be possible)
Your road runner may still get hot and loud when taking crazy video call with greenscreen effects
You may not be able to do some corporate tasks – think about GPU-heavy things like video processing
You may have hard time connecting to the corporate VPN

When dealing with corporate VPN, use things like SSLH or FileProxy for avoiding VPN without split tunneling (also known as TCP over File System). There are many more tricks and improvements you can apply. I cover them in my talk.

Level 4

Let’s now deal with more devices and the GPU. I use two solutions for that: vSpatial and NoMachine.

First, vSpatial lets you connect to your jump host with browser and application for VR goggles. This way, you can work from basically any device you can think of. Another bonus is that vSpatial lets you forward audio and camera, so you can take your meetings from the jump host. However, as of today, vSpatial supports only 20 FPS for your webcam, so people will notice that something is off. We’ll deal with that in a second.

Second, NoMachine lets you connect to the jump host over VNC-like protocol which lets you use GPU easily. You can now do basically anything you can think of.

Pros:

You can do everything the same way you did while in the office (assuming your VDI can do the things your corporate laptop could)
You can work from any device
You can take your hardware anywhere you wish

Cons:

You may need to run video conferencing software on your road runner (which may not be possible)
Your road runner may still get hot and loud when taking crazy video call with greenscreen effects
You may have hard time connecting to the corporate VPN

Nothing stops you from having more jump hosts. For instance, you can host a dedicated Mac mini in Cyberlynk and connect to it.

Also, if you need more screens with your road runner, you can try portable monitor. You can also emulate screens with IddSampleDriver or its Rust version, or Better Display. This way you can literally move between machines with not a single window changing its position.

Level 5

Let’s finally deal with our road runners becoming loud and hot. We basically want to offload whole video processing to the jump host so that our device doesn’t need to encode/decode any video conferencing stuff.

To do that, use VDO.Ninja. This lets you “forward” your audio and camera to any jump host. Just use OBS or Splitcam on the jumphost to receive the video, apply any greenscreen effects you need, and forward it to the conference call.

Pros:

You can do everything the same way you did while in the office (assuming your VDI can do the things your corporate laptop could)
You can work from any device
You can take your hardware anywhere you wish
Your hardware stays cool and quiet no matter what you do

Cons:

You may have hard time connecting to the corporate VPN

This is how my final setup looks like. I don’t do anything on my road runner. Nothing. I just connect to the jump host with RDP + vSpatial + NoMachine and I forward my audio + video with VDO.Ninja. No matter if I code, process videos, record stuff, attend conference call (or many of them), remove my webcam background, or anything else – my road runner has constant and predictable load. I can also switch to any other road runner easily.

Double-thin-client architecture

Yet another trick you can apply is what I call “double-thin-client” architecture. Instead of booting up your road runner natively, you can use native boot and get yourself a portable SSD drive like Samsung T7.

This way, you can install your operating system once, and carry it with you wherever you go. You just plug the drive to the USB, boot the operating system from the drive, and you have your environment up and running. You can then replace your road runner with another machine easily. I actually did that many times. My Windows installation survived like 4 different laptops already. Best thing is, you can simply clone the drive to another one when you get a new SSD, so you can upgrade your hardware in place with no failures.

You can use similar approach for your jump host. Do not install things locally. Just boot from VHD and this way you can easily take backup of your dedicated host.

Other improvements

There are many more things you can consider for improving your remote work. Some ideas include Bluetooth devices (obviously) and VNC server installed on a road runner, so you can walk around your home and control the device from your mobile phone. This can be useful when you’re already dialed into a call and you don’t want to rejoin from another device. Simply take your Bluetooth headset with you to another room and watch the screen of your road runner from a mobile phone.

Another idea is automatically capturing meeting notes. Since you take calls on your jump host, you can run some notetaker AI application that will automatically capture whatever you need and send it forward.

You can use Firefox with Multi-Account Containers and Tree Style Tab. They really make your work way faster. Too bad it’s just for Firefox.

You may also want to use VirtuaWin to have many desktops. I find it much better to use than “tasks” in Windows, mostly thanks to keyboard shortcuts. And if you need to run applications that need to actively interact with your desktop (like Puppeteer or UIPath), just use Desktops.

Tricks for mobile phones

You can also work on your mobile phone. First, you can get a silicone keyboard to type easily. This is really great and you can actually work from your mobile phone. I once travelled for two days with my mobile phone only and I was still able to do coding and debugging.

You can dial in over GSM. This may simplify the way you take online meetings.

Next, you can run many clones of one application with Multiple Accounts. This way, you can separate your corporate stuff from your personal one on your road runner.

You can virtualize mobile phone inside a mobile phone. Just consider applications like VMOS or Virtual Master – Android Clone. You can try Limbo x86 PC Emulator.

You can turn your mobile phone into a mobile workstation with Samsung DeX.

Short note on Apple devices

I sometimes get asked why I don’t use Apple devices. If you watch my conference recording, you can see that I always use Windows and Android. This is surprising to some people because they consider me a “power user” and yet it seems like I don’t use Apple devices. Let me give you some reasons.

First, I do use them. I really like my Mac with silicon chip as it’s really great CPU + GPU in one box. However, I don’t use it as a road runner for three reasons:

It’s heavy. It’s over 25% heavier than my Windows road runner. While it’s not a big deal as it’s still relatively light, I just don’t want to carry additional kilograms
It is loud and hot. I don’t like that it gets so hot when I don’t do crazy things. This is especially irritating when working in bed
Most importantly, Macs do not support many screens easily. Right now, I have 5 displays on my desk (not counting the monitor one). When I travel, I often plug additional screens to my laptop. I just don’t accept that Macbook Air doesn’t handle 3 external screens

When it comes to iPhone, I have these reasons to prefer Android:

iPhone doesn’t record calls. This is a major obstacle for which there is no easy workaround
iPhone can’t run many instances of an app. I want to run them independently because I need to log in with multiple accounts. Some applications support that natively, but I need a generic solution
Foldable phone is really a game changer
I have some “sophisticated” applications running on my mobile phone, for instance web server and Node.js applications that automate things around text messages. I simply can’t have that on iPhone
Many things are not available for iPhone, like some of my VPNs, port forwarding, etc. Android just works

That’s it. I’ve been using iPhone for some time and it just doesn’t cut. For Macbook, I may use it as a road runner one day when it improves the hardware specification (sic!).

Availability Anywhere Part 23 — RDP over VR goggles with no PCVR

afish — Wed, 21 Feb 2024 21:10:54 +0000

This is the twentieth third part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

In this part we’re going to see how we can work remotely from VR goggles with no PCVR around. Effectively, we can work remotely with goggles and a bluetooth keyboard, nothing else.

My workstation

My home workstation is quite powerful. I have 5 physical screens which I use to display 4 different screens. I also use VirtuaWin to have many desktops. I also work remotely from a dedicated machine to which I connect using MS RDP (mstsc.exe).

My typical setup works like this:

My main physical screen is 24” with FullHD resolution. I duplicate this physical screen to another physical screen which is 10” and has touch capabilities. I use the touch to draw diagrams.
My two additional side screens are 32” with 4K resolution
My fifth screen is 10” big and I use it to display all the instant messengers I need (slacks, discords, meets etc). That’s basically a browser window that I configure to be visible on all of the desktops.
I have 11 desktops configured in VirtuaWin.
To connect remotely to my machine, I move to some other VirtuaWin desktop and use four physical screens (FullHD + 2x4K + touch-enabled screen) to open mstsc.exe.
When I need to connect to some other machine, I basically do CTRL+ALT+HOME to unfocus the current RDP, then I switch to another VirtuaWin desktop, and there I connect to the server. This way I can quickly switch between multiple machines I have

This setup works great for me. I can work remotely from any place around the planet. When I’m not at home, I can use my road runner laptop with two physical screens or my mobile phone to connect over RDP to the remote server. I do various things over RDP – I code, I write/read documents, I postprocess videos, I record videos (yes, I record them over RDP!), I browse the Internet. Typical office work.

I wanted to replicate this setup as much as possible with VR goggles. I managed to do that with Meta Quest 3 and vSpatial. Let’s see how I did that.

What we need

First, you need the goggles. I used Meta Quest 3 and they were quite good. I can’t complain on the image quality and I definitely can do all the stuff I need. The battery lasts for around 90-120 minutes which is okayish.

Next, you need to have a machine with many physical displays. That’s the most tricky part if you want to configure it entirely over RDP and use free version of vSpatial. I use two dedicated machines: one is Windows Server and the other is Mac. I first connect to Windows Server from my mobile phone over the regular RDP and log in as User1. At this point, I have a session in Windows Server that has just one display. Next, I use NoMachine to connect from Windows Server to my Mac. In my Mac, I create multiple physical screens using Better Display. Next, I connect from my Mac to my Windows Server and log in as User2 using the regular RDP application. At this point, I have a session in Windows Server that has multiple physical screens. All of that done entirely from my mobile phone, so I can do that when I’m travelling. While it works and allows for any setup of monitors, it’s probably good enough to just use 4 screens from paid vSpatial. Up to you.

Next, we need the vSpatial server. I install it in Windows Server for User2. I then log in to Windows Server using vSpatial application in Meta Quest 3 goggles. The vSpatial client correctly recognizes all 4 displays and now I have access to a regular Windows machine. I can install VirtuaWin and configure desktops the regular way. It all works just like on the screenshot below:

You can see one screen at the top and three screens in the middle. This is nearly the same setup as I have at home.

Finally, I just pair my bluetooth keyboard with touchpad and that’s it. CTRL+ALT+HOME works, so I can work the same way as at home.

Now, sky is the limit. Since I managed to RDP into a regular Windows machine, I can RDP anywhere I need, I can install any application I need, and I can do all of that using my goggles and the bluetooth keyboard only. I want to stress that there is no PCVR here. The Windows Machine with vSpatial server is completely virtualized and has no dedicated graphics card. Also, I don’t use powerful Internet connection. I just connect over regular WiFi. This works from mobile hotspot as well as it’s just RDP.

When it comes to online meetings, I can take them from my mobile phone. If I want to share the screen, I can just join the same meeting from my mobile phone and from my remote machine, and I can share the screen from Windows Server. However, you can also start browser on VR and join meeting like in Google Meet. The meeting will work even after you switch to vSpatial.

Quirks

There are quirks, obviously.

First, Meta Quest 3 seems to be based on Android. Bluetooth keyboards don’t work 100% properly with Android when you RDP to Windows. One issue is that the WIN key doesn’t work. Another, some shortcuts (like ALT+TAB) are captured by the host (goggles in this case).

To solve the lack of WIN key, I use PowerToys. I just remap CAPS LOCK to LEFT WIN on my Windows Server. You can also use AutoHotkey with the following script:

*CapsLock::
    Send {Blind}{LWin Down}
Return

*CapsLock up::
    Send {Blind}{LWin Up}
Return

The script rebinds CAPS LOCK to LEFT WIN. No matter which method you use, you can just use CAPS LOCK in all the shortcuts (like maximizing the window with LEFT WIN+ARROW UP which becomes CAPS LOCK+ARROW UP). While it’s a bit painful, it works. You need to install AutoHotkey in all the remote machines you connect to.

I didn’t solve the ALT+TAB issue with AutoHotkey. I solved it with PowerToys by remapping SCROLL LOCK to TAB. I think you could use AutoHotkey to remap it as well.

Second, vSpatial has a concept of active (focused) desktop. If you worked on one screen and want to click on another one, the mouse click changes the focus. You need to click again to do the actual click you want. This is not a big deal and I got used to it very fast.

Third, I think vSpatial is slightly slower than the regular RDP. I don’t find this as a problem, though, as it’s fast enough for me.

Fourth, the goggles don’t last for long (up to two hours). On the other hand, maybe it’s better for your eyes. You can also buy an additional strap with built-it powerbank.

Summary

It took me quite a while to figure out how to configure this setup. However, now I can truly work using goggles and no laptop around. That sounds like a great approach when traveling – imagine sitting at the airport and having multiple screens, just like at home. Now, the only issue is around the goggles. I hope one day they will be smaller, lighter, and will have stronger batteries. You can always plug them into powerbank if needed, though.

Availability Anywhere Part 22 — Customer Experience Improvement Program restarts

afish — Sun, 11 Feb 2024 10:10:10 +0000

This is the twentieth second part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

I was recently investigating a case of a computer restarting every Sunday at 3AM UTC. I couldn’t figure out what was going on and I suspected Customer Experience Improvement Program as I found the following event in event viewer:

User Logoff Notification for Customer Experience Improvement Program

- 
- 
		 
		1074 
		0 
		4 
		0 
		0 
		0x8080000000000000 
		 
		404828 
		 
		 
		System 
		computerName 
		 
	
- 
		C:\Windows\system32\shutdown.EXE (computerName) 
		computerName 
		No title for this reason could be found 
		0x800000ff 
		restart 
		System is scheduled to reboot in 10 minutes. Please save your work. 
		NT AUTHORITY\SYSTEM

I checked many solutions on the Internet and none of them helped. However, later I realized it was not due to Customer Experience Improvement Program. After carefully checking the event viewer, I found the following:

The process C:\Windows\system32\shutdown.EXE (computerName) has initiated the restart of computer computerName on behalf of user NT AUTHORITY\SYSTEM for the following reason: No title for this reason could be found
Reason Code: 0x800000ff
Shutdown Type: restart
Comment: System is scheduled to reboot in 10 minutes. Please save your work.

- 
- 
		 
		7002 
		0 
		4 
		1102 
		0 
		0x2000200000000000 
		 
		404868 
		 
		 
		System 
		computerName 
		 
	
- 
		2 
		S-1-5-21-1801674531-515967899-839522115-19733726

So it seems that something triggered the restart. I wasn’t sure what that was and I just solved it by cancelling the restart. Just run this batch script to cancel any shutdowns every 30 seconds:

:start
shutdown /a
timeout 30
goto start

It worked good enough.

Availability Anywhere Part 21 — Fixed mstsc.exe broken UI

afish — Sat, 16 Dec 2023 07:32:14 +0000

This is the twentieth first part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

I recently installed a set of November 23′ updates for Windows 10 and mstsc.exe disappeared! That was definitely unexpected and I couldn’t find a solution on the Internet. I decided to copy mstsc.exe from some other machine and it wasn’t as simple as possible.

TL;DR; Those are the files that you may need:

C:\Windows\System32\mstsc.exe
C:\Windows\System32\mstscax.dll
C:\Windows\System32\en-US\mstsc.exe.mui
C:\Windows\System32\en-US\mstscax.dll.mui
C:\Windows\System32\wbem\en-US\mstsc.mfl
C:\Windows\System32\wbem\en-US\mstscax.mfl
C:\Windows\SystemResources\mstsc.exe.mun
C:\Windows\SystemResources\mstscax.dll.mun

You may need similar files from C:\Windows\SysWOW64 or for different locales like pl-PL.

When you skip files in SystemResources, then your mstsc.exe will look like this (attaching this image so Google can index it and people have easier time finding things like “mstsc.exe no options” or “mstsc.exe UI broken” or “mstsc.exe no settings” or “mstsc.exe no show options”):

Copying files into C:\Windows\SystemResources is not as simple as it may seem. That directory is owned by TrustedInstaller and even SYSTEM has no permissions to write there. You can run shell like in this StackOverflow answer which I copy here just in case (run this in PowerShell started as Administrator):

Install-Module -Name NtObjectManager
Start-Service -Name TrustedInstaller
$parent = Get-NtProcess -ServiceName TrustedInstaller
$proc = New-Win32Process cmd.exe -CreationFlags NewConsole -ParentProcess $parent

Even though whoami shows nt authority\system, this effectively runs as TrustedInstaller.

If you wonder how I found out about these files, I used ProcessMonitor and looked for CreateFile operation that failed.

Availability Anywhere Part 19 — Banning RDP and SSH attacks

afish — Fri, 13 Jan 2023 09:00:05 +0000

This is the nineteenth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

If you expose RDP or OpenSSH to the wide Internet, you’ll most likely get automated attacks. There is a way to block these attacks with firewall, but I didn’t find a nice solution to do so, so I created my own.

The idea is as follows: we periodically scan event log to get failed authentication attempts. We extract the IP address, and then ban it if it happened too often.

Event for RDP is in Security with ID 4625:

- 
- 
   
  4625 
  0 
  0 
  12544 
  0 
  0x8010000000000000 
   
  27785286 
   
   
  Security 
  COMPUTER 
   
  
- 
  S-1-0-0 
  - 
  - 
  0x0 
  S-1-0-0 
  administrator 
  domainname 
  0xc000006d 
  %%2313 
  0xc000006a 
  3 
  NtLmSsp 
  NTLM 
  LOCALPCNAME 
  - 
  - 
  0 
  0x0 
  - 
  ADDRESS 
  PORT

For OpenSSH we extract these two events: Applications and Services -> OpenSSH -> Admin for IDs 1 and 2:

- 
- 
   
  1 
  0 
  1 
  0 
  0 
  0x8000000000000000 
   
  46166 
   
   
  OpenSSH/Admin 
  COMPUTER 
   
  
- 
  sshd 
  fatal: Timeout before authentication for ADDRESS port PORT

and

- 
- 
   
  2 
  0 
  2 
  0 
  0 
  0x8000000000000000 
   
  45523 
   
   
  OpenSSH/Admin 
  COMPUTER 
   
  
- 
  sshd 
  error: maximum authentication attempts exceeded for invalid user root from ADDRESS port PORT ssh2 [preauth]

We ban them with the following:

while($True){	
	write-host (Get-Date)
	$timeFrame = (Get-Date).AddMinutes(-15)
	$attackThreshold = 20
	$maxEvents = 10000

	$logsWithPatterns = @(
		@("Security", 4625, @("Source Network Address")),
		@("OpenSSH/Admin", 2, @("maximum authentication attempts")),
		@("OpenSSH/Admin", 1, @("Timeout before authentication"))
	)

	$newIpAddressesToBan = $logsWithPatterns | %{
		$log = $_[0]
		$id = $_[1]
		$patterns = $_[2]
		write-host $log, " ", $id, " ", $patterns
		
		Get-WinEvent $log -maxevents $maxEvents | ? {$_.id -eq $id } | ?{$_.TimeCreated -ge $timeFrame } | %{ $_.message} | select-string  $patterns | %{ [regex]::match($_,'(\d{1,4}[.]\d{1,4}[.]\d{1,4}[.]\d{1,4})') } | ?{ $_.Success} | %{$_.Groups[0].Value} | group-object | ?{$_.Count -ge $attackThreshold} | %{$_.Name}
	}

	$newIpAddressesToBan = @(($newIpAddressesToBan | %{ $_.Substring(0, $_.lastIndexOf(".")) + ".0/255.255.255.0"}) | sort | get-unique)

	$existingBannedIpAddresses = @((get-netfirewallrule -displayname "Rule for banning" | Get-NetFirewallAddressFilter).RemoteAddress | sort | get-unique)

	$allIpAddresses = (($newIpAddressesToBan + $existingBannedIpAddresses) | sort | get-unique)

	netsh advfirewall firewall set rule name="Rule for banning" new remoteip=([string]::Join(",", $allIpAddresses))
	
	sleep -s 60
}

I get at most 10000 events from the last 15 minutes, and I look for at least 20 attempts from a given IP. I extract IPv4 addresses with a regexp. I also replace them with a network subnet with mask of 24 bits.

Just run this script in the background with Administrator privileges and it will automatically add new IP addresses to the firewall. You need to create the rule Rule for banning manually beforehand as you need. If you need some other patterns or event logs, just add them to the collection at the beginning.

Availability Anywhere Part 18 — Binding same port for multiple docker containers

afish — Fri, 06 Jan 2023 09:00:44 +0000

This is the eighteenth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Let’s say that you run a project with Docker that uses a webserver (like a frontend application). This application will need to talk to some other service, like backend or database. We already know hw to forward port from host to the container. This way we can host whatever service locally, and talk to it from the container. We can also expose the webserver port (be it 8080) to the host, so we can connect to it from our browser running locally.

Let’s now consider similar problem but in the opposite direction. Let’s say that we have multiple webservers running in multiple dockers, and we want to be able to connect to all of them simultaneously from the host. How to do that?

The problem we typically face is how we bind ports. When using docker, we typically do the following:

docker -p 8080:80

with docker-compose we would do this:

ports:
  - "8080:80"

However, we can’t expose multiple applications this way because only one binding is allowed. Technically, lines above try to bind port 80 in the container to the port 8080 on the default IP address of the Docker daemon (which is 0.0.0.0) or 0.0.0.0 for docker-compose. This effectively binds all available interfaces we have. So, how to deal with this? Let’s see multiple steps we can take.

First step – changing port bindings

That’s the easiest solution. Just change port bindings (to something like 8081:80) and off you go. The problem is that you need to keep a mental map of the bindings, you may need to change the connection strings, and you need to remember not to push these changes to the repository. The last one is especially tricky because you need to change the files for docker-compose that normally you want push to the repository.

Second step – using environment variables

12-Factor App Methodology tells us to use environment variables as much as possible. Based on that, we should change docker commands and docker-compose files to accept environment variables for port bindings. Since we can put these values in .env file and not push them to the repository, we can effectively change our configuration easily without changing the source code. However, since the connection strings are often in files that we need to push to the repository, this may still be not enough.

Unfortunately, this won’t work well if we decide to run the frontend locally (instead of running it in the docker). Since the frontend connects to the default port, this port won’t be available on the host (or will be mapped to some other application).

Third step – use separate network interfaces

Separate network interfaces can give us even more. When we bind ports like 8080:80, we actually bind them as in 0.0.0.0:8080:80. We already mentioned that 0.0.0.0 means “bind all interfaces”. However, we can change this to some other IP address. The question is which one to use.

Windows lets us create new loopback network interface with hdwwiz.exe. See the last section of this post for the instructions how to do that. Linux and Mac let us do similar with aliases and new loopback interfaces. What’s more, we can assign some new IP address to the interface (like 192.168.123.5) and use hosts file to map subdomain like MYPROJECT.localhost to that address.

This way we can easily separate the traffic. We now bind all ports to 192.168.123.5:80:80, change connection strings to sql://MYPROJECT.localhost:PORT, and finally connect to the application from the browser easily on MYPROJECT.localhost:80. We can also configure Multi-Account Containers in Firefox or FoxyProxy in Firefox/Chrome to connect via Socks proxy forwarded from the docker container, so we would just connect as we were in the docker.

The cool part is that this will work the same way regardless of running that in the docker or locally. If we run inside the docker, then MYPROJECT.localhost:PORT will resolve to something like 127.0.0.1:PORT which works well because we forward port from the host to the docker container. However, if we run it locally, then it will resolve to 192.168.123.5:PORT that is accessible from the host thanks to the interface.

What’s more, we can modify docker-compose and docker commands transparently and push changes to the repository. Other developers on the team won’t be affected, and we will be able to separate traffic from multiple projects locally. If some other developer doesn’t have new interface configured and hosts file modified, then MYPROJECT.localhost will resolve to 127.0.0.1 and will still work.

Creating new network interface with IP address

In Windows

Run hdwwiz.exe
Install the hardware that I manually select from the list (Advanced)
Network adapters
Microsoft -> Microsoft KM-TEST Loopback Adapter
Go to network adapters. Rename the new adapter to MYPROJECT. Manually set the IP address to something like 192.168.123.5. Set mask to 255.255.255.255. Set DNS IP to 8.8.8.8
Edit C:\windows\system32\drivers\etc\hosts and add the following line: 192.168.123.5 MYPROJECT.localhost

In Linux

ifconfig lo:1 192.168.123.5/24

In mac

ifconfig lo0 alias 192.168.123.5

Availability Anywhere Part 17 — Splitting physical monitor into multiple

afish — Fri, 23 Dec 2022 09:00:14 +0000

This is the seventeenth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Let’s say that we have a big physical screen that we would like to split into multiple screens. We would like to support a proper full-screen behavior, so not just like splitting a screen into windows, but screens that are completely independent from each other.

We can emulate this with OBS and Indirect Display Driver Sample. Install the driver as specified in GitHub to get additional 5 virtual monitors (not desktops).

Now, the trick is to juggle with OBS to manage screens. Currently, I have 3 physical screens numbered 1, 2, and 3. They are physically configured like this:

------- ------- -------
|     | |     | |     |
|  1  | |  2  | |  3  |
|     | |     | |     |
------- ------- -------

I have 5 additional virtual screens numbered 4, 5, 6, 7, 8. I turn off screen number 8, and other screens put to the left of my middle screen (2), like this:

------- ---- ---  ------- -------
|     | | 4 | 5 | |     | |     |
|  1  | ---- ---  |  2  | |  3  |
|     | | 6 | 7 | |     | |     |
------- ---------- ------- ------

Now, I start OBS, create a scene of the size of screen 1. I create 4 display captures for monitors 4, 5, 6, 7. I put them in the corners.
Finally, I select “Full Screen Projector Preview” and put it on the screen 1. This way I can move my mouse from screen 2 to the left to enter screens 5+7 and then 4+6. If I move the mouse pointer more to the left, then I end up in screen 1, but I don’t want to do it.

This works as physical monitors where I can use only screens 2-7. I can’t use screen 1 (as it would obscure screens 4-7), but that’s not a problem. I have 6 screens effectively at this point. Full screen works as expected. The problem is with connecting over RDP with multiple screens support, because then screen 1 obscures all screens 4-7. We could fix that by running similar OBS in the remote environment.

You can achieve the same in Mac with BetterDisplay.

Availability Anywhere Part 16 — Forwarding port from host to docker

afish — Sat, 17 Dec 2022 09:00:23 +0000

This is the sixteenth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Let’s say that we want to expose a port from the host machine to the docker container. For instance, we have a locally installed database and we want to access it from the docker container. If we’re on Linux, then we can use --network host and that will do the trick for us. However, this option is not supported on Mac or Windows. Let’s see how to do it in that case.

We are going to run an OpenSSH server inside the docker network, connect to it from the host, forward a remote port, and then connect to localhost.

Let’s start with creating a directory for the docker project:

mkdir -p ssh_tunnel

Let’s create SSH keys.

ssh-keygen -b 2048 -t rsa -f ssh_tunnel/tunnel_rsa

Next, create the configuration for OpenSSH. Just a default configuration with port forwarding enabled (AllowTcpForwarding set to yes). This is in file ssh_tunnel/sshd_config:

#	$OpenBSD: sshd_config,v 1.104 2021/07/02 05:11:21 dtucker Exp $

# This is the sshd server system-wide configuration file.  See
# sshd_config(5) for more information.

# This sshd was compiled with PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin

# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented.  Uncommented options override the
# default value.

Port 2222
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::

#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_ecdsa_key
#HostKey /etc/ssh/ssh_host_ed25519_key

# Ciphers and keying
#RekeyLimit default none

# Logging
#SyslogFacility AUTH
#LogLevel INFO

# Authentication:

#LoginGraceTime 2m
#PermitRootLogin prohibit-password
#StrictModes yes
#MaxAuthTries 6
#MaxSessions 10

#PubkeyAuthentication yes

# The default is to check both .ssh/authorized_keys and .ssh/authorized_keys2
# but this is overridden so installations will only check .ssh/authorized_keys
AuthorizedKeysFile	.ssh/authorized_keys

#AuthorizedPrincipalsFile none

#AuthorizedKeysCommand none
#AuthorizedKeysCommandUser nobody

# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes

# To disable tunneled clear text passwords, change to no here!
PasswordAuthentication no
#PermitEmptyPasswords no

# Change to no to disable s/key passwords
#KbdInteractiveAuthentication yes

# Kerberos options
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no

# GSSAPI options
#GSSAPIAuthentication no
#GSSAPICleanupCredentials yes

# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the KbdInteractiveAuthentication and
# PasswordAuthentication.  Depending on your PAM configuration,
# PAM authentication via KbdInteractiveAuthentication may bypass
# the setting of "PermitRootLogin without-password".
# If you just want the PAM account and session checks to run without
# PAM authentication, then enable this but set PasswordAuthentication
# and KbdInteractiveAuthentication to 'no'.
#UsePAM no

#AllowAgentForwarding yes
# Feel free to re-enable these if your use case requires them.
AllowTcpForwarding yes
GatewayPorts no
X11Forwarding no
#X11DisplayOffset 10
#X11UseLocalhost yes
#PermitTTY yes
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#PermitUserEnvironment no
#Compression delayed
#ClientAliveInterval 0
#ClientAliveCountMax 3
#UseDNS no
PidFile /config/sshd.pid
#MaxStartups 10:30:100
#PermitTunnel no
#ChrootDirectory none
#VersionAddendum none

# no default banner path
#Banner none

# override default of no subsystems
Subsystem	sftp	internal-sftp

# Example of overriding settings on a per-user basis
#Match User anoncvs
#	X11Forwarding no
#	AllowTcpForwarding no
#	PermitTTY no
#	ForceCommand cvs server

And the ssh_tunnel/Dockerfile:

FROM lscr.io/linuxserver/openssh-server:latest

COPY sshd_config /config/ssh_host_keys/sshd_config

We could mount the volume instead, but I couldn’t get it to work with file permissions in Mac. Copying the file is good enough.

That’s it. We can now start the machinery. First, build the image:

docker build -t image_ssh ssh_tunnel

Now, let’s start or run the container with OpenSSH server:

docker start container_ssh 2>/dev/null || docker run -d \
  --name=container_ssh \
  -e TZ=Etc/UTC \
  -e "PUBLIC_KEY=YOUR_GENERATED_PUBLIC_KEY_HERE" \
  -p 127.0.0.1:58222:2222 \
  -p 127.0.0.1:YOUR_APPLICATION_PORT:YOUR_APPLICATION_PORT \
  -e USER_NAME=tunnel \
  --restart unless-stopped \
  image_ssh

Now, let’s clear the saved SSH fingerprint (in case when we restarted everything), kill existing session (if there is one), and connect to the container:

ssh-keygen -R '[localhost]:58222'
pkill -f "ssh -i $(pwd)/ssh_tunnel/tunnel_rsa tunnel@localhost -p 58222"
ssh -i "$(pwd)"/ssh_tunnel/tunnel_rsa tunnel@localhost -p 58222 -4 -o StrictHostKeyChecking=no -R YOUR_DEPENDENCY_PORT:127.0.0.1:YOUR_DEPENDENCY_PORT -fN

YOUR_DEPENDENCY_PORT is the port you’d like to forward to the container. This could be 5432 of your PostgreSQL server hosted locally, for instance. You can also see that we expose two ports from the container: 2222 as 58222 in order to connect to the OpenSSH server, and YOUR_APPLICATION_PORT which can be another application or dependency that you’d like to expose to another docker container or to external world.

Let’s now start or run our application:

docker start container_application 2>/dev/null || docker run \
  --name=container_application \
  --network 'container:container_ssh' \
  image_application

This way the application starts in the same network, so it can access localhost:YOUR_DEPENDENCYPORT that will go to the host via OpenSSH. At the same time, your container exposes YOUR_APPLICATION_PORT the regular way, so you can then configure additional tunnels.

You can clean all things up with the following (mind that it may remove some of your unused things):

ssh-keygen -R '[localhost]:58222'
pkill -f "ssh -i $(pwd)/ssh_tunnel/tunnel_rsa tunnel@localhost -p 58222"
docker stop container_application
docker rm container_application
docker rmi --force image_application
docker stop container_ssh 
docker rm container_ssh 
docker rmi image_ssh 
docker system prune
docker volume prune

Tested with Amazon Linux 2 and Ventura Mac with M1 chip. Should work with Windows as well, as long as you install OpenSSH client on your host (or use putty instead) and have some decent terminal. If you prefer to use PowerShell, then go with this code:

$location = (get-location).path
$Key = "${location}\ssh_tunnel\tunnel_rsa"
Icacls $Key /c /t /Inheritance:d
TakeOwn /F $Key
Icacls $Key /c /t /Grant:r ${env:UserName}:F
Icacls $Key /c /t /Remove:g Administrator "Authenticated Users" BUILTIN\Administrators BUILTIN Everyone System Users
Icacls $Key

docker build -t image_ssh ssh_tunnel

docker start container_ssh

if($LASTEXITCODE -ne 0){
docker run -d `
  --name=container_ssh`
  -e TZ=Etc/UTC `
  -e "PUBLIC_KEY=YOUR_GENERATED_PUBLIC_KEY_HERE" `
  -p 127.0.0.1:58222:2222 `
  -p 127.0.0.1:YOUR_APPLICATION_PORT:YOUR_APPLICATION_PORT `
  -e USER_NAME=tunnel `
  --restart unless-stopped `
  image_ssh
}

sleep 5

ssh-keygen -R '[localhost]:58222'
$replacedLocation = $location.replace("\", "/")
(Get-WmiObject win32_process -filter "Name='ssh.exe' AND CommandLine LIKE '%${replacedLocation}/ssh_tunnel/tunnel_rsa tunnel@localhost -p 58222%'").Terminate()
$sshStartBlock = { param([string]$pwd) ssh -i "$pwd/ssh_tunnel/tunnel_rsa" tunnel@localhost -p 58222 -4 -o StrictHostKeyChecking=no -R YOUR_DEPENDENCY_PORT:127.0.0.1:YOUR_DEPENDENCY_PORT -fN }
start-job -ScriptBlock $sshStartBlock -ArgumentList $replacedLocation

docker start container_application
if($LASTEXITCODE -ne 0){
docker run `
  --name=container_application `
  --network 'container:container_ssh' `
  image_application
}

This was tested with English-based Windows Server 2016 Datacenter. You may need to replace user names for different locales. Also, connecting to localhost should work with IPv4. If it defaults to IPv6, then change it to 127.0.0.1

Cleaning in PowerShell:

ssh-keygen -R '[localhost]:58222'
$location = (get-location).path
$replacedLocation = $location.replace("\", "/")
(Get-WmiObject win32_process -filter "Name='ssh.exe' AND CommandLine LIKE '%${replacedLocation}/ssh_tunnel/tunnel_rsa tunnel@localhost -p 58222%'").Terminate()
docker stop container_application
docker rm container_application
docker rmi --force image_application
docker stop container_ssh 
docker rm container_ssh 
docker rmi image_ssh 
docker system prune
docker volume prune

What’s more, we can make this dance with keys even simpler by using sish. That’s basically an OpenSSH server that allows for empty password authentication and other stuff, just like serveo.net

Availability Anywhere Part 15 — TCP over Serial Port

afish — Sat, 03 Dec 2022 09:00:54 +0000

This is the fifteenth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

You can’t forward a named pipe between the host and the guest VM, but we can forward the named pipe as a serial port to the VM. So the host writes to the named pipe, and the VM reads from the serial port. Let’s see the server now:

using System;
using System.IO.Ports;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Threading;
using System.IO;

namespace PipeProxy
{
    class PortServer
    {
        public static void Start(string[] args)
        {
            var destinationIp = args[1].Split(':')[0];
            var destinationPort = int.Parse(args[1].Split(':')[1]);
            var portName = args[2];

            Console.WriteLine($"Routing to {destinationIp}:{destinationPort} via {portName}");

            Console.WriteLine("Connecting");
            //In theory I can set baud rate to 268435456
            //In practice the max is 115200
            SerialPort port = new SerialPort(portName, 115200, Parity.None, 8, StopBits.One)
            {
                ReadBufferSize = 10000000,
                WriteBufferSize = 10000000
            };

            while (true)
            {
                try
                {
                    while (true)
                    {
                        Thread.Sleep(1000);
                        if (port.IsOpen)
                        {
                            continue;
                        }
                        else
                        {
                            port.Open();
                            Console.WriteLine("Connected");
                        }

                        // I'm forcing IPv4 (IPv6 breaks with Cisco AnyConnect)
                        IPEndPoint remoteEP = new IPEndPoint(Dns.GetHostEntry(destinationIp).AddressList.First(a => a.AddressFamily == AddressFamily.InterNetwork), destinationPort);
                        Socket senderSocket = new Socket(remoteEP.AddressFamily, SocketType.Stream, ProtocolType.Tcp);

                        senderSocket.Connect(remoteEP);
                        Console.WriteLine("Socket connected to {0}", senderSocket.RemoteEndPoint);

                        new Thread(() =>
                        {
                            var socket = new PortSocket(senderSocket, port, r => { }, s => { });
                            socket.Start();
                        }).Start();
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine("Exception " + e);
                }
            }
        }
    }
	
    public class PortSocket
    {
        private Socket clientSocket;
        private SerialPort port;
        private Action received;
        private Action sent;
        public PortSocket(Socket socket, SerialPort port, Action received, Action sent)
        {
            this.clientSocket = socket;
            this.port = port;
            this.received = received;
            this.sent = sent;
        }

        public void Start()
        {
            Thread clientThread = new Thread(() =>
            {
                try
                {
                    var buffer = new byte[100000];
                    while (true)
                    {
                        var read = clientSocket.Receive(buffer);

                        if (read == 0)
                        {
                            port.Close();
                            return;
                        }

                        while (true)
                        {
                            try
                            {
                                port.Write(buffer, 0, read);
                                sent(read);
                                break;
                            }
                            catch (Exception e2)
                            {
                                Console.WriteLine("Exception when writing to pipe" + e2);
                            }
                        }
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine("Exception in client thread: " + e);
                }
            });

            try
            {
                Thread senderThread = new Thread(KeepReading);
                senderThread.Start();
            }
            catch (Exception e)
            {
                Console.WriteLine("Exception  " + e);
            }

            clientThread.Start();
            clientThread.Join();
        }

        private void KeepReading()
        {
            int totalRead = 0;
            int toRead = 100024;

            byte[] bytes = new byte[toRead];
            try
            {
                while (true)
                {
                    int howMuchRead = port.Read(bytes, 0, toRead);

                    if (howMuchRead == 0)
                        break;

                    totalRead += howMuchRead;
                    received(howMuchRead);
                    clientSocket.Send(bytes, 0, howMuchRead, SocketFlags.None);
                }
            }
            catch (Exception e)
            {
                Console.WriteLine("Exception while reading" + e);
            }
        }
    }
}

And here is the benchmark:

Pipe -> Serial (115200 baud rate)
Mode: PipeProxyVirtualMachine
1       0.993
10      2.364
100     2.452
1000    9.119
10000   85.12
100000  844.847

It works, but slows down linearly with the size. Why? That’s because the serial port has a max bandwidth (that we call baud rate). Here it is 115200 (around 112 kBps). While this works much faster for small packets, this won’t give a decent performance for RDP with bigger bandwidth requirements.

Can we do better? One solution would be to emulate a COM port that has a bigger bandwidth. However, I don’t know a solution yet.

Availability Anywhere Part 14 — TCP over Named Pipe

afish — Sat, 26 Nov 2022 09:00:52 +0000

This is the fourteenth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

We built a very nice TCP over file System solution. Let’s now implement something similar, based on named pipes. Here comes the code:

using System;
using System.Collections.Generic;
using System.IO.Pipes;
using System.Linq;
using System.Net;
using System.Net.Sockets;
using System.Threading;

namespace PipeProxy
{
    class Program
    {
        static void Main(string[] args)
        {
            if (args.Length < 2)
            {
                Exit(Usage());
            }

            if (args[0] == "client")
            {
                Client.Start(args);
            }
            else if (args[0] == "pipe_server")
            {
                PipeServer.Start(args);
            }
            else
            {
                Exit(Usage());
            }
        }

        private static string Usage()
        {
            return "PipeProxy.exe client local_port pipe_name\nPipeProxy.exe pipe_server destination_ip:destination_port pipe_name\nPipeProxy.exe port_server destination_ip:destination_port por_name";
        }

        private static void Exit(string message)
        {
            Console.WriteLine(message);
            Environment.Exit(0);
        }
    }
	
    public class Client
    {
        private static Random random = new Random();

        public static void Start(string[] args)
        {
            var localPort = int.Parse(args[1]);
            var pipeName = args[2];

            Console.WriteLine($"Routing from {localPort} via {pipeName}");

            IPEndPoint localEndPoint = new IPEndPoint(0, localPort);
            Socket listener = new Socket(localEndPoint.AddressFamily, SocketType.Stream, ProtocolType.Tcp);
            listener.Bind(localEndPoint);
            listener.Listen(100);

            var totalSent = new Dictionary();
            var totalReceived = new Dictionary();
            var totalExceptions = new Dictionary();
            var localDnss = new[] {"localhost"};


            foreach (var localDns in localDnss)
            {
                totalSent[localDns] = 0;
                totalReceived[localDns] = 0;
                totalExceptions[localDns] = 0;
            }

            new Thread(() =>
            {
                while (true)
                {
                    Console.Write(DateTime.Now);
                    Console.Write(" E/S/R:\t");
                    Console.WriteLine(string.Join("\t", localDnss.Select(dns => $"{dns}: {totalExceptions[dns]}/{totalSent[dns]}/{totalReceived[dns]}")));

                    Thread.Sleep(3000);
                }
            }).Start();

            while (true)
            {
                try
                {
                    while (true)
                    {
                        Socket socket = listener.Accept();
                        Console.WriteLine("New connection accepted to be scattered");

                        new Thread(() => Start(socket, localDnss, pipeName, random.Next(), totalSent, totalReceived, totalExceptions)).Start();
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine("Exception " + e);
                }
            }
        }

        private static void Start(Socket clientSocket, string[] localDnss, string pipeName, int identifier,
            Dictionary sent, Dictionary received, Dictionary exceptions)
        {
            try
            {
                Console.WriteLine("Connecting");
                var pipeClient = new NamedPipeClientStream(".", pipeName,  PipeDirection.InOut, PipeOptions.Asynchronous);
                pipeClient.Connect();
                Console.WriteLine("Connected");

                var fileSocket = new PipeSocket(clientSocket, pipeClient, r => received[localDnss[0]]+= r, s => sent[localDnss[0]]+=s);
                fileSocket.Start();
            }
            catch (Exception e)
            {
                Console.WriteLine("Exception " + e);
            }
        }
    }
    class PipeServer
    {
        public static void Start(string[] args)
        {
            var destinationIp = args[1].Split(':')[0];
            var destinationPort = int.Parse(args[1].Split(':')[1]);
            var pipeName = args[2];

            Console.WriteLine($"Routing to {destinationIp}:{destinationPort} via {pipeName}");

            NamedPipeServerStream pipeServer = new NamedPipeServerStream(pipeName, PipeDirection.InOut, NamedPipeServerStream.MaxAllowedServerInstances, PipeTransmissionMode.Byte, PipeOptions.Asynchronous);

            while (true)
            {
                try
                {
                    while (true)
                    {
                        Thread.Sleep(1000);

                        pipeServer.WaitForConnection();
                        // I'm forcing IPv4 (IPv6 breaks with Cisco AnyConnect)
                        IPEndPoint remoteEP = new IPEndPoint(Dns.GetHostEntry(destinationIp).AddressList.First(a => a.AddressFamily == AddressFamily.InterNetwork), destinationPort);
                        Socket senderSocket = new Socket(remoteEP.AddressFamily, SocketType.Stream, ProtocolType.Tcp);

                        senderSocket.Connect(remoteEP);
                        Console.WriteLine("Socket connected to {0}", senderSocket.RemoteEndPoint);

                        new Thread(() =>
                        {
                            var socket = new PipeSocket(senderSocket, pipeServer, r => { }, s => { });
                            socket.Start();
                        }).Start();
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine("Exception " + e);
                    if (e.Message.Contains("being closed"))
                    {
                        pipeServer = new NamedPipeServerStream(pipeName, PipeDirection.InOut, NamedPipeServerStream.MaxAllowedServerInstances, PipeTransmissionMode.Byte, PipeOptions.Asynchronous);
                    }
                }
            }
        }
    }
	
    public class PipeSocket
    {
        private Socket clientSocket;
        private Stream pipeSocket;
        private Action received;
        private Action sent;
        public PipeSocket(Socket socket, Stream pipeSocket, Action received, Action sent)
        {
            this.clientSocket = socket;
            this.pipeSocket = pipeSocket;
            this.received = received;
            this.sent = sent;
        }

        public void Start()
        {
            Thread clientThread = new Thread(() =>
            {
                try
                {
                    var buffer = new byte[100000];
                    while (true)
                    {
                        var read = clientSocket.Receive(buffer);

                        if (read == 0)
                        {
                            pipeSocket.Close();
                            return;
                        }

                        while (true)
                        {
                            try
                            {
                                pipeSocket.Write(buffer, 0, read);
                                sent(read);
                                break;
                            }
                            catch (Exception e2)
                            {
                                Console.WriteLine("Exception when writing to pipe" + e2);
                            }
                        }
                    }
                }
                catch (Exception e)
                {
                    Console.WriteLine("Exception in client thread: " + e);
                }
            });

            try
            {
                Thread senderThread = new Thread(KeepReading);
                senderThread.Start();
            }
            catch (Exception e)
            {
                Console.WriteLine("Exception  " + e);
            }

            clientThread.Start();
            clientThread.Join();
        }

        private void KeepReading()
        {
            int totalRead = 0;
            int toRead = 100024;

            byte[] bytes = new byte[toRead];
            try
            {
                while (true)
                {
                    int howMuchRead = pipeSocket.Read(bytes, 0, toRead);

                    if (howMuchRead == 0)
                        break;

                    totalRead += howMuchRead;
                    received(howMuchRead);
                    clientSocket.Send(bytes, 0, howMuchRead, SocketFlags.None);
                }
            }
            catch (Exception e)
            {
                Console.WriteLine("Exception while reading" + e);
            }
        }
    }
}

And now the benchmark:

Mode: PipeProxy
1       0.687
10      2.024
100     1.998
1000    2.039
10000   2.157
100000  3.131

This makes it a little slower than the TCP over file System run locally. However, we can’ run this solution between the host and the guest VM (at least I don’t know how to route named pipes to the VM), but we’ll use this code for something else next time.