This is the sixth part of the Bit Twiddling series. For your convenience you can find other parts in the table of contents in Par 1 — Modifying Android application on a binary level

Today we’re going to solve the problem when the remote server disconnects GUI when we lock the workstation or disconnect.

Let’s start by explaining the problem a little bit more. Imagine that you connect like this:

Local ---> Remote

If you now lock the Local workstation (with WIN+L) or disconnect, then the GUI on Remote will break. The user on Remote will not be logged out, the applications will continue to work, but the UI will not be there. This effectively breaks things like VNC or apps that take screenshots or click on the screen.

Similarly, the same happens when you connection with a jump host:

Local ---> Windows Server 2019 ---> Remote

If you now disconnect Local or lock Local workstation, then the GUI on Remote will break. However, it’s different if you use Windows Server 2016:

Local ---> Windows Server 2016 ---> Remote

If you now disconnect Local or lock the workstation, then the GUI on Remote will still be there. This suggests that something has changed in Windows Server 2019. This is not surprising as this is the time when Microsoft implemented RemoteFX heavily into their RDP to support cameras, USB redirection, improve security and much more.

At this point we have one solution – just use the Windows Server 2016 jump host and it fixes the issue. However, this way you won’t use the camera (it doesn’t work in 2016 typically) and you need to keep another machine along the way (this could be a virtual machine on your Remote, though). Let’s see if we can do better.

Fixing the workstation lock

The first thing to notice is that if you suspend the mstsc.exe on Local (with Task Manager or debugger) and then lock the workstation, then Remote doesn’t lose the UI. This suggests that disconnecting the GUI is an explicit action of the RDP implementation. However, we can easily fix that.

mstsc.exe registers for session notifications with WTSRegisterSessionNotification. We can see that with ApiMonitor:

We can simply hook this method to not register for any notifications. We can use that with the following WinDBG script:

.sympath srv*C:\tmp*http://msdl.microsoft.com/download/symbols
.reload
bu	WTSAPI32!WTSRegisterSessionNotification	""e @rip 0x48 0xc7 0xc0 0x01 0x00 0x00 0x00 0xC3; qd""
g

The shell code we put here is simply:

move rax, 1
ret

You can run mstsc.exe, attach the debugger, add the breakpoint, resume the application, and then connect to the server. The debugger will detach shortly after you open the connection and you can check that now you can lock the workstation and the GUI will still be there.

Fixing the disconnection

Fixing the case on disconnect is slightly harder. Once again, when we pause the mstsc.exe, then the remote session still works even if we keep mstsc.exe paused for a long time. This suggests that the session is up as long as the connection is alive. Even though we paused the mstsc.exe application, the operating system keeps the connection up on the TCP level. We can exhibit that.

The idea is to run a proxy on the Remote. We’ll then connect like this:

Local ---> Proxy ---> Remote

The proxy must just route the connection to the RDP server (on the port 3389),but do not lose the connection to Remote when the connection from Local disappears. This way we can simply kill the mstsc.exe on Local, and then the connection will still be alive and the GUI will persist. However, if you close mstsc.exe gracefully, then the client will terminate the GUI on the remote server. Therefore, just kill the client. You can also put your machine to sleep and the UI should still work.

This is the fifth part of the Bit Twiddling series. For your convenience you can find other parts in the table of contents in Par 1 — Modifying Android application on a binary level

Today we’re going to fix the audio latency in mstsc.exe. Something that people really ask about on the Internet and there is no definite solution. I’ll show how to hack mstsc.exe to fix the latency. First, I’ll explain why existing solutions do not work and what needs to be done, and at the end of this post I provide an automated PowerShell script that does the magic.

What is the issue

First, a disclaimer. I’ll describe what I suspect is happening. I’m not an expert of the RDP and I didn’t see the source code of mstsc.exe. I’m just guessing what happens based on what I see.

RDP supports multiple channels. There is a separate channel for video and another one for audio. Unfortunately, these two channels are not synchronized which means that they are synchronized on the client on a best effort basis (or rather sheer luck). To understand why it’s hard to synchronize two independent streams, we need to understand how much different they are.

Video is something that doesn’t need to be presented “for some time”. We can simply take the last video frame and show it to the user. If frames are buffered, we just process them as fast as possible to present the last state. You can see that mstsc.exe does exactly that by connecting to a remote host, playing some video, and then suspending the mstsc.exe process with ProcessExplorer. When you resume it after few seconds, you’ll see the video moving much faster until the client catches up with the latest state.

When it comes to audio, things are much different. You can’t just jump to the latest audio because you’d loose the content as it would be unintelligible. Each audio piece has its desired length. When you get delayed, you could just skip the packets (and lose some audio), play it faster (which would make it less intelligible for some time), or just play it as it goes. However, to decide what to do, you would need to understand whether the piece you want to play is delayed or not. You can’t tell that without timestamps or time markers, and I believe RDP doesn’t send those.

As long as you’re getting audio packets “on time”, there is no issue. You just play them. The first part is if you get them “in time”. This depends on your network quality and on the server that sends you the sound. From my experience, Windows Server is better when it comes to speed of sending updates. I can see my mouse moving faster and audio delayed less often when I connect to the Windows Server than Windows 10 (or other client edition). Therefore, use Windows Server where possible. Just keep in mind that you’ll need CAL licenses to send microphone and camera to the server which is a bummer (client edition lets you do that for free). Making the packets to be sent as fast as possible is the problem number one.

However, there is another part to that. If your client gets delayed for whatever reason (CPU spike, overload, or preemption), your sound will effectively slow down. You will just play it “later” even though you received it “on time”. Unfortunately, RDP cannot detect whether this happened because there are no timestamps in the stream. As far as I can tell, this is the true reason why your sound is often delayed. You can get “no latency audio” over the Internet and I had it many times. However, the longer you run the client, the higher the chance that you’ll go out of sync. This is the problem number two.

Therefore, we need to “resync” the client. Let’s see how to do it.

Why existing solutions don’t work and what fix we need

First, let me explain why existing solutions won’t work. Many articles on the Internet tell you to change the audio quality to High in Group Policy and enforce the quality in your rdp.file by setting audioqualitymode:i:2. This fixes the problem number one (although I don’t see much difference to be honest), but it doesn’t address the problem number two.

Some other articles suggest other fixes on the remote side. All these fixes have one thing in common – they don’t fix the client. If the mstsc.exe client cannot catch up when it gets delayed, then the only thing you can do is to reset the audio stream. Actually, this is how you can fix the delay easily – just restart the audio service:

net stop audiosrv & timeout 3 & net start audiosrv

I add the timeout to give some time to clear the buffer on the client. Once the buffer is empty, we restart the service and then the audio should be in sync. Try this to verify if it’s physically possible to deliver the audio “on time” in your networking conditions.

Unfortunately, restarting the audio service have many issues. First, it resets the devices on the remote end, so your audio streams may break and you’ll need to restart them. Second, this simply takes time. You probably don’t want to lose a couple of seconds of audio and microphone when you’re presenting (and unfortunately, you’ll get delayed exactly during that time).

What we need is to fix the client to catch up when there is a delay. However, how can we do that when we don’t have any timestamps? Well, the solution is simple – just drop some audio packages (like 10%) periodically to sync over time. This will decrease the audio quality to some extent and won’t fix the delay immediately, but after few seconds we’ll get back on track. Obviously, you could implement some better heuristics and solutions. There is one problem, though – we need to do that in the mstsc.exe itself. And here comes the low level magic. Let’s implement the solution that drops the audio frames to effectively “resync” the audio.

How to identify

We need to figure out how the sound is played. Let’s take ApiMonitor to trace the application. Luckily enough, it seems that the waveOutPrepareHeader is used, as we can see in the screenshot:

Let’s break there win WinDBG:

bu 0x00007ffb897d3019

kb

 # RetAddr               : Args to Child                                                           : Call Site														
00 00007ffb`897d1064     : 00000252`6d68beb8 00000252`6d68beb8 00000000`000014ac 00000252`6d68be00 : mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x6d														
01 00007ffb`897e7ee8     : 00000000`00000000 00000073`bfc7fcb9 00000252`6d68be00 00000252`7e69df80 : mstscax!CRdpWinAudioWaveoutPlayback::vcwaveWritePCM+0xec														
02 00007ffb`898e73bf     : 00000000`00000001 00000000`00000003 00000073`bfc7d055 00000073`00001000 : mstscax!CRdpWinAudioWaveoutPlayback::RenderThreadProc+0x2c8														
03 00007ffc`02e57344     : 00000000`000000ac 00000252`6d68be00 00000000`00000000 00000000`00000000 : mstscax!CRdpWinAudioWaveoutPlayback::STATIC_ThreadProc+0xdf														
04 00007ffc`046826b1     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x14														
05 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21

We can see a method named mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite. Let’s see it:

u mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x6d		
									
mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite:											
00007ffb`897d2fac 48895c2410      mov     qword ptr [rsp+10h],rbx											
00007ffb`897d2fb1 55              push    rbp											
00007ffb`897d2fb2 56              push    rsi											
00007ffb`897d2fb3 57              push    rdi											
00007ffb`897d2fb4 4883ec40        sub     rsp,40h											
00007ffb`897d2fb8 488bf2          mov     rsi,rdx											
00007ffb`897d2fbb 488bd9          mov     rbx,rcx											
00007ffb`897d2fbe 488b0543287500  mov     rax,qword ptr [mstscax!WPP_GLOBAL_Control (00007ffb`89f25808)]											
00007ffb`897d2fc5 488d2d3c287500  lea     rbp,[mstscax!WPP_GLOBAL_Control (00007ffb`89f25808)]											
00007ffb`897d2fcc 483bc5          cmp     rax,rbp											
00007ffb`897d2fcf 740a            je      mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x2f (00007ffb`897d2fdb)											
00007ffb`897d2fd1 f6401c01        test    byte ptr [rax+1Ch],1											
00007ffb`897d2fd5 0f85e8000000    jne     mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x117 (00007ffb`897d30c3)											
00007ffb`897d2fdb 83bbc000000000  cmp     dword ptr [rbx+0C0h],0											
00007ffb`897d2fe2 7418            je      mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x50 (00007ffb`897d2ffc)											
00007ffb`897d2fe4 488b8bb8000000  mov     rcx,qword ptr [rbx+0B8h]											
00007ffb`897d2feb 4885c9          test    rcx,rcx											
00007ffb`897d2fee 740c            je      mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x50 (00007ffb`897d2ffc)											
00007ffb`897d2ff0 48ff15a9295c00  call    qword ptr [mstscax!_imp_EnterCriticalSection (00007ffb`89d959a0)]											
00007ffb`897d2ff7 0f1f440000      nop     dword ptr [rax+rax]											
00007ffb`897d2ffc 488b4b70        mov     rcx,qword ptr [rbx+70h]											
00007ffb`897d3000 4885c9          test    rcx,rcx											
00007ffb`897d3003 0f84f2000000    je      mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x14f (00007ffb`897d30fb)											
00007ffb`897d3009 41b830000000    mov     r8d,30h											
00007ffb`897d300f 488bd6          mov     rdx,rsi											
00007ffb`897d3012 48ff15e7a87900  call    qword ptr [mstscax!_imp_waveOutPrepareHeader (00007ffb`89f6d900)]											
00007ffb`897d3019 0f1f440000      nop     dword ptr [rax+rax]

Okay, we can see that this method passes the audio packet to the API. When we look at WAVEHDR structure, we can see that it has the following fields:

typedef struct wavehdr_tag {
  LPSTR              lpData;
  DWORD              dwBufferLength;
  DWORD              dwBytesRecorded;
  DWORD_PTR          dwUser;
  DWORD              dwFlags;
  DWORD              dwLoops;
  struct wavehdr_tag  *lpNext;
  DWORD_PTR          reserved;
} WAVEHDR, *LPWAVEHDR;

This is exactly what we see in ApiMonitor. Seems like the dwBufferLength is what we might want to change. When we shorten this buffer, we’ll effectively make the audio last shorter. We can do that for some of the packets to not break the quality much, and then all should be good.

We can verify that this works with this breakpoint:

bp 00007ffb`897d3012 "r @$t0 = poi(rdx + 8); r @$t1 = @$t0 / 2; ed rdx+8 @$t1; g"

Unfortunately, this makes the client terribly slow. We need to patch the code in place. Effectively, we need to inject a shellcode.

First, we need to allocate some meory with VirtualAllocEx via .dvalloc.

.dvalloc 1024

The debugger allocates the memory. In my case the address is 25fb8960000.

The address of the WinAPI function is in the memory, so we need to remember to extract the pointer from the address:

00007ffb`897d3012 48ff15e7a87900  call    qword ptr [mstscax!_imp_waveOutPrepareHeader (00007ffb`89f6d900)]

Now we need to do two things: first, we need to patch the call site to call our shellcode instead of [mstscax!_imp_waveOutPrepareHeader (00007ffb89f6d900)]. Second, we need to construct a shell code that fixes the audio packet for some of the packets, and then calls [mstscax!_imp_waveOutPrepareHeader (00007ffb89f6d900)] correctly.

To do the first thing, we can do the absolute jump trick. We put the address in the rax register, push it on the stack, and then return. This is the code:

mov rax, 0x25fb8960000	;Move the address to the register
push rax		;Push the address on the stack
ret			;Return. This takes the address from the stuck and jumps
nop			;Nops are just to not break the following instructions when you disassemble with u
nop
nop
nop

We can compile the code with Online assembler and we should get a shell code. We can then put it in place with this line:

e	mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+5d		0x48 0xB8 0x00 0x00 0x96 0xb8 0x5f 0x02 0x00 0x00 0x50 0xC3 0x90 0x90 0x90 0x90

Unfortunately, this patch is long. We need to break few lines and then restore them in the shellcode. So our shell code starts with the lines that we broke:

mov r8d, 0x30	;Preserved code
mov rdx,rsi	;Preserved code

Next, we need to preserve our working registers:

push rbx
push rdx

Okay, now we can do the logic. We want to modify the buffer length. However, we don’t want to do it for all of the packets. We need some source of randomness, like time, random values, or something else. Fortunately, the WAVEHDR structure has the field dwUser which may be “random enough” for our needs. Let’s take that value modulo some constant, and then change the packet length only for some cases.

First, let’s preserve the buffer length for the sake of what we do later:

mov rax, [rdx + 8]	;Load buffer length (that's the second field of the structure)
push rax		;Store buffer length on the stack

Now, let’s load dwUser and divide it by some constant like 23:

mov rax, [rdx + 16]	;Load dwUser which is the fourth field
mov rbx, 23		;Move the constant to the register
xor rdx, rdx		;Clear upper divisor part
div rbx			;Divide

Now, we can restore the buffer length to rax:

pop rax	;Restore buffer length

At this point we have rax with the buffer length, and rdx with the remainder. We can now compare the reminder and skip the code modifying the pucket length if needed:

cmp rdx, 20	;Compare with 20	
jbe 0x17	;Skip the branch

We can see that we avoid the buffer length modification if the remainder is at most 20. Effectively, we have 20/22 = 0.909% chance that we won’t modify the package. This means that we modify something like 9% of the packages, assuming the dwUser has a good distribution. The code is written in this way so you can tune the odds of changing the packet.

Now, let’s modify the package. We want to divide the buffer length by 2, however, we want to keep it generic to be able to experiment with other values:

mov rbx, 1	;Move 1 to rbx to multiply by 1/2
xor rdx, rdx	;Clear remainder
mul rbx		;Multiply
mov rbx, 2	;Store 2 to rbx to multiply by 1/2
xor rdx, rdx	;Clear remainder
div rbx		;Divide

You can play with other values, obviously. From my experiments, halving the value works the best.

Now it’s rather easy. rax has the new buffer length or the original one if we decided not to modify it. Let’s restore other registers:

pop rdx
pop rbx

Let’s update the buffer length:

mov [rdx + 8], rax

Now, we need to prepare the jump addresses. First, we want to put the original return address of the method mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite which is mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+0x6d:

mov rax, 0x00007ffb897d3019
push rax

Now, we can jump to the WinAPI method:

push rbx
mov rbx, 0x00007ffbe6c6a860
mov rax, [rbx]
pop rbx
push rax
ret

That’s it. The final shellcode looks like this:

mov r8d, 0x30
mov rdx,rsi
push rbx
push rdx
mov rax, [rdx + 8]
push rax
mov rax, [rdx + 16]
mov rbx, 23
xor rdx, rdx
div rbx
pop rax
cmp rdx, 20
jbe 0x17
mov rbx, 1
xor rdx, rdx
mul rbx
mov rbx, 2
xor rdx, rdx
div rbx
pop rdx
pop rbx
mov [rdx + 8], rax
mov rax, 0x00007ffb897d3019
push rax
push rbx
mov rbx, 0x00007ffbe6c6a860
mov rax, [rbx]
pop rbx
push rax
ret

We can implant it with this:

e	25f`b8960000		0x41 0xB8 0x30 0x00 0x00 0x00 0x48 0x89 0xF2 0x53 0x52 0x48 0x8B 0x42 0x08 0x50 0x48 0x8B 0x42 0x10 0x48 0xC7 0xc3 0x17 0x00 0x00 0x00 0x48 0x31 0xD2 0x48 0xF7 0xF3 0x58 0x48 0x83 0xFA 0x14 0x76 0x1A 0x48 0xC7 0xC3 0x01 0x00 0x00 0x00 0x48 0x31 0xD2 0x48 0xF7 0xE3 0x48 0xC7 0xC3 0x02 0x00 0x00 0x00 0x48 0x31 0xD2 0x48 0xF7 0xF3 0x5A 0x5B 0x48 0x89 0x42 0x08 0x48 0xB8 0x19 0x30 0x7D 0x89 0xFB 0x7F 0x00 0x00 0x50 0x48 0xBB 0x60 0xA8 0xC6 0xE6 0xFB 0x7F 0x00 0x00 0x50 0xC3

Automated fix

We can now fix the code automatically. We need to do the following:

• Start mstsc.exe
• Find it’s process ID
• Attach the debugger and find all the addresses: free memory, mstsc.exe method, WinAPI method
• Construct the shellcode
• Attach the debugger and patch the code
• Detach the debugger

We can do all of that with PowerShell. Here is the code:

Function Run-Mstsc($rdpPath, $cdbPath, $numerator, $denominator){
	$id = get-random
	$code = @"
using System;
using System.Collections;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Threading;
	
namespace MstscPatcher
{
	public class Env$id {
		public static void Start() {
			Process[] processes = Process.GetProcessesByName("mstsc");
			RunProcess("mstsc.exe", "$rdpPath");
			Thread.Sleep(3000);
			Process[] processes2 = Process.GetProcessesByName("mstsc");
			var idToPatch = processes2.Select(p => p.Id).OrderBy(i => i).Except(processes.Select(p => p.Id).OrderBy(i => i)).First();
			Patch(idToPatch);
		}
		
		public static void Patch(int id){
			Console.WriteLine(id);
			var addresses = RunCbd(id, @"
.sympath srv*C:\tmp*http://msdl.microsoft.com/download/symbols
.reload
!address
u mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+100
.dvalloc 1024
qd
			");
			
			string freeMemoryAddress = addresses.Where(o => o.Contains("Allocated 2000 bytes starting at")).First().Split(' ').Last().Trim();
			Console.WriteLine("Free memory: " + freeMemoryAddress);
			
			var patchAddress = addresses.SkipWhile(o => !o.StartsWith("mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite:"))
				.SkipWhile(o => !o.Contains("r8d,30h"))
				.First().Split(' ').First().Trim();
			Console.WriteLine("Patch address: " + patchAddress);
			var returnAddress = (Convert.ToUInt64(patchAddress.Replace(((char)96).ToString(),""), 16) + 0x10).ToString("X").Replace("0x", "");
			Console.WriteLine("Return address: " + returnAddress);
			
			var winApiAddress = addresses.SkipWhile(o => !o.Contains("[mstscax!_imp_waveOutPrepareHeader"))
				.First().Split('(')[1].Split(')')[0].Trim();
			Console.WriteLine("WinAPI address: " + winApiAddress);
			
			Func<string, IEnumerable<string>> splitInPairs = address => address.Where((c, i) => i % 2 == 0).Zip(address.Where((c, i) => i % 2 == 1), (first, second) => first.ToString() + second.ToString());			
			Func<string, string> translateToBytes = address => string.Join(" ", splitInPairs(address.Replace(((char)96).ToString(), "").PadLeft(16, '0')).Reverse().Select(p => "0x" + p));
						
			var finalScript = @"
.sympath srv*C:\tmp*http://msdl.microsoft.com/download/symbols
.reload
e	" + patchAddress + @"	0x48 0xB8 " + translateToBytes(freeMemoryAddress) + @" 0x50 0xC3 0x90 0x90 0x90 0x90	
e	" + freeMemoryAddress + @"	0x41 0xB8 0x30 0x00 0x00 0x00 0x48 0x89 0xF2 0x53 0x52 0x48 0x8B 0x42 0x08 0x50 0x48 0x8B 0x42 0x10 0x48 0xC7 0xc3 $denominator 0x00 0x00 0x00 0x48 0x31 0xD2 0x48 0xF7 0xF3 0x58 0x48 0x83 0xFA $numerator 0x76 0x1A 0x48 0xC7 0xC3 0x01 0x00 0x00 0x00 0x48 0x31 0xD2 0x48 0xF7 0xE3 0x48 0xC7 0xC3 0x02 0x00 0x00 0x00 0x48 0x31 0xD2 0x48 0xF7 0xF3 0x5A 0x5B 0x48 0x89 0x42 0x08 0x48 0xB8 " + translateToBytes(returnAddress) + @" 0x50 0x53 0x48 0xBB " + translateToBytes(winApiAddress) + @" 0x48 0x8B 0x03 0x5B 0x50 0xC3	
qd
			";
			Console.WriteLine(finalScript);
			RunCbd(id, finalScript);
		}
		
		public static string[] RunCbd(int id, string script) {
			Console.WriteLine(script);
			File.WriteAllText("mstsc.txt", script);
			string output = "";
			Process process = RunProcess("$cdbPath", "-p " + id + " -cf mstsc.txt", text => output += text + "\n");
			process.WaitForExit();
			File.Delete("mstsc.txt");
			
			return output.Split('\n');
		}
		
		public static Process RunProcess(string fileName, string arguments, Action<string> outputReader = null){
			ProcessStartInfo startInfo = new ProcessStartInfo
			{
				FileName = fileName,
				Arguments = arguments,
				UseShellExecute = outputReader == null,
				RedirectStandardOutput = outputReader != null,
				RedirectStandardError = outputReader != null
			};
			
			if(outputReader != null){
				var process = new Process{
					StartInfo = startInfo
				};
				process.OutputDataReceived += (sender, args) => outputReader(args.Data);
				process.ErrorDataReceived += (sender, args) => outputReader(args.Data);

				process.Start();
				process.BeginOutputReadLine();
				process.BeginErrorReadLine();
				return process;
			}else {
				return Process.Start(startInfo);
			}
		}
	}
}
"@.Replace('$id', $id)
	$assemblies = ("System.Core","System.Xml.Linq","System.Data","System.Xml", "System.Data.DataSetExtensions", "Microsoft.CSharp")
	Add-Type -referencedAssemblies $assemblies -TypeDefinition $code -Language CSharp
	iex "[MstscPatcher.Env$id]::Start()"
}


Run-Mstsc "my_rdp_settings.rdp".Replace("\", "\\") "cdb.exe".Replace("\", "\\") "0x14" "0x17"

We compile some C# code on the fly. First, we find existing mstsc.exe instances (line 16), then run the new instance (line 17), wait a bit for the mstsc.exe to spawn a child process, and then find the id (lines 19-20). We can then patch the existing id.

First, we look for addresses. We do all the manual steps we did above to find the memory address, and two function addresses. The script is in lines 27-32. Notice that I load symbols as we need them and CDB may not have them configured on the box.

We can now parse the output. We first extract the allocated memory in lines 35-36.

Next, we look for the call site. We dump the whole method, and then find the first occurrence of mov 8d,30h. That’s our call site. This is in lines 38-41.

Next, we calculate the return address which is 16 bytes further. This is in lines 42-43.

Finally, I calculate the WinAPI method address. I extract the location of the pointer for the method (lines 45-47).

Next, we need to construct the shell code. This is exactly what we did above. We just need to format addresses properly (this is in helper methods in lines 49-50), and then build the script (lines 52-558). We can run it and that’s it. The last thing is customization of the values. You can see in line 108 that I made two parameters to change numerator and denominator for the odds of modifying the package. This way you can easily change how many packets are broken. The more you break, the faster you resynchronize, however, the worse the sound is.

That’s it. I verified that on Windows 10 22H2 x64, Windows 11 22H2 x64, Windows Server 2016 1607 x64, and Windows Server 2019 1809 x64, and it worked well. Your mileage may vary, however, the approach should work anywhere. Generally, to make this script work somewhere else, you just need to adjust how we find the call site, the return address, and the address of the WinAPI function. Assuming that the WinAPI is still called via the pointer stored in memory, then you won’t need to touch the machine code payload.

Below is the script for x86 bit (worked on Windows 10 10240 x86). Main differences are in how we access the data structure as the pointer is on the stack (and not in the register).

Function Run-Mstsc($rdpPath, $cdbPath, $numerator, $denominator){
	$id = get-random
	$code = @"
using System;
using System.Collections;
using System.Collections.Generic;
using System.Diagnostics;
using System.IO;
using System.Linq;
using System.Threading;
	
namespace MstscPatcher
{
	public class Env$id {
		public static void Start() {
			Process[] processes = Process.GetProcessesByName("mstsc");
			RunProcess("mstsc.exe", "$rdpPath");
			Thread.Sleep(3000);
			Process[] processes2 = Process.GetProcessesByName("mstsc");
			var idToPatch = processes2.Select(p => p.Id).OrderBy(i => i).Except(processes.Select(p => p.Id).OrderBy(i => i)).First();
			Patch(idToPatch);
		}
		
		public static void Patch(int id){
			Console.WriteLine(id);
			var addresses = RunCbd(id, @"
.sympath srv*C:\tmp*http://msdl.microsoft.com/download/symbols
.reload
!address
u mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite+100
.dvalloc 1024
qd
			");
			
			string freeMemoryAddress = addresses.Where(o => o.Contains("Allocated 2000 bytes starting at")).First().Split(' ').Last().Trim();
			Console.WriteLine("Free memory: " + freeMemoryAddress);
			
			var patchAddress = addresses.SkipWhile(o => !o.StartsWith("mstscax!CRdpWinAudioWaveoutPlayback::vcwaveOutWrite:"))
				.SkipWhile(o => !(o.Contains("dword ptr [ebx+3Ch]") && o.Contains("push")))
				.First().Split(' ').First().Trim();
			Console.WriteLine("Patch address: " + patchAddress);
			var returnAddress = (Convert.ToUInt64(patchAddress.Replace(((char)96).ToString(),""), 16) + 0x9).ToString("X").Replace("0x", "");
			Console.WriteLine("Return address: " + returnAddress);
			
			var winApiAddress = addresses.SkipWhile(o => !o.Contains("[mstscax!_imp__waveOutPrepareHeader"))
				.First().Split('(')[1].Split(')')[0].Trim();
			Console.WriteLine("WinAPI address: " + winApiAddress);
			
			Func<string, IEnumerable<string>> splitInPairs = address => address.Where((c, i) => i % 2 == 0).Zip(address.Where((c, i) => i % 2 == 1), (first, second) => first.ToString() + second.ToString());			
			Func<string, string> translateToBytes = address => string.Join(" ", splitInPairs(address.Replace(((char)96).ToString(), "").PadLeft(8, '0')).Reverse().Select(p => "0x" + p));
						
			var finalScript = @"
.sympath srv*C:\tmp*http://msdl.microsoft.com/download/symbols
.reload
e	" + patchAddress + @"	0xB8 " + translateToBytes(freeMemoryAddress) + @" 0x50 0xC3 0x90 0x90	
e	" + freeMemoryAddress + @"	0xFF 0x73 0x3C 0x53 0x52 0x8B 0x53 0x3C 0x52 0x8B 0x42 0x04 0x50 0x8B 0x42 0x0C 0xBB $denominator 0x00 0x00 0x00 0x31 0xD2 0xF7 0xF3 0x58 0x83 0xFA $numerator 0x76 0x12 0xBB 0x01 0x00 0x00 0x00 0x31 0xD2 0xF7 0xE3 0xBB 0x02 0x00 0x00 0x00 0x31 0xD2 0xF7 0xF3 0x5A 0x89 0x42 0x04 0x5A 0x5B 0xB8 " + translateToBytes(returnAddress) + @" 0x50 0x53 0xBB " + translateToBytes(winApiAddress) + @" 0x8B 0x03 0x5B 0x50 0xC3	
qd
			";
			Console.WriteLine(finalScript);
			RunCbd(id, finalScript);
		}
		
		public static string[] RunCbd(int id, string script) {
			Console.WriteLine(script);
			File.WriteAllText("mstsc.txt", script);
			string output = "";
			Process process = RunProcess("$cdbPath", "-p " + id + " -cf mstsc.txt", text => output += text + "\n");
			process.WaitForExit();
			File.Delete("mstsc.txt");
			
			return output.Split('\n');
		}
		
		public static Process RunProcess(string fileName, string arguments, Action<string> outputReader = null){
			ProcessStartInfo startInfo = new ProcessStartInfo
			{
				FileName = fileName,
				Arguments = arguments,
				UseShellExecute = outputReader == null,
				RedirectStandardOutput = outputReader != null,
				RedirectStandardError = outputReader != null
			};
			
			if(outputReader != null){
				var process = new Process{
					StartInfo = startInfo
				};
				process.OutputDataReceived += (sender, args) => outputReader(args.Data);
				process.ErrorDataReceived += (sender, args) => outputReader(args.Data);

				process.Start();
				process.BeginOutputReadLine();
				process.BeginErrorReadLine();
				return process;
			}else {
				return Process.Start(startInfo);
			}
		}
	}
}
"@.Replace('$id', $id)
	$assemblies = ("System.Core","System.Xml.Linq","System.Data","System.Xml", "System.Data.DataSetExtensions", "Microsoft.CSharp")
	Add-Type -referencedAssemblies $assemblies -TypeDefinition $code -Language CSharp
	iex "[MstscPatcher.Env$id]::Start()"
}

Run-Mstsc "my_rdp_settings.rdp".Replace("\", "\\") "cdb.exe".Replace("\", "\\") "0x14" "0x17"

Some keyword to make this easier to find on the Internet

Her some keywords to make this article easier to find on the Internet.

audio latency in mstsc.exe
audio latency in rdp
audio delay in mstsc.exe
audio delay in rdp
laggy sound in rdp
sound desynchronized in rdp
sound latency in rdp
slow audio
how to fix audio in rdp

Enjoy!

We use case classes in Scala and the Jackson library for XML serialization. We have a class that has a list of some nested class. When serializing it, we would like to name the nested element properly.

We start with these classes:

case class Person(val Name: String)
 
@JsonPropertyOrder(Array("People"))
case class Base
(
	val People: List[Person]
)

When we try to serialize it, we’ll get something like this:

< Base>
    < People>
        < People>
            < Name>Some name here</Name>
        </People>
    </People>
</Base>

You can see that we have Base -> People -> People instead of Base -> People -> Person. We can try to fix it with regular annotations:

case class Base
(
	@JacksonXmlElementWrapper(localName = "People")
	@JacksonXmlProperty(localName = "Person")
	val People: List[Person]
)

It now serializes correctly. However, when you try to deserialize it, you get the following exception:

Exception in thread "main" com.fasterxml.jackson.databind.JsonMappingException: Could not find creator property with name 'Person'

This fails because we use JacksonXmlProperty to rename the property and this gets handled incorrectly.

The issue is with how Scala implements case classes. Fields are always stored as private fields with getters (and setters if you use var), and the annotations are propagated to the constructor parameters.

My way of fixing it was the following:

• Do not propagate annotations to the constructor
• Create another constructor that has a different signature that the default one
• Mark the constructor as @JsonCreator

This way, Jackson creates the object with default values using the new constructor, and then sets the fields with reflection. So, this is how the code should look like:

case class Base
(
	@(JacksonXmlElementWrapper @field @getter)(localName = "People")
	@(JacksonXmlProperty @field @getter)(localName = "Person")
	val People: List[Person]
) {
   @JsonCreator
   def this(
	v1: List[Person]
	ignored: String
   ) = {
    this(v1)
   }
}

You could go with some different constructor if needed (like a parameterless one and pass nulls explicitly).

This works with Jackson 2.13.0.

This is the fourth part of the Bit Twiddling series. For your convenience you can find other parts in the table of contents in Par 1 — Modifying Android application on a binary level

Today we’re going to disable CTRL+ALT+HOME shortcut in mstsc. We want to disable the shortcut so it doesn’t “unfocus” the RDP session but still works inside the connection. I needed that for easier work in nested RDPs and I couldn’t find any decent RDP client for Windows that wouldn’t have this shortcut. The only one I found was Thincast but it lacked some other features as well.

Let’s begin.

Finding the entry point

The hardest part is as always finding “the place”. I took API Monitor which is like strace for Windows. I started the mstsc.exe, connected to some machine, pressed CTRL+ALT+HOME and observed what happened. After some digging here and there I found that when I press CTRL+ALT+HOME, the WM_USER+19 message is sent inside the application:

This is clearly a hint what’s going on. We can see that the key combination is captured by mstscax.dll which is the ActiveX part of the RDP. We also found the call stack and the address 0x000007ffec2d82fe2.

Analyzing the code

Now we can use WinDBG or whatever other debugger to figure out what’s going on. I attached to the running mstsc.exe, added a breakpoint with bu 0x00007ffec2d82fe2 and then pressed CTLR+ALT+HOME. As expected, the breakpoint was hit and I could observe the call stack:

0:003> kb
 # RetAddr               : Args to Child                                                           : Call Site
00 00007ffe`c2d27655     : 00000000`00000000 00000000`00000000 00000000`00000024 00000000`00008000 : mstscax!CTSCoreEventSource::InternalFireAsyncNotification+0xca
01 00007ffe`c2d262a5     : 00000000`00000100 00000000`00000000 00000000`00000024 00000135`bc82ea28 : mstscax!CTSInput::IHPostMessageToMainWindow+0x1c5
02 00007ffe`c2d261c8     : 00000000`00000001 00000000`01470001 00000000`003f0b28 00000000`00000113 : mstscax!CTSInput::IHInputCaptureWndProc+0x85
03 00007fff`5a04ef75     : 00000000`00000001 00000033`4faff2d0 00000000`003f0b28 00000000`80000022 : mstscax!CTSInput::IHStaticInputCaptureWndProc+0x58
04 00007fff`5a04e69d     : 00000000`00000000 00007ffe`c2d26170 00000033`4f5d3800 00007fff`3f646aae : USER32!UserCallWinProcCheckWow+0x515
05 00007fff`3f64ab32     : 00007ffe`c2d26170 00000000`00000000 00000000`ffffffff 00007fff`3f646a35 : USER32!DispatchMessageWorker+0x49d
06 00007fff`3f643997     : 00000000`00000000 00000000`00000000 00000135`bc830b00 00007fff`00000000 : apimonitor_drv_x64+0xab32
07 00000135`bc8fb89f     : 00000135`bc25da6a 00000000`00000001 00000000`00000001 00000000`00000001 : apimonitor_drv_x64+0x3997
08 00000135`bc25da6a     : 00000000`00000001 00000000`00000001 00000000`00000001 00000135`bd33e0e0 : 0x00000135`bc8fb89f
09 00000000`00000001     : 00000000`00000001 00000000`00000001 00000135`bd33e0e0 00007ffe`c2cd761c : 0x00000135`bc25da6a
0a 00000000`00000001     : 00000000`00000001 00000135`bd33e0e0 00007ffe`c2cd761c 00000033`4faff480 : 0x1
0b 00000000`00000001     : 00000135`bd33e0e0 00007ffe`c2cd761c 00000033`4faff480 00000135`bba40000 : 0x1
0c 00000135`bd33e0e0     : 00007ffe`c2cd761c 00000033`4faff480 00000135`bba40000 00000000`00000083 : 0x1
0d 00007ffe`c2cd761c     : 00000033`4faff480 00000135`bba40000 00000000`00000083 00000000`7ffef000 : 0x00000135`bd33e0e0
0e 00007ffe`c2cd7444     : 00000000`00000000 4e478f48`d4f63727 00000000`000051c0 0000aa52`0fad086d : mstscax!PAL_System_CondWait+0x1cc
0f 00007ffe`c2cf0f75     : 00000000`00000400 00000000`00000000 00000033`4faff8c0 00000135`bd35a720 : mstscax!CTSThreadInternal::ThreadSignalWait+0x34
10 00007ffe`c2cf1f8d     : 00000000`00000000 00000000`00000000 00000135`bd35a720 00000000`00000400 : mstscax!CTSThread::internalMsgPump+0x6d
11 00007ffe`c2d8693c     : 00000000`00000000 00007ffe`c2cee22d 00000135`bd344640 00007ffe`c2fdf3f0 : mstscax!CTSThread::internalThreadMsgLoop+0x14d
12 00007ffe`c3128550     : 00007ffe`c3475808 00000033`4faff8c0 00000000`00000000 00000135`ba29b8c0 : mstscax!CTSThread::ThreadMsgLoop+0x1c
13 00007ffe`c2fdeca8     : 00000135`bd35a720 00000135`ba29b8c0 00000135`bd35a720 00000135`ba29b748 : mstscax!CSND::SND_Main+0x148
14 00007ffe`c2fe73c2     : 00000135`bd35d100 00000135`bd35a720 00000033`4f67e3f0 00000000`00000000 : mstscax!CTSThread::TSStaticThreadEntry+0x258
15 00007fff`5a1f7344     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : mstscax!PAL_System_Win32_ThreadProcWrapper+0x32
16 00007fff`5b7a26b1     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : KERNEL32!BaseThreadInitThunk+0x14
17 00000000`00000000     : 00000000`00000000 00000000`00000000 00000000`00000000 00000000`00000000 : ntdll!RtlUserThreadStart+0x21

Perfect! We can see the method names and they are really useful. We can immediately see that some IHStaticInputCaptureWndProc method captures the regular message from the OS, then it calls the window procedure IHInputCaptureWndProc, and then the method posts the message to the main mstsc.exe window in IHPostMessageToMainWindow.

This shows us the way. We can also ask the WinDBG to get the call stack with addresses to find out that mstscax!CTSInput::IHInputCaptureWndProc+0x85 is mstscax+0x562a5. We can now dump the code:

0:004> u mstscax+0x562a5-0x85 mstscax+0x562a5
mstscax+0x56220:
00007ffe`c2d26220 4053            push    rbx
00007ffe`c2d26222 55              push    rbp
00007ffe`c2d26223 56              push    rsi
00007ffe`c2d26224 57              push    rdi
00007ffe`c2d26225 4154            push    r12
00007ffe`c2d26227 4155            push    r13
00007ffe`c2d26229 4156            push    r14
00007ffe`c2d2622b 4157            push    r15
00007ffe`c2d2622d 4881ece8000000  sub     rsp,0E8h
00007ffe`c2d26234 488b05cdc37500  mov     rax,qword ptr [mstscax!DllUnregisterServer+0x6f1308 (00007ffe`c3482608)]
00007ffe`c2d2623b 4833c4          xor     rax,rsp
00007ffe`c2d2623e 48898424d0000000 mov     qword ptr [rsp+0D0h],rax
00007ffe`c2d26246 33ed            xor     ebp,ebp
00007ffe`c2d26248 0f57c0          xorps   xmm0,xmm0
00007ffe`c2d2624b 448bf5          mov     r14d,ebp
00007ffe`c2d2624e 896c245c        mov     dword ptr [rsp+5Ch],ebp
00007ffe`c2d26252 f30f7f442470    movdqu  xmmword ptr [rsp+70h],xmm0
00007ffe`c2d26258 498bf1          mov     rsi,r9
00007ffe`c2d2625b 418bd8          mov     ebx,r8d
00007ffe`c2d2625e 4c8be2          mov     r12,rdx
00007ffe`c2d26261 488bf9          mov     rdi,rcx
00007ffe`c2d26264 488b059df57400  mov     rax,qword ptr [mstscax!DllUnregisterServer+0x6e4508 (00007ffe`c3475808)]
00007ffe`c2d2626b 4c8d2d96f57400  lea     r13,[mstscax!DllUnregisterServer+0x6e4508 (00007ffe`c3475808)]
00007ffe`c2d26272 4c8bbc2450010000 mov     r15,qword ptr [rsp+150h]
00007ffe`c2d2627a 493bc5          cmp     rax,r13
00007ffe`c2d2627d 740a            je      mstscax+0x56289 (00007ffe`c2d26289)
00007ffe`c2d2627f f6401c01        test    byte ptr [rax+1Ch],1
00007ffe`c2d26283 0f85c6010000    jne     mstscax+0x5644f (00007ffe`c2d2644f)
00007ffe`c2d26289 39af88030000    cmp     dword ptr [rdi+388h],ebp
00007ffe`c2d2628f 0f85f1020000    jne     mstscax+0x56586 (00007ffe`c2d26586)
00007ffe`c2d26295 4d8bcf          mov     r9,r15
00007ffe`c2d26298 4c8bc6          mov     r8,rsi
00007ffe`c2d2629b 8bd3            mov     edx,ebx
00007ffe`c2d2629d 488bcf          mov     rcx,rdi
00007ffe`c2d262a0 e8eb110000      call    mstscax+0x57490 (00007ffe`c2d27490)
00007ffe`c2d262a5 85c0            test    eax,eax

Great. We can see that the line 00007ffe`c2d262a0 calls some internal method and then tests if the output of the call is equal to 0 (this is the meaning of test eax,eax). We can now comment out this line and see what happens (mind the empty line):

a 00007ffe`c2d262a0
xor eax,eax
nop
nop
nop

g

We come back to mstsc.exe and we can see that CTRL+ALT+HOME doesn’t unfocus the window anymore! We can also check that the combination is handled properly inside the connection so it still unfocuses nested RDPs.

Automation

Automating this is pretty straightforward with cdb. We could modify the dll in place, but that would affect all the RDP connections we ever make. If we want to disable CTRL+ALT+HOME for some of them only, then this is what we can do:

First, create the file mstsc.txt with the following code:

a mstscax+0x562a0
xor eax, eax
nop
nop
nop

qd

Next, create batch file no_home.bat

cdb.exe -p %1 -cf mstsc.txt

Finally, run it like this:

no_home.bat PROCESS_ID_OF_MSTSC_YOU_WANT_TO_HACK

Changing shortcut to something else

How about we change shortcut instead of getting rid of it entirely? Let’s disassemble the method a bit more:

0:001> u mstscax+0x57655-0x1c5 mstscax+0x57655
mstscax!CTSInput::IHPostMessageToMainWindow:
00007ffe`c2d27490 48895c2408      mov     qword ptr [rsp+8],rbx
00007ffe`c2d27495 48896c2410      mov     qword ptr [rsp+10h],rbp
00007ffe`c2d2749a 4889742418      mov     qword ptr [rsp+18h],rsi
00007ffe`c2d2749f 57              push    rdi
00007ffe`c2d274a0 4156            push    r14
00007ffe`c2d274a2 4157            push    r15
00007ffe`c2d274a4 4883ec30        sub     rsp,30h
00007ffe`c2d274a8 33db            xor     ebx,ebx
00007ffe`c2d274aa 4d8bf1          mov     r14,r9
00007ffe`c2d274ad 498be8          mov     rbp,r8
00007ffe`c2d274b0 8bf2            mov     esi,edx
00007ffe`c2d274b2 488bf9          mov     rdi,rcx
00007ffe`c2d274b5 399958030000    cmp     dword ptr [rcx+358h],ebx
00007ffe`c2d274bb 0f8582000000    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0xb3 (00007ffe`c2d27543)
00007ffe`c2d274c1 81fa00010000    cmp     edx,100h
00007ffe`c2d274c7 0f8489000000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0xc6 (00007ffe`c2d27556)
00007ffe`c2d274cd b901000000      mov     ecx,1
00007ffe`c2d274d2 8d82fcfeffff    lea     eax,[rdx-104h]
00007ffe`c2d274d8 3bc1            cmp     eax,ecx
00007ffe`c2d274da 0f47cb          cmova   ecx,ebx
00007ffe`c2d274dd 399f24030000    cmp     dword ptr [rdi+324h],ebx
00007ffe`c2d274e3 0f84c3040000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x51c (00007ffe`c2d279ac)
00007ffe`c2d274e9 448bc3          mov     r8d,ebx
00007ffe`c2d274ec 81fe04010000    cmp     esi,104h
00007ffe`c2d274f2 0f84c7040000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x52f (00007ffe`c2d279bf)
00007ffe`c2d274f8 8bd3            mov     edx,ebx
00007ffe`c2d274fa 85c9            test    ecx,ecx
00007ffe`c2d274fc 0f85d1040000    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0x543 (00007ffe`c2d279d3)
00007ffe`c2d27502 8bc3            mov     eax,ebx
00007ffe`c2d27504 85d2            test    edx,edx
00007ffe`c2d27506 0f8558010000    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0x1d4 (00007ffe`c2d27664)
00007ffe`c2d2750c 85c0            test    eax,eax
00007ffe`c2d2750e 0f8550010000    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0x1d4 (00007ffe`c2d27664)
00007ffe`c2d27514 8bc3            mov     eax,ebx
00007ffe`c2d27516 4585c0          test    r8d,r8d
00007ffe`c2d27519 0f85c8040000    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0x557 (00007ffe`c2d279e7)
00007ffe`c2d2751f 85c0            test    eax,eax
00007ffe`c2d27521 0f85c0040000    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0x557 (00007ffe`c2d279e7)
00007ffe`c2d27527 488b6c2458      mov     rbp,qword ptr [rsp+58h]
00007ffe`c2d2752c 8bc3            mov     eax,ebx
00007ffe`c2d2752e 488b5c2450      mov     rbx,qword ptr [rsp+50h]
00007ffe`c2d27533 488b742460      mov     rsi,qword ptr [rsp+60h]
00007ffe`c2d27538 4883c430        add     rsp,30h
00007ffe`c2d2753c 415f            pop     r15
00007ffe`c2d2753e 415e            pop     r14
00007ffe`c2d27540 5f              pop     rdi
00007ffe`c2d27541 c3              ret
00007ffe`c2d27542 cc              int     3
00007ffe`c2d27543 81fe13010000    cmp     esi,113h
00007ffe`c2d27549 0f851f010000    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0x1de (00007ffe`c2d2766e)
00007ffe`c2d2754f bb01000000      mov     ebx,1
00007ffe`c2d27554 ebd1            jmp     mstscax!CTSInput::IHPostMessageToMainWindow+0x97 (00007ffe`c2d27527)
00007ffe`c2d27556 8b8128030000    mov     eax,dword ptr [rcx+328h]
00007ffe`c2d2755c be00800000      mov     esi,8000h
00007ffe`c2d27561 483be8          cmp     rbp,rax
00007ffe`c2d27564 0f8470010000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x24a (00007ffe`c2d276da)
00007ffe`c2d2756a 8b8744030000    mov     eax,dword ptr [rdi+344h]
00007ffe`c2d27570 483be8          cmp     rbp,rax
00007ffe`c2d27573 0f84dc010000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x2c5 (00007ffe`c2d27755)
00007ffe`c2d27579 8b8748030000    mov     eax,dword ptr [rdi+348h]
00007ffe`c2d2757f 483be8          cmp     rbp,rax
00007ffe`c2d27582 0f8440020000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x338 (00007ffe`c2d277c8)
00007ffe`c2d27588 8b874c030000    mov     eax,dword ptr [rdi+34Ch]
00007ffe`c2d2758e 483be8          cmp     rbp,rax
00007ffe`c2d27591 0f84a4020000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x3ab (00007ffe`c2d2783b)
00007ffe`c2d27597 8b8750030000    mov     eax,dword ptr [rdi+350h]
00007ffe`c2d2759d 483be8          cmp     rbp,rax
00007ffe`c2d275a0 0f8408030000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x41e (00007ffe`c2d278ae)
00007ffe`c2d275a6 4883fd24        cmp     rbp,24h
00007ffe`c2d275aa 0f8471030000    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x491 (00007ffe`c2d27921)
00007ffe`c2d275b0 4883fd2d        cmp     rbp,2Dh
00007ffe`c2d275b4 0f856dffffff    jne     mstscax!CTSInput::IHPostMessageToMainWindow+0x97 (00007ffe`c2d27527)
00007ffe`c2d275ba 8d4de5          lea     ecx,[rbp-1Bh]
00007ffe`c2d275bd 48ff15acec5b00  call    qword ptr [mstscax!_imp_GetKeyState (00007ffe`c32e6270)]
00007ffe`c2d275c4 0f1f440000      nop     dword ptr [rax+rax]
00007ffe`c2d275c9 6685c6          test    si,ax
00007ffe`c2d275cc 0f8455ffffff    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x97 (00007ffe`c2d27527)
00007ffe`c2d275d2 8d4de4          lea     ecx,[rbp-1Ch]
00007ffe`c2d275d5 48ff1594ec5b00  call    qword ptr [mstscax!_imp_GetKeyState (00007ffe`c32e6270)]
00007ffe`c2d275dc 0f1f440000      nop     dword ptr [rax+rax]
00007ffe`c2d275e1 6685c6          test    si,ax
00007ffe`c2d275e4 0f843dffffff    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x97 (00007ffe`c2d27527)
00007ffe`c2d275ea e8c581fcff      call    mstscax!CClientUtilsWin32::UT_IsRunningInAppContainer (00007ffe`c2cef7b4)
00007ffe`c2d275ef 85c0            test    eax,eax
00007ffe`c2d275f1 0f8430ffffff    je      mstscax!CTSInput::IHPostMessageToMainWindow+0x97 (00007ffe`c2d27527)
00007ffe`c2d275f7 488b050ae27400  mov     rax,qword ptr [mstscax!WPP_GLOBAL_Control (00007ffe`c3475808)]
00007ffe`c2d275fe 4c8d3d03e27400  lea     r15,[mstscax!WPP_GLOBAL_Control (00007ffe`c3475808)]
00007ffe`c2d27605 493bc7          cmp     rax,r15
00007ffe`c2d27608 7430            je      mstscax!CTSInput::IHPostMessageToMainWindow+0x1aa (00007ffe`c2d2763a)
00007ffe`c2d2760a f6401c01        test    byte ptr [rax+1Ch],1
00007ffe`c2d2760e 742a            je      mstscax!CTSInput::IHPostMessageToMainWindow+0x1aa (00007ffe`c2d2763a)
00007ffe`c2d27610 80781904        cmp     byte ptr [rax+19h],4
00007ffe`c2d27614 7224            jb      mstscax!CTSInput::IHPostMessageToMainWindow+0x1aa (00007ffe`c2d2763a)
00007ffe`c2d27616 e82de60000      call    mstscax!RdpWppGetCurrentThreadActivityIdPrefix (00007ffe`c2d35c48)
00007ffe`c2d2761b 488b0de6e17400  mov     rcx,qword ptr [mstscax!WPP_GLOBAL_Control (00007ffe`c3475808)]
00007ffe`c2d27622 4c8d0557ca5c00  lea     r8,[mstscax!WPP_f5f71bb7bac236b27f26969128cc1e12_Traceguids (00007ffe`c32f4080)]
00007ffe`c2d27629 448bc8          mov     r9d,eax
00007ffe`c2d2762c bafd000000      mov     edx,0FDh
00007ffe`c2d27631 488b4910        mov     rcx,qword ptr [rcx+10h]
00007ffe`c2d27635 e85ecdfcff      call    mstscax!WPP_SF_D (00007ffe`c2cf4398)
00007ffe`c2d2763a 488b8f60050000  mov     rcx,qword ptr [rdi+560h]
00007ffe`c2d27641 4533c0          xor     r8d,r8d
00007ffe`c2d27644 488b01          mov     rax,qword ptr [rcx]
00007ffe`c2d27647 418d5010        lea     edx,[r8+10h]
00007ffe`c2d2764b 488b4048        mov     rax,qword ptr [rax+48h]
00007ffe`c2d2764f ff1523f75b00    call    qword ptr [mstscax!_guard_dispatch_icall_fptr (00007ffe`c32e6d78)]
00007ffe`c2d27655 c7871404000001000000 mov dword ptr [rdi+414h],1

The most interesting line is:

00007ffe`c2d275a6 4883fd24        cmp     rbp,24h

0x24 is the code key of HOME. We can replace it with something else, like insert which is 2D:

e mstscax+0x575A9 0x2D

And there you go. You can now press CTRL+ALT+HOME to unfocus nested RDP and CTRL+ALT+INSERT to unfocus the outer one. This gives you two levels of unfocusing.

This is the twentieth seventh part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Last time we discussed how to work remotely. Let’s now look at that from a different perspective and see various tools that we can use and how they work.

Sessions, input, and output

Before seeing the tools let’s understand some basics of how computers work. This will be important to understand what tools we need and how they should work. I’ll be talking mostly from the perspective of Windows OS, but the same applies to other systems.

Whenever you try to use a computer, you need to have a session. Session is an object that controls your human interfaces, most importantly input and output. Depending on the version of Windows, you may be able to create many sessions.

The main session (the one you have when you “sit at a computer”) is called CONSOLE. You can see it with qwinsta command:

>qwinsta
 SESSIONNAME       USERNAME                 ID  STATE   TYPE        DEVICE
 services                                    0  Disc
 console                                     1  Conn

CONSOLE is what you create when you use your computer “normally”. This session handles your keyboard, mouse, and (most importantly) your monitors.

You can create more sessions. Whenever you use mstsc.exe, you connect to a computer and create a session:

SESSIONNAME       USERNAME                 ID  STATE   TYPE        DEVICE
 rdp-tcp#112       user                     2  Active

Obviously, when you create a new session, it must have some way of getting input (mouse and keyboard) and producing output (monitors).

It’s important to understand that one physical monitor can be in one session only (at least I have no idea how to use it in many sessions and I don’t think that would make sense). After all, you just have “one picture” on the monitor and there is no point in showing many sessions in it. Sure, you could fake it to some extent maybe to do picture-in-picture, but I really don’t see a decent use case for that.

While all of that sounds super clear, good understanding is crucial for picking the right tools.

What do we really need

Let’s now think what we really need to do the remote work.

Regarding sessions:

• Do we want to connect to an existing session? In other words, do we want to connect to the state that the “person sitting next to the computer” sees?
• Do we want to create a new session? This will start things from scratch just like we “restarted” the computer

However, the more important question we need to consider is how do we want to interact with the session?

• Do we want to control it at the destination (“remote”)?
• Do we want to control it from both places (“remote” and “local”)? This is “shared” control.
• Or maybe only “local” should control it? This is “takeover” control.
• Do you want the keyboard shortcuts pressed on “local” to be sent to “remote”?

If you read this carefully, you may realize some interesting facts that may change the way you look for the remote work software:

• If you want to create a session and control it from “local” then you need RDP-like (mstsc.exe-like) application
• If you want to connect to the existing session and control it from “local” then you need VNC-like software
• If you want to connect to the existing session and control it from “remote” then you basically need a screen share

While all these things may seem rather trivial, they have crucial impact on what we’re actually trying to achieve. Let’s see the typical scenarios:

• You want to connect to a remote server (like VDI, Azure VM, etc.) – in that case you want to create a new session and you want to control it from “local” which is RDP-like software
• You want to help someone else with their computer – in that case you connect to the existing session and you want to let “remote” control it which is a screen share
• You want to play a game on your VR goggles from your PVCR – in that case you connect to the existing session and you want to control it from “local” or from both places which is a VNC-like software
• You want to use VR goggles to work from your laptop – in that case you connect to the existing session and you want to let “remote” control it which is a screen share

Notice that a regular “monitor” can be thought of a remote connection to the existing session that is controlled from “remote” so it’s basically a screen share. However, you could have a TV that you use to connect to the existing session and control it from “local” with your remote control (no pun intended) which is a VNC-like software.

Multiple monitors

Now we can ask the question of the monitors. This is crucial as here are many ways we can configure our machines.
Let’s assume that our “local” has 3 screens and our “remote” has 2 physical screens. So the geometry in “remote” may look like (1920, 1080, 0, 0) and (1920, 1080, 1920, 0) (which is (width, height, left, top)). Similarly, geometry in “local” may be (1920, 1080, 0, 0) + (1920, 1080, -1920, 0) + (1920, 1080, 1920, 1080).

Let’s consider two scenarios.

Creating new session in remote

If we create a new session in remote then we need to provide monitors for it. Effectively, it doesn’t matter what monitors are there in “remote” because we won’t use them. The only important thing that matters here is that we have 3 monitors in “local”. To understand what’s going on, we now need to decide on how many virtual monitors we want to create in “remote” and how we want to map them in “local”. There are typically the following options:

• 1:1 – You create one monitor in “remote” and you map it to one monitor in “local” – this is how most RDP clients work
• 1:n – You create one monitor in “remote” and you show it on multiple monitors in “local” – this is how xfreerdp workd for me. Sometimes this is called “span”
• n:n – You create as many monitors in “remote” as you have in “local” – this is how mstsc.exe works when you select “use all displays”. Sometimes this is called “multimon”
• m:m – You create fewer monitors in “remote” then you have in “local” but each monitor is mapped to one monitor. This is how mstsc.exe works when you modify the .rdp file directly
• m:n – You create some monitors in “remote” and you map them to some configuration in “local”. I don’t know any software that would do this. Notice that this doesn’t mean that you use entire “local” monitor. You could come up with any geometry that you need.

Whenever I talk about “multiple monitors” I mean the n:n or m:m scenario from the above.

Connecting to he existing session in remote

In this scenario the “remote” has 2 monitors. You now want to map them somehow in your machine. You have the following options:

• 1:1 – You show only one “remote” monitor in one “local”
• m:1 – You show all “remote” monitors in one “local” (typically one window) – this is how most VNC apps work
• m:m – You show “remote” monitors separately in “local”. This rarely works on Windows
• m:n – You configure custom geometry. I’m not aware of any software that does that

Let’s now see the software for various scenarios.

Software

Applications that create new session (RDP-like) (and therefore “local” must control the session):

• mstsc.exe – sometimes called Terminal Services or Microsoft Remote Desktop – this supports 1:1, n:n, and m:m screen configurations
• msrds.exe – so called Azure Virtual Desktop https://learn.microsoft.com/en-us/azure/virtual-desktop/users/connect-windows?pivots=remote-desktop-msi#install-the-client – this supports 1:1 and n:n (it may support more)
• Remote Desktop App for Windows https://apps.microsoft.com/detail/9wzdncrfj3ps?rtc=1&hl=pl-pl&gl=PL – this supports 1:1
• FreeRDP https://www.freerdp.com/ – this supported 1:1 and 1:n in my case
• Thincast Remote Desktop client https://thincast.com/en/products/client – this supports 1:1 and n:n
• Alter, mRemoteNG – these support 1:1 to the best of my knowledge
• Keep in mind that many applications use mstsc.exe behind the scenes

I’m not aware of an RDP-like application that would work in VR goggles.

VNC-like applications:

• TightVNC, TigerVNC, UltraVNC and similar – they typically support 1:1 and “shared” control. You can often change the geometry on the server (I didn’t see any client that would let me change the geometry, though).
• vSpatial – this supports n:n, and “shared” or “takeover”
• noVNC – this is a web VNC client. It supports 1:1 and “shared”. It’s just an HTML file with some JS, so you can easily adjust it to your needs
• Immersed – this supports n:n and “shared”
• Virtual Desktop, Multirooms – they support 1:1 and “shared”

Screen share applications:

• vdo.ninja – this shares one screen and works in browser, so you can share as many screens as you need
• Google Meets, Teams, Amazon Chime, Zoom, Webex and many others – they share one screen and work in browser or in dedicated apps. You can open up many bridges to share many screens

And now the most important part (kind of TL;DR):

• You want to have the “true” RDP experience with multiple monitors – mstsc.exe or Thincast Remote Desktop Client
• You want to interact with some remote session from your computer – any VNC client that works for your case (and your mileage may vary significantly) – TightVNC, UltraVNC, TigerVNC, noVNC, many more. Also, vSpatial
• You just want to see the remote session only – any screen share or VNC. Specifically, you don’t need Immersed or other “built-in” VR apps that stream the screen. You can just open up a browser and use vdo.ninja!

Being that said. If you just want to use your goggles as a “monitor” for your laptop, then you don’t need any CPU/GPU-heavy applications for that. Just share the screen in browser and it works.

And if you want to offload that even more, then use RDP + Screen share:

• Take your road runner and connect over RDP to a remote machine. This way you can create a session there with monitors (and you can use IndirectDisplayDriver or BetterDisplay to create virtual monitors)
• Use your VR goggles to connect to see the screen shared from the remote machine (with screen share solutions like browser, VNC, vSpatial or whatever else)

This way your road runner does nearly nothing. It just connects to the remote machine. Similarly, your VR goggles don’t work hard. They just run a browser or something light.

This is the twentieth sixth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Over the years I improved my remote work setup significantly. This blog post summarizes what various approaches I took and how I configured my setup over the years. One note before moving on – I assume you don’t have any “hardware” but the computer (so no programmable boards, weird mobile devices, confidential hardware etc.).

Level 1

First approach is what most of us consider “remote work”. You just take your corporate laptop with you and you work remotely. You probably have a corporate VPN that lets you connect to the company’s resources from wherever you are.

Pros:

• You can do everything the same way you did while in the office

Cons:

• You can’t work from devices that are not onboarded with your corporate infrastructure – mobile phones, tablets, private laptops, VR goggles, etc.
• You need to take the corporate laptop with you anywhere you go
• You work from a device that is probably hot and loud working crazy on your office stuff (we all know the bloatware they install on the machines…)
• You can rarely pick your hardware, so your laptop is probably big and bulky (and yes, Macbook Air is big and bulky)
• You may not be able to connect to the VPN in some places due to weird networking conditions

While this is enough to actually do the work, I don’t like this approach. First, I always want to be able to be online (that doesn’t mean that I’m online all the time) no matter where I am and what I have on me. Specifically, I want to be able to access the infrastructure and do all my stuff using my mobile phone, private laptop, etc. I mean, I want to be able to do coding or video processing from my mobile phone while on a train (yes, I did that).

Second, I don’t like taking my corporate machine with me. This is especially troublesome when I need to travel for conferences for which I need to take more hardware with me (like 2 laptops and many mobile phones). Taking another laptop on the plane is just cumbersome. This is sometimes even dangerous depending on where you go and when they want to control your corporate machines. Not to mention, that your corporate policy may prohibit you from taking the corporate laptop to some places.

Third, I don’t like when I can’t pick my hardware. My laptop must be light and quiet, especially when I sit in a deckchair or lie down.

Level 1.5

Sometimes you cannot connect to the VPN while working remotely. This is due to weird networking, policy, or simply geolocation block.

You can fix that by getting a networking device that will connect to some other VPN and expose a tunnel for you. Typical solution is to get a router with built-in VPN. You can see this tutorial to learn how to do that.

However, I don’t recommend taking a router for that. Just get yourself a decent Windows x86 tablet, for instance Dell Venue 8 Pro and install SoftEther VPN. First, such a tablet is a “full Windows machine” so you can do whatever is needed to get connected (think about captive portals or weird passwords). Second, since this is a tablet, you can take it with you wherever you need.

Level 2

Let’s not bring the corporate machine with us at all. Get yourself a corporate virtual machine (typically called virtual desktop or VDI) and do all the work from it. Next, to work remotely, you just need to connect to the VDI from your device (which we’ll call road runner) using RDP + SSH. To do that, you may need to onboard your road runner with your corporate VPN. However, there are many solutions to work that around, so from now on we assume your road runner is not onboarded with the corporate bloatware.

Pros:

• You can do almost everything the same way you did while in the office
• You can work from your laptop
• You can take your hardware anywhere you wish

Cons:

• You may still be unable to work from mobile phones, tablets, VR goggles, etc.
• You may need to run video conferencing software on your road runner (which may not be possible)
• Your road runner may still get hot and loud when taking crazy video call with greenscreen effects
• You may not be able to do some corporate tasks – think about GPU-heavy things like video processing

This is much better now since we can connect from our road runner, so we can get a light laptop like Acer Swift 5 which is 1kg/2.2lb (your Macbook Air is at least 25% heavier).

When it comes to video conferences, they may be tricky. First, your webcam may not work over RDP. Second, RDP introduces noticeable audio delay. To fix that, you may need to run the conference software locally (I recommend doing that in browser). While it solves the problem, it also makes your road runner loud and hot.

Last but not least, if your company cannot prepare a VDI for you, you can create it on your own. Just use Sysinternal’s disk2vhd. Keep in mind it may cause some troubles with Active Domain and IdP in general, but it works.

Level 3

Let’s finally address the problem that we can’t work from any device. Let’s add a jump host. Get yourself a decent Windows machine and put it somewhere on the Internet. I generally recommend Hetzner (but whatever you choose, just get yourself a dedicated host). And BTW, choose Intel CPU instead of AMD (your virtual machines will thank you for that). Next, connect to that machine with plain RDP. This way, you can connect to it using any road runner – be it laptop, mobile phone, tablet, or whatever else. And since it can be a mobile phone, just get yourself a decent foldable. Yes, that really makes a difference when you’re working remotely.

Pros:

• You can do almost everything the same way you did while in the office
• You can work from nearly any device
• You can take your hardware anywhere you wish

Cons:

• You may need to run video conferencing software on your road runner (which may not be possible)
• Your road runner may still get hot and loud when taking crazy video call with greenscreen effects
• You may not be able to do some corporate tasks – think about GPU-heavy things like video processing
• You may have hard time connecting to the corporate VPN

When dealing with corporate VPN, use things like SSLH or FileProxy for avoiding VPN without split tunneling (also known as TCP over File System). There are many more tricks and improvements you can apply. I cover them in my talk.

Level 4

Let’s now deal with more devices and the GPU. I use two solutions for that: vSpatial and NoMachine.

First, vSpatial lets you connect to your jump host with browser and application for VR goggles. This way, you can work from basically any device you can think of. Another bonus is that vSpatial lets you forward audio and camera, so you can take your meetings from the jump host. However, as of today, vSpatial supports only 20 FPS for your webcam, so people will notice that something is off. We’ll deal with that in a second.

Second, NoMachine lets you connect to the jump host over VNC-like protocol which lets you use GPU easily. You can now do basically anything you can think of.

Pros:

• You can do everything the same way you did while in the office (assuming your VDI can do the things your corporate laptop could)
• You can work from any device
• You can take your hardware anywhere you wish

Cons:

• You may need to run video conferencing software on your road runner (which may not be possible)
• Your road runner may still get hot and loud when taking crazy video call with greenscreen effects
• You may have hard time connecting to the corporate VPN

Nothing stops you from having more jump hosts. For instance, you can host a dedicated Mac mini in Cyberlynk and connect to it.

Also, if you need more screens with your road runner, you can try portable monitor. You can also emulate screens with IddSampleDriver or its Rust version, or Better Display. This way you can literally move between machines with not a single window changing its position.

Level 5

Let’s finally deal with our road runners becoming loud and hot. We basically want to offload whole video processing to the jump host so that our device doesn’t need to encode/decode any video conferencing stuff.

To do that, use VDO.Ninja. This lets you “forward” your audio and camera to any jump host. Just use OBS or Splitcam on the jumphost to receive the video, apply any greenscreen effects you need, and forward it to the conference call.

Pros:

• You can do everything the same way you did while in the office (assuming your VDI can do the things your corporate laptop could)
• You can work from any device
• You can take your hardware anywhere you wish
• Your hardware stays cool and quiet no matter what you do

Cons:

• You may have hard time connecting to the corporate VPN

This is how my final setup looks like. I don’t do anything on my road runner. Nothing. I just connect to the jump host with RDP + vSpatial + NoMachine and I forward my audio + video with VDO.Ninja. No matter if I code, process videos, record stuff, attend conference call (or many of them), remove my webcam background, or anything else – my road runner has constant and predictable load. I can also switch to any other road runner easily.

Double-thin-client architecture

Yet another trick you can apply is what I call “double-thin-client” architecture. Instead of booting up your road runner natively, you can use native boot and get yourself a portable SSD drive like Samsung T7.

This way, you can install your operating system once, and carry it with you wherever you go. You just plug the drive to the USB, boot the operating system from the drive, and you have your environment up and running. You can then replace your road runner with another machine easily. I actually did that many times. My Windows installation survived like 4 different laptops already. Best thing is, you can simply clone the drive to another one when you get a new SSD, so you can upgrade your hardware in place with no failures.

You can use similar approach for your jump host. Do not install things locally. Just boot from VHD and this way you can easily take backup of your dedicated host.

Other improvements

There are many more things you can consider for improving your remote work. Some ideas include Bluetooth devices (obviously) and VNC server installed on a road runner, so you can walk around your home and control the device from your mobile phone. This can be useful when you’re already dialed into a call and you don’t want to rejoin from another device. Simply take your Bluetooth headset with you to another room and watch the screen of your road runner from a mobile phone.

Another idea is automatically capturing meeting notes. Since you take calls on your jump host, you can run some notetaker AI application that will automatically capture whatever you need and send it forward.

You can use Firefox with Multi-Account Containers and Tree Style Tab. They really make your work way faster. Too bad it’s just for Firefox.

You may also want to use VirtuaWin to have many desktops. I find it much better to use than “tasks” in Windows, mostly thanks to keyboard shortcuts. And if you need to run applications that need to actively interact with your desktop (like Puppeteer or UIPath), just use Desktops.

Tricks for mobile phones

You can also work on your mobile phone. First, you can get a silicone keyboard to type easily. This is really great and you can actually work from your mobile phone. I once travelled for two days with my mobile phone only and I was still able to do coding and debugging.

You can dial in over GSM. This may simplify the way you take online meetings.

Next, you can run many clones of one application with Multiple Accounts. This way, you can separate your corporate stuff from your personal one on your road runner.

You can virtualize mobile phone inside a mobile phone. Just consider applications like VMOS or Virtual Master – Android Clone. You can try Limbo x86 PC Emulator.

You can turn your mobile phone into a mobile workstation with Samsung DeX.

Short note on Apple devices

I sometimes get asked why I don’t use Apple devices. If you watch my conference recording, you can see that I always use Windows and Android. This is surprising to some people because they consider me a “power user” and yet it seems like I don’t use Apple devices. Let me give you some reasons.

First, I do use them. I really like my Mac with silicon chip as it’s really great CPU + GPU in one box. However, I don’t use it as a road runner for these reasons:

• It’s heavy. It’s over 25% heavier than my Windows road runner. While it’s not a big deal as it’s still relatively light, I just don’t want to carry additional kilograms
• It is loud and hot. I don’t like that it gets so hot when I typical stuff. This is especially irritating when working in bed
• It doesn’t support booting from VHD which I need for my public speaking stuff
• Most importantly, Macs do not support many screens easily. Right now, I have 5 displays on my desk (not counting the monitor one). When I travel, I often plug additional screens to my laptop. I just don’t accept that Macbook Air doesn’t handle 3 external screens

When it comes to iPhone, I have these reasons to prefer Android:

• iPhone doesn’t record calls. This is a major obstacle for which there is no easy workaround
• iPhone can’t run many instances of an app. I want to run them independently because I need to log in with multiple accounts. Some applications support that natively, but I need a generic solution
• Foldable phone is really a game changer
• I have some “sophisticated” applications running on my mobile phone, for instance web server and Node.js applications that automate things around text messages. I simply can’t have that on iPhone
• Many things are not available for iPhone, like some of my VPNs, port forwarding, etc. Android just works better

That’s it. I’ve been using iPhone for some time and it just doesn’t cut. For Macbook, I may use it as a road runner one day when it improves the hardware specification (sic!).

This is the twentieth part of the Types and Programming Languages series. For your convenience you can find other parts in the table of contents in Part 1 — Do not return in finally

Sometimes you may ask yourself how hard you work. You could count the hours of work but we all know that some things are harder and some others are easier. One hour isn’t the same effort in different activities. Similarly, it’s easier to work when you’re not supervised and there is no time pressure. With deadlines ahead of you, the same amount of work now becomes harder and more stressful. Another factor is when you can do your work. Maybe you can work 24/7 and do it when you just feel like it, or maybe you need to stick to a strict schedule.

Being that said, there are many factors that affect “how hard” the work is. I was considering that recently and this is the formula I think works in my case. Let’s call this “Work Complexity Model”:

Let’s say that at your work you need to do coding, doc writing, and mentoring. Therefore, your set of activities would have these three elements. You then need to asses the cost of context switch which is your personal coefficient. It doesn’t matter per se and do not compare it with others. You can use it to compare your effort in different months when you move between projects or tasks.

Next, you need to decide on the period, for instance a single quarter.

Next, for each activity you need to measure how many increments you have. If you need to deliver your work at the end of the sprint, then you would have six increments (two increments each month). If you need to deliver something every day, then you would have ninety increments.

You then need to measure the size of the increment. Obviously, this is very subjective and it’s up to you to define how hard a particular piece of work is. Technically, this should be the amount of energy (physical and mental) you spent on the task. Since it’s hard to measure, you can just count the number of hours dedicated to the task and then multiply that by how intense and frustrating the work was.

Finally, you need to include the length of the timeframe to do the work. If you can work asynchronously 24/7, then your timeframe would be the whole period. If you can do your work during work days 9-5, then it’s just these working hours.

Let’s say that you can do coding 24/7, same for doc writing, but the mentoring you can do only on Mondays 9-5. You need to deliver your coding artifacts every other week, your doc writing twice a week, and your mentoring every Monday. Therefore, this would be your complexity over 3 months.

See that the formula has the following features:

• It shows that context switching has some cost that scales non-linearly with the number of activities
• Number of increments affects the result much more than the size of the increments. That’s because supervision tends to slow us much more
• The length of the timeframe is also included in the formula

It’s up to you what your values for C and S are. The goal of this formula is not to give you some absolute scale. It’s much more to compare your different projects to have some numbers showing you how hard it was, as we know that our memory misleads us often.

This is the twentieth fifth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Today we’re going to make our VR experience even better. This is continuation of the previous part.

As mentioned before, there are issues with ALT+TAB and WINDOWS key. I remapped CAPS LOCK to WINDOWS with PowerToys, but today we’re doing something better.

First, I was looking for a good full-size keyboard with touchpad that I could use with VR. The best thing I found is Perixx PERIBOARD-313. Touchpad is in some weird position but I got used to that. It has this irritating Fn key, lacks Right WINDOWS, and the space is quite short, but generally the keyboard is good enough.

However, it’s a wired keyboard. You can plug that into your VR headset and it works well. The keyboard has additional USB slots, so you can plug more devices to your VR this way. Still, we’d like to make it wireless. To do that, I’m using Bluetooth Adapter for Keyboard & Mouse (BT-500). This adapter has a couple of interesting features. First, it turns your wired keyboard or mouse into a wireless one. Second, it can remap keys on the fly, add macros, or have custom settings that you can switch between easily.

I’m using the adapter to remap Left WINDOWS to CAPS LOCK, and TAB to SCROLL LOCK. This is my setup:

map add l_com caps_lock
map add tab scroll_lock
save
reboot

Next, I remap CAPS LOCK to Left WINDOWS back using PowerToys. I do the same for SCROLL LOCK and TAB.

Finally, I use another adapter to turn any wired mouse into wireless one. You could go with Bluetooth mouse instead. With these things in place, I now have a “regular” wireless keyboard that I can use with VR. Even though it’s Android behind the scenes, all is just like a regular Windows laptop.

As a side note, I tried remapping keys to F13 and similar special keys. Unfortunately, Quest 3 doesn’t handle these properly and my remote machine I connect to over vSpatial doesn’t receive the keys.

This is the twentieth fourth part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

Today we’re going to solve a problem of RDP session moving your windows around when you connect from multiple machines. Let’s see how it works.

Problem statement

First, a couple of assumptions what I’m trying to solve:

• I’m using mstsc.exe to connect to the RDP server. mstsc.exe is the regular Windows RDP client (so called the “blue” client). I’m using the regular application, not the UWP one
• I have two machines that I use to connect to the RDP server. These machines are Client1 and Client2. Both Clients have three screens attached. While I’m using physical screens, you’re free to use IddSampleDriver or vSpatial virtual screens. I think the same should apply to other solutions like spacedesk and similar. The same should apply to screens created by BetterDisplay (when you RDP from Mac to your Client1).
• Screen1 is “at front”, Screen2 is “on the right”, Screen3 is “on the left”. The actual ordering doesn’t matter but I’ll refer to this later in this post
• I use Client1 to connect to the RDP server and I open maximized Browser on Screen1 and maximized Notepad on Screen2. I then disconnect and connect to the RDP server from Client2. At this point, I want Browser to still be on Screen1 (“at front”) and Notepad on Screen2 (“on the right”).
• Screen1 is my “main display” and it has the taskbar. I want the same in the RDP server (so taskbar should be on Screen1)

I’m going to use screenshots to explain my setup. They are a little trickier to let’s walk one by one.

My setup

First, my Client1 has five physical screens in total. However, I use Screen4 to duplicate my Screen1, and I use Screen5 to keep things outside of RDP. Therefore, I only want to use Screen1, Screen2, and Screen3 in the RDP session (and obviously Screen4 which is implicitly showing Screen1). I also have 5 additional virtual screens created with IddSampleDriver and 2 more virtual screens created with vSpatial. I don’t use these virtual screens (they are turned off).

This is what “Display” shows me which from now on I’ll call “Display Setup”:

So this is the mapping:

• Screen1 (“at front”) is Display Setup Screen 1
• Screen2 (“on the right”) is Display Setup Screen 12
• Screen3 (“on the left”) is Display Setup Screen 11
• Screen4 (duplicating Screen1) is Display Setup Screen 9
• Screen5 (kept outside of RDP) is Display Setup Screen 10
• Display Setup Screen 2-6 are IddSampleDriver virtual screens
• Display Setup Screen 7-8 are vSpatial virtual screens

Now I want to use screens Screen1 + Screen2 + Screen3 in RDP session only. Scott Hanselman explains how to do it. We first need to list the screens with mstsc /l and this is what I get (which from now I’ll call “MSTSC Setup”):

So here comes the mapping:

• Screen1 (“at front”) is MSTSC Setup Screen 44
• Screen2 (“on the right”) is MSTSC Setup Screen 46
• Screen3 (“on the left”) is MSTSC Setup Screen 47
• Screen4 (duplicating Screen1) is MSTSC Setup Screen 44 (the same as Screen1)
• Screen5 (kept outside of RDP) is MSTSC Setup Screen 45

It’s important to notice that numbers do not match between Display Setup and MSTSC Setup. This is very important and makes the whole trouble. Let’s now solve the problems.

Using only a subset of displays in the RDP session

This is actually easy. When you see Scott Hanselman’s blog, it mentions the parameter called selectedmonitors. You use it to specify the comma-separated-list of monitors that you want to use for the RDP session.

In my case, I need to use MSTSC Setup Screen 44, MSTSC Setup Screen 46, and MSTSC Setup Screen 47. So the parameter in my case is:

selectedmonitors:s:44,46,47

After connecting to RDP, this is what “Display” shows (which I’ll refer to as Display RDP Setup):

In textual form each line is screenId: width x height; (xBegin, yBegin, xEnd, yEnd):

44: 1920 x 1080; (0, 0, 1919, 1079)
45: 1080 x 1920; (363, 1080, 1442, 2999)
46: 2560 x 1440; (1920, 0, 4479, 1439)
47: 2560 x 1440; (-2560, 0, -1, 1439)

So, here is the mapping:

• Screen1 (“at front”) is Display RDP Setup Screen 1
• Screen2 (“on the right”) is Display RDP Setup Screen 2
• Screen3 (“on the left”) is Display RDP Setup Screen 3

You can see that there are only 3 screens in the RDP session. Exactly as I wanted.

Controlling the main display in RDP

selectedmonitors parameter controls this. The main display is the first monitor from this list. So, if you have selectedmonitors:s:x,y,z, then the screen x is going to be the main display in RDP. This doesn’t need to match your main display on the host. In my case, my parameter is set to selectedmonitors:s:44,46,47, so the main display in RDP is going to be MSTSC Setup Screen 44.

Making RDP not move windows

This is the trickiest part. Remember that I open maximized Browser on Screen1 and maximized Notepad on Screen2. Based on our mappings above, the Browser is on Screen1 = Display Setup Screen 1 = Display RDP Setup Screen 1 = MSTSC Setup Screen 44. Similarly, Notepad is on Screen2 = Display Setup Screen 12 = Display RDP Setup Screen 2 = MSTSC Setup Screen 46.

Now, these are the things that I believe are correct. I found them empirically and they work for me across 4 different laptops, but your mileage my vary.

• The order of screens in selectedmonitors parameter doesn’t matter (apart from choosing the “main display” explained in the previous section)
• The numbers from Display Setup do not matter! The numbers from MSTSC Setup do not matter! The only thing that matters is the order of screens from MSTSC Setup
• The resolutions of the screens do not matter! Windows remembers maximized windows positions not based on the geometry (like “the window’s top left corner is in pixel 0,0) but rather as “the window is maximized on Display RDP Setup Screen 1“
• You can’t control numbers from MSTSC Setup programmatically. You can only control them by physically moving your screens to different ports/connectors/docking stations/USB adapters, etc.
• To get the windows not move, you need to have screen “passed to RDP” in the same order between Client1 and Client2 (between different clients)

That’s it. It may be a little bit cryptic, so let’s delve into how I think it works.

mstsc.exe uses some API to enumerate displays and then applies logic to merge windows or reorder them. Notice, that in my case Display Setup shows 12 screens while MSTSC Setup shows only 4. That’s because 7 virtual screens are turned off, and Screen4 duplicates Screen1.

Now, it seems that mstsc.exe enumerates the screens, and then for each screen checks if it was selected in selectedmonitors. If yes, then it registers the screen in the RDP session. So the code looks conceptually like:

foreach(screen in enumerateScreens()):
    if(screen.id in selectedmonitors) then
        registerScreenInRdpSession(screen)
    end
end

This explains why the order of screens in selectedmonitors doesn’t matter. Assuming that, this is what we need to get:

• We want the Browser to still be presented on the Screen1 (“at front”). This screen was registered first in the loop above when connecting from Client1. Therefore, when Client2 enumerates the screens in mstsc.exe, Screen1 must be the first one that matches the if condition. In other words, Screen1 can be in whatever position according to MSTSC Setup and can have any number, but there must be no other selectedmonitors screen earlier in the list of mstsc.exe /l
• We don’t care about resolutions. Screen1 on Client1 can have different resolution than Screen1 on Client2. The Browser will still be “maximized on the Display RDP Setup Screen 1
• We want the Notepad to still be presented on the Screen2 (“on the right”). Same logic applies: mstsc.exe continues enumerating the screens and the second screen that gets picked must be the screen on the right

Now, how do we control the order of screens enumerated by mstsc.exe /l? The short answer is: we can’t do that programmatically. We need to physically change cables! What’s worse, this order breaks when you change settings like Duplicate screen or Extend desktop to this screen.

How to change cables physically? There are solutions:

• Just plug cable to some other slot in your laptop/PC
• Use docking station and change the order of cables (or move some monitors to your laptop and some through your docking station)
• Use another docking station! Yes, you can have many of them (I actually have 2, that’s a long story)
• Use USB adapters like i-tec USB Type C to DisplayPort Adapter, 4K Video, Supports 2 External Displays. I have two of them and they are great. Your mileage may vary, obviously
• Try hacking mstsc.exe with things like API Monitor (I haven’t tried that but that seems plausible)

I tested this setup across four different laptops – one with physical screens, two with virtual screens by IddSampleDriver, one with virtual screens by vSpatial. In my case, windows are not moved anymore. I can literally switch from one computer to another with different screen resolutions but all the windows stay where they were. And yes, I had to switch my cables like crazy. In my case it was even worse, because Screen1 was attached directly (and so was enumerated first by Display Setup) but Screen4 was attached by an adapter (and was taking precedence in mstsc.exe /l). It works, though.

This is the twentieth third part of the Availability Anywhere series. For your convenience you can find other parts in the table of contents in Part 1 – Connecting to SSH tunnel automatically in Windows

In this part we’re going to see how we can work remotely from VR goggles with no PCVR around. Effectively, we can work remotely with goggles and a bluetooth keyboard, nothing else.

My workstation

My home workstation is quite powerful. I have 5 physical screens which I use to display 4 different screens. I also use VirtuaWin to have many desktops. I also work remotely from a dedicated machine to which I connect using MS RDP (mstsc.exe).

My typical setup works like this:

• My main physical screen is 24” with FullHD resolution. I duplicate this physical screen to another physical screen which is 10” and has touch capabilities. I use the touch to draw diagrams.
• My two additional side screens are 32” with 4K resolution
• My fifth screen is 10” big and I use it to display all the instant messengers I need (slacks, discords, meets etc). That’s basically a browser window that I configure to be visible on all of the desktops.
• I have 11 desktops configured in VirtuaWin.
• To connect remotely to my machine, I move to some other VirtuaWin desktop and use four physical screens (FullHD + 2x4K + touch-enabled screen) to open mstsc.exe.
• When I need to connect to some other machine, I basically do CTRL+ALT+HOME to unfocus the current RDP, then I switch to another VirtuaWin desktop, and there I connect to the server. This way I can quickly switch between multiple machines I have

This setup works great for me. I can work remotely from any place around the planet. When I’m not at home, I can use my road runner laptop with two physical screens or my mobile phone to connect over RDP to the remote server. I do various things over RDP – I code, I write/read documents, I postprocess videos, I record videos (yes, I record them over RDP!), I browse the Internet. Typical office work.

I wanted to replicate this setup as much as possible with VR goggles. I managed to do that with Meta Quest 3 and vSpatial. Let’s see how I did that.

What we need

First, you need the goggles. I used Meta Quest 3 and they were quite good. I can’t complain on the image quality and I definitely can do all the stuff I need. The battery lasts for around 90-120 minutes which is okayish.

Next, you need to have a machine with many physical displays. That’s the most tricky part if you want to configure it entirely over RDP and use free version of vSpatial. I use two dedicated machines: one is Windows Server and the other is Mac. I first connect to Windows Server from my mobile phone over the regular RDP and log in as User1. At this point, I have a session in Windows Server that has just one display. Next, I use NoMachine to connect from Windows Server to my Mac. In my Mac, I create multiple physical screens using Better Display. Next, I connect from my Mac to my Windows Server and log in as User2 using the regular RDP application. At this point, I have a session in Windows Server that has multiple physical screens. All of that done entirely from my mobile phone, so I can do that when I’m travelling. While it works and allows for any setup of monitors, it’s probably good enough to just use 4 screens from paid vSpatial. Up to you.

Next, we need the vSpatial server. I install it in Windows Server for User2. I then log in to Windows Server using vSpatial application in Meta Quest 3 goggles. The vSpatial client correctly recognizes all 4 displays and now I have access to a regular Windows machine. I can install VirtuaWin and configure desktops the regular way. It all works just like on the screenshot below:

You can see one screen at the top and three screens in the middle. This is nearly the same setup as I have at home.

Finally, I just pair my bluetooth keyboard with touchpad and that’s it. CTRL+ALT+HOME works, so I can work the same way as at home.

Now, sky is the limit. Since I managed to RDP into a regular Windows machine, I can RDP anywhere I need, I can install any application I need, and I can do all of that using my goggles and the bluetooth keyboard only. I want to stress that there is no PCVR here. The Windows Machine with vSpatial server is completely virtualized and has no dedicated graphics card. Also, I don’t use powerful Internet connection. I just connect over regular WiFi. This works from mobile hotspot as well as it’s just RDP.

When it comes to online meetings, I can take them from my mobile phone. If I want to share the screen, I can just join the same meeting from my mobile phone and from my remote machine, and I can share the screen from Windows Server. However, you can also start browser on VR and join meeting like in Google Meet. The meeting will work even after you switch to vSpatial.

Quirks

There are quirks, obviously.

First, Meta Quest 3 seems to be based on Android. Bluetooth keyboards don’t work 100% properly with Android when you RDP to Windows. One issue is that the WIN key doesn’t work. Another, some shortcuts (like ALT+TAB) are captured by the host (goggles in this case).

To solve the lack of WIN key, I use PowerToys. I just remap CAPS LOCK to LEFT WIN on my Windows Server. You can also use AutoHotkey with the following script:

*CapsLock::
    Send {Blind}{LWin Down}
Return

*CapsLock up::
    Send {Blind}{LWin Up}
Return

The script rebinds CAPS LOCK to LEFT WIN. No matter which method you use, you can just use CAPS LOCK in all the shortcuts (like maximizing the window with LEFT WIN+ARROW UP which becomes CAPS LOCK+ARROW UP). While it’s a bit painful, it works. You need to install AutoHotkey in all the remote machines you connect to.

I didn’t solve the ALT+TAB issue with AutoHotkey. I solved it with PowerToys by remapping SCROLL LOCK to TAB. I think you could use AutoHotkey to remap it as well.

Second, vSpatial has a concept of active (focused) desktop. If you worked on one screen and want to click on another one, the mouse click changes the focus. You need to click again to do the actual click you want. This is not a big deal and I got used to it very fast.

Third, I think vSpatial is slightly slower than the regular RDP. I don’t find this as a problem, though, as it’s fast enough for me.

Fourth, the goggles don’t last for long (up to two hours). On the other hand, maybe it’s better for your eyes. You can also buy an additional strap with built-it powerbank.

Summary

It took me quite a while to figure out how to configure this setup. However, now I can truly work using goggles and no laptop around. That sounds like a great approach when traveling – imagine sitting at the airport and having multiple screens, just like at home. Now, the only issue is around the goggles. I hope one day they will be smaller, lighter, and will have stronger batteries. You can always plug them into powerbank if needed, though.