Custom memory allocation in C# Part 9 — Hijacking new in x64

This is the ninth part of the Custom memory allocation series. For your convenience you can find other parts in the table of contents in Part 1 — Allocating object on a stack

We already hijacked new in C#. Last time we did this for 32-bit applications, today we will modify the code to support x64. Let’s go.

Object allocation

Let’s start with object allocation. We had the helper method (which was not used anyway) for getting pointer to the object. Just for keeping everything clean, we can fix it:

No big changes, just replaced int with IntPtr.

Now, let’s modify RawAllocate:

Once again, we remove unnecessary casting to int and use IntPtr all the way.

Hijacking new

Now we need to modify method getting address of .NET allocation method:

We replace integers with long integers, add two more offsets (for x64), and change casting to long instead of int. We can use the same code for x86 now as we can always cast to the wider types.

Last part, hijacking the operator:

Different cast for offset calculation, nothing else. We rely on the method placement when subtracting addresses, if the methods would be in different order, we would need to do the absolute jump (e.g., via push + ret trick).

You should also notice call to UnlockPage which we do in the following way:

Testing

Just for the sake of completeness, here is the debugging session for doing the same manually:

Getting .NET allocation method:

As we can see, now we are calling different method clr!JIT_TrialAllocSFastMP_InlineGetThread (00007ffb`b1052510)

Let’s grab our allocator:

And now modify the allocation:

Let’s verify:

Jump is in place, we are good.

This code works for x86 and x64, both Debug and Release modes. Tested with Windows 10 x64 1703 and .NET Framework 4.5.2.