Custom memory allocation in C# Part 13 — In-place serialization

This is the thirteenth part of the Custom memory allocation series. For your convenience you can find other parts in the table of contents in Part 1 — Allocating object on a stack

Recently Jean-Bernard Pellerin showed nice tricks for binary serialization in place. There is also a description of .NET Framework hack using unions for transforming any object into bytes. Using typed references we can actually do it a little differently.

using System;
using System.IO;

public class Program
{
    public static void Main()
    {
        unsafe
        {
            var bytes = new byte[0];
            Class s = new Class
            {
                x = 0xBADF00D,
                y = 0xFEED233

            };
            TypedReference typedReference = __makeref(bytes);
            int* objectAddress = (int*)(*(int*)(*(int*)(&typedReference)) - 4);

            var arrayAddress = (int)objectAddress;
            *(int*)(arrayAddress + 8) = Int32.MaxValue;

            typedReference = __makeref(s);
            objectAddress = (int*)(*(int*)(*(int*)(&typedReference)) + 4);

            var structureAddress = (int)objectAddress;
            var structureLength = 8;

            var elementAddress = structureAddress - arrayAddress - 12;

            Console.WriteLine($"Array length: {bytes.Length.ToString("X")}");
            Console.WriteLine($"Array address: {arrayAddress.ToString("X")}");
            Console.WriteLine($"Structure to serialize address: {structureAddress.ToString("X")}");

            for (int i = 0; i < structureLength; ++i)
            {
                Console.WriteLine(bytes[elementAddress + i].ToString("X"));
            }
            Console.WriteLine();

            using (var stream = new FileStream(@"file.txt", FileMode.OpenOrCreate))
            {
                stream.Write(bytes, elementAddress, structureLength);
                stream.Close();
            }

            using (var stream = new FileStream(@"file.txt", FileMode.OpenOrCreate))
            using (var reader = new BinaryReader(stream))
            {
                Class s2 = new Class();
                s2.x = reader.ReadInt32();
                s2.y = reader.ReadInt32();
                Console.WriteLine($"{s2.x} {s2.y}");
            }
        }
    }
}

class Class
{
    public int x;
    public int y;
}

using System;

using System.IO;

public class Program

{

public static void Main()

{

unsafe

{

var bytes = new byte[0];

Class s = new Class

{

x = 0xBADF00D,

y = 0xFEED233

};

TypedReference typedReference = __makeref(bytes);

int* objectAddress = (int*)(*(int*)(*(int*)(&typedReference)) - 4);

var arrayAddress = (int)objectAddress;

*(int*)(arrayAddress + 8) = Int32.MaxValue;

typedReference = __makeref(s);

objectAddress = (int*)(*(int*)(*(int*)(&typedReference)) + 4);

var structureAddress = (int)objectAddress;

var structureLength = 8;

var elementAddress = structureAddress - arrayAddress - 12;

Console.WriteLine($"Array length: {bytes.Length.ToString("X")}");

Console.WriteLine($"Array address: {arrayAddress.ToString("X")}");

Console.WriteLine($"Structure to serialize address: {structureAddress.ToString("X")}");

for (int i = 0; i < structureLength; ++i)

{

Console.WriteLine(bytes[elementAddress + i].ToString("X"));

}

Console.WriteLine();

using (var stream = new FileStream(@"file.txt", FileMode.OpenOrCreate))

{

stream.Write(bytes, elementAddress, structureLength);

stream.Close();

}

using (var stream = new FileStream(@"file.txt", FileMode.OpenOrCreate))

using (var reader = new BinaryReader(stream))

{

Class s2 = new Class();

s2.x = reader.ReadInt32();

s2.y = reader.ReadInt32();

Console.WriteLine($"{s2.x} {s2.y}");

}

class Class

{

public int x;

public int y;

}

We have a simple class stored on the heap. It contains two integers which we initialize to something “readable”.

The trick here is to allocate an empty array on the heap which we will use to pass it to stream using existing API (without spans, slices or whatever else). Array is empty to not consume any (unnecessary) memory.

We then cheat and modify the array size so CLR doesn’t scream that we want to read bytes beyond the end. Effectively, our array becomes a pointer, something very similar to using arrays in C++ (which can be used as pointers in some places).

Finally, we just need to calculate the address of our structure and pass indexes to the stream API. We can see that it effectively serializes the object with no additional allocation.

Pros: It works (at least on my machine).
Cons: It doesn’t work with structures currently. To access structures we need to have two things in mind: stack grows towards lower addresses so effectively we would need to read the memory using negative indexes through the array. Also, it doesn’t work if we allocate more than 2GB of memory.