This is the eighteenth part of the .NET Inside Out series. For your convenience you can find other parts in the table of contents in Part 1 – Virtual and non-virtual calls in C#

We saw in Part 8 — Handling Stack Overflow Exception in C# with VEH how to handle SOE in C#. Obviously, that method is very unsafe and shouldn’t be used in production. But what can we do if we really need to handle SOE?

The answer is — nothing. .NET will always kill the CLR, we cannot stop that, we cannot do anything about that. We can try handling the situation to log errors but it is very dangerous and there is no reliable way to do so. Let’s try, at least.

Let’s begin with the following application in C#:

using System;

namespace StackOverflowGenerator
{
    class Program
    {
        public static void Main(string[] args)
        {
        }

        public static int Start(string args)
        {
            Console.WriteLine("Looping: " + args);
            Start(args);
            return 0;
        }
    }
}

Nothing big, we just loop and print the argument. The thing here is we cannot run this application just like that. To handle StackOverflowException we need to use custom CLR loader. So let’s use this C++ code:

#include <metahost.h>
#include <string>

#pragma comment(lib, "mscoree.lib")

#import "mscorlib.tlb" raw_interfaces_only				\
    high_property_prefixes("_get","_put","_putref")		\
    rename("ReportEvent", "InteropServices_ReportEvent")

using namespace mscorlib;
using namespace std;

LONG WINAPI VehExceptionHandler(_EXCEPTION_POINTERS* pExceptionInfo) {
	printf("%x\n", pExceptionInfo->ExceptionRecord->ExceptionCode);
	//0xE053534F - Soft SOE
	//0xE0434352 - CLR Exception
	//0xE06D7363 - C++ Exception
	if(pExceptionInfo->ExceptionRecord->ExceptionCode == 0xE053534f) {
		puts("SOE!");
	}

	return EXCEPTION_CONTINUE_SEARCH;
}

int main()
{
	HRESULT hr;
	ICLRMetaHost *pMetaHost = NULL;
	ICLRRuntimeInfo *pRuntimeInfo = NULL;
	ICLRRuntimeHost *pClrRuntimeHost = NULL;

	// build runtime
	hr = CLRCreateInstance(CLSID_CLRMetaHost, IID_PPV_ARGS(&pMetaHost));
	hr = pMetaHost->GetRuntime(L"v4.0.30319", IID_PPV_ARGS(&pRuntimeInfo));
	hr = pRuntimeInfo->GetInterface(CLSID_CLRRuntimeHost, IID_PPV_ARGS(&pClrRuntimeHost));

	// configure SOE handling
	ICLRControl *pCLRControl = NULL;
	pClrRuntimeHost->GetCLRControl(&pCLRControl);

	ICLRPolicyManager* pCLRPolicyManager = NULL;
	hr = pCLRControl->GetCLRManager(IID_ICLRPolicyManager, (PVOID*)&pCLRPolicyManager);
	hr = pCLRPolicyManager->SetActionOnFailure(FAIL_StackOverflow, eRudeUnloadAppDomain);

	// start runtime
	hr = pClrRuntimeHost->Start();

	// add VEH
	AddVectoredExceptionHandler(1, VehExceptionHandler);

	// execute managed assembly
	DWORD pReturnValue;
	hr = pClrRuntimeHost->ExecuteInDefaultAppDomain(
		L"..\\StackOverflowGenerator\\bin\\Debug\\StackOverflowGenerator.exe",
		L"StackOverflowGenerator.Program",
		L"Start",
		L"Argument",
		&pReturnValue);

	puts("I'm done");
}

Let’s go through this line by line. Initially we just import some libraries to have an access to CLR methods. Next, in lines 27-35 we just load .NET 4.5 (it’s still based on version 4). You can load different .NET here (for instance version 2).

Next, in line 43 we tell .NET to unload application domain in case of SOE. The name says eRudeUnloadAppDomain which should suggest you that this is not nice.

Next, in line 46 we start the runtime and in line 49 we add VEH handler to report the SOE.

Handler is in lines 13-22. If you run this application you see the following output:

Looping: Argument
...
Looping: Argument
e053534f
SOE!
e0434352
e06d7363
I'm done

There are some more exceptions afterwards related to closing the application but they are not interesting. You can see multiple exceptions being reported. These are:

0xE053534F - Soft SOE
0xE0434352 - CLR Exception
0xE06D7363 - C++ Exception

What is “Soft SOE”? There are actually two types of StackOverflowException. One is reported by MMU and has code 0xC00000FD. The other one is thrown by .NET when CLR realizes there is not enough stack, and the exception code is 0xE053534F.

So we can see that we cannot handle SOE but we can at least run some code when it happens and not kill the application. If you comment out line 43 then the application will get terminated as soon as SOE is thrown.

However, this is very tricky. Notice that we do printf and puts in VEH handler. Is it safe? Well, not really. Let’s modify the code:

public static int Start(string args)
{
	Start(args);
	return 0;
}

And now our application crashes and VEH “is not called”. It is called, in fact, but it allocates some memory on the stack and crashes the process. Let’s change it to

LONG WINAPI VehExceptionHandler(_EXCEPTION_POINTERS* pExceptionInfo) {
	return EXCEPTION_CONTINUE_SEARCH;
}

And now we get this output:

Process is terminated due to StackOverflowException.

If you attach the debugger you can actually see that this time we get

Exception thrown at 0x00270448 in StackOverflowCatcherViaShim.exe: 0xC00000FD: Stack overflow (parameters: 0x00000001, 0x002F2FFC).

To sum up:

• You cannot stop .NET application from terminating in case of SOE (unless you do the magic from Part 8)
• You can stop the process from terminating (and continue running in native space)
• You can execute code on SOE (soft and hard) but it is unreliable (and very dangerous)

So what should you do in reality? If you really need to handle SOE then it seems like you could set some flag atomically in VEH and pause the thread. Then, some other thread could observe the flag, dump the memory when needed, and clean up. You probably cannot do anything on the thread with insufficient stack as it is easy to crash the process.