.NET Inside Out Part 8 — Handling Stack Overflow Exception in C# with VEH

This is the eighth part of the .NET Inside Out series. For your convenience you can find other parts in the table of contents in Part 1 – Virtual and non-virtual calls in C#

If you are interested in the topic see the talk page

Today we are going to swallow Stack Overflow Exception in C# using Vectored Exception Handling. Let’s begin.

Table of Contents

Why?

As specified in exception’s documentation, you are not allowed to catch Stack Overflow Exception (SOE) in .NET 2.0+. Why would you even try to do that? Well, normally you should detect and avoid it but imagine that you are writing a testing framework. If some of the tested code has a bug and results in SOE your runner will fail instead of showing nice message with stacktrace. You could in theory change this by reworking shim loading CLR but today we will do it using VEH.

I hope you read the Allocating object on a stack and Generating Func from a bunch of bytes posts because I use those tricks a lot.

How?

VEH is like an external exception handler, similar to Application_Error method. When something goes wrong you are notified with exception details. It is a Windows mechanism so don’t count on nice object which you can utilize from C#, you need to work with pointers and low level stuff. There can be multiple VEH handlers and each of them can indicate either to continue execution or continue search. Continuing execution means restoring thread to the state at the exception (or to any different state by changing CPU registers) and carrying on. Continuing search means executing next VEH handler, SEH handler or unhandled exception handler.

We are going to register new VEH handler, remember the point where we start calling method causing SOE (it is like adding catch in the code), catching the exception and redirecting the CPU to known location. Let’s go.

Implementation

We start with this faulty method:

You can uncomment any part you like. Dividing by zero should be caught by C# exception handler, however, SO should be ignored and crash your process. Ideally, if we get division by zero, C# handler should run, otherwise (if it is a SOE), our VEH handler should be in play.

We need the following helper methods:

P/Invoke to register VEH:

Methods to get addresses of machine codes:

Some space to hold registers to restore:

First, easy part — exception handler:

Lots of magic here!

First, method CatchingVeh is an actual VEH handler. It has very weird signature – it is because of VEH handler signature. Basically, the pointer to exception structure is passed via the stack, however, C# calling convention would access it via fourth parameter so we need to ad dummy parameters. Otherwise we would need to access the exception via asm code which is tedious so let’s make a different hack (see later).

We first get exception code. You can easily traverse structures on your own using pointers so there is nothing special here. We compare exception code with value 0xC00000FD which is a Stack Overflow Exception code. If the code is different, we just continue searching for different handler, eventually going to ordinary C# catch handler.

Next we start toying with CPU. ExceptionInfo has a pointer to ContextRecord structure which holds CPU registers at the time of throwing the exception. If we indicate that we want to continue execution, those registers will be restored from this structure. So we can modify EIP here and effectively jump to some other place. We jump to our catch handler (or in this sample just to line right after calling the faulty method). We also restore other registers so we are in the same state as before calling faulty method. So: we change EIP to do jump. We change ESP and EBP to effectively drop frames from the stack. We restore ESI, EDI and EBX to be in the same state and to adhere to calling convention.

This was easy, now comes the tricky part. We need to fill EipEspEbpEsiEdiEbxHolder with register values before calling the faulty method. And for this we need to use asembler language.

Even more magic here…

First, we get addresses of methods we use.

Next, we need to fix CatchingVeh function. If you disassembled it you would see that it uses ret 8 instruction at the end which drops two integers from the stack. To adhere to calling convention we must drop exactly one integer so we need to manually modify the code to use ret 4

Next, we register VEH handler and get address of array holding registers. We need to have machine address because we are going to fill it from asm. We pass this address via delegate parameters (see invocation at the end) so it lands in edx register (recall that ecx is a this parameter).

And then we begin our magic. We want to get the eip register but we cannot read it directly so we use a trick. At the end of machine code we have a helper function. When we call it, return address is pushed on the stack so we can read it from there and return via eax.

So we call this method at the beginning and when we are back we have the eip value in eax register. We just need to calculate the actual address after the method call (yes, we just count the bytes by hand). Next, we store other registers which we can easily access.

Finally, we just do absolute call via eax register and return. That’s it.

Summary

We can now handle the SOE easily using the VEH. We could do some more magic to analyze the reason behind the exception, capture the stack trace etc, but this is an exercise for you. And don’t forget to remove the VEH handler!