So the first thing to tackle here was that Mupen64Plus is pretty heavily threaded. I’m still a little wary about implementing pthreads into DEX, because adding multiple threads interacting with memory etc simultaneously introduces a whole lot of scary edge cases that I didn’t want to deal with.
Luckily, Mupen64Plus has a fallback which uses coroutines to jump around the code. Except their coroutine library uses setjump and longjump, which is just, not happening in C#. We don’t have enough control over execution to make it happen.
Enter Asyncify.
Emscripten has this neat fiber API which is more or less exactly what I needed. It’s pretty standard stuff - it lets you pause execution at a spot, go over to do something else with a brand new stack, then trampoline back to the original yield to continue whatever it was you were doing. Currently they’ve got this as a Binaryen pass, but it used to be an LLVM pass - either way, the principle is the same.
For every function that might* call into something which yields execution, add some instrumentation after that call to check whether we did yield, and if so immediately return instead of continuing with what you were doing. Then add instrumentation to every function which calls that function, and every function which calls that function, on and on up the call tree. This has the effect that immediately after you yield, your whole stack just unwinds like dominoes all the way back up to the top.
Then, as you unwind, if you make note of the local variables (stack pointer, the index of the function that was the unwind source, etc), you have everything you need to “rewind” the stack and continue execution like nothing happened.
This increases your module size like a motherfucker, because it has to instrument every function in the whole module which might be able to yield. So not just any function which calls a function which calls a function which yields… any function which calls a function pointer, because what if the function it points to has a yield, and any function which calls that function which calls a function pointer… yeah. Basically triples the size of your code. But it works. And there are ways to reduce that cost which you can go read up on if you’re interested.
