More backstory. I realized pretty early on that it wouldn’t be that big of a challenge to write a homebrew shader_c compiler for a few reasons:
shader_c used to contain DXBC. It doesn’t anymore. S&box is on Vulkan, so shader_c has SPIR-V now. GLSL to SPIR-V compilers are dime-a-dozen (I chose to use glslang).
The wonderful folks who maintain VRF have already done the majority of the legwork in reverse engineering the shader_c format. It turned out to not be comprehensive enough to write the full compiler, so I had to do a little bit of reversing on my own - but once the hard stuff like compression has been figured out, it’s really just a matter of diffing compiled shaders to see what changed.
There’s actually a lot of stuff you can fuck up in the shader_c binary and s&box will still accept it. Obviously there are fields here and there which will crash the game if they’re even slightly incorrect, but my vertex layout was just dead wrong for the first half of the project without me realizing and the engine was able to cope.
So I got to work. Turns out the format’s pretty intuitive all things considered. Combos are organized how you’d expect (static top-level combos, then dynamic combos, then individual SPIR-V sources beneath that).
Each SPIR-V input file is bundled with some reflection metadata which the game uses to create the Vulkan pipeline layout, and then there’s some top-level information about the set of globals and textures and samplers and whatnot, and where to put them when they get mapped down to static->dynamic->SPIR-V.
Each dynamic combo definition is trailed with some renderstate information, so I can just enumerate all the possible permutations of renderstates I might need and control them C#-side with Graphics.SetCombo.
I’ll be publishing the DexGL source at some point soon. What I’ve gleaned from the format is not perfect, but it’s leaps and bounds more comprehensive than what VRF is able to read right now and it’s functional enough that it compiles all my shaders without issue.
