Provide public DXIL specification

[Jasper-Bekkers]

This may be a controversial topic, but I figured I'd bring it up regardless.

Right now DXC is the defacto implementation for a compiler emitting valid DXIL, however, various projects are running into trouble trying to support DXIL alongside - say - SPIR-V. Think, RenderDoc, https://github.com/gfx-rs/naga, https://github.com/embarkstudios/rust-gpu, the Microsoft provided Mesa3d implementation etc.

SPIR-V has two things going for it making it a prime target for shader compiler language experimentation:

• It has an official specification
• It has machine readable representation of that specification (for example: https://github.com/KhronosGroup/SPIRV-Headers/blob/master/include/spirv/unified1/spirv.core.grammar.json)
• It's easy to generate (it's just a series of u32's).

I realize this is not a small ask, however, I do think it's important that it's made & discussed publicly now with DXC being part of a more open ecosystem.

Edit:

This is mostly a request for:

• A spec around the binary serialization format used for DXIL
• A machine readable set of data around types and functions etc (I think some of this stuff is already being generated from python scripts already, in this repository).

[seanbaxter]

I agree. I'm beginning to implement a DXIR codegen. I have three big problems:

1. No HLSL specification. No docs of quality at all. I don't know how to even write HLSL to create the disassembly I need to implement. Eg is new/linear cbuffer layout, which is all throughout the DXIR source code, but is not documented for HLSL.
2. DXIL.rst needs a lot more work. There's no docs for many of the special functions. Forensically determining what they do is very hard from reading the source. I'm not Sherlock Holmes.
   The GLSL and SPIR-V specifications, which each cover one side of the same coin, make building a compiler relatively smooth.
3. No docs at all on DXIR, or what you can expect from the provided passes to lower IR to DXIR. Eg, Can I make an llvm GlobalVariable and use GEP and load/store to access it, and will the passes remove those and generate DxilInst_CBufferLoad and the like? SPIR-V uses Variable declarations for all interface variable and marks pointer types with storage classes, then allows you to use normal IO instructions. That's a lot easier to lower for, because it's consistent over all address spaces/storage classes.

[llvm-beanz]

I totally understand this request. DXIL.rst is intended to be a spec for the format, but it completely misses the binary encoding for LLVM IR as bitcode (and bitcode is awful to work with since it is bit-packed and uses odd abbreviations).

Realistically I don't see us elaborating that documentation ever, but I do think there are two things we're doing that will help this:

1. Generating DXIL from modern LLVM will enable generating DXIL from something that isn't DXC's out-of-date fork.
2. We are working on a spec for HLSL over here, which is hopefully going to keep us more honest about a lot of things.

In the interest of being honest about our resources and team priorities I'm going to close this issue. Sorry that I don't have better news.