Here’s a little project which runs GPT2 inference with every operation rolled out by hand to be inspectable and debuggable, in C++, with no dependences. Ideal for learning about the Transformer mechanism.

https://github.com/binaryfoundry/picoGPT-cpp