RASP and TRACR

RASP (Restricted Access Sequence Processing Language) is introduced in Thinking Like Transformers arXiv:2106.06981 cs.LG by Gail WeissYoav GoldbergEran Yahav to set out a computational model for transformer-encoders. This is to support describing and reasoning about capabilities identified in transformer LLMs.

TRACR (TRAnsformer Compiler for RASP) is a compiler for generating transformer weights from RASP programs. Developed in [[DeepMind]] by David LindnerJános KramárMatthew RahtzThomas McGrathVladimir Mikulik

https://github.com/deepmind/tracr https://www.alignmentforum.org/posts/2J6fFHQZkWxFcjL6c/tracr-compiled-transformers-as-a-laboratory-for-1 https://arxiv.org/abs/2301.05062