Bertology #

The term Bertology is based on the BERT large language model. We still don’t understand very well how this and other large language models can do what they do, Bertology became used as a term for this type of research. Huggingface provides ways access to internal model parameters to support this research. Useful results of this research includes Interpreting GPT - the Logit Lens by Nostalgebraist and Ecco a python library developed by Jay Alammar.