Language and bert

Jacob Devlin’s guest lecture at Stanford about Bidirectional Encoder Representations from Transformers (BERT), which was the SOTA Language model and a major paper.

Implementation:
Huggingface is a great library and has a lot of pretrained Transformer models you can play around with or implement. Simple Transformers is a simpler library for implementing and finetuning your own NLP model. It also lets you pull models from Huggingface.

For the latest State of the Art (SOTA) language models you can check benchmarks on GLUE and SuperGLUE.