In this post, we’ll show how to add to the neural net the notion of position of the tokens. Simple but powerful.

One simple mathematical trick. The most cleaver matrix multiplication of the gen AI revolution. What enabled ultra fast self attention.