![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/10\/gpt-1.png?resize=150%2C150&ssl=1\")
<\/a><\/div>GPT From Scratch #1: Intro<\/a>You probably use AI, but do you understand it? Get ready to dive into the internals of what started the (gen) AI revolution: GPT. <\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/11\/gpt-2.png?resize=150%2C150&ssl=1\")
<\/a><\/div>GPT From Scratch #2: The Training Set<\/a>Don’t understate the importance of building a proper training set. It is a critical part of the process, and in GPT’s case, a beautifully cleaver one as well. <\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/11\/gpt-3.png?resize=150%2C150&ssl=1\")
<\/a><\/div>GPT From Scratch #3: The Bigram Model<\/a>The simplest model we can use to predict the next character is a Bigram Model. But if implemented as a neural net, the building blocks will stay the same up to GPT. <\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/11\/gpt-4.png?resize=150%2C150&ssl=1\")
<\/a><\/div>GPT From Scratch #4: The Mathematical Trick Behind Self Attention<\/a>One simple mathematical trick. The most cleaver matrix multiplication of the gen AI revolution. What enabled ultra fast self attention. <\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/11\/gpt-5.png?resize=150%2C150&ssl=1\")
<\/a><\/div>GPT From Scratch #5: Positional Encodings<\/a>In this post, we’ll show how to add to the neural net the notion of position of the tokens. Simple but powerful. <\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/11\/gpt-6.png?resize=150%2C150&ssl=1\")
<\/a><\/div>GPT From Scratch #6: Coding Self Attention<\/a>This is where we get to understand the ~20 most important and impactful lines of code which started the gen AI revolution. <\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/11\/gpt-7.png?resize=150%2C150&ssl=1\")
<\/a><\/div>GPT From Scratch #7: Building a GPT<\/a>Self Attention is the heart of Transformers, the T of GPT. But there are few additional critical parts to the transformer architecture that actually made it shine.<\/div><\/li>\n<\/ul>\n\n\nDeep Learning Gymnastic Series<\/h2>\n\n\n
![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2023\/07\/image-31-1.png?resize=150%2C150&ssl=1\")
<\/a><\/div>Deep Learning Gymnastics #1: Tensor Broadcasting<\/a>Master broadcasting like a pro and learn how a single trick can make your deep learning code faster, cleaner, and more elegant.<\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2023\/12\/image-7.png?resize=150%2C150&ssl=1\")
<\/a><\/div>Deep Learning Gymnastics #2: Tensor Indexing<\/a>Learn how smart indexing lets you build batches, embeddings, and masked ops efficiently in modern DL frameworks.<\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2024\/02\/image-1-1536x1149-1.png?resize=150%2C150&ssl=1\")
<\/a><\/div>Deep Learning Gymnastics #3: Tensor (re)Shaping<\/a>Your tensors aren\u2019t the right shape? Learn how to reshape, squeeze, and stack them like a deep learning gymnast.<\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2024\/03\/image-4-1024x784-1.png?resize=150%2C150&ssl=1\")
<\/a><\/div>Deep Learning Gymnastics #4: Master Your (LLM) Cross Entropy<\/a>Use all the gymnastics tricks we’ve learned in order to master (LLM) cross-entropy in PyTorch and TensorFlow.<\/div><\/li>\n<\/ul>\n\n\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2025\/11\/image-64.png?resize=312%2C110&ssl=1\")
<\/figure>\n\n\n\nDeep Dive Into Logistic Regression Series<\/h2>\n\n\n\n
<\/p>\n\n\n
![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2017\/12\/Screenshot-2025-07-11-at-15.28.16.png?fit=300%2C92&ssl=1\")
<\/a><\/div>Deep Dive Into Logistic Regression: Part 1<\/a>Learn the fundamental theory behind logistic regression.<\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2018\/02\/Deep-Dive-Into-Logistic-Regression-Part-2-Philippe-Adjimans-blog-2018-03-03-21-35-24.png?fit=300%2C29&ssl=1\")
<\/a><\/div>Deep Dive Into Logistic Regression: Part 2<\/a>Want to know how to implement Stochastic Gradient Descent for Logistic regression able to learn millions of parameters\u00a0using the hashing trick and per-coordinate adaptive learning rate with a tiny memory footprint? This post is for you.<\/div><\/li>\n![\"\"](\"https:\/\/i0.wp.com\/philippeadjiman.com\/blog\/wp-content\/uploads\/2018\/04\/Vowpal-Wabbit-1.png?fit=300%2C150&ssl=1\")
<\/a><\/div>Deep Dive Into Logistic Regression: Part 3<\/a>In this third and last post of this series, we present the use of a very effective and powerful library to build logistic regression models (among others) in practice: Vowpal Wabbit.<\/div><\/li>\n<\/ul>","protected":false},"excerpt":{"rendered":"GPT From Scratch Deep Learning Gymnastic Series Deep Dive Into Logistic Regression Series<\/p>\n","protected":false},"author":1,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"nf_dc_page":"","footnotes":""},"class_list":["post-1856","page","type-page","status-publish","hentry"],"jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/pages\/1856","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/comments?post=1856"}],"version-history":[{"count":20,"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/pages\/1856\/revisions"}],"predecessor-version":[{"id":2247,"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/pages\/1856\/revisions\/2247"}],"wp:attachment":[{"href":"https:\/\/philippeadjiman.com\/blog\/wp-json\/wp\/v2\/media?parent=1856"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}