46
1
10Gb Ethernet: what I had to (re)learn (gilesthomas.com)
3
LLM from scratch, part 33 – what I learned from the appendices (gilesthomas.com)
1
LLM from scratch (32l) – Interventions: updated instruction fine-tuning results (gilesthomas.com)
1
How an LLM becomes more coherent as we train it (gilesthomas.com)
2
LLM from scratch, part 32k – Interventions: gradient accumulation (gilesthomas.com)
2
Provision: LLM-powered server setup from Markdown (provision.sh)
2
LLM from scratch, part 32j – trying to train a better model in the cloud (gilesthomas.com)
1
Writing an LLM from scratch, part 32i – Interventions: what is in the noise? (gilesthomas.com)
2
Writing an LLM from scratch, part 32h – Interventions: full fat float32 (gilesthomas.com)
1
Writing an LLM from scratch, part 32g – Interventions: weight tying (gilesthomas.com)
5
Writing an LLM from scratch, part 32f – Interventions: weight decay (gilesthomas.com)
3
Writing an LLM from scratch, part 32e – Interventions: the learning rate (gilesthomas.com)
1
Writing an LLM from scratch, part 32d – Interventions: adding attention bias (gilesthomas.com)
1
Writing an LLM from scratch, part 32c – Interventions: removing dropout (gilesthomas.com)
2
Writing an LLM from scratch, part 32B – Interventions: gradient clipping (gilesthomas.com)
1
Writing an LLM from scratch, part 32a – Interventions: training a baseline model (gilesthomas.com)
1
Getting a Custom PyTorch LLM onto the Hugging Face Hub (gilesthomas.com)
1
Writing an LLM from scratch, part 31 – the models are now on Hugging Face (gilesthomas.com)
1
Writing an LLM from scratch, part 30 – digging into the LLM-as-a-judge results (gilesthomas.com)
2
LLM from scratch, part 29 – using DDP to train a base model in the cloud (gilesthomas.com)
46
LLM from scratch, part 28 – training a base model from scratch on an RTX 3090 (gilesthomas.com)
1
Writing an LLM from scratch, part 27 – what's left, and what's next? (gilesthomas.com)
3
Writing an LLM from scratch, part 26 – evaluating the fine-tuned model (gilesthomas.com)
1
Writing an LLM from scratch, part 25 – instruction fine-tuning (gilesthomas.com)
1
Writing an LLM from scratch, part 24 – the transcript hack (gilesthomas.com)
2
Retro Language Models: Rebuilding Karpathy's RNN in PyTorch (gilesthomas.com)
1
Writing an LLM from scratch, part 23 – fine-tuning for classification (gilesthomas.com)
60
Writing an LLM from scratch, part 22 – training our LLM (gilesthomas.com)
2
Revisiting Karpathy's 'Unreasonable Effectiveness of Recurrent Neural Networks' (gilesthomas.com)
1
Writing an LLM from scratch, part 21 – perplexed by perplexity (gilesthomas.com)
7
Writing an LLM from scratch, part 20 – starting training, and cross entropy loss (gilesthomas.com)
2
How Do LLMs Work? (gilesthomas.com)
63
The maths you need to start understanding LLMs (gilesthomas.com)
2
What AI chatbots are doing under the hood (gilesthomas.com)
1
LLM from scratch, part 18 – residuals, shortcut connections, and the Talmud (gilesthomas.com)
1
The fixed length bottleneck and the feed forward network (gilesthomas.com)
3
Writing an LLM from scratch, part 17 – the feed-forward network (gilesthomas.com)
1
Writing an LLM from scratch, part 16 – layer normalisation (gilesthomas.com)
1
Leaving PythonAnywhere (gilesthomas.com)
2
Writing an LLM from scratch, part 15 – from context vectors to logits (gilesthomas.com)
1
Writing an LLM from scratch, part 14 – the complexity of self-attention at scale (gilesthomas.com)
41
Writing an LLM from scratch, part 13 – attention heads are dumb (gilesthomas.com)
1
Writing an LLM from scratch, part 12 – multi-head attention (gilesthomas.com)
1
Writing an LLM from scratch, part 11 – batches (gilesthomas.com)
2
The Business of the AI Labs (omega-prime.co.uk)
48
Writing an LLM from scratch, part 10 – dropout (gilesthomas.com)
1
Adding /Llms.txt (gilesthomas.com)
3
Writing an LLM from scratch, part 9 – causal attention (gilesthomas.com)
95
Writing an LLM from scratch, part 8 – trainable self-attention (gilesthomas.com)
114
It’s still worth blogging in the age of AI (gilesthomas.com)
97
The benefits of learning in public (gilesthomas.com)
12
Ask HN: If you have a blog, do you have an RSS feed?
2
Do reasoning LLMs need their own Philosophical Language? (gilesthomas.com)
1
LLM Quantisation Weirdness (gilesthomas.com)
2