Gaurav Tadkapally
projectseverything from scratchwriteups

large language models

gpt-2 + pre-trainingmulti-head attn · layer-norm
llama-2rope embeddings · rms-norm · silu
llama-3 and llama-3.1rope embeddings · group query attn
lstm

machine learning

linear regressionnumpy-only
logistic regressionnumpy-only
neural network w/ backpropnumpy-only
k-means