tinyGPT | Matt G

Try It

The trained model (int8 ONNX, ~8 MB) runs entirely in your browser— no server, no API call. It's a small, deliberately undertrained character model, so expect word-like, dream-logic text rather than coherent prose; that's the from-scratch transformer actually generating, one character at a time.

Prompt

Temperature: 0.80Length: 200

Output will appear here…

Loading model…

Why I Made It

I was about to get on a plane and wanted a small AI lab assistant I could actually use for a research project mid-flight — no Wi-Fi, no GPU, no cloud API to call. That single constraint shaped every design decision: the whole thing had to train and run on a plain CPU, offline, on whatever laptop I happened to have with me. So instead of wrapping an existing model, I built a language model from scratch that's tiny enough to fit those limits and learn from my own research notes.

It's deliberately minimal. The goal was never to compete with a frontier model — it was to have a genuinely portable, self-contained assistant I understand end to end, and to learn how a transformer actually works by writing every piece of it myself.

What It Does

tinyGPT is a character-level GPT. You point it at your own text and PDF files, it builds a tokenizer over those characters, trains on the material, and then generates text in the same style — a small, personal language model that never leaves your machine. Because it runs on CPU, you can train and sample from it anywhere, including 35,000 feet up with the Wi-Fi off.

How It's Built

Written from scratch in PyTorch with no nn.Transformer shortcut. The architecture is the classic decoder-only stack: a token embedding plus a learned position embedding, then a series of transformer blocks, each with multi-head causal self-attention and a feed-forward network wrapped in residual connections and layer norm, finished by a final layer norm and a language-model head. A custom character tokenizer maps the source text to and from integer IDs. Training is a straightforward AdamW loop with periodic validation-loss checks that save the best checkpoint. A small data-extraction layer pulls raw text out of PDFs and text files so the model can learn from real research material.

Architecture

  tinyGPT — decoder-only transformer (from scratch)
  ═══════════════════════════════════════════════════

  your notes / PDFs ──► char tokenizer ──► integer IDs
                                              │
                                              ▼
                          ┌──────────────────────────────────┐
                          │  token embedding + position embed │
                          └───────────────────┬───────────────┘
                                              │
                          ┌───────────────────▼───────────────┐
                          │   Transformer Block  × N           │
                          │                                    │
                          │   ┌──────────────────────────┐     │
                          │   │ multi-head causal         │     │
                          │   │ self-attention  (+ resid) │     │
                          │   └──────────────────────────┘     │
                          │   ┌──────────────────────────┐     │
                          │   │ feed-forward MLP (+ resid)│     │
                          │   └──────────────────────────┘     │
                          │        layer norm throughout       │
                          └───────────────────┬───────────────┘
                                              │
                                    final layer norm
                                              │
                                       LM head (logits)
                                              │
                                              ▼
                                  next-character probabilities
                                     (sample → generate)

  Runs on CPU. Trains offline. Learns from your own text.

Every layer is hand-written in PyTorch — the point was to understand and control the whole stack, and to keep it light enough to run with no network and no GPU.

Dev Notes

Problem Solved

Needed a usable AI assistant with zero connectivity and no GPU, on a plane, for a research project. Off-the-shelf models assume a server or an API; this one assumes nothing but a CPU.

New Tech Learned

Building a transformer by hand — attention math, causal masking, positional encoding, and a training/validation loop — instead of calling a library abstraction.

What's Next

Training it better: more and cleaner data, longer runs, and tuning the model size and context window to push output quality up while keeping it CPU-friendly.