How do you turn a model checkpoint into a working app?

A checkpoint is a file of numbers that only makes sense with the exact code and settings that produced it. An app is something a person can use on a bad day without reading your notebook. The gap between the two is short for a small model, and much of it is ordinary software care.

This project takes the tiny model from project six and carries it across. The script 14_checkpoint_to_app.py does every step and then shuts its server down, using only Python's standard library, PyTorch and safetensors, on this computer's local address (127.0.0.1). It took 26 seconds on an Apple M5 Max.

Step 1: package it

A training checkpoint made with torch.save uses Python's pickle format, and loading a pickle can run arbitrary code. safetensors stores only the tensors. The script saves the weights there and writes a small description next to them.

import hashlib, json
from pathlib import Path
from safetensors.torch import load_file, save_file

def sha(path):
    return hashlib.sha256(Path(path).read_bytes()).hexdigest()

def package(state, cfg, chars, folder):
    Path(folder).mkdir(parents=True, exist_ok=True)
    state = {k: v.clone() for k, v in state.items() if k != "out.weight"}   # shared with tok.weight
    save_file(state, f"{folder}/model.safetensors")
    meta = {"name": "mini-gpt-anne", "version": "1.0.0", "cfg": cfg, "chars": chars,
            "limits": "Writes novel-like nonsense. Not for facts.",
            "sha256": sha(f"{folder}/model.safetensors")}
    Path(f"{folder}/model.json").write_text(json.dumps(meta, ensure_ascii=False))

The limits line matters. Whoever picks this up later should learn what it must not be used for from the file itself. If you plan to share it with others, the licence and model card have their own steps.

The size barely changed: 3.29 MB for the training checkpoint and 3.28 MB packaged. Larger checkpoints that also store optimizer state shrink more, since the app needs only the weights.

Step 2: prove it is the same model

Three tests, each short. Keep them with the app and run them every time something changes.

Step 3: wrap it in a small server

The server has three routes: / for a small web page, /health for a status check and /generate for the model. Most of its code does not concern the model at all. It concerns what a request may ask for.

MAX_CHARS, MAX_PROMPT = 400, 200

def do_POST(self):
    length = int(self.headers.get("Content-Length", 0))
    if self.path != "/generate":
        return self.send(404, {"error": "not found"})
    if length > 2000:
        return self.send(413, {"error": "request too large"})
    try:
        req = json.loads(self.rfile.read(length))
        prompt, n, seed = str(req.get("prompt", "")), int(req.get("chars", 100)), int(req.get("seed", 1))
    except (ValueError, TypeError):
        return self.send(400, {"error": "send JSON with prompt and chars"})
    if not prompt or len(prompt) > MAX_PROMPT or not 1 <= n <= MAX_CHARS:
        return self.send(422, {"error": f"prompt 1-{MAX_PROMPT} characters, chars 1-{MAX_CHARS}"})
    with LOCK:                               # one generation at a time; others wait
        text = generate(model, meta, prompt, n, seed)
    self.send(200, {"text": text, "version": meta["version"]})

Without the limits, one request for 99,999 characters would tie up the machine for everyone. Tested through the real server: a good request returned 200, an empty prompt 422, too many characters 422, an unknown path 404. The answer through the server matched the earlier saved text exactly, which also proves that the whole chain works.

Python's documentation warns that http.server is not recommended for production, since it implements only basic security checks. It is a good teaching tool, on your own machine. Do not expose it to the internet.

Step 4: measure it with several people at once

One user tells you little. Measured here with 100 generated characters per request on the CPU. p50 is the median wait, half of requests were faster. p95 is what the slowest one in twenty faced.

Throughput stays flat at about six per second and waiting grows in step with the number of users. That comes straight from the LOCK line: requests form a line, and nothing fails. It is the normal first shape of a served model. Your options are to batch several requests together, run several copies or generate fewer characters. The process peaked at 250 MB. For a real language model, a dedicated server such as vLLM or llama.cpp does batching and streaming for you, and these same four steps still apply.

What is still missing before real people use it?

This project stops at a local demonstration, and nothing below was tested. Before you let anyone outside use an app like this, you need:

The model is the smallest part of that table. That is the usual finding at the end of this kind of project: training is the visible step, and the app's trustworthiness rests on all the plain work around it. This is where the series ends, and where the larger roadmap can help you choose what to climb next.