How do you train a small language model on a rented GPU, step by step?

Our tiny model guide stays on your laptop. This one moves to real hardware. You will rent a GPU server, train a model whose only ancestor is random numbers, and chat with it. It assumes you can open a terminal and copy commands, and nothing more.

The steps use nanochat, Andrej Karpathy's small training project, and Lambda's on-demand cloud as the example, since nanochat's author uses it. Prices were read on November 2, 2026; any provider with Ubuntu machines and SSH works in much the same way.

Pick your run

Always do the test first. The 40 minutes is an estimate from the project's note that a 12-layer run takes about 5 minutes on eight GPUs and that one GPU is about eight times slower. Your first run shows how long setup takes you, and every minute counts.

Step 1: rent the machine

Create an account with the provider and add a payment method and an SSH key, which is the file that lets your computer log in without a password. In Lambda's console, launch an instance with the GPU count you chose. Billing begins once it passes its health checks. Copy the IP address it shows, then connect:

ssh ubuntu@YOUR_IP
nvidia-smi

nvidia-smi lists the GPUs and their memory. Check that you got what you are paying for. nanochat's defaults assume 80 GB per GPU.

Step 2: install

git clone https://github.com/karpathy/nanochat.git
cd nanochat
curl -LsSf https://astral.sh/uv/install.sh | sh
source $HOME/.local/bin/env
uv sync --extra gpu
source .venv/bin/activate

uv is the tool that installs the Python packages the project needs, including PyTorch, the library doing the training. Before you run the curl line, you can read the script at that address if you like; it comes from the official uv project.

Step 3: the test run

Start a screen session first. It keeps the job running on the server if your laptop sleeps or your connection drops.

screen -S test
python -m nanochat.dataset -n 32
python -m scripts.tok_train
python -m scripts.base_train --depth=12 --model-tag=d12 --save-every=1000

The first line downloads 32 data files of about 100 MB each. The second trains the tokenizer. The third trains a 12-layer model and saves a checkpoint, a snapshot of the model, every 1,000 steps. Press Ctrl-a then d to detach; screen -r test brings it back.

Watch two things in the log. The validation bits per byte should keep falling, and it is explained in how to evaluate your model. The text samples, printed every 2,000 steps, should get less silly. If you run out of graphics memory, cut --device-batch-size in half, from 32 to 16 and so on, until it fits.

Step 4: the full run

On an eight-GPU machine, the whole pipeline is one line:

screen -L -Logfile runs/speedrun.log -S speedrun bash runs/speedrun.sh

The script downloads the data, trains the tokenizer, pretrains a 24-layer model, evaluates it, fine-tunes it for conversation and evaluates it again. Inside, it uses torchrun, PyTorch's launcher, which starts one training process per GPU so that the eight share the work. To follow along from another login, run tail -f runs/speedrun.log.

When it finishes:

source .venv/bin/activate
python -m scripts.chat_cli

Try a story, a poem, and a question about you. Expect fluent nonsense on the last, since this model knows very little. Its author compares it to talking with a kindergartener.

Step 5: save it, then stop paying

From your own computer, copy the results, then terminate:

scp -r ubuntu@YOUR_IP:~/.cache/nanochat/chatsft_checkpoints ./nanochat-model/
scp -r ubuntu@YOUR_IP:~/.cache/nanochat/tokenizer ./nanochat-model/

Pretraining checkpoints are in base_checkpoints next to those if you want them too. Open a checkpoint folder to confirm that the files arrived, and then terminate the instance in the console. Lambda charges nothing for data going out, though other providers may. Note that nanochat's default dataset is licensed for non-commercial use only, which matters if you plan to publish the model.

What goes wrong most often, and how do you avoid paying for it?

The machine keeps billing after training ends. Lambda bills by the minute from launch until termination, used or not, and other providers work similarly. Note your finish time, and set a phone alarm for it. A forgotten eight-GPU node costs about $766 a day. Check the provider's own rule about what "stop" means, because it may differ from "terminate".

Storage outlives the machine. A network filesystem keeps billing until you delete it, even with no instance attached. On Lambda you must attach it when you create the instance, or do without it and copy your results off before terminating.

A crash loses hours of work. By default nanochat saves only at the end of pretraining (--save-every is -1). For anything long, set a value, and use --resume-from-step to restart from the saved step. The full speedrun script does not set this, so add it to the script's training line if you want the insurance.

A dropped connection kills the job. Without screen or a similar tool, closing the terminal can end the run. Start the session before the job, every time.

Out-of-memory errors. Your GPU has less memory than the defaults expect. Lower --device-batch-size. nanochat compensates automatically so that training still behaves the same, only more slowly.

Spot instances vanish. They are the route to the low figure of about $15 that the project mentions, but providers can take them back. Use them only with regular checkpoints that you copy elsewhere, and treat the saving as uncertain.

Mixed precision errors. The speedrun's --fp8 setting needs an H100 or newer GPU. On older cards, remove the flag and the run uses bf16, a higher-precision format.