How do you download and run a model from Hugging Face in LM Studio?

Hugging Face is the largest public library of AI models: companies and researchers upload their models there, and anyone can download the open ones. LM Studio is a desktop app that runs those models on your own computer, with a chat window much like ChatGPT's. The two fit together: LM Studio has a built-in downloader that searches Hugging Face for you.

Running a model locally means your prompts and documents never leave the machine, and the model keeps working without an internet connection. The trade-off is that your computer does all the work, so the size of the model you can run depends on your memory.

The vocabulary you need

Parameters are the numbers a model learned during training. An "8B" model has about 8 billion of them. More parameters usually means a more capable model that needs more memory.

GGUF is a file format designed to load quickly and run efficiently on ordinary hardware. It was created by the developer of llama.cpp, the engine behind many local AI apps. A GGUF file holds the model itself plus a standard block of information about it, such as its name and how to format prompts.

Quantization stores each parameter with fewer bits, the way a photo can be saved at lower quality to make a smaller file. In file names it appears as codes such as Q8_0, Q4_K_M or Q2_K: the number after Q is roughly the bits per parameter. LM Studio's documentation describes these as copies of the same model at different levels of fidelity.

Worked example: Llama 3 8B Instruct

This walk-through uses Meta's Llama 3 8B Instruct, published in GGUF form by LM Studio's own community account. At 8 billion parameters, it is one of the smaller models that still handles everyday questions well, according to its model card.

1. Install LM Studio from lmstudio.ai and open it.
2. Open the Discover tab, or press Cmd+2 on a Mac or Ctrl+2 on Windows and Linux.
3. Search. Type lmstudio-community llama 3 8b instruct. You can also search for an exact user/model name, or paste a full Hugging Face address such as https://huggingface.co/lmstudio-community/Meta-Llama-3-8B-Instruct-GGUF.
4. Choose a file. The model page lists several download options. Pick a 4-bit file such as Q4_K_M for a first try; the table further down shows roughly how large each option is.
5. Download and wait for it to finish. Models are stored in a folder you can change from the My Models tab.
6. Load and chat. Load the downloaded model into memory from the chat view and send a message. The model card recommends LM Studio's "Llama 3" preset, which formats your messages the way the model expects.

Then test it with something that shows what a small local model can and cannot do:

You are a helpful assistant. In five bullet points, explain how a heat pump works to someone with no technical background. Then tell me one thing about heat pumps you are not sure of.

If the answer arrives at a comfortable reading speed, the model fits your machine. If it crawls, try a smaller quantization or a smaller model.

How do you pick the right file format and size?

Format first. LM Studio's downloader shows the files it can run, and for most computers these are GGUF files. On Hugging Face you can list every model that has GGUF files with the GGUF filter at hf.co/models?library=gguf, and the site has a viewer that shows each file's details before you download it. If a model you want has no GGUF version, look for one converted by its maker or by a well-known community account.

Then size. The file has to fit in your computer's memory with room to spare for the app, the conversation and everything else you have open. You can estimate the size of a quantized file from the parameter count and the bits per weight listed in Hugging Face's GGUF documentation. For an 8-billion-parameter model, the estimate in gigabytes is close to the bits-per-weight number:

These are estimates from simple arithmetic (parameters × bits ÷ 8); real files are a little larger or smaller because some parts of a model are stored at higher precision.

LM Studio recommends staying at 4 bits or above whenever your computer can handle it. A practical rule: start with Q4_K_M; move up to Q5 or Q6 if you have memory to spare; drop to Q3 only if Q4 will not load. When you are choosing between two models of different sizes, run the same few prompts through both rather than relying on the size alone: your own tasks are the best test.