Guide · portable · any OS

AI on a USB stick

Carry a small local AI on a USB drive or external SSD and run it on any computer. What is realistic, the options, exact steps, and how to leave no trace.

FreeWindows · macOS · Linux

Short version: a USB drive can carry a small AI model and the program to run it, so you can use it on a computer that has nothing installed. It works best with a fast external SSD, a small model, and a computer with at least 8 GB of RAM. Expect modest speed, not cloud quality.

Jump to: What is realistic · Formatting · Options · Windows · macOS · Linux · Size and time · Safety and cleanup

What is realistic

Small models only. Models of 1-4 billion parameters (about 1-3 GB) are practical on most hosts. 7-8B models (about 4-5 GB) need a host with 16 GB of RAM.
The computer you plug into does the work. The drive only stores things. Speed depends on the host's RAM, CPU and graphics, not on the drive, except at load time.
Drive speed matters when loading. A model is read from the drive into memory each time it starts. A cheap USB 2 stick can take minutes for a 4 GB model; a USB 3 SSD takes seconds to under a minute. After loading, drive speed barely matters.
No GPU assumption. A borrowed laptop may have only a CPU; expect a few words per second from a small model.
Different CPUs and systems. Windows, macOS and Linux programs are separate downloads. A drive can hold all three side by side, but a Mac with Apple Silicon and an Intel PC need different builds (verify for the tool you choose).

Formatting the drive

exFAT is the usual choice for use across Windows, macOS and Linux: it reads and writes on all three and has no 4 GB file limit. (Linux supports exFAT in recent kernels; older systems may need an extra package, verify.)
Avoid FAT32 for models: it cannot hold a single file larger than 4 GB.
NTFS works on Windows and reads on macOS but writing on macOS needs extra software; ext4 is Linux-only. Use them only if you stay on one system.
Formatting erases everything on the drive. Copy anything you need first. In Windows: right-click the drive in File Explorer, Format, choose exFAT. In macOS: Disk Utility, Erase, ExFAT. In Linux: a disk tool such as GNOME Disks, or mkfs.exfat after carefully checking the device name.
Linux/macOS programs copied to exFAT lose the executable permission flag. You may need to run them with an explicit interpreter or copy them to the computer's own disk first (see steps below).

Your options

Option	How portable	Notes
llamafile (github.com/Mozilla-Ocho/llamafile)	Best. One file contains the program and the model; the same file runs on several systems.	Download a single `.llamafile`; it starts a local chat page in your browser. Windows needs the file to end in `.exe` and has a 4 GB limit on executable files (larger models are loaded as separate weights, verify the current instructions).
Ollama with `OLLAMA_MODELS` on the drive	Good for models; the program still has to be installed on each host.	Models live on the drive, the program on the computer. Good when you can install software.
LM Studio	Partly. Installing it on each host is normal; moving its models folder to the drive is possible in its settings.	Whether a fully portable install works depends on the version. Verify in its documentation.
llama.cpp binaries (github.com/ggml-org/llama.cpp)	Good. Release downloads are zip archives you can unpack onto the drive with a GGUF model.	No install needed. Pick the build matching the OS and CPU (or GPU) of the host.

Steps per OS

Windows host

llamafile:

Download a small .llamafile (for example a 1-4B model) from the llamafile project page onto the drive.
Rename it so it ends with .exe if it does not already.
Open a terminal in the drive's folder and run it (for example E:\ai\model.llamafile.exe). A browser page opens at a local address.

Ollama with models on the drive (Ollama installed on the host):

setx OLLAMA_MODELS "E:\ai\ollama-models"

Quit Ollama from the tray, start it again, then ollama pull llama3.2. Change the drive letter to match. Note that setx stores the setting on that computer permanently; on a borrowed machine, set it for one terminal session only with set OLLAMA_MODELS=E:\ai\ollama-models and start the server from that window with ollama serve (verify).

llama.cpp: unzip the Windows release into E:\ai\llama.cpp, put a .gguf file next to it, and run:

llama-cli.exe -m model.gguf -c 4096

macOS host

llamafile: in Terminal, make it executable and run it. If exFAT blocks the permission change, copy it to the Mac's Downloads first.

cp /Volumes/MYDRIVE/ai/model.llamafile ~/Downloads/
chmod +x ~/Downloads/model.llamafile
~/Downloads/model.llamafile

macOS may block a downloaded program the first time; the allow option is in System Settings, Privacy & Security. Delete the copy afterwards.

Ollama with models on the drive:

OLLAMA_MODELS=/Volumes/MYDRIVE/ai/ollama-models ollama serve

In a second terminal window: ollama run llama3.2. If the Ollama app is already running, quit it first so the new setting is used.

llama.cpp: unpack the macOS release to the drive, then ./llama-cli -m model.gguf -c 4096. If the file will not run from exFAT, copy the folder to the Mac first.

Linux host

llamafile:

cp /media/$USER/MYDRIVE/ai/model.llamafile /tmp/
chmod +x /tmp/model.llamafile
/tmp/model.llamafile

(Some systems mount drives without permission to run programs, which is why we copy it first. Remove the copy afterwards.)

Ollama with models on the drive: run a private server instance instead of the system service, so nothing changes on the host:

OLLAMA_MODELS=/media/$USER/MYDRIVE/ai/ollama-models ollama serve

If the system service already uses port 11434, stop it temporarily or use OLLAMA_HOST=127.0.0.1:11500 for your instance and the same variable in the client terminal. Ollama must be installed on the host for this to work.

llama.cpp: unpack the Linux release from the project to the drive (or copy to the host's disk), then ./llama-cli -m model.gguf -c 4096.

Size and time table

Approximate. Download sizes are for 4-bit versions; load times are for reading the file from the drive into memory and depend on the exact drive and host.

Model size	File size	Host RAM wanted	Load from USB 2 stick	Load from USB 3 SSD
1B	about 1 GB	4 GB	about 1 minute	a few seconds
3-4B	about 2-3 GB	8 GB	2-3 minutes	5-15 seconds
7-8B	about 4-5 GB	16 GB	4-6 minutes	10-30 seconds
14B	about 9 GB	24-32 GB	not practical	20-60 seconds

Verify: these load times are estimates from typical drive speeds (USB 2 is about 30 MB/s in practice, a USB 3 SSD often several hundred MB/s). Measure your own drive.

Not sure what a host can run? Use the Can my PC run this AI model? calculator.

Safety and cleanup on a borrowed machine

Your chats are not on the drive only. Most programs keep chat history in the host user's profile folder, and the browser page used by llamafile keeps its own data. Prefer a command-line session that you close, and do not type private or confidential information on a computer you do not control.
Ask first. Use someone else's computer only with permission, and do not run programs on a work or school machine where that is against policy.
Close the server when you finish (Ctrl+C in its terminal, or quit Ollama), so nothing keeps listening.
Delete copies you made on the host (the llamafile copy in Downloads or /tmp), and empty the trash or recycle bin.
Check for leftovers: Ollama may leave logs and a .ollama folder in the user profile (history and keys, verify), and a browser keeps history for local pages. Clear them if you are able to.
Eject safely before pulling the drive, especially after downloading, to avoid damaged model files.
Download models only from sources you trust (the project's own pages, ollama.com, Hugging Face) and keep a copy of the file name and source for later.
Consider encrypting a drive that holds your own documents. Models themselves are public files and need no protection.

Check the project pages for current instructions. Portable options change quickly. Details marked "verify" were not confirmed on every system. Last checked: 2026-11-07.