Mirror

Backups

Real, verified pull commands for every local AI model AI Cockpit supports — see the full guide for the searchable/organized version. Key tool installers below are still mirrored here directly in case the original source is unreachable.

Local AI Models Guide — real, working pull commands for every use case

A curated list of small local AI models covering every common AI use case — chat,

code, reasoning, images, video, voice, translation, and more — that run on modest

consumer hardware. Every command below is real and verified: Ollama tags checked

against Ollama's live registry, LocalAI ids checked against LocalAI's live model

gallery (checked 2026-08-02). No files are hosted here — these commands download

directly from Ollama's registry or Hugging Face.

How to read this: organized by System RAM needed (8/12/16/24 GB), then by

VRAM needed (1/2/3/4 GB) within each. If your machine has more RAM/VRAM than a

given section, everything in the smaller sections below it will run on your machine

too — not just the section matching your exact specs.

8 GB RAM

1 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Qwen 2.5 0.5B

Good for: Basic automation, blazing fast edge processing
Watch out for: Extremely limited knowledge base
Ollama: ollama pull qwen2.5:0.5b (~379 MB)
LocalAI: local-ai run qwen3-0.6b (~462 MB) — Qwen3 0.6B replaces Qwen2.5 0.5B — newer generation, same tiny-chat role, already verified in gallery
https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Qwen 3 0.6B

Good for: Basic reasoning, ultra-fast routing
Watch out for: Low factual retrieval capability
Ollama: ollama pull qwen3:0.6b (you already have this)
LocalAI: local-ai run qwen3-0.6b (~462 MB)
https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

SmolLM2 360M

Good for: Prototyping, basic text routing
Watch out for: Essentially no world knowledge
Ollama: ollama pull smollm2:360m (you already have this)
LocalAI: local-ai run qwen3-0.6b (~462 MB) — no SmolLM2-360M in gallery; nearest ultra-tiny chat model
https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Qwen2 0.5B

Good for: Basic logic, blazingly fast generation, simple tasks
Watch out for: Very small knowledge base, limited reasoning
Ollama: ollama pull qwen2:0.5b (~336 MB)
LocalAI: local-ai run qwen3-0.6b (~462 MB) — Qwen2 (old gen) not in gallery; Qwen3 0.6B is the current tiny generalist
https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Gemma 3 1B

Good for: Strict instruction following
Watch out for: Often refuses benign prompts
Ollama: ollama pull gemma3:1b (you already have this)
LocalAI: local-ai run gemma-3-1b-it (~769 MB)
https://huggingface.co/ggml-org/gemma-3-1b-it-GGUF/resolve/main/gemma-3-1b-it-Q4_K_M.gguf

Llama 3.2 1B

Good for: Summarization, basic tool calling (can call external tools/functions mid-conversation)
Watch out for: Hallucinates (makes up wrong-but-confident-sounding facts) on complex reasoning
Ollama: ollama pull llama3.2:1b (you already have this)
LocalAI: local-ai run llama-3.2-1b-instruct:q4_k_m (~770 MB)
https://huggingface.co/hugging-quants/Llama-3.2-1B-Instruct-Q4_K_M-GGUF/resolve/main/llama-3.2-1b-instruct-q4_k_m.gguf

TinyLlama 1.1B

Good for: Fast drafting, straightforward NLP
Watch out for: Low factual accuracy, older architecture
Ollama: ollama pull tinyllama (you already have this)
LocalAI: local-ai run qwen3-0.6b (~462 MB) — TinyLlama not in gallery; Qwen3 0.6B is the modern equivalent-tier chat model
https://huggingface.co/MaziyarPanahi/Qwen3-0.6B-GGUF/resolve/main/Qwen3-0.6B.Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

Qwen 2.5 Coder 0.5B

Good for: Inline code completions, fast syntax checking
Watch out for: Struggles with large logic blocks
Ollama: ollama pull qwen2.5-coder:0.5b (~379 MB)
LocalAI: local-ai run opencoder-1.5b-base (~1.3 GB) — no 0.5B coder in gallery; OpenCoder 1.5B is the smallest real code model
https://huggingface.co/QuantFactory/OpenCoder-1.5B-Base-GGUF/resolve/main/OpenCoder-1.5B-Base.Q4_K_M.gguf

Audio — Text to Speech

Turns written text into spoken audio — a synthetic voice reading your text out loud.

Piper TTS

Good for: Blazing fast text-to-speech, low footprint
Watch out for: Less emotive than Bark
LocalAI: local-ai run piper-en_US-lessac-medium-crispasr (~30 MB)
https://huggingface.co/LocalAI-Community/piper-voices-GGUF/resolve/main/piper-en_US-lessac-medium-f16.gguf

Embedding / RAG

Doesn't chat at all — turns text into a mathematical "fingerprint" used for search. This is the engine behind "search my own documents" features (RAG).

All-MiniLM-L6-v2

Good for: Extremely lightweight local document indexing
Watch out for: Lower semantic accuracy than Nomic
Ollama: ollama pull all-minilm (~44 MB)
LocalAI: local-ai run all-MiniLM-L6-v2 (~86 MB)

BGE-m3

Good for: Multilingual (handles multiple languages) vector search (Hebrew/Spanish support)
Watch out for: Does not generate text
Ollama: ollama pull bge-m3 (~1.1 GB)
LocalAI: local-ai run bge-m3-colbert (~2.1 GB)

Nomic-Embed-Text

Good for: Creating vector embeddings for local document search
Watch out for: Does not generate text
Ollama: ollama pull nomic-embed-text (you already have this)
LocalAI: local-ai run nomic-embed-text-v1.5 (~261 MB)
https://huggingface.co/mradermacher/nomic-embed-text-v1.5-GGUF/resolve/main/nomic-embed-text-v1.5.f16.gguf

2 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

BitNet 2B-4T

Good for: Extremely fast CPU inference
Watch out for: Highly experimental
Ollama: not available — 'bitnet:2b-4t' doesn't exist in Ollama's registry, no substitute found
LocalAI: not available — no BitNet or 1-bit-architecture model in the gallery at all — genuinely no substitute for this specific architecture

Qwen 2.5 1.5B

Good for: Strong multilingual (handles multiple languages), balanced generalist
Watch out for: Sometimes loses context
Ollama: ollama pull qwen2.5:1.5b (you already have this)
LocalAI: local-ai run qwen3-1.7b (~1.2 GB) — Qwen3 1.7B replaces Qwen2.5 1.5B, same tier, already verified
https://huggingface.co/MaziyarPanahi/Qwen3-1.7B-GGUF/resolve/main/Qwen3-1.7B.Q4_K_M.gguf

Qwen 3 1.7B

Good for: Agentic (able to plan and use tools on its own, not just chat) workflows, basic classification
Watch out for: Factual accuracy on obscure topics
Ollama: ollama pull qwen3:1.7b (you already have this)
LocalAI: local-ai run qwen3-1.7b (~1.2 GB)
https://huggingface.co/MaziyarPanahi/Qwen3-1.7B-GGUF/resolve/main/Qwen3-1.7B.Q4_K_M.gguf

Qwen2 1.5B

Good for: Multilingual (handles multiple languages) translation, general chat
Watch out for: Advanced coding or complex logic
Ollama: ollama pull qwen2:1.5b (~892 MB)
LocalAI: local-ai run qwen3-1.7b (~1.2 GB) — old-gen Qwen2 not in gallery; Qwen3 1.7B is the current equivalent
https://huggingface.co/MaziyarPanahi/Qwen3-1.7B-GGUF/resolve/main/Qwen3-1.7B.Q4_K_M.gguf

SmolLM2 1.7B

Good for: Local text manipulation, rewriting
Watch out for: Lacks nuanced conversational abilities
Ollama: ollama pull smollm2:1.7b (you already have this)
LocalAI: local-ai run smollm2-1.7b-instruct (~1007 MB)
https://huggingface.co/HuggingFaceTB/SmolLM2-1.7B-Instruct-GGUF/resolve/main/smollm2-1.7b-instruct-q4_k_m.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

DeepSeek Coder 1.3B

Good for: Code completion, debugging snippets
Watch out for: Not suited for general chat
Ollama: ollama pull deepseek-coder:1.3b (~740 MB)

Qwen 2.5 Coder 1.5B

Good for: FPC, PHP, bash snippets, local code assistance
Watch out for: Requires clear context
Ollama: ollama pull qwen2.5-coder:1.5b (you already have this)
LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — real, current small code-instruct model
https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Reasoning / Logic

Built to "think through" harder problems step by step — math, logic puzzles, multi-step decisions — instead of just pattern-matching an answer. Often slower because it works through the problem first.

DeepSeek-R1 1.5B

Good for: Deep reasoning for its size, math
Watch out for: Can over-think simple prompts
Ollama: ollama pull deepseek-r1:1.5b (you already have this)
LocalAI: local-ai run deepseek-r1-distill-qwen-1.5b (~1.0 GB)
https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-1.5B-GGUF/resolve/main/DeepSeek-R1-Distill-Qwen-1.5B-Q4_K_M.gguf

Vision Analysis

Can look at an image you give it and describe, answer questions about, or read text from it. Doesn't generate new images — just understands existing ones.

Moondream2 1.8B

Good for: Fast local Img querying
Watch out for: Text generation is basic
Ollama: ollama pull moondream (~1.6 GB)
LocalAI: local-ai run moondream2 (~3.5 GB)
https://huggingface.co/moondream/moondream2-gguf/resolve/main/moondream2-text-model-f16.gguf
https://huggingface.co/moondream/moondream2-gguf/resolve/main/moondream2-mmproj-f16.gguf

Audio — Speech to Text

Turns spoken audio (a recording, podcast, video) into written text — transcription.

Whisper (Small)

Good for: Transcribing podcasts/videos locally
Watch out for: Not a generation model
LocalAI: local-ai run whisper-small (~465 MB)
https://huggingface.co/ggerganov/whisper.cpp/resolve/main/ggml-small.bin

Translation

Translates text (or sometimes speech) from one language to another. Some are conversational generalists that translate okay; these are dedicated specialists.

Madlad400 3B

Good for: Machine translation (400+ languages)
Watch out for: Not suitable for conversational chat
Ollama: ollama pull translategemma (~3.1 GB) — substituted for 'madlad400:3b' (doesn't exist in Ollama's registry)

Summarization

Condenses a long document into a short summary. Dedicated summarizers do this more reliably than asking a general chat model to "summarize this."

BART-Large-CNN

Good for: Dedicated summarization model (non-conversational)
Watch out for: Strictly for summarization
LocalAI: local-ai run llama-chat-summary-3.2-3b (~1.9 GB) — BART-Large-CNN not in gallery; Llama-Chat-Summary 3.2 3B is a real, current dedicated summarization finetune
https://huggingface.co/bartowski/Llama-Chat-Summary-3.2-3B-GGUF/resolve/main/Llama-Chat-Summary-3.2-3B-Q4_K_M.gguf

Pegasus-X-Base

Good for: Long-document abstractive summarization
Watch out for: Hard to configure properly
LocalAI: local-ai run llama-chat-summary-3.2-3b (~1.9 GB) — Pegasus-X not in gallery; same real summarization substitute as BART
https://huggingface.co/bartowski/Llama-Chat-Summary-3.2-3B-GGUF/resolve/main/Llama-Chat-Summary-3.2-3B-Q4_K_M.gguf

3 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Gemma 2 2B

Good for: Step-by-step reasoning
Watch out for: Strict alignment (built-in safety behavior)
Ollama: ollama pull gemma2:2b (you already have this)

Ministral-3 3B

Good for: Solid generalist
Watch out for: Overshadowed by Llama 3.2
Ollama: ollama pull ministral-3 (~5.6 GB)
LocalAI: local-ai run mistralai_ministral-3-3b-instruct-2512-multimodal (~3.6 GB) — real 2512 release, already applied
https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-GGUF/resolve/main/Ministral-3-3B-Instruct-2512-Q4_K_M.gguf
https://huggingface.co/unsloth/Ministral-3-3B-Instruct-2512-GGUF/resolve/main/mmproj-F32.gguf

Llama 3.2 3B

Good for: Nuanced chat, tool calling (can call external tools/functions mid-conversation)
Watch out for: Can loop tools
Ollama: ollama pull llama3.2 (you already have this)
LocalAI: local-ai run llama-3.2-3b-instruct-uncensored (~2.1 GB)
https://huggingface.co/bartowski/Llama-3.2-3B-Instruct-uncensored-GGUF/resolve/main/Llama-3.2-3B-Instruct-uncensored-Q4_K_M.gguf

Qwen 2.5 3B

Good for: Highly versatile generalist
Watch out for: Slightly less capable at complex math
Ollama: ollama pull qwen2.5:3b (you already have this)
LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — plain Qwen2.5 3B not in gallery; Falcon3 3B is a real, current 3B-class generalist
https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

Stable Code 3B

Good for: Fast completion for HTML, CSS, JS
Watch out for: Lacks complex logical reasoning
Ollama: ollama pull stable-code:3b (~1.5 GB)
LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — Stable Code 3B not in gallery; OpenCoder covers the same lightweight-completion role
https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Qwen 2.5 Coder 3B

Good for: High-quality scripting, FPC/Lazarus syntax fixes
Watch out for: Less conversational depth
Ollama: ollama pull qwen2.5-coder:3b (you already have this)
LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — closest real small code model in the current gallery
https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Starcoder2 3B

Good for: Offline IDE autocomplete integration
Watch out for: General knowledge is poor
Ollama: ollama pull starcoder2:3b (you already have this)
LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — StarCoder2 not in gallery at any size; OpenCoder is the real small-code substitute
https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Image Generation

Creates brand-new images from a text description ("a cat wearing a top hat"). The opposite of Vision Analysis.

LCM (Latent Consistency)

Good for: Extremely fast generation (2-4 steps)
Watch out for: Lower fidelity than base SD
LocalAI: local-ai run sd-1.5-ggml (~1.5 GB) — LCM fast-inference variant not in gallery; sd-1.5-ggml is the real small SD1.5-family image generator
https://huggingface.co/second-state/stable-diffusion-v1-5-GGUF/resolve/main/stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf

Audio — Sound/FX Gen

Generates non-music sound effects or ambient audio from a text description.

AudioLDM

Good for: Generates sound effects from text
Watch out for: Not meant for high fidelity vocal tracks
LocalAI: local-ai run acestep-cpp-turbo-4b (~7.6 GB) — AudioLDM (sound effects from text) not in gallery; AceStep is a real music/audio generation model — not a perfect substitute (music vs. arbitrary SFX) but the closest real audio-gen entry
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-5Hz-lm-4B-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/Qwen3-Embedding-0.6B-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-v15-turbo-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/vae-BF16.gguf

Voice Cloning

Text-to-speech that mimics a specific person's voice, given a short sample of them speaking.

XTTS-v2

Good for: Clones voices locally with high accuracy
Watch out for: Can stutter on low VRAM splits
LocalAI: local-ai run qwen3-tts-cpp-1.7b-base (~2.2 GB) — XTTS-v2 not in gallery; Qwen3-TTS is a real current voice-cloning-capable TTS model
https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF/resolve/main/qwen-talker-1.7b-base-Q8_0.gguf
https://huggingface.co/Serveurperso/Qwen3-TTS-GGUF/resolve/main/qwen-tokenizer-12hz-Q8_0.gguf

4 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Gemma 3 4B

Good for: Robust logical deductions
Watch out for: Heavy memory usage during scaling
Ollama: ollama pull gemma3:4b (you already have this)
LocalAI: local-ai run gemma-3-4b-it (~3.1 GB)
https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/main/gemma-3-4b-it-Q4_K_M.gguf
https://huggingface.co/lmstudio-community/gemma-3-4b-it-GGUF/resolve/main/mmproj-model-f16.gguf

Qwen 3 4B

Good for: Advanced logic, agentic (able to plan and use tools on its own, not just chat) routing
Watch out for: Slightly large for 4GB VRAM
Ollama: ollama pull qwen3:4b-instruct (~2.3 GB)
LocalAI: local-ai run qwen3-4b (~2.3 GB)
https://huggingface.co/MaziyarPanahi/Qwen3-4B-GGUF/resolve/main/Qwen3-4B.Q4_K_M.gguf

Reasoning / Logic

Phi-3 Mini 3.8B

Good for: High logic, reasoning, following strict formats
Watch out for: Can hallucinate (makes up wrong-but-confident-sounding facts) on niche facts
Ollama: ollama pull phi3 (~2.0 GB)
LocalAI: local-ai run qwen3-4b (~2.3 GB) — Phi-3-mini not in gallery; Qwen3 4B covers the same reasoning-chat tier
https://huggingface.co/MaziyarPanahi/Qwen3-4B-GGUF/resolve/main/Qwen3-4B.Q4_K_M.gguf

Phi-4-mini 3.8B

Good for: Intricate reasoning, strict formatting
Watch out for: Requires explicit prompts
Ollama: ollama pull phi4-mini (~2.3 GB)
LocalAI: local-ai run qwen3-4b (~2.3 GB) — Phi-4-mini not in gallery; Qwen3 4B is the real current equivalent
https://huggingface.co/MaziyarPanahi/Qwen3-4B-GGUF/resolve/main/Qwen3-4B.Q4_K_M.gguf

Image Generation

Creates brand-new images from a text description ("a cat wearing a top hat"). The opposite of Vision Analysis.

OpenJourney

Good for: Midjourney-style stylized renders
Watch out for: Requires specific trigger words
LocalAI: local-ai run dreamshaper (~2.0 GB) — OpenJourney (Midjourney-style) not in gallery; Dreamshaper is a real stylized SD1.5-family alternative
https://huggingface.co/Lykon/DreamShaper/resolve/main/DreamShaper_8_pruned.safetensors

Stable Diffusion 1.5

Good for: Offline Img generation, highly moddable
Watch out for: Strict 512x512 limit on 4GB VRAM
LocalAI: local-ai run sd-1.5-ggml (~1.5 GB) — real, current small SD1.5 gguf build
https://huggingface.co/second-state/stable-diffusion-v1-5-GGUF/resolve/main/stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf

12 GB RAM

4 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Llama 3.1 8B (Q4)

Good for: High-tier chat, deep contextual understanding
Watch out for: Context window (how much text it can "remember" in one conversation) eats RAM quickly
Ollama: ollama pull llama3.1 (~4.6 GB)
LocalAI: local-ai run fusechat-llama-3.1-8b-instruct (~4.6 GB)
https://huggingface.co/bartowski/FuseChat-Llama-3.1-8B-Instruct-GGUF/resolve/main/FuseChat-Llama-3.1-8B-Instruct-Q4_K_M.gguf

Mistral 7B v0.3

Good for: Versatile tasks, reliable function calling (can call external tools/functions mid-conversation)
Watch out for: Older architecture compared to Llama 3.1
Ollama: ollama pull mistral (you already have this)
LocalAI: local-ai run mistral-7b-instruct-v0.3 (~4.1 GB)
https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf

Neural Chat 7B

Good for: Instruction following, highly responsive
Watch out for: Outdated compared to late-2025/2026 models
Ollama: ollama pull neural-chat (~3.8 GB)
LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — Neural Chat 7B (old Intel finetune) not in gallery; Falcon3 3B covers the same instruction-following chat role at a smaller footprint
https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf

Mistral 7B v0.3 (Q4)

Good for: Versatile tasks, reliable function calling (can call external tools/functions mid-conversation)
Watch out for: Slightly less capable than Llama 3.1 out of the box
Ollama: ollama pull mistral (you already have this)
LocalAI: local-ai run mistral-7b-instruct-v0.3 (~4.1 GB)
https://huggingface.co/MaziyarPanahi/Mistral-7B-Instruct-v0.3-GGUF/resolve/main/Mistral-7B-Instruct-v0.3.Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

CodeLlama 7B

Good for: Bash and basic C++ generation
Watch out for: Surpassed by newer models
Ollama: ollama pull codellama (~3.6 GB)
LocalAI: local-ai run codellama-7b (~3.6 GB)
https://huggingface.co/TheBloke/CodeLlama-7B-GGUF/resolve/main/codellama-7b.Q4_0.gguf

Starcoder2 7B

Good for: Deep offline IDE integration
Watch out for: Not conversational
Ollama: ollama pull starcoder2:7b (you already have this)
LocalAI: local-ai run deepseek-coder-v2-lite-instruct (~9.7 GB) — StarCoder2 not in gallery; DeepSeek-Coder-V2-Lite is the real larger code substitute
https://huggingface.co/LoneStriker/DeepSeek-Coder-V2-Lite-Instruct-GGUF/resolve/main/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

Reasoning / Logic

DeepSeek-R1 7B (Q4)

Good for: Distilled (a smaller model trained to copy a bigger one's behavior) from Qwen, excellent logical flow
Watch out for: Can hallucinate (makes up wrong-but-confident-sounding facts) system setups
Ollama: ollama pull deepseek-r1:7b (~4.4 GB)
LocalAI: local-ai run deepseek-r1-distill-qwen-7b (~4.4 GB)
https://huggingface.co/bartowski/DeepSeek-R1-Distill-Qwen-7B-GGUF/resolve/main/DeepSeek-R1-Distill-Qwen-7B-Q4_K_M.gguf

DeepSeek-R1 8B

Good for: State-of-the-art step-by-step logic, mathematical proofs, complex deductions
Watch out for: Produces long 'thought' chains before answering
Ollama: ollama pull deepseek-r1:8b (~4.9 GB)
LocalAI: local-ai run deepseek-r1-distill-llama-8b (~4.6 GB)
https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/resolve/main/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf

DeepSeek-R1 8B (Q4)

Good for: State-of-the-art step-by-step logic, mathematical proofs
Watch out for: Produces long 'thought' chains
Ollama: ollama pull deepseek-r1:8b (~4.9 GB)
LocalAI: local-ai run deepseek-r1-distill-llama-8b (~4.6 GB)
https://huggingface.co/unsloth/DeepSeek-R1-Distill-Llama-8B-GGUF/resolve/main/DeepSeek-R1-Distill-Llama-8B-Q4_K_M.gguf

Audio — Text to Speech

Turns written text into spoken audio — a synthetic voice reading your text out loud.

Bark

Good for: Realistic, emotive Text-to-Speech
Watch out for: Resource intensive
LocalAI: local-ai run piper-en_US-lessac-medium-crispasr (~30 MB) — Bark not in gallery; Piper is the real, fast TTS already verified — less emotive but real and working
https://huggingface.co/LocalAI-Community/piper-voices-GGUF/resolve/main/piper-en_US-lessac-medium-f16.gguf

Translation

Translates text (or sometimes speech) from one language to another. Some are conversational generalists that translate okay; these are dedicated specialists.

Aya 23 8B

Good for: Highly accurate multi-language (Spanish, Hebrew, English)
Watch out for: Needs heavy RAM offloading
Ollama: ollama pull aya (~4.5 GB)
LocalAI: local-ai run aya-23-8b (~4.7 GB)
https://huggingface.co/bartowski/aya-23-8B-GGUF/resolve/main/aya-23-8B-Q4_K_M.gguf

SeamlessM4T

Good for: Text-to-Text, Audio-to-Audio translation
Watch out for: Heavy resource footprint
LocalAI: not available — no SeamlessM4T or equivalent any-to-any speech translation model in the gallery — genuinely no substitute

TowerInstruct 7B

Good for: Professional-grade translation workflows
Watch out for: Focuses strictly on EU/Major languages
Ollama: ollama pull translategemma (~3.1 GB) — substituted for 'towerinstruct' (doesn't exist in Ollama's registry)
LocalAI: not available — no dedicated translation-specialist model in the gallery — use a strong generalist (Qwen3 8B) with a translation prompt instead

Storytelling / Creative Writing

Tuned for fiction, roleplay, and narrative writing rather than factual accuracy or following instructions strictly.

Llama 3.1 8B

Good for: Roleplay, world-building, narrative generation
Watch out for: Context window (how much text it can "remember" in one conversation) eats RAM in long stories
Ollama: ollama pull llama3.1 (~4.6 GB)
LocalAI: local-ai run fusechat-llama-3.1-8b-instruct (~4.6 GB)
https://huggingface.co/bartowski/FuseChat-Llama-3.1-8B-Instruct-GGUF/resolve/main/FuseChat-Llama-3.1-8B-Instruct-Q4_K_M.gguf

Nous-Hermes-2 7B

Good for: Highly conversational, dramatic narrative flows
Watch out for: Slightly older dataset
Ollama: ollama pull nous-hermes2 (~5.7 GB)
LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — Nous-Hermes-2 not in gallery; Falcon3 3B for general conversational storytelling
https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf

16 GB RAM

4 GB VRAM

General Chat

The all-purpose assistant type — ask it questions, have a conversation, get help writing something. This is what most people mean by "chatting with AI."

Gemma 2 9B (Q3/Q4)

Good for: Top-tier nuance
Watch out for: Will severely bottleneck a 4GB VRAM card
Ollama: ollama pull gemma2:9b (~5.1 GB)

Qwen 2.5 7B (Q4)

Good for: SOTA generalist in the 7B class
Watch out for: Requires at least 16GB RAM for split
Ollama: ollama pull qwen2.5:7b (~4.4 GB)
LocalAI: local-ai run qwen3-8b (~4.7 GB) — Qwen3 8B replaces Qwen2.5 7B, already verified
https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Qwen2 7B (Q4)

Good for: Multilingual (handles multiple languages) coding, robust mathematics
Watch out for: Heavy RAM usage when context window (how much text it can "remember" in one conversation) expands
Ollama: ollama pull qwen2 (~4.1 GB)
LocalAI: local-ai run qwen3-8b (~4.7 GB) — old-gen plain Qwen2 not in gallery at all; Qwen3 8B is the current equivalent-tier generalist
https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Code Generation

Writes and explains programming code. Good for a developer who wants suggestions, bug fixes, or a starting point — not a replacement for understanding what the code does.

DeepSeek Coder 6.7B

Good for: Advanced multi-file coding
Watch out for: Resource intensive
Ollama: ollama pull deepseek-coder:6.7b (~3.6 GB)

DeepSeek Coder V2 (Lite)

Good for: Reading complex source code, debugging FPC/Lazarus
Watch out for: Slower startup
Ollama: ollama pull deepseek-coder-v2 (~8.3 GB)
LocalAI: local-ai run deepseek-coder-v2-lite-instruct (~9.7 GB)
https://huggingface.co/LoneStriker/DeepSeek-Coder-V2-Lite-Instruct-GGUF/resolve/main/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

Phind-CodeLlama 34B (Heavy Q2) / 7B

Good for: Answering deep programming questions, analyzing user code inputs
Watch out for: The 7B version is less capable than Qwen Coder
Ollama: ollama pull phind-codellama (~17.7 GB)
LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — Phind-CodeLlama not in gallery; OpenCoder for lightweight code Q&A
https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Phind-CodeLlama 7B

Good for: Answering deep programming questions
Watch out for: Requires precise prompting
Ollama: ollama pull phind-codellama (~17.7 GB)
LocalAI: local-ai run opencoder-1.5b-instruct (~1.3 GB) — Phind-CodeLlama not in gallery; OpenCoder for lightweight code Q&A
https://huggingface.co/QuantFactory/OpenCoder-1.5B-Instruct-GGUF/resolve/main/OpenCoder-1.5B-Instruct.Q4_K_M.gguf

Qwen 2.5 Coder 7B

Good for: Advanced multi-file coding, analyzing existing codeblocks
Watch out for: Resource heavy for large context
Ollama: ollama pull qwen2.5-coder:7b (you already have this)
LocalAI: local-ai run deepseek-coder-v2-lite-instruct (~9.7 GB) — real larger code model already verified elsewhere in this doc
https://huggingface.co/LoneStriker/DeepSeek-Coder-V2-Lite-Instruct-GGUF/resolve/main/DeepSeek-Coder-V2-Lite-Instruct-Q4_K_M.gguf

Reasoning / Logic

Qwen 2.5 7B

Good for: High accuracy fact retrieval, excellent multilingual (handles multiple languages) (Spanish/Hebrew/English)
Watch out for: Context offloading uses significant System RAM
Ollama: ollama pull qwen2.5:7b (~4.4 GB)
LocalAI: local-ai run qwen3-8b (~4.7 GB) — Qwen3 8B replaces Qwen2.5 7B, already verified
https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Qwen 3 8B

Good for: Advanced logic, agentic (able to plan and use tools on its own, not just chat) routing
Watch out for: Requires RAM offloading
Ollama: ollama pull qwen3:8b (~4.9 GB)
LocalAI: local-ai run qwen3-8b (~4.7 GB)
https://huggingface.co/MaziyarPanahi/Qwen3-8B-GGUF/resolve/main/Qwen3-8B.Q4_K_M.gguf

Vision Analysis

Can look at an image you give it and describe, answer questions about, or read text from it. Doesn't generate new images — just understands existing ones.

BakLLaVA 7B

Good for: High detail Img captioning
Watch out for: Requires high System RAM
Ollama: ollama pull bakllava (~4.4 GB)
LocalAI: local-ai run qwen3-vl-2b-instruct (~1.8 GB) — BakLLaVA not in gallery; Qwen3-VL 2B is a real, current small vision model
https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen3-VL-2B-Instruct-Q4_K_M.gguf
https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/mmproj-F16.gguf

Llava 1.5 7B

Good for: Analyzing uploaded Imgs, OCR (OCR (reading text out of images))
Watch out for: Slow projection phase
Ollama: ollama pull llava (~4.4 GB)
LocalAI: local-ai run qwen3-vl-2b-instruct (~1.8 GB) — Llava 1.5 not in gallery; Qwen3-VL 2B covers image+text QA
https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen3-VL-2B-Instruct-Q4_K_M.gguf
https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/mmproj-F16.gguf

Qwen-VL-Chat 7B

Good for: Excellent OCR (OCR (reading text out of images)) and multilingual (handles multiple languages) visual QA
Watch out for: Slightly less conversational than Llava
Ollama: ollama pull qwen3-vl (~5.7 GB) — substituted for 'qwen-vl' (doesn't exist in Ollama's registry)
LocalAI: local-ai run qwen3-vl-2b-instruct (~1.8 GB) — the old Qwen-VL-Chat naming isn't in gallery; Qwen3-VL is the current real version of the same family
https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/Qwen3-VL-2B-Instruct-Q4_K_M.gguf
https://huggingface.co/unsloth/Qwen3-VL-2B-Instruct-GGUF/resolve/main/mmproj-F16.gguf

Image Generation

Creates brand-new images from a text description ("a cat wearing a top hat"). The opposite of Vision Analysis.

SDXL Turbo

Good for: Fast 1-4 step 1024x1024 generation
Watch out for: Pushes 4GB VRAM to absolute maximum
LocalAI: local-ai run sd-1.5-ggml (~1.5 GB) — SDXL Turbo not in gallery; sd-1.5-ggml is the real small image-gen model that fits 4GB VRAM (SDXL-class models generally don't)
https://huggingface.co/second-state/stable-diffusion-v1-5-GGUF/resolve/main/stable-diffusion-v1-5-pruned-emaonly-Q4_0.gguf

Music Generation

Creates instrumental music tracks from a text description.

MusicGen (Small)

Good for: Generates instrumental tracks from text
Watch out for: Takes a long time for 10 seconds of audio
LocalAI: local-ai run acestep-cpp-turbo-4b (~7.6 GB) — real, current music generation model in the gallery
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-5Hz-lm-4B-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/Qwen3-Embedding-0.6B-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-v15-turbo-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/vae-BF16.gguf

MusicGen (Small/Medium)

Good for: Generates short instrumental music tracks from text descriptions
Watch out for: Takes a long time for even 10 seconds of audio
LocalAI: local-ai run acestep-cpp-turbo-4b (~7.6 GB) — real, current music generation model in the gallery
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-5Hz-lm-4B-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/Qwen3-Embedding-0.6B-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/acestep-v15-turbo-Q8_0.gguf
https://huggingface.co/Serveurperso/ACE-Step-1.5-GGUF/resolve/main/vae-BF16.gguf

Structured Data / Office

Good at pulling structured information (tables, JSON, forms) out of messy text, or formatting output to fit a specific structure.

Command-R 8B (Q4)

Good for: Excellent at RAG (RAG (feeding it your own documents to answer from)) and formatting enterprise JSON
Watch out for: Rigid conversational style
Ollama: ollama pull command-r (~17.4 GB)
LocalAI: local-ai run command-r-v01:q1_s (~7.9 GB) — real quantized build, already applied
https://huggingface.co/dranger003/c4ai-command-r-v01-iMat.GGUF/resolve/main/ggml-c4ai-command-r-v01-iq1_s.gguf

Storytelling / Creative Writing

Tuned for fiction, roleplay, and narrative writing rather than factual accuracy or following instructions strictly.

Dolphin 2.9 (Llama 3 8B)

Good for: Uncensored (no built-in content filters — use responsibly) creative writing, raw sci-fi/thriller generation
Watch out for: Can produce overly graphic content unprompted
Ollama: ollama pull dolphin-llama3 (~4.3 GB)
LocalAI: local-ai run dolphin-2.9-llama3-8b (~4.6 GB)
https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b-gguf/resolve/main/dolphin-2.9-llama3-8b-q4_K_M.gguf

Gemma 2 9B (Q4)

Good for: Exceptional creative writing, deep character nuance (Sci-Fi, Thrillers)
Watch out for: Strict safety alignment (built-in safety behavior) can interrupt creative flow
Ollama: ollama pull gemma2:9b (~5.1 GB)

24 GB RAM

4 GB VRAM

Video Generation

Creates short video clips from a text description. Still an early, slow, low-resolution capability on consumer hardware.

Zeroscope v2 576w

Good for: Short (2-3 second) low-res video generation
Watch out for: Generates frame-by-frame; heavily relies on System RAM
LocalAI: local-ai run wan-2.1-t2v-1.3b-ggml (~7.3 GB) — Zeroscope not in gallery; Wan2.1 1.3B is a real, current small text-to-video model
https://huggingface.co/calcuis/wan-gguf/resolve/main/wan2.1_t2v_1.3b-q8_0.gguf
https://huggingface.co/Comfy-Org/Wan_2.1_ComfyUI_repackaged/resolve/main/split_files/vae/wan_2.1_vae.safetensors
https://huggingface.co/city96/umt5-xxl-encoder-gguf/resolve/main/umt5-xxl-encoder-Q8_0.gguf

Structured Data / Office

Good at pulling structured information (tables, JSON, forms) out of messy text, or formatting output to fit a specific structure.

Mistral Nemo 12B (Q3)

Good for: Massive context (128k) for processing large tabular text/CSVs
Watch out for: Very slow on 4GB VRAM
Ollama: ollama pull mistral-nemo (~6.6 GB)

Storytelling / Creative Writing

Tuned for fiction, roleplay, and narrative writing rather than factual accuracy or following instructions strictly.

Mythomax-L2 13B (Q2)

Good for: Exceptional multi-character roleplay
Watch out for: Extremely slow due to heavy system RAM usage
Ollama: ollama pull nous-hermes2 (~5.7 GB) — substituted for 'mythomax' (doesn't exist in Ollama's registry)
LocalAI: local-ai run falcon3-3b-instruct (~1.9 GB) — MythoMax roleplay finetune not in gallery; Falcon3 3B for general storytelling/chat (won't match its uncensored roleplay style specifically)
https://huggingface.co/bartowski/Falcon3-3B-Instruct-GGUF/resolve/main/Falcon3-3B-Instruct-Q4_K_M.gguf

Mirrored here

⬇ Download Official source