Term
Ollama
Ollama is a tool for running language models locally — Llama, Mistral or Qwen — operated via a CLI, with an OpenAI-compatible HTTP server for applications.
Ollama — explained in more detail
Ollama is a compact runtime that lets language models run directly on your own hardware — no cloud, no token costs, no data leaving the device.
Operation
The main interface is a slim command line: ollama run llama3 pulls the model from a curated library and starts a chat. Available models range from small (Llama 3 8B, Phi-3) to large (Qwen 72B, DeepSeek), each in quantised variants of different precision.
API server
Alongside the CLI, Ollama runs an HTTP server with an OpenAI-compatible interface. Tools originally built against the OpenAI API — Continue.dev, Aider, custom scripts — can be pointed at the local instance just by swapping endpoint and model name.
Requirements
Speed and feasible model size depend directly on the hardware. 7B models run with 8 GB of RAM; 70B models need 64 GB plus a strong GPU. Anyone planning to work locally should check the machine’s spec before settling on a model.
Entdecke mehr
Saving Tokens with Claude: 6 Principles That Make Experts Twice as Fast
How I turned my CLAUDE.md from a style guide into a token budget — 6 principles for lower cost, less waiting, and more honest reporting.
GlossarGPT4All
GPT4All is an open-source desktop application by Nomic AI for running large language models locally and offline on your own computer. It works without a GPU and without a cloud connection, so data never leaves the device.
LexikonRunning LLMs locally — hardware, tools, models
How to run language models on your own hardware — VRAM requirements, tooling (Ollama, LM Studio, llama.cpp, vLLM) and which models fit which GPU.