Lokales LLM
Tools zum Betreiben von Sprachmodellen auf eigener Hardware.
- GPT4All Lokales LLM
GPT4All is an open-source desktop application by Nomic AI for running large language models locally and offline on your own computer. It works without a GPU and without a cloud connection, so data never leaves the device.
- Jan Lokales LLM
Jan is an open-source desktop application that lets you run large language models locally and fully offline on your own computer. It uses the llama.cpp engine and is positioned as a privacy-friendly ChatGPT alternative.
- llama.cpp Lokales LLM
llama.cpp is an open-source C++ library for efficient LLM inference on CPU and GPU — the engine many local tools like Ollama and LM Studio are built on.
- Llamafile Lokales LLM
Llamafile is a Mozilla project that packages a local language model as a single executable file — model weights and inference engine in one. That file runs on several operating systems without installation.
- LM Studio Lokales LLM
LM Studio is a desktop application for running LLMs locally with a graphical interface — including a model browser, chat interface and OpenAI-compatible server.
- Ollama Lokales LLM
Ollama is a tool for running language models locally — Llama, Mistral or Qwen — operated via a CLI, with an OpenAI-compatible HTTP server for applications.
- Open WebUI Lokales LLM
Open WebUI is an open-source chat interface for local and remote LLMs — a ChatGPT-like browser UX that runs as a Docker container and talks to Ollama, OpenAI-compatible APIs and more.
- TensorRT-LLM Lokales LLM
TensorRT-LLM is an open-source library from NVIDIA that optimizes LLM inference on NVIDIA GPUs. It compiles models into highly optimized runtime engines and offers in-flight batching, a paged KV cache and quantization for high throughput at low latency.
- Text-Generation-WebUI Lokales LLM
Text-Generation-WebUI (oobabooga) is an open-source web interface for running LLMs locally. It bundles several inference backends, chat and completion modes and an OpenAI-compatible API under one common UI.
- TGI (Text Generation Inference) Lokales LLM
TGI (Text Generation Inference) is an open-source toolkit from Hugging Face for serving large language models efficiently. It provides a production-ready inference server with continuous batching, token streaming and optimised attention methods.
- vLLM Lokales LLM
vLLM is an open-source high-throughput inference server for LLMs — known for PagedAttention, continuous batching, and an OpenAI-compatible API endpoint, widely used in production self-hosting setups.
- whisper.cpp Lokales LLM
whisper.cpp is an open-source C++ port of OpenAI's Whisper speech recognition — a compact native binary for local transcription on CPU, CUDA, Metal or Vulkan, with no Python dependency.