Main area
Agency
ki-werkzeuge

Pipeline Frameworks — LangChain, LlamaIndex and when you actually need them

Redaktion ·

What this is about — and why the choice matters

The moment you build an LLM application that does more than “prompt in, answer out”, you face the same question: do you reach for a pipeline framework like LangChain or LlamaIndex, or do you wire the steps together yourself? Both frameworks promise to take boilerplate off your hands — document loaders, embedding calls, vector DB integration, tool use, memory, retry logic. Both are widely adopted, and both have undergone fundamental redesigns over the past two years.

This article maps the field: what these frameworks really do, when adopting one is worth it, when DIY wins — and which criticisms come up so often that they need to be taken seriously. Status: May 2026, after the LangChain and LangGraph 1.0 stable releases in October 2025.

What these frameworks actually do

Pipeline frameworks sit between your code and the LLM APIs. Instead of speaking directly to the OpenAI or Anthropic API, you describe a pipeline of steps — generate an embedding, query a vector DB, rerank, fill a prompt template, call the model, parse the output — and the framework runs them, with default implementations for most building blocks.

Three classes of work come up most often:

RAG pipelines

An incoming question gets translated into an embedding, searched in a vector database, the hits sorted by reranking, embedded into a prompt and sent to the model. That requires document loaders (PDF, HTML, Markdown), chunking strategies, support for multiple vector DBs and a clean separation between “indexing” and “query”.

Agent loops

A model gets tools (function calling), decides which to call, evaluates the result, decides again. This is less about data chains and more about state, loops, stop conditions and human-in-the-loop approval.

Chained calls

Several LLM calls in sequence — first a classifier, then a router, then a generator. Parsing, validating, possibly retrying between steps. Classic case: content generation with a quality check.

Frameworks differ in which of these three they treat as the core case. That’s the most important dividing line.

LangChain — the generalist suite

LangChain is the best-known framework, started in 2022, stabilised at version 1.0 in October 2025. It covers all three classes: document loaders, vector DB adapters, prompt templates, output parsers, memory, tool use, agent loops. Over 600 integrations with model providers, vector DBs and external tools.

With 1.0, LangChain consolidated its architecture: the old chains and agents abstractions are deprecated, everything now flows through create_agent, which uses LangGraph as its runtime under the hood. Legacy features have been moved into langchain-classic; the main package focuses on four pillars: agents, models, messages, tools. Standardised “content blocks” make model outputs provider-agnostic.

LangGraph 1.0 shipped on the same day and is the runtime for anything that needs loops, state, or pause/resume. If you build a simple agent, you call create_agent and never see LangGraph directly; if you need more control, you drop one level deeper and describe the graph yourself — with nodes, edges, checkpointing and human-approval gates.

Strengths: broadest ecosystem, solid tool integration, LangSmith as a mature observability solution, LangGraph as a clean abstraction for complex agent workflows.

Weaknesses: a history of breaking changes — 0.1, 0.2, 0.3, 1.0 each brought their own migrations. Code from 2024 takes more migration work in 2026 than expected. Plenty of boilerplate for simple cases where three direct API calls would have done.

LlamaIndex — the RAG specialist

LlamaIndex was optimised for one use case from day one: getting data into an LLM. Document loaders for 160+ data sources, mature chunking strategies (hierarchical, sentence-window, semantic), built-in reranker, a Composite Retrieval API for multiple indices side by side.

The mental hurdle is lower than with LangChain: a RAG prototype runs in five to ten lines of code. Anyone working in a document-heavy setup — knowledge base, technical docs, contracts — gets to production faster with LlamaIndex.

LlamaIndex evolved heavily towards agent workflows in 2025/26. “Agent Workflows” is the counterpart to LangGraph — multi-step, stateful, with loops and parallel paths. LlamaParse is a related product for VLM-based document understanding (PDFs with tables, forms, images).

Strengths: more stable upgrade path than LangChain, less boilerplate for RAG, stronger built-ins for retrieval engineering (auto-merging, sub-question decomposition, RAPTOR).

Weaknesses: no first-party observability solution like LangSmith — monitoring comes via third parties (Langfuse, Arize Phoenix). Outside retrieval-heavy scenarios (e.g. complex multi-agent setups without a document focus), LangGraph is often the better fit.

LangGraph — the graph runtime for agents

LangGraph isn’t a framework in the classical sense, it’s a runtime for stateful multi-agent workflows. Instead of a linear chain (A → B → C), you describe a graph with nodes, edges and state, with first-class support for loops, branching and pause/resume.

When do you need that? The moment an agent tries something, evaluates the result and decides whether to try again or move on — i.e. any non-trivial agent loop. Also wherever human-in-the-loop comes in: an agent generates a proposal, pauses, a human approves, the agent continues. That’s only possible with pure chain frameworks via tricks; LangGraph builds it in.

LangGraph is now the official recommendation for non-trivial agents within the LangChain ecosystem. In LangChain 1.0, agents are LangGraph under the hood anyway — start with create_agent, and when you need more control, drop down a layer without migrating.

Haystack — the enterprise alternative

Haystack from deepset is the most relevant alternative outside the LangChain universe. Open-source, Python, modular pipelines built from “components” (embedder, retriever, reranker, generator, router). Current version: 2.28 (May 2026). The focus is on production-grade RAG, semantic search and agent workflows with “explicit control over retrieval, routing, memory, and generation”.

Haystack comes from the search/question-answering world, and it shows in the design: less experimental, fewer breaking changes, more clarity per component interface. 2.x adds LLMRanker (LLM-based reranking framed as semantic reasoning rather than similarity scoring), agent state handling and a “Haystack Enterprise Starter” for hosted deployments.

Strengths: stable API, transparent component model, suited for teams that want production-grade rather than experimental.

Weaknesses: smaller ecosystem than LangChain — fewer integrations, fewer tutorials, fewer Stack Overflow answers. If you want to stay on the well-trodden path, LangChain/LlamaIndex is the safer pick.

Comparison table

| Framework | Core strength | API stability | Observability | Fits | |---|---|---|---|---| | LangChain v1 | Generalist, biggest ecosystem | Historically volatile; v1 stable since 10/2025 | LangSmith (proprietary, 1st-party) | broad tool/model integration | | LlamaIndex | Document-focused RAG | More stable upgrade path | 3rd-party (Langfuse, Phoenix) | knowledge bases, technical docs | | LangGraph | Stateful agent loops | 1.0 with backwards-compat | LangSmith | multi-agent, human-in-the-loop | | Haystack 2.x | Modular production RAG | Stable, predictable releases | 3rd-party + deepset Cloud | enterprise RAG, search/QA |

When to adopt — and when not to

Pipeline frameworks pay off the moment you hit at least three of the following: multiple document sources, multiple vector DBs (or likely switches), multiple LLM providers, memory/state across calls, reranking, multi-step agent loops, human-in-the-loop approval, observability across multiple steps.

Rule of thumb: with fewer than five pipeline steps and a stable use case, DIY is often faster — and easier to maintain long-term. A simple Q&A bot over a known knowledge base needs no framework. Three direct API calls (embed, search, generate) fit into 30 lines of Python; every additional abstraction is risk.

You usually cross the threshold when:

  • Multiple use cases share infrastructure (customer support + internal search + eval pipeline) — shared component reuse pays off.
  • Tool use grows beyond two or three tools — orchestrating function calling cleanly is faster with a framework.
  • You need human-in-the-loop, checkpointing or pause/resume — building that yourself is doable but takes weeks.
  • You need observability across the whole pipeline — LangSmith out of the box beats three weeks of homebrew logging.

Below that threshold: write your own code that does exactly what it should. Three extra classes are better than an inherited migration.

Common criticism — and how seriously to take it

“Boilerplate avalanche”: true for LangChain v0, much improved in v1. LlamaIndex has always been leaner. For pure RAG prototypes, the criticism lands less in 2026 than it did in 2024.

“API instability”: historically valid — LangChain ran two years of rolling migrations. With v1.0 the promise has shifted: stable major release, breaking changes only across major versions. Anyone starting in 2026 should see far less migration pain than someone who started in 2023. LlamaIndex and Haystack were always more measured here.

“Lock-in”: real but overstated. Building with LangChain components means writing code against LangChain interfaces — moving to LlamaIndex or DIY is days of work, not weeks, because the concepts (embedding, retriever, prompt, tool) translate. Lock-in becomes real once you invest deeply in framework-specific features (LangGraph persistence, LangSmith tracing).

“Performance overhead”: rarely the bottleneck in practice — LLM latency dominates anyway. Exception: high-frequency inline calls without network hops (local models, embedding pipelines), where 50–100 ms of framework overhead per step can become a throughput problem.

Practical pitfalls

Underestimating version dependencies. LangChain ships several sub-packages within one major release cycle (langchain, langchain-core, langchain-openai, langchain-community, langgraph…) that must be compatible with each other. Pin all versions to the same release, otherwise you’ll land in subtle incompatibilities.

Treating frameworks as a substitute for understanding. “I’ll use LangChain so I don’t have to understand how embeddings work” — works for a while, then doesn’t. Frameworks abstract mechanics, they don’t replace them. If your reranking returns bad results, the framework won’t help you — you need to understand the RAG mechanics.

Picking multi-agent setups by default. “We’ll go with three agents — a researcher, a writer, a reviewer.” Ninety percent of the time, a single agent with good prompts and clear tools is faster, cheaper and more stable. Multi-agent setups pay off only once a single agent demonstrably fails at task separation.

The eval gap. A framework speeds up building, not testing. Without an evaluation set and metrics (faithfulness, context precision/recall), you can’t tell whether your pipeline is getting better or worse when you swap a step.

Conclusion

There’s no clear winner in 2026 — but there’s a clear division of labour:

For RAG with a document focus and quick prototypes, LlamaIndex is the most direct choice. For complex agent workflows with state, loops and human-in-the-loop, LangGraph (often via LangChain v1) is the standard. For production-grade, stable RAG pipelines with clear component separation, Haystack 2.x is the calmer alternative. Pure LangChain without LangGraph mainly pays when you need a generalist with the largest integration ecosystem.

More important than the framework choice is the honest answer to the question one step earlier: does the project even need a framework? With a stable use case at three to five steps, DIY wins — less code, no migrations, mechanics fully understood. Frameworks pay off when you carry enough complexity that doing it yourself takes weeks instead of hours — or when observability, multi-agent and persistence features tip the balance.

If you’re at the decision point: prototype the smallest meaningful pipeline without a framework first. If it runs fast and you don’t miss anything, you’ve saved time. If you spend five days and only stumble over boilerplate, the framework adoption is justified — and you now know exactly which features you actually need.

Themenuebersicht

KI-WerkzeugePipeline-Frameworks