Solutions for
Please select
Try the Web App
ki-konzepte

Embeddings Simply Explained

Redaktion ·

Embeddings Simply Explained

A computer does not understand words. It understands numbers. An embedding is the bridge between the two: it turns a text — a word, a sentence, an entire document — into a list of numbers, a vector. And not arbitrarily, but in such a way that the numbers capture the meaning of the text.

That sounds abstract but is the foundation of almost all modern semantic search, from RAG to product recommendations. This article explains what an embedding concretely is, why similar meanings end up close together, and what you use this for in practice.

What an Embedding Is

An embedding is a vector representation of data — a list of floating point numbers. An embedding model takes your text and outputs this list. How long the list is depends on the model: OpenAI’s text-embedding-3-small, for example, produces 1,536 dimensions, text-embedding-3-large even 3,072 (source: OpenAI Embeddings Guide, accessed 2026-06-06).

Each of these numbers is a coordinate in a high-dimensional space. You can picture it like coordinates on a map — except instead of two axes (longitude, latitude) there are a thousand or more. The decisive point: these coordinates encode the semantic properties of the text. Content with similar meaning gets similar coordinates.

The formal definition is in the glossary entry on embedding.

Why Similar Meanings Sit Close Together

This is where it gets useful. Google states the core principle this way: the distance between any two items can be calculated mathematically and interpreted as a measure of their relative similarity (source: Google ML Crash Course, accessed 2026-06-06).

Concretely: in embedding space, hot dog and shawarma land close together because they are conceptually similar (both savory street food). Apple strudel sits farther away because it is a different category (dessert). The model has learned from huge amounts of text which terms appear in similar contexts — and exactly that proximity is reflected in the coordinates.

Distance is usually measured via cosine similarity. It measures the angle between two vectors, not their absolute length — two texts point in the same direction when they mean something similar. OpenAI puts it succinctly: small distances suggest high relatedness, large distances suggest low relatedness. Because OpenAI’s embeddings are normalized to unit length, cosine similarity can even be computed as a simple dot product.

The effect is powerful: you can compute similarity of meaning without the texts sharing a single word. Car and automobile sit close together even though no letter matches — classic keyword search would fail here.

What Embeddings Are Used For

From this one ability — translating meaning into measurable proximity — several applications follow (source: OpenAI Embeddings Guide, accessed 2026-06-06):

  • Semantic search. You convert the query and all documents into embeddings and return the documents whose vectors are closest to the query. Matches by meaning, not by exact word overlap.
  • RAG retrieval. This exact step is at the heart of retrieval-augmented generation: before a language model answers, it retrieves the relevant knowledge snippets via embedding similarity. More on this in the lexicon entry on understanding RAG.
  • Classification. You assign a text to the category whose example embedding is closest.
  • Clustering. You automatically group similar texts without defining categories in advance.
  • Recommendations. You suggest articles or products whose embeddings resemble those of the one currently viewed.
  • Anomaly detection. You find outliers whose vector sits far from everything else.

Embedding Models vs. Chat Models

A common misconception: embedding models and chat models (LLMs) are not the same. A chat model generates text — you give a prompt, it writes an answer. An embedding model generates no text. It only returns the vector. Full stop.

Embedding models are usually significantly smaller, faster, and cheaper than chat models because their task is narrower. That makes them ideal for use at scale: you embed your entire knowledge base once (thousands of documents) and then, per query, only need to embed the query and compare.

In a RAG pipeline both work together: the embedding model finds the relevant passages, the chat model formulates the answer from them.

The Connection to Vector Databases

When you have thousands or millions of embeddings, you need a place that stores them and finds the nearest neighbors to a query vector lightning fast. A normal database is unsuited for this — it cannot efficiently search by vector proximity.

This is exactly what vector databases are for. They store embeddings and use specialized indexing methods (such as approximate nearest neighbor) to deliver the most similar from millions of vectors in milliseconds. Without this infrastructure, semantic search at scale would not be practical. Which options exist and how they differ is covered in the lexicon entry on vector databases compared.

FAQ

What is an embedding in one sentence?

An embedding is a list of numbers (a vector) that places the meaning of a text or image in space such that similar content sits close together and dissimilar content far apart.

How do you measure the similarity of two embeddings?

Usually via cosine similarity, which measures the angle between two vectors. Small distance means high content relatedness, large distance low. For normalized vectors this is equivalent to the dot product.

Is an embedding model the same as ChatGPT?

No. A chat model generates text. An embedding model only returns a vector and never writes an answer. Embedding models are smaller, faster, and cheaper and are used for search, retrieval, and classification.

What do I concretely need embeddings for?

For semantic search, RAG retrieval, classification, clustering, recommendations, and anomaly detection. Everywhere you want to compute similarity of meaning instead of just matching keywords.

Why do I need a vector database for this?

Because normal databases cannot efficiently search by vector proximity. Vector databases store embeddings and, with specialized indexes, find the most similar from millions of vectors in milliseconds.

Themenuebersicht

KI-KonzepteLLM-Grundlagen