Embeddings & Vector Databases

An embedding turns a piece of text into a list of numbers that captures its meaning. Text with similar meaning produces similar numbers, which means a computer can compare meaning arithmetically instead of matching words.

What an Embedding Actually Is

Run three sentences through an embedding model and you get three lists of numbers back:

"The dog chased the ball."    → [ 0.021, -0.134,  0.087, ... ]
"The puppy ran after a toy."  → [ 0.019, -0.128,  0.091, ... ]
"Quarterly revenue rose 4%."  → [-0.203,  0.331, -0.045, ... ]

The first two lists are nearly identical, even though they share no words beyond "the". The third is nowhere near either of them. The model was never told that a puppy is a dog. It learned that from the way the words are used across its training data.

Each number in the list is a coordinate on one axis. Dimensions is just how many numbers are in the list - 1,536 dimensions means 1,536 numbers describing one piece of text. Think of a two-dimensional chart with "dog" and "puppy" plotted close together and "car" far away, then imagine that chart with 1,536 axes instead of two. No one can name what each individual axis represents, and that does not matter. What matters is the distance between points.

That two-dimensional chart, drawn out:

The clusters form without anyone defining them. No rule says a van belongs with a car, and nothing tags "quarterly revenue" as finance. Position is a by-product of how the words were used across the training corpus, which is why an embedding generalises to vocabulary the system has never been configured for.

The same applies to whole paragraphs, not just single words. A paragraph about parental leave policy and a paragraph about time off for new parents land near each other, regardless of the vocabulary each one uses.

Why This Matters

Embeddings are what make semantic search possible: matching on meaning rather than on characters.

Take the query "Who runs the company?" against a document that says "Sarah Chen was appointed Chief Executive Officer in 2024."

• Keyword search looks for the words "runs" and "company". The document contains neither. It returns nothing, and the system reports that it has no information.
• Semantic search embeds the query, and that vector lands close to text about executives and leadership. The document is returned even though the wording does not overlap at all.

This is the same reason a search for "CEO" surfaces "Chief Executive Officer", and why a support query about "my card was declined" finds an article titled "Payment authorisation failures".

Keyword search still wins in one important case: exact identifiers. A search for part number AX-4471 or an invoice reference needs literal matching, and an embedding model will happily return something that merely looks similar. Production systems therefore combine both approaches. How that combination is built, tuned, and evaluated belongs to the retrieval pipeline - see RAG.

Choosing an Embedding Model

How to read those columns in practice:

• Dimensions trade quality against cost. More dimensions capture finer distinctions, but every extra dimension is more storage, more memory, and slightly slower search. 3,072 dimensions is not automatically better than 1,536 for your content - it is measurably better on some benchmarks and indistinguishable on many real corpora.
• Max input length caps how much text you can embed in a single call, and it is measured in tokens. A model with a ~512 token limit will silently truncate a long document, so anything past the cut-off is invisible to search.
• Hosted or self-hosted is usually decided by data residency, volume, and who operates it. Hosted APIs are cheap per call and cost nothing when idle. Self-hosted open-weight models remove per-token fees and keep text inside your boundary, at the price of running the serving infrastructure.
• Test on your own content. Public benchmark rankings are a shortlist, not an answer. Embed a few hundred of your real documents, run your real queries, and compare.

Vector Databases

A vector database stores the embeddings and finds the closest matches to a query vector quickly. Each record holds three things:

Metadata is what makes a result citable. Without it, the system can return exactly the right passage and still be unusable in an enterprise setting: no source to show the user, no way to date the answer, and no way to restrict results to documents that person is allowed to see. Metadata has to be attached at index time. You cannot reconstruct it afterwards from the vector.

The honest way to frame this choice is not "which product scores highest" but which of three postures fits:

• A dedicated vector database gives the best retrieval features and scale characteristics. The cost is a second system with its own security model, its own bill, and a synchronisation job that someone has to own.
• The database you already run (pgvector being the common case) adds no new vendor, no new access model, and no new backup story. Vectors sit in the same transaction as the rows they describe. This is genuinely sufficient for a large share of enterprise workloads and is consistently underrated.
• A platform-native index keeps the index synced to the source tables automatically and inherits the governance already in place - Unity Catalog on Databricks, roles and grants on Snowflake. It removes the hardest operational problem in the list, staleness, in return for keeping retrieval on that platform.

How Text Becomes a Searchable Vector

Both paths run through the same embedding model. That single shared box is the whole of the rule in the warning above: swap it on one path only and every comparison downstream is meaningless.

How Similarity Search Works

The standard measure is cosine similarity, which compares the direction two vectors point in rather than how far apart their endpoints are. Vectors pointing the same way score close to 1, unrelated ones sit near 0, and opposing meanings go negative. The practical effect is that a one-line summary and a three-page document on the same subject still match well, because length changes a vector's magnitude but not much of its direction.

At scale, the search is not exhaustive. Comparing a query against every vector in a 50 million record index on every request would be far too slow, so vector databases build an index structure (HNSW is the most common) that narrows the search to a promising neighbourhood. This is approximate nearest neighbour search: it can occasionally miss a true best match in exchange for answering in milliseconds. Almost every production vector search is approximate, and the accuracy-versus-latency balance is a tuning parameter, not a fixed property.

What happens to those matches next - filtering, re-ranking, blending with keyword results, and assembling them into a prompt - is the retrieval pipeline, covered in RAG.

Operating Embeddings in Production

Three things reliably cause trouble once a system is live.

Staleness. The index is a copy, and copies drift. When a source document is updated, the index still holds the old vector and the old text until something re-embeds it. The system then answers confidently from a version of the truth that no longer exists, and cites a document that appears entirely legitimate. Options are a scheduled full re-index (simple, wasteful), change-driven incremental updates (embed only what changed, needs reliable change capture), or a platform-native sync such as Databricks Delta Sync. Whichever you pick, store a last-indexed timestamp in metadata so answers can be dated and freshness can be monitored.

Re-indexing when the model changes. Switching embedding model, or accepting a version upgrade, invalidates the entire index. For a few thousand documents that is a short batch job. For several million it is a budgeted piece of work, normally run as a dual-index cutover: build the new index alongside the live one, verify retrieval quality against it, switch reads across, then drop the old index. Treat the embedding model as a long-lived architectural commitment rather than a setting.

Index size and cost. Storage scales with the number of text units multiplied by dimensions multiplied by bytes per number, plus the original text, plus metadata, plus the search index structure, which is frequently held in memory. At 1,536 dimensions and 4 bytes per number, one record carries roughly 6 KB of vector before anything else. A million records is a few gigabytes and unremarkable. Ten million records at 3,072 dimensions is a different budget conversation. The levers are fewer or larger text units, a lower dimension count (several models support truncating dimensions with modest quality loss), and quantisation to store numbers at reduced precision.

One smaller cost worth tracking: every query is also an embedding call. Individually trivial, but at high query volume it is a recurring line item, and caching vectors for repeated queries removes most of it.

Key Takeaways

• An embedding converts text into a list of numbers that encodes meaning, so semantically similar content sits close together. Dimensions is simply how many numbers are in that list.
• Embeddings enable matching by meaning instead of by characters, which is why a search for "CEO" finds "Chief Executive Officer". Keyword matching still wins on exact identifiers, so production retrieval combines both.
• Queries and documents must be embedded with the same model and the same version. A mismatch with equal dimensions returns confidently wrong results and raises no error.
• A vector store record is vector plus original text plus metadata. Metadata is what makes an answer citable and permission-aware, and it must be captured at index time.
• Start with the vector store attached to the platform your data already lives on. A dedicated vector database is a justified move when scale or retrieval features demand it, not a default.
• Budget for staleness and re-indexing before launch. Changing the embedding model means re-embedding everything, so choose it as an architectural decision.