What an embedding actually commits you to

Embedding choice is made early, usually quickly, and then becomes load-bearing. Six months later the index has grown, several systems query it, and the cost of changing your mind is no longer a configuration change.

This is not an argument for agonising over the initial choice. It is an argument for setting it up so the choice can be revisited cheaply.

What is actually locked in

The vectors themselves. They are only comparable to vectors produced by the same model, so a change means re-embedding the entire corpus rather than migrating it.

The dimensionality, which determines storage and query cost at your corpus size, and which is difficult to reason about before the corpus is large.

Behaviour on your specific vocabulary. Internal product names, abbreviations and domain terms are handled differently by different models, and this shows up in retrieval quality long before anyone connects it to the embedding layer.

Keep the raw text, always

The single most useful habit is unglamorous: store the original text alongside the vector, with its provenance.

Teams that discard the source and keep only vectors have made re-embedding a data recovery exercise. Teams that kept the text can re-embed the whole corpus as a batch job whenever they want to evaluate an alternative, which means the decision stays open.

This costs storage, which is cheap, and it buys optionality, which is not.

Evaluate on your vocabulary, not a benchmark

General benchmarks tell you how a model behaves on general language. Your corpus is not general. It contains the names of your services, the abbreviations your teams use in tickets and the phrasing your policies are written in.

Build a set of real queries against your own documents, with the passages that should answer them, and measure retrieval directly. That set is reusable forever: it evaluates the embedding model, the chunking strategy, the reranker and anything else you change later.

It is also the only thing that makes "we should try a different embedding model" a question with an answer rather than an opinion with advocates.

Hybrid is usually the pragmatic answer

Semantic search finds things that mean the same thing in different words. Keyword search finds the exact error code someone pasted in.

Most real query sets contain both kinds, and systems that combine the two outperform either alone on the mixed reality of what people actually ask. This is not a sophisticated technique, and it is skipped surprisingly often because semantic search demos better.

The operational view

Embedding is part of a chain that ends in an answer someone acts on, and it has both a quality dimension and a cost dimension. Re-embedding a large corpus is a real expense; carrying oversized vectors on every query is a smaller expense repeated indefinitely.

Both belong in the same evaluation loop as the rest of the agent's behaviour, which is why evals score latency and cost alongside correctness rather than treating them as separate concerns.

Next

The developer documentation covers embedding and retrieval mechanics. The Agent Orchestration System pages cover the evaluation layer that keeps decisions like this reversible.