Vector Search

How does a vector search actually run?

In three steps, and only the middle one is exotic. Passages are converted to embeddings in advance and stored. At query time the question is embedded with the same model. Then the index returns the passages whose vectors sit closest, where closeness is a distance calculation: OpenAI’s documentation puts it plainly, “small distances suggest high relatedness and large distances suggest low relatedness”.

The dimensions involved are not small. OpenAI’s current models produce 1,536 values per passage for text-embedding-3-small and 3,072 for text-embedding-3-large, which is why practical systems use approximate nearest-neighbour indexes rather than comparing everything to everything. Approximate is the operative word, and it has a consequence worth knowing: these indexes trade a small amount of recall for a large amount of speed, so a passage that genuinely is the best match can be missed.

Retrieval failures are not always about the page. Nobody publishes the recall figures for the indexes behind consumer assistants, so how often that happens is unknown from outside. It is a real source of variation and it cannot be separated from the others by anybody who is not running the index. It is worth holding that alongside every claim in this field about why a page was or was not cited: some of the answer is a property of an index nobody outside the vendor can see.

What does vector search reliably get wrong?

Exactness. Two strings that a person would treat as completely different, a version number and its successor, two model codes in the same range, a brand and a near-homonym, can sit close together because the model was never trained to distinguish them. Nothing in the ranking is aware that one of them is right and the other is a different product.

This is why serious retrieval systems keep a keyword index beside the vector one and combine the results rather than replacing one with the other. It is also why a specification that must be matched precisely belongs in a sentence with context around it, not alone in a table cell where it has nothing to be near. The hybrid arrangement has a name in the literature and several in the marketplace, and the detail worth checking when somebody sells you one is which half breaks the tie.

A system that ranks by vector similarity and uses keywords only as a filter behaves very differently from one that does the reverse, and vendors rarely volunteer which they built. For a publisher the consequence is small and specific: write the exact string somewhere it has neighbours. A model number inside a sentence explaining what it is can be found by both halves of a hybrid system, and the same number alone in a specification table can reliably be found only by one.

Why does none of this appear in a search console?

Because the index belongs to somebody else. Vector retrieval inside an assistant runs on an index the vendor built, using an embedding model the vendor chose, and none of them report which of your passages were embedded, retrieved or discarded. Google reports impressions and clicks for its own surfaces and nothing at all about the vector step underneath them.

So the mechanism this entry describes is one you can write for and cannot observe, which is worth stating plainly before anybody buys a tool claiming otherwise. What can be observed is the outcome: whether you appear in answers, measured across a panel of prompts over time.

Related terms

• EmbeddingAn embedding is a list of numbers representing a piece of text, positioned so that texts with similar meaning sit close together.
• Semantic SearchSemantic search retrieves results by meaning rather than by matching the words a searcher typed.
• RAG (Retrieval-Augmented Generation)RAG, or retrieval-augmented generation, is the technique of fetching documents at question time and giving them to a language model to answer from, rather than relying on what the model learned in training.
• ChunkingChunking is the step that splits a page into passages so a retrieval system can select one without the rest.