Embedding

What are the numbers, concretely?

A fixed-length list of floating point values, and the length is a property of the model rather than of your text. OpenAI’s current models produce 1,536 values for text-embedding-3-small and 3,072 for text-embedding-3-large, whether the input is one word or a page. Every passage becomes a point in that space, and retrieval becomes a search for near points. The values themselves are not interpretable.

No single dimension means “about pricing”, and there is no way to read a number out of the list and act on it: meaning lives in the geometry of the whole vector, which is why an embedding cannot be edited or audited the way a keyword list can. Two consequences follow for anyone trying to influence retrieval. The model producing the embedding is chosen by the engine rather than by you, so the same page sits in a different space for every assistant. And a passage’s position is fixed by what it says, so there is no lever that moves it closer to a query without changing the words.

Why does this end the era of matching words?

Because distance in embedding space tracks meaning rather than spelling. A page about “how often assistants name our brand” can be retrieved for “am I visible in ChatGPT” without sharing a single content word, and a page stuffed with an exact phrase gains nothing from the repetition. That is the mechanism underneath every measurement in this glossary that says keyword density does not work.

It also explains why sub-queries matter more than head terms. The engine embeds the search it generated, not the words your reader typed, so the text worth writing is the text that answers the question rather than the text that repeats it.

Where does an embedding lose precision?

Precision on anything the model treats as interchangeable. Product codes, version numbers, exact prices and rare names are the classic casualties: two strings that differ in one digit can land close together, which is why retrieval systems that need exact matching keep a keyword index alongside the vector one rather than replacing it.

The practical version for a page: spell out the thing that must be matched exactly, in text, near words that give it context. A version number alone in a table cell is the hardest kind of fact for this machinery to return correctly. The same holds for anything a reader would want to copy: if it matters, give it a sentence.

Related terms

• ChunkingChunking is the step that splits a page into passages so a retrieval system can select one without the rest.
• RAG (Retrieval-Augmented Generation)RAG, or retrieval-augmented generation, is the technique of fetching documents at question time and giving them to a language model to answer from, rather than relying on what the model learned in training.
• GroundingGrounding is the step that ties a model’s answer to sources it retrieved, so the reply can cite them rather than assert them.
• Query Fan-OutQuery fan-out is Google’s own term for how AI Overviews and AI Mode answer a question: the engine breaks the question into subtopics and issues many searches at once, then writes one answer from the results.