The central idea
Coordinates are rarely meaningful by themselves. Relative direction and neighborhood structure are what downstream systems use.
Meaning becomes useful geometry.Interactive machine-learning explainer
Why similar meanings cluster in embedding space
Rotate the space, select a concept, inspect cosine neighbors, run a retrieval query, and move “bank” between financial and river contexts. These are synthetic teaching vectors, not output from a production model.
Mode
rivermoney
View
An embedding turns an item into a direction with useful neighbors
An embedding model maps text, images, users, products, or other objects into numerical vectors. The goal is not to store a dictionary definition in each coordinate. The goal is to arrange directions so a distance or similarity function becomes useful for a downstream task.
In this lab, every labeled sphere has a synthetic three-dimensional vector. Terms from the same teaching category point in similar directions, so cosine similarity ranks them as neighbors. Real embedding systems often use hundreds or thousands of dimensions, and their geometry is learned from data rather than hand-placed clusters.
Select “dog” and the explorer connects it to nearby animal concepts. Switch to Retrieval and the selected sphere becomes a query vector: the system scores every stored item, sorts by similarity, and displays the top k. This is the core geometric operation behind many semantic search and retrieval-augmented generation systems.
Switch to Context and move “bank” between two neighborhoods. A single static word vector struggles with polysemy because “bank” can refer to finance or a river edge. Contextual language models produce token representations influenced by surrounding words, so the same token can land in different regions for different sentences.
The scene is intentionally honest about its limits. Three dimensions make relationships visible, but projecting a high-dimensional space into two or three dimensions can invent, compress, or hide neighborhoods. Use a visualization to form questions, then test those questions against the original vectors and task metrics.
Similarity is task-shaped
A model trained for sentence retrieval may group paraphrases. A product model may group substitute items. “Similar” reflects objectives, examples, and negative pairs.
Dense representation
A vector stores distributed features across many coordinates rather than assigning one human-readable meaning to each dimension.
Neighborhood query
Nearest-neighbor search retrieves items whose vectors score highly against a query under cosine similarity, dot product, or another metric.
Context changes position
Modern contextual embeddings represent a token in its sentence, which helps separate senses that a single word-level vector would merge.
What the explorer computes
Normalize direction
The lab normalizes a copy of each vector before computing cosine similarity. Point size and distance from the origin do not directly increase the score.
Score every candidate
For clarity, the browser compares the query with every teaching vector. Large systems use approximate nearest-neighbor indexes to avoid exhaustive search while preserving high recall.
Sort and take top k
The neighbor slider controls how many high-scoring items remain visible. Increasing k improves recall but can introduce weakly related context and consume a retrieval budget.
Inspect, do not worship, the score
A high geometric score does not prove factual relevance, freshness, authority, or permission to act. Retrieval quality also depends on chunking, filters, metadata, reranking, and evaluation examples.
From a query to retrieved context
Input
A user question, document, image, or product request enters the system with optional metadata and policy scope.
Raw intentEmbed
The same embedding model maps the query and candidate items into a compatible vector space.
Comparable vectorsSearch
An index finds high-scoring neighbors, often with metadata filters for source, user, time, or permissions.
Candidate recallRerank
A stronger scorer can compare the query and candidates jointly, trading extra latency for better ordering.
Precision passUse evidence
The application presents sources or passes selected context to a model while preserving attribution and access controls.
Grounded responseWhere a beautiful projection can mislead
Projection distortion
Real embeddings occupy many dimensions. PCA, t-SNE, UMAP, and learned projections preserve different properties. A clean 3D gap may be a projection artifact, while important high-dimensional separation may disappear.
Similarity is not truth
Nearby text can share topic, style, sentiment, or wording while disagreeing factually. A retrieval system needs source quality, freshness, authority, and task-specific relevance checks.
Global maps hide local behavior
A model can look organized overall and still fail on the exact queries a product serves. Evaluate neighborhoods with real examples, hard negatives, subgroup slices, and downstream outcomes.
Coordinates can encode bias
Training data and objectives shape the space. Sensitive associations, language coverage, domain imbalance, and representation quality need direct audits rather than visual reassurance.
How this geometry shows up in personal agents
A personal agent can embed messages, task receipts, browser observations, and user-approved documents so a new request can retrieve relevant prior evidence. The value comes from narrowing a search space, not from treating vector memory as unquestionable truth.
For a text-message AI assistant, embeddings can connect a terse follow-up such as “what happened with that order?” to an earlier task receipt even when the wording differs. Identity, time, and conversation scope still need structured filters.
A computer-use cache can use vectors to retrieve observations related to a target or task. Freshness and provenance remain separate dimensions: a semantically similar cached page may be stale or belong to another account.
When an AI website-building agent retrieves design requirements, deployment notes, and prior decisions, reranking can prioritize the evidence most directly tied to the current edit. The final action should still be checked against current files and live deployment state.
Super connects these retrieval ideas to messaging and computer-use workflows where the agent needs useful memory without losing source, scope, or outcome verification.
Use the same compatible embedding model for queries and candidates.
Choose a similarity metric that matches training and index configuration.
Filter by user, permissions, source type, and freshness before or during search.
Evaluate recall with real queries and known relevant items.
Include hard negatives that look similar but are wrong for the task.
Rerank when top-k cosine order is not precise enough.
Preserve source attribution through retrieval and generation.
Measure latency, index recall, relevance, and downstream task quality separately.
Frequently asked questions
Are the points generated by a real embedding model?
No. They are deterministic synthetic teaching vectors arranged to make cosine neighborhoods, retrieval, and context movement easy to inspect. The lab never claims they are measurements from a production model.
Why use cosine similarity?
Cosine similarity compares vector direction and is common when magnitude should not dominate. The correct metric depends on how the embedding model was trained and how the index expects vectors to be normalized.
Why does “bank” move?
The slider illustrates contextual representation. In “the bank approved a loan,” surrounding tokens favor a financial sense. In “we sat on the river bank,” context favors a geographic sense. Modern transformers produce context-dependent token states.
Do real embeddings form clean clusters?
Sometimes locally, but not as neatly as a teaching diagram. Spaces can contain overlapping topics, anisotropy, hubs, language differences, and task-specific structure. A projection adds further distortion.
Is nearest-neighbor search enough for RAG?
No. Strong retrieval may include query rewriting, chunking, metadata filters, hybrid lexical search, reranking, source-quality controls, freshness checks, and evaluation against task-specific relevant documents.
What should I measure?
At the retrieval layer, measure recall at k, precision, ranking metrics, latency, and index behavior. At the product layer, measure whether the retrieved evidence improves accurate, attributable outcomes for real user tasks.
Primary references
Efficient Estimation of Word Representations in Vector SpaceMikolov and colleagues on learning distributed word representations.
Sentence-BERTReimers and Gurevych on semantically meaningful sentence embeddings for similarity search.
Attention Is All You NeedThe transformer architecture underlying modern contextual representations.
Visualizing Data Using t-SNEA primary reference on nonlinear dimensionality reduction and visualization.
UMAPA dimension-reduction method with different local and global tradeoffs.
Three.js DocumentationThe WebGL rendering library used for this interactive teaching scene.
Explore the neighborhood. Verify the task.
Embedding geometry is powerful when it narrows evidence for a real workflow and stays connected to source, scope, freshness, and evaluation.
Explore Super