Embeddings & Vector Search

What it is

An embedding is a list of numbers that captures the meaning of a piece of content. Similar content produces nearby vectors, so "reset my password" and "I can't log in" land close together even though they share no words. Embeddings power semantic search, the retrieval step in RAG, recommendations, clustering, deduplication and anomaly detection.

Vector search finds the nearest vectors to a query quickly, even across hundreds of millions of items, using approximate nearest-neighbor indexes. The engineering questions are which embedding model to use, how to store and filter vectors, and how to balance recall, latency and cost.

How it works

1. Choose a model

   Pick an embedding model suited to your language, domain and content type (text, code, images).

2. Embed content

   Convert each item or chunk into a vector and store it with its ID and metadata.

3. Build an index

   Create an approximate nearest-neighbor index such as HNSW or IVF for fast lookup.

4. Query

   Embed the query with the same model and retrieve the closest vectors, applying metadata filters.

5. Tune

   Measure recall and latency; adjust index parameters, dimensions, quantization and hybrid weighting.

Key concepts

Cosine similarity

The usual measure of how close two vectors point; the basis of most semantic search.

Dimensions

The length of each vector. More dimensions can capture more nuance but cost more storage and compute.

ANN indexes

HNSW, IVF and product quantization trade a little accuracy for large speed gains.

Quantization

Compressing vectors (for example to 8-bit or binary) to cut memory with modest recall loss.

Metadata filtering

Restricting search by tenant, date, type or permissions alongside similarity.

Multi-vector retrieval

Late-interaction approaches such as ColBERT keep token-level vectors for sharper matching.

Re-embedding

Changing embedding models means re-processing the whole corpus — plan for it.

Tools & platforms

Skills employers look for

• Selecting and benchmarking embedding models on domain-specific data.
• Designing schemas that combine vectors, metadata and access control.
• Tuning ANN index parameters for recall, latency and memory.
• Implementing hybrid search with keyword and vector scores.
• Planning re-indexing, versioning and incremental updates.
• Choosing between a dedicated vector database and extending an existing database.

Enterprise use cases

Common pitfalls

• Adding a new database you don't need — pgvector or your existing search engine is often enough.
• Mixing embedding models — query and document vectors must come from the same model and version.
• Relying on vectors alone — exact identifiers and rare terms need keyword search too.
• Ignoring index cost — memory for large, high-dimensional indexes adds up quickly.