What are Vector Embeddings? High-Dimensional Semantic Mathematics Explained

At the core of modern semantic search, Retrieval-Augmented Generation (RAG), recommendation engines, and vector databases lies the concept of Vector Embeddings.

An embedding model (such as text-embedding-3-large, BGE-M3, or e5-mistral-7b) maps unstructured human data—such as text, source code, images, or audio—into a dense, high-dimensional vector space ${\mathbb{R}}^D$ (where $D$ typically ranges from $768$ to $3,072$ dimensions).

In this coordinate space, semantic similarity is encoded directly as geometric proximity. Texts with similar conceptual meanings are positioned close to each other in Euclidean or Cosine space, regardless of whether they share any exact keyword vocabulary.

---

1. Dense Semantic Vectors vs Sparse Lexical Vectors

In information retrieval, representations fall into two major paradigms:

The Geometry of Semantic Vector Spaces

In a well-trained semantic space, directional vectors encode fundamental relational concepts:

${\vec{v}}_{\text{King}}-{\vec{v}}_{\text{Man}}+{\vec{v}}_{\text{Woman}}\approx {\vec{v}}_{\text{Queen}}$

${\vec{v}}_{\text{Paris}}-{\vec{v}}_{\text{France}}+{\vec{v}}_{\text{Germany}}\approx {\vec{v}}_{\text{Berlin}}$

${\vec{v}}_{\text{PostgreSQL}}-{\vec{v}}_{\text{Relational}}+{\vec{v}}_{\text{Document}}\approx {\vec{v}}_{\text{MongoDB}}$

---

2. Pooling Strategies: CLS Token vs Mean Pooling

Embedding models extract a single fixed-length vector $\vec{v}\in {\mathbb{R}}^D$ from a variable-length token sequence $[t_1,t_2,\dots ,t_N]$ using one of two pooling techniques:

A. Mean Pooling (Recommended for Modern Models)

Mean pooling averages the token hidden states across the sequence length, weighted by the attention mask:

${\vec{v}}_{\text{mean}}=\frac{{\sum }_{i=1}^N{m}_i\cdot {\mathbf{h}}_i}{{\sum }_{i=1}^N{m}_i}$

Where $m_i\in \{0,1\}$ is the attention mask bit and ${\mathbf{h}}_i\in {\mathbb{R}}^{d_{\text{model}}}$ is the output vector of the final transformer layer for token $i$. Mean pooling captures rich information from all tokens across the document.

B. CLS Token Pooling (Legacy BERT)

Takes the hidden state of the first special classification token:

${\vec{v}}_{\text{CLS}}={\mathbf{h}}_0$

---

3. Matryoshka Representation Learning (MRL)

A cutting-edge innovation in modern embeddings (used in OpenAI text-embedding-3 and modern open-source models) is Matryoshka Representation Learning (MRL) (Kusupati et al., 2022).

In standard embeddings, truncating a 1536-dimensional vector to 512 dimensions destroys vector geometry. MRL trains the model so that the first $K$ dimensions form an optimal, fully functional vector space on their own:

• Infrastructure teams can index only the first 256 or 512 dimensions in vector database RAM (saving 75% of memory costs and speeding up search by $4\times$), then re-score the top 50 candidates using the full 1536 dimensions.

---

4. Production Failure Modes: Token Truncation Blindspots

Failure Mode: Poor Semantic Retrieval Caused by Silent Document Truncation

• Symptom: In an enterprise RAG system, searching for specific legal disclaimers located at the bottom of long vendor agreements yields 0 relevant vector matches.
• Root Cause: The embedding model has an unconfigured max_sequence_length = 512 tokens. When 10-page documents were ingested, the embedding pipeline silently truncated everything after page 1, completely discarding the legal terms from the generated vector!
• Resolution: Implement hierarchical or chunked document ingestion (e.g. 500-token semantic chunks with 10% overlap) before generating vector embeddings, and verify embedding model context limits.

---

5. Summary & Key Takeaways

1. Embeddings Map Concepts to Geometry: Vector embeddings encode unstructured semantic meaning as high-dimensional coordinates in ${\mathbb{R}}^D$.
2. Dense vs Sparse: Dense vectors excel at conceptual similarity and synonyms; sparse vectors (BM25) excel at exact keywords, serial numbers, and code identifiers.
3. Mean Pooling Captures Global Context: Averaging token hidden states preserves semantic nuances across the entire document chunk.
4. Matryoshka Embeddings Enable Tiered Search: MRL allows truncating dimensions (e.g. 1536 to 256) to slash vector database RAM costs by 75% with minimal recall degradation.