Embeddings

Video: Watch this lesson

Before we can do vector search, we need to turn our text into vectors. We call this process embedding: we embed text into a vector space. The vectors we get back are also called "embeddings."

Word embeddings and sentence embeddings

This idea comes from word2vec. The model learns to place words as points in a multi-dimensional space. Words with similar meanings land close to each other.

Imagine a 2D space where "enroll" and "join" are near each other and "Docker" is far away:

        · enroll
       · join
                   · Docker

The same idea works for entire sentences:

Q1: "I just discovered the course. Can I still join it?"
Q2: "I just found out about the program. Can I still enroll?"

These two are close - they mean the same thing.

Q3: "How do I run Docker on Windows?"

This one is far away from Q1 and Q2.

Now imagine all 1200 documents in our FAQ dataset. Each one becomes a point in this space. When a user asks a question, we embed it into the same space and find the closest documents. Those nearest neighbors are our search results.

The model encodes the whole sentence, not the words in isolation. So it can tell apart the same word in different contexts.

Take the word "judge." In "the judge ruled out the possibility of crime" (legal) it gets one vector. In "LLM-as-a-judge approach to evaluate LLMs" (ML evaluation) it gets a different one. The surrounding context changes the embedding.

So an embedding model takes text in and returns a fixed-length array of numbers. We train it so that texts with similar meanings get similar vectors.

We'll use sentence-transformers, a popular open-source library for embeddings. It runs locally on your machine, so there are no API costs.

Install sentence-transformers

Recent versions of sentence-transformers install the GPU-enabled PyTorch package, which pulls large NVIDIA CUDA libraries. It takes a few gigabytes of disk space, even if you don't have a GPU.

To avoid this, you need to explicitly tell uv to install the CPU-only PyTorch wheels.

Add a CPU-only PyTorch index to pyproject.toml:

[tool.uv.sources]
torch = { index = "pytorch-cpu" }
name = "pytorch-cpu"
url = "https://download.pytorch.org/whl/cpu"

Install the library:

uv add sentence-transformers

If your pyproject.toml already specifies the requests package, make sure it is pinned to requests>=2.28.1, because pytorch-cpu requires this version to install correctly.

In this way, uv will automatically resolve PyTorch using the CPU-only index.

We will also see how to trim it down even more for using in production in the ONNX Embedder lesson later.

Choosing a model

Sentence-transformers supports many models. The right one depends on your task, your language, and the resources you have. Larger models are usually slower, so for our FAQ dataset of short English texts a small model is enough. Try a few on your own data and keep the one that works best.

We'll use all-MiniLM-L6-v2:

• 384-dimensional vectors (compact)
• Fast on CPU
• Good quality for general English text
• Uses cosine similarity (we'll explain this below)

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

The first time you run this, it downloads the model (~80 MB) and the tokenizer from HuggingFace. The tokenizer turns text into something the model can read. After that, both load from a local cache.

Trying it with simple examples

Let's see how embeddings work on a few examples.

We'll start with a query:

q1 = "Can I still join the course after the start date?"
v1 = model.encode(q1)

v1 is a vector, an array of 384 numbers. Each number stands for some concept the model learned. We can't read off what any one of them means. But two vectors with similar values point to texts about similar things.

Encode our document:

d  = "You don't need to register. You're accepted. You can also just start learning and submitting homework without registering."
dv = model.encode(d)

Next, we compare the query against the document using dot product:

v1.dot(dv)

We get 0.32.

Now we try an unrelated query:

q2 = "How to install Docker on Windows?"
v2 = model.encode(q2)

This time the similarity with the document should be much smaller:

v2.dot(dv)

And we get 0.01.

The first score for q1 vs d (0.32) is higher, so that query is more similar to the document about registration. The second score for q2 vs d sits near 0, because installing Docker has nothing to do with registration. A score near 0 means the two vectors are about as different as they can be.

That's the whole idea behind vector search: similar texts get similar vectors, and a dot product tells us how similar.

Cosine similarity

The all-MiniLM-L6-v2 model outputs normalized vectors - vectors with unit length. When both vectors are normalized, the dot product equals cosine similarity. That's why the model documentation says it "uses cosine similarity."

Cosine similarity measures the angle between two vectors, ignoring their length:

• 1.0 = same direction (similar)
• 0.0 = perpendicular (unrelated)
• -1.0 = opposite direction (opposite meaning)

Formally, if theta is the angle between two vectors, cosine similarity is cos(theta):

• cos(0) = 1 - vectors point in the same direction
• cos(90) = 0 - vectors are perpendicular
• cos(180) = -1 - vectors point in opposite directions

Because our vectors are normalized, the dot product gives us cosine similarity directly. This is why we can use v1.dot(dv) to compare texts.

In practice, we rarely get cosine similarity below 0. The embedding model maps text to a region of the vector space where most vectors have positive components. There's no concept of "opposite meaning" that maps to a vector pointing the other way.

← What is Vector Search | Embedding Our Dataset →