Practical Example how we use embedding in python

How to Create Embeddings in Python: A Beginner-Friendly Guide

When I first came across embeddings, the idea sounded more complicated than it actually is.

Terms like vectors, dimensions, and semantic similarity can make embeddings feel intimidating.

But the basic idea is quite simple:

An embedding converts text into a list of numbers that represents its meaning.

"Python is easy to learn"
        ↓
Embedding Model
        ↓
[0.21, -0.14, 0.67, 0.32, ...]

That list of numbers is called a vector.

Once text is converted into vectors, we can compare those vectors and find text that has a similar meaning.


What Are We Going to Build?

In this example, we will:

  1. Install an embedding model
  2. Create some text
  3. Convert the text into embeddings
  4. Convert a user's question into an embedding
  5. Compare the embeddings
  6. Find the most similar text

We will use the sentence-transformers library.


Step 1: Install the Library

First, install the required packages:

pip install sentence-transformers numpy

Step 2: Load an Embedding Model

from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

Here we are loading a pre-trained embedding model.

The model takes text as input and produces a vector as output.


Step 3: Create Some Text

Let's create a small collection of documents:

documents = [
    "React is a JavaScript library for building user interfaces.",
    "Python is widely used for data analysis and machine learning.",
    "MongoDB is a NoSQL database.",
    "Redis is commonly used for caching data."
]

These could represent documents, paragraphs, product descriptions, FAQs, or chunks from a PDF.


Step 4: Generate Embeddings

Now we convert our documents into vectors:

embeddings = model.encode(documents)

We can check the result:

print(embeddings.shape)

You will get something similar to:

(4, 384)

What does this mean?

4 documents
   ↓
Each document
   ↓
384 numbers

So each document has been represented by a vector containing 384 numbers.

For example, one vector might look something like:

[0.021, -0.143, 0.562, 0.091, ...]

The actual values are generated by the model.


Step 5: Create an Embedding for the User's Question

Now imagine a user asks:

"What can I use to build a web interface?"

We also convert this question into an embedding:

query = "What can I use to build a web interface?"

query_embedding = model.encode(query)

Now we have:

Documents
    ↓
Document embeddings

User question
    ↓
Query embedding

The important part is that both the documents and the question are represented as vectors.


Step 6: Compare the Vectors

Now we want to know:

Which document is most similar to the user's question?

One common method for this is Cosine Similarity.

We can calculate it using NumPy:

import numpy as np

def cosine_similarity(a, b):
    return np.dot(a, b) / (
        np.linalg.norm(a) * np.linalg.norm(b)
    )

Now compare the query with every document:

scores = []

for i, embedding in enumerate(embeddings):

    score = cosine_similarity(
        query_embedding,
        embedding
    )

    scores.append((score, documents[i]))

Step 7: Sort the Results

We can sort the results from highest similarity to lowest:

scores.sort(reverse=True)

Then print them:

for score, document in scores:
    print(f"{score:.4f} -> {document}")

The result might look something like:

0.72 -> React is a JavaScript library for building user interfaces.
0.31 -> Python is widely used for data analysis and machine learning.
0.18 -> Redis is commonly used for caching data.
0.12 -> MongoDB is a NoSQL database.

The exact numbers can vary depending on the model and input.

The important thing is that the React document is ranked highest because its meaning is most related to the question.


Complete Python Code

Here is the complete example in one place:

import numpy as np
from sentence_transformers import SentenceTransformer


# Load embedding model
model = SentenceTransformer("all-MiniLM-L6-v2")


# Documents
documents = [
    "React is a JavaScript library for building user interfaces.",
    "Python is widely used for data analysis and machine learning.",
    "MongoDB is a NoSQL database.",
    "Redis is commonly used for caching data."
]


# Create embeddings for documents
embeddings = model.encode(documents)


# User query
query = "What can I use to build a web interface?"


# Create embedding for query
query_embedding = model.encode(query)


# Cosine similarity
def cosine_similarity(a, b):
    return np.dot(a, b) / (
        np.linalg.norm(a) * np.linalg.norm(b)
    )


# Compare query with documents
results = []

for i, embedding in enumerate(embeddings):

    score = cosine_similarity(
        query_embedding,
        embedding
    )

    results.append((score, documents[i]))


# Sort by similarity
results.sort(reverse=True)


# Display results
for score, document in results:
    print(f"{score:.4f} -> {document}")

What Did We Actually Do?

The whole process can be simplified to:

Document
   ↓
Embedding Model
   ↓
Vector

For the user:

Question
   ↓
Embedding Model
   ↓
Query Vector

Then:

Query Vector
     ↓
Compare with
     ↓
Document Vectors
     ↓
Similarity Score
     ↓
Most Similar Document

That's the basic idea behind semantic search.


But What About Large Applications?

The example above compares the query with every document:

Query
 ↓
Document 1
Document 2
Document 3
Document 4
...

This is fine for learning and small datasets.

But imagine having millions of documents.

Comparing the query with every single vector would not be the ideal approach.

This is where vector databases and efficient vector-search algorithms become useful.

For example:

  • Qdrant
  • Pinecone
  • Weaviate
  • Chroma
  • pgvector

They can store embeddings and perform similarity searches efficiently.


How Embeddings Fit Into RAG

This is also where embeddings become extremely useful in RAG systems.

Suppose you have a large PDF.

The typical process is:

PDF
 ↓
Split into chunks
 ↓
Create embeddings
 ↓
Store vectors
 ↓
Vector Database

When the user asks a question:

User Question
      ↓
Create Query Embedding
      ↓
Vector Search
      ↓
Find Relevant Chunks
      ↓
Send Chunks to LLM
      ↓
Generate Answer

So embeddings are one of the building blocks that make semantic retrieval possible.


Embeddings Are Not the Answer

This is an important point to understand.

An embedding does not directly give you the answer.

It represents text as numbers so that we can compare its meaning with other text.

Think of the responsibilities like this:

Embedding Model
    ↓
Convert text into vectors

Vector Search
    ↓
Find similar information

LLM
    ↓
Generate the final answer

Final Takeaway

If you are just starting with embeddings, don't worry too much about the mathematical details initially.

Focus on understanding this flow:

Text
 ↓
Embedding
 ↓
Vector
 ↓
Similarity Comparison
 ↓
Relevant Information

Once this becomes clear, concepts like Vector Search, Vector Databases, and RAG become much easier to understand.

And that's the main reason embeddings are so important in modern AI applications.

Comments

Popular Posts