Embeddings: what they are and their applications

We know that with the emergence of various technologies, there is a large increase in the number of terms we hear about, embeddings is one of them, but what are they?

embeddings, which in English means “to embed”, is a term used in AI and Natural Language Processing (NLP). It refers to the process of “embedding” or “embedding” complex information (such as words, sentences or documents) into a vector space.

This means that data that would be difficult to process directly is transformed into a numerical form (vectors), which the models Machine Learning can understand and use for tasks such as classification and semantic analysis.

When combined with vector databases, they enable systems to analyze large volumes of unstructured data. This allows for the extraction of relevant information and complex queries quickly and effectively.

This data transformation technique is essential in building scalable solutions, as the vector representation facilitates the search and recovery of information, in addition to compressing the information and still maintaining the relationship with its original content.

How works

We are aware that embeddings are vectors for machine understanding based on texts, phases, documents. But how do we transform this information into vectors?

Vectors are formed by using AI models trained to identify contexts, classifying them based on the approximation of the context in numbers, which typically range from -1 to 1. The value 1 indicates the closest proximity, with thousands of comparison parameters.

These models are typically trained on large volumes of text, identifying patterns of competition between words that frequently appear in similar contexts, such as “cat” and “animal.” During training, the model learns to map these words to numeric vectors in a multidimensional space, so that words with related meanings or similar contexts are positioned closer to each other in this vector space.

The goal is to make words or phrases with similar meanings closer together in the “space” of the vectors. For example, “cat” and “dog” should be represented by close vectors, while “cat” and “car” will be further apart.

Embedding example | Image: https://arize.com/blog-course/embeddings-meaning-examples-and-how-to-compute/

How is the similarity between two vectors calculated, comparing, for example, a text with several vectors from the trained model?

Mathematically, the cosine similarity technique is normally used to compare two vectors. Cosine similarity provides a value in the range [-1,1], with 1 being the closest context value and -1 the furthest [1]

Cosine similarity equation | Image: Wikipedia

Two vectors with 98% similarity based on the cosine of the angle between the vectors | Image: Richmond Alake

embeddings, in practice

PDF Analysis with QA (Question Answering): embeddings are used in document analysis systems, such as PDFs, to perform Question and Answer (QA) tasks. Companies that deal with large volumes of documents, such as contracts or reports, can use embeddings to automatically locate relevant passages in a text. For example, when analyzing a PDF contract, the embeddings allow you to semantically map content and identify passages related to questions such as “What is the validity period of this contract?” or “What are the customer’s payment obligations?” A generative AI model can then use these passages to interpret the context and generate natural language responses with greater accuracy.

Product Recommendation (E-commerce): Platforms like Amazon and Netflix use embeddings to recommend products or movies based on users' preferences and past behaviors. For example, when recommending movies, embeddings are used to capture the style, genre and characteristics of the films the user has watched, suggesting new content based on vector similarity.

Sentiment Analysis (Customer Service): Companies use embeddings to analyze sentiment in customer feedback or messages. For example, when analyzing a set of social media comments or customer emails, embeddings help to automatically identify whether the sentiment is positive, negative or neutral, allowing a quick and appropriate response.

Conclusion

embeddings have proven to be a powerful and growing tool in several industries, transforming the way we interact with unstructured data. Their ability to represent complex information numerically has led to improvements in document analysis systems, recommendations and even customer service.

As a technology that is constantly evolving, it is expected that, over time, it will be increasingly integrated into intelligent and scalable solutions. Furthermore, with the trend towards reducing computing costs and the advancement of processing and storage infrastructures, it is becoming increasingly feasible to scale these solutions efficiently and at low cost.

References

https://builtin.com/machine-learning/cosine-similarity#:~:text=Cosine%20similarity%20is%20a%20measurement,within%20an%20inner%20product%20space

https://arize.com/blog-course/embeddings-meaning-examples-and-how-to-compute