Sentence Embeddings in NLP (SBERT & USE)

Sentence Embeddings are advanced text representation techniques in Natural Language Processing (NLP) used to convert entire sentences, paragraphs, or documents into dense numerical vectors. Unlike word embeddings that represent individual words, sentence embeddings capture the semantic meaning of complete sentences. 

Sentence embeddings are widely used in: 

The two most popular sentence embedding models are: 

These models significantly improved semantic understanding in NLP systems. 

What are Sentence Embeddings? 

Sentence embeddings are dense vector representations of complete sentences. 

Example: 

Sentence 1: 

"I love Natural Language Processing" 

Sentence 2: 

"NLP is my favorite subject" 

Although the sentences use different words, they have similar meanings. 

Sentence embedding models generate vectors that place semantically similar sentences close together in vector space. 

SBERT (Sentence-BERT) 

SBERT was developed to improve sentence similarity tasks using BERT. 

Traditional BERT is computationally expensive for sentence comparison because each pair of sentences must be processed together. 

SBERT solves this by generating fixed-size sentence embeddings independently. 

Architecture of SBERT 

SBERT uses: 

Working Principle of SBERT 

Steps: 

Example 

Sentence: 

"Artificial Intelligence is transforming healthcare" 

Output vector: 

[0.24, 0.67, 0.91, ...] 

Pooling in SBERT 

Pooling converts token embeddings into a single sentence vector. 

Common pooling methods: 

SBERT Objective 

SBERT uses cosine similarity for semantic comparison. 

Formula 

Universal Sentence Encoder (USE) 

USE (Universal Sentence Encoder) was developed by Google. 

It generates embeddings for: 

USE is optimized for semantic similarity and transfer learning tasks. 

Deep Averaging Network (DAN) 

DAN averages word embeddings and passes them through dense neural layers. 

It is: 

Working Principle of USE 

Steps: 

Example 

Sentence: 

"Machine Learning is powerful" 

Embedding: 

[0.12, 0.89, 0.56, ...] 

Sentence Similarity 

Sentence embeddings are commonly used for similarity comparison. 

Example: 

Sentence 1: 

"I love AI" 

Sentence 2: 

"I enjoy Artificial Intelligence" 

These sentences receive high cosine similarity scores. 

Similarity Formula 

Higher similarity indicates closer semantic meaning. 

Practical Implementation Using SBERT 

Install Libraries 

``` 

Hide  

pip install sentence-transformers 

``` 

Example: 

``` 

Import library 

from sentence_transformers import SentenceTransformer from sklearn.metrics.pairwise import cosine_similarity 

Load SBERT model 

model = SentenceTransformer('all-MiniLM-L6-v2') 

Sample sentences 

sentences = [ 

"I love Natural Language Processing", 
 
"NLP is very interesting", 
 
"Football is a popular sport" 
 

] 

Generate embeddings 

embeddings = model.encode(sentences) 

Print embedding shape 

print("Embedding Shape:\n") 

print(embeddings.shape) 

Calculate similarity 

similarity = cosine_similarity([embeddings[0]], [embeddings[1]]) 

print("\nSimilarity Score:\n") 

print(similarity) 

``` 

Practical Implementation Using USE 

Install Libraries 

``` 

pip install tensorflow tensorflow-hub 

``` 

Example: 

``` 

Import libraries 

import tensorflow_hub as hub from sklearn.metrics.pairwise import cosine_similarity 

Load USE model 

model = hub.load( 

"https://tfhub.dev/google/universal-sentence-encoder/4" 
 

) 

Sample sentences 

sentences = [ 

"Artificial Intelligence is powerful", 
 
"AI is transforming industries" 
 

] 

Generate embeddings 

embeddings = model(sentences) 

Similarity calculation 

similarity = cosine_similarity( 

[embeddings[0]], 
 
[embeddings[1]] 
 

) 

print("Similarity Score:\n") 

print(similarity) 

``` 

Conclusion 

Sentence embeddings represent one of the most important advancements in semantic representation for Natural Language Processing. Models such as SBERT and USE generate dense contextual vectors capable of capturing the semantic meaning of complete sentences. SBERT provides highly accurate semantic similarity using transformer-based architectures, while USE offers efficient and scalable sentence encoding solutions. These models play a critical role in modern AI systems including semantic search engines, conversational agents, recommendation systems, and intelligent information retrieval applications.