Word2Vec in NLP

Word2Vec is one of the most important and revolutionary word embedding techniques in Natural Language Processing (NLP). It was introduced by Google in 2013 and is used to convert words into dense numerical vectors that capture semantic and syntactic relationships between words. 

Unlike traditional text representation methods such as Bag of Words (BoW) and TF-IDF, Word2Vec learns the meaning of words based on their surrounding context. 

Word2Vec is widely used in: 

Word2Vec is a shallow neural network-based embedding model that transforms words into dense vector representations.The main objective of Word2Vec is:Words with similar meanings should have similar vector representations. 

For example: This demonstrates that Word2Vec captures semantic relationships between words. 

Why Word2Vec is Important 

Traditional methods such as BoW generate sparse vectors and cannot understand semantic meaning. 

Example: 

BoW treats them as unrelated words. 

Word2Vec learns their semantic relationship using vector space representations. 

Working Principle of Word2Vec 

Word2Vec learns embeddings by analyzing neighboring words in sentences. 

The main idea: 

Context defines meaning. 

Example sentence: 

"The cat sits on the mat" 

Context words around “sits” help the model understand its meaning. 

CBOW (Continuous Bag of Words) 

CBOW predicts the missing target word using surrounding context words. 

Example of CBOW 

Sentence: 

Input context: 

Prediction: 

CBOW is generally: 

CBOW Objective Function 

Skip-Gram Model 

Skip-Gram works opposite to CBOW. It predicts surrounding context words using the target word. 

Example of Skip-Gram 

Input word: 

Predicted context: 

Skip-Gram is generally: 

Skip-Gram Objective Function 

Neural Network Structure of Word2Vec 

Word2Vec uses a shallow neural network with: 

The hidden layer weights become the final word embeddings. 

Word2Vec Prediction Formula 

One-Hot Encoding Problem 

Before Word2Vec, words were represented using one-hot vectors. 

Example vocabulary: 

One-hot vectors: 

Problems: 

Word2Vec solves these issues using dense embeddings. 

Dense Vector Representation 

Word2Vec generates dense low-dimensional vectors. 

Example: 

Similar words have similar vectors. 

Semantic Relationships in Word2Vec 

Word2Vec captures semantic relationships mathematically. 

Example: This is called vector arithmetic. 

Example Problem 

Sentence 1: 

"I went to the bank" 

Sentence 2: 

"The river bank is beautiful" 

Word2Vec generates the same vector for “bank” in both sentences. 

Practical Implementation Using Python 

Problem Statement 

Train a Word2Vec model on sample sentences and generate word embeddings. 

Install Libraries 

``` 

Hide 

pip install gensim 

``` 

Example: 

``` 

#Import library 

from gensim.models import Word2Vec 

#Sample sentences 

sentences = [ 

["i", "love", "nlp"], 
 
["machine", "learning", "is", "powerful"], 
 
["deep", "learning", "improves", "ai"], 
 
["nlp", "uses", "machine", "learning"], 
 
["artificial", "intelligence", "is", "future"] 
 

] 

#Train Word2Vec model 

model = Word2Vec( 

sentences, 
 
vector_size=100, 
 
window=5, 
 
min_count=1, 
 
workers=4 
 

) 

#Display vocabulary 

print("Vocabulary:\n") 

print(model.wv.index_to_key) 

#Display vector for a word 

print("\nWord Vector for 'learning':\n") 

print(model.wv['learning']) 

#Find similar words 

print("\nMost Similar Words to 'learning':\n") 

print(model.wv.most_similar('learning')) 

``` 

Finding Similar Words 

Word2Vec can identify semantically similar words. 

Example: 

``` 

Hide 

print(model.wv.most_similar('machine')) 

``` 

Cosine Similarity in Word2Vec 

Similarity between words is measured using cosine similarity. 

Formula: 

Higher cosine similarity means words are more semantically related. 

Conclusion 

Word2Vec is one of the foundational embedding techniques in Natural Language Processing that revolutionized text representation by introducing dense semantic word embeddings. Unlike traditional sparse representations, Word2Vec captures relationships between words using neural network-based learning approaches such as CBOW and Skip-Gram. Although modern contextual embedding models such as BERT and GPT have surpassed Word2Vec in performance, Word2Vec remains an essential concept for understanding semantic embeddings and the evolution of modern NLP systems.