Bag of Words (BoW) in NLP

Bag of Words (BoW) is one of the simplest and most fundamental text representation techniques used in Natural Language Processing (NLP). It converts textual data into numerical vectors so that machine learning algorithms can process and analyze the text. 

The Bag of Words model represents a document based on the frequency of words present in it, without considering grammar, sentence structure, or word order. 

It is widely used in: 

The Bag of Words model treats a document as a collection (“bag”) of words. 

It focuses only on: 

It ignores: 

Example 

Sentence 1: 

Sentence 2: 

Bag of Words treats both sentences as identical because the same words are present. 

Working Principle of Bag of Words 

The Bag of Words model works in the following stages. 

Step 1: Text Collection 

Example documents: 

Document 1: "I love NLP" 
 
Document 2: "I love Machine Learning" 

Step 2: Text Preprocessing 

Common preprocessing operations include: 

Example: 

Before preprocessing: 

"I LOVE NLP!!!" 

After preprocessing: 

["love", "nlp"] 

Step 3: Build Vocabulary 

Vocabulary contains all unique words from the dataset. 

Vocabulary: 

["i", "love", "nlp", "machine", "learning"] 

Step 4: Convert Text into Vectors 

Each document is represented as a frequency vector. 

This matrix is called the Document-Term Matrix (DTM). 

2.3.3 Mathematical Representation of BoW 

Suppose: 

Example Vector 

Sentence: 

Vector: 

Types of Bag of Words 

1. Binary Bag of Words 

Represents only presence or absence. 

Example: 

2. Frequency Bag of Words 

Represents word count frequencies. 

Example: 

Vector: 

Applications of Bag of Words 

CountVectorizer in Scikit-Learn 

In Python, Bag of Words is commonly implemented using: 

``` 

Hide 

CountVectorizer 

``` 

from Scikit-learn. 

CountVectorizer automatically: 

Practical Implementation Using Python 

Problem Statement 

Convert text documents into Bag of Words vectors. 

``` 

#Import library 

from sklearn.feature_extraction.text import CountVectorizer 

#Sample documents 

documents = [ 

"I love NLP", 
 
"NLP is very interesting", 
 
"I love machine learning", 
 
"Machine learning and NLP are powerful" 
 

] 

#Create CountVectorizer object 

vectorizer = CountVectorizer() 

#Convert text into Bag of Words vectors 

X = vectorizer.fit_transform(documents) 

#Print vocabulary 

print("Vocabulary:\n") 

print(vectorizer.get_feature_names_out()) 

#Print vector representation 

print("\nBag of Words Matrix:\n") 

print(X.toarray()) 

``` 

Binary Bag of Words Example 

``` 

from sklearn.feature_extraction.text import CountVectorizer 

documents = [ 

    "I love NLP", 

    "NLP is amazing" 

] 

# Binary vectorization 

vectorizer = CountVectorizer(binary=True) 

X = vectorizer.fit_transform(documents) 

print(vectorizer.get_feature_names_out()) 

print(X.toarray()) 

``` 

Conclusion 

Bag of Words is one of the most fundamental and widely used text representation techniques in Natural Language Processing. It converts textual documents into numerical vectors based on word frequency and forms the basis for many classical machine learning algorithms. Although it ignores semantic meaning and contextual relationships, Bag of Words remains an important introductory technique due to its simplicity, speed, and effectiveness in basic NLP tasks.