NumPy

Python tutorial · PySpark.in

NUMPY

NumPy (Numerical Python) is the fundamental Python library for numerical computing, used heavily in Data Science, Machine Learning, Deep Learning, and Scientific Computing.

It provides:

Fast computations – vectorized operations instead of loops.
Handles large datasets efficiently.
Array operations are memory and CPU optimized.
Integration with other libraries – Pandas, SciPy, Scikit-Learn.
Supports advanced math & linear algebra used in ML.

NumPy Installation

```

Hide

pip install numpy

```

Importing NumPy

```

import numpy as np

```

NumPy Array (ndarray)

The core of NumPy is the ndarray, a powerful multi-dimensional array object.

Creating Arrays

(A) From List

```

arr = np.array([1, 2, 3])

print(arr)

```

(B) 2D Array

```

arr2 = np.array([[1, 2, 3], [4, 5, 6]])

print(arr2)

```

Array Creation Functions

np.zeros() — Array of zeros

```

import numpy as np

x=np.zeros(2,3))

print(x)

```

np.ones() — Array of ones

```

import numpy as np

x=np.ones((3, 3))

print(x)

```

np.full() — Array with fixed value

```

import numpy as np

x=np.full((2,2), 7)

print(x)

```

np.eye() — Identity matrix

```

import numpy as np

x=np.eye(4)

print(x)

```

np.arange() — Like range()

```

import numpy as np

x=np.arange(0, 10, 2)

print(x)

```

np.linspace() — Evenly spaced numbers

```

import numpy as np

x=np.linspace(0, 1, 5)

print(x)

```

Random Arrays

```

import numpy as np

x=np.random.rand(3,3) # uniform distribution

y=np.random.randn(3,3) # normal distribution

z=np.random.randint(1,10, size=(2,3))

print(x)

print(y)

print(z)

```

Array Attributes

```

arr = np.array([[1,2,3],[4,5,6]])

print(arr)

```

Attribute	Meaning	Example Output
arr.shape	Dimensions	(2,3)
arr.ndim	Number of dimensions	2
arr.size	Total elements	6
arr.dtype	Data type	int32
arr.itemsize	Bytes per element	4

Indexing & Slicing

1D slicing

```

arr = np.array([10,20,30,40,50])

arr[1:4] # [20, 30, 40]

print(arr)

```

2D indexing

```

arr = np.array([[1,2,3],[4,5,6]])

arr[1,2] # 6

print(arr)

```

2D slicing

```

x=arr[:, 1] # entire column 1

y=arr[0, :] # first row

z=arr[0:2, 1:3] # sub-matrix

print(x)

print(y)

print(x)

```

Vectorized Operations (Fast)

NumPy does operations without loops, making it extremely fast.

```

arr = np.array([1,2,3])

x=arr + 5 # [6,7,8]

y=arr * 2 # [2,4,6]

z=arr ** 2 # [1,4,9]

p=arr / 2 # [0.5,1,1.5]

print(arr)

print(x)

print(z)

print(p)

```

Universal Functions (ufuncs)

Mathematical

```

x=np.sqrt(arr)

y=np.exp(arr)

z=np.log(arr)

p=np.sin(arr)

q=np.cos(arr)

r=np.tan(arr)

print(x)

print(y)

print(z)

print(p)

print(q)

print(r)

```

Statistical Functions

```

x=np.min(arr)

y=np.max(arr)

z=np.mean(arr)

p=np.median(arr)

q=np.std(arr)

r=np.var(arr)

s=np.sum(arr)

print(x)

print(y)

print(z)

print(p)

print(q)

print(r)

print(s)

```

Axis-wise operations

```

arr = np.array([[1,2],[3,4]])

x=np.sum(arr, axis=0) # column-wise

y=np.sum(arr, axis=1) # row-wise

print(arr)

print(x)

print(y)

```

Array Reshaping & Manipulation

reshape

```

x=arr.reshape(2,2) # This is its current shape, but it's a valid reshape

# For example, to reshape it to 4 rows and 1 column:

# arr.reshape(4,1)

# Or to 1 row and 4 columns:

# arr.reshape(1,4)

print(x)

```

ravel (flatten)

```

y=arr.ravel()

print(y)

```

transpose (T)

```

z=arr.T

print(z)

```

concatenate

```

arr1 = np.array([[1, 2], [3, 4]])

arr2 = np.array([[5, 6], [7, 8]])

z=np.concatenate((arr1, arr2), axis=0)

print(arr1)

print(arr2)

print(z)

```

stack

```

x=np.hstack((arr1, arr2))

y=np.vstack((arr1, arr2))

print(x)

print(y)

```

Copy vs View

view (shallow copy)

```

b = arr.view()

print(b)

```

Changes in b will reflects in arr.

copy (deep copy)

```

b = arr.copy()

print(b)

```

Changes in b will not affect arr.

Boolean Indexing

```

arr = np.array([10, 20, 30, 40])

x=arr[arr > 25] # [30, 40]

print(arr)

print(x)

```

NumPy Broadcasting

Broadcasting allows operations on arrays of different shapes.

Example

```

arr = np.array([1,2,3])

x=arr + 5 # adds 5 to every element

print(arr)

print(x)

```

2D + 1D

```

A = np.array([[1,2,3],[4,5,6]])

b = np.array([1,1,1])

x=A + b

print(A)

print(b)

print(x)

```

Linear Algebra (Important for ML)

Use np.linalg module.

Matrix multiplication

```

B = np.array([[7, 8], [9, 10], [11, 12]]) # Example B matrix

A @ B

x=np.dot(A, B)

print(B)

print(x)

```

Determinant

```

A_square = np.array([[1, 2], [3, 4]])

det_A = np.linalg.det(A_square)

print("Square Matrix A:\n", A_square)

print("Determinant of A:", det_A)

```

Inverse

```

inverse_A_square = np.linalg.inv(A_square)

print("Inverse of A_square:\n", inverse_A_square)

```

Eigenvalues & Eigenvectors

```

eigenvalues, eigenvectors = np.linalg.eig(A_square)

print("Eigenvalues of A_square:", eigenvalues)

print("Eigenvectors of A_square:\n", eigenvectors)

```

Random Module (Important for ML Models)

```

x=np.random.seed(42)

y=np.random.rand(3)

z=np.random.randint(1, 100, 10)

print(x)

print(y)

print(z)

```

Memory Efficiency (Why NumPy is Fast)

NumPy is fast because:

Uses contiguous memory
Written in C/C++
Avoids Python loops
Broadcasting removes loops automatically

Some Advanced NumPy

a. Fancy Indexing

```

arr = np.array([10,20,30,40])

x=arr[[0,3]] # [10,40]

print(arr)

print(x)

```

b. Sorting

```

x=np.sort(arr)

y=arr.argsort()

print(x)

print(y)

```

Unique values

```

X=np.unique(arr)

print(x)

```

NaN Handling

```

x=np.isnan(arr)

y=np.nanmean(arr)

z=np.nan_to_num(arr)

print(x)

print(y)

print(z)

```

Save/Load Arrays

```

x=np.save("data.npy", arr)

y=np.load("data.npy")

print(x)

print(y)

```

Masking and Conditional Operations

Replace values or create masks for analysisExample:

```

data = np.array([1, 2, 3, 4, 5])

mask = data % 2 == 0

data[mask] = 0

print(data) # [1, 0, 3, 0, 5]

```

Broadcasting in Data Science

Apply operations on arrays of different shapes
Example: Feature scaling

```

X = np.array([[1,2],[3,4],[5,6]])

X_scaled = (X - X.mean(axis=0)) / X.std(axis=0)

print(X)

```

Aggregations (axis-wise operations)

Important for column-wise statistics in datasetsExample:

```

data = np.array([[1,2,3],[4,5,6],[7,8,9]])

np.mean(data, axis=0) # mean per column

np.sum(data, axis=1) # sum per row

```

Working with Missing Data

Handle NaN values, common in real-world datasets

```

data = np.array([1, np.nan, 3, np.nan])

data[np.isnan(data)] = np.nanmean(data)

print(data)

```

Random Data Generation

Synthetic data creation for ML testing, simulations, bootstrapping

```

x = np.random.normal(loc=0, scale=1, size=(100, 5))

y = np.random.randint(0, 2, 100)

print(x)

print(y)

```

Linear Algebra & Matrix Operations

Fundamental for ML algorithms
Operations: dot product, transpose, inverse, pseudo-inverse, eigenvalues, SVD

```

x = np.array([[1,2],[3,4]])

y = np.array([5,6])

z=np.dot(X.T, X) # matrix multiplication

p=np.linalg.pinv(X) @ y # linear regression weights

print(x)

print(y)

print(z)

print(p)

```

Reshaping and Manipulating Arrays

Reshape, flatten, stack, concatenate, split arrays for feature engineering

```

arr = np.arange(12)

x=arr.reshape(3,4)

y=np.vstack([arr, arr])

print(arr)

print(x)

print(p)

```

Statistical Functions

Core for EDA (Exploratory Data Analysis)

```

x=np.mean, np.median, np.std, np.var, np.min, np.max

y=np.percentile(arr, 25) # quartiles

print(x)

print(y)

```

Histograms and Binning

Useful for data distribution analysis

```

x=np.histogram(arr, bins=5)

print(x)

```

Set Operations

Unique values, intersection, union, differences – used in categorical data analysis

```

a = np.array([1,2,3])

b = np.array([2,3,4])

x=np.union1d(a,b)

y=np.intersect1d(a,b)

print(a)

print(b)

print(x)

print(y)

```

Performance Tips

Use vectorized operations instead of loops
Avoid Python lists when performing heavy numeric operations
Use np.where() for conditional logic

```

arr = np.array([1,2,3,4,5])

x=np.where(arr % 2 == 0, 0, arr)

print(arr)

print(x)

```

Saving & Loading Data

Store preprocessed datasets efficiently for ML pipelines

```

Y=np.save("X.npy", X)

X = np.load("X.npy")

print(X)

print(Y)

```

Real-Life NumPy Uses

1. Machine Learning

Input preprocessing
Handling tensors before using TensorFlow or PyTorch
Efficient math operations
Train-test split (with random arrays)

2. Data Analysis

Data cleaning
Matrix operations
Statistical calculations
Exploratory Data Analysis (EDA) stats)

3. Image Processing

Images are arrays of shape (height, width, channels).

NumPy

NUMPY

It provides:

NumPy Installation

Importing NumPy

NumPy Array (ndarray)

Creating Arrays

(A) From List

(B) 2D Array

Array Creation Functions

np.zeros() — Array of zeros

np.ones() — Array of ones

np.full() — Array with fixed value

np.eye() — Identity matrix

np.arange() — Like range()

np.linspace() — Evenly spaced numbers

Random Arrays

Array Attributes

Indexing & Slicing

1D slicing

2D indexing

2D slicing

Vectorized Operations (Fast)

Universal Functions (ufuncs)

Mathematical

Statistical Functions

Axis-wise operations

Array Reshaping & Manipulation

reshape

ravel (flatten)

transpose (T)

concatenate

stack

Copy vs View

view (shallow copy)

copy (deep copy)

Boolean Indexing

NumPy Broadcasting

Example

2D + 1D

Linear Algebra (Important for ML)

Matrix multiplication

Determinant

Inverse

Eigenvalues & Eigenvectors

Random Module (Important for ML Models)

Memory Efficiency (Why NumPy is Fast)

Some Advanced NumPy

a. Fancy Indexing

b. Sorting

Unique values

NaN Handling

Save/Load Arrays

Masking and Conditional Operations

Replace values or create masks for analysisExample:

Broadcasting in Data Science

Aggregations (axis-wise operations)

Important for column-wise statistics in datasetsExample:

Working with Missing Data

Random Data Generation

Linear Algebra & Matrix Operations

Reshaping and Manipulating Arrays

Statistical Functions

Histograms and Binning

Set Operations

Performance Tips

Saving & Loading Data

Real-Life NumPy Uses

1. Machine Learning

2. Data Analysis

3. Image Processing

More Python tutorials