Deep Learning - EigenValues & EigenVectors

Vector

In simple sentence we can say that vector is quantity with magnitude and direction - is a mathematical object with both magnitude (length) and direction. It can be represented as a list of numbers, or an arrow, and is used to represent quantities like displacement, velocity, or force. 

Every vector which is a list of numbers - has a direction if we plotted it on X-Y axis chart.

Eigenvector

Vector is an Eigenvectors when a linear transformation (such as multiplying it to a scalar) is performed on them, their direction does not change.

Eigenvalue 

The scalar that is used to transform (stretch/compress) an Eigenvector is called eigen value.

Eigenvectors and eigenvalues are used to reduce noise in data. They can help us improve efficiency in computationally intensive tasks. They also eliminate features that have a strong correlation between them and also help in reducing over-fitting.

Practical usages:

One of the commonly used scenario is - eigenvectors and eigenvalues are used in facial recognition techniques such as EigenFaces.

Data that contains a large amount of noise is being gathered to find important or meaningful patterns within the data. Finding patter can be very difficult. Eigenvectors and eigenvalues can be used to construct spectral clustering for this purpose

While building the forecasting models which are expected to get trained on images, sound and/or textual contents, the input feature sets can end up having a large set of features. To understand and visualize data may also be difficult - with more than 3 dimensions.

In such scenario we can use one-hot encoding to transform values in textual features to separate numerical columns which can end up taking a large amount of space on a disk.

Example

For a matrix A of size n, find Eigenvalues of size n. We need to find Eigenvector and Eigenvalues of A such that:

Ax−λx=0 where (x = Eigenvector and λ = Eigenvalues) :

Find λ Such that Determinant(A — λ I) = 0

# Calculate Eigenvalues in python 
# numpy.linalg.eig module takes in a square matrix as the input and returns eigen values and eigen vectors.
# It also raises an LinAlgError if the eigenvalue computation does not converge.

import numpy as np
from numpy import linalg as LA

input = np.array([[2,-1],[4,3]])

w, v = LA.eig(input)        

Will share more details in next articles ...

Keep learning - keep sharing