Chapter 5-Numpy-Pandas.pptx python programming

Usage of Numpy for numerical Data
NumPy (Numerical Python) is a fundamental library for
numerical computing in Python. It provides support for arrays,
matrices, and a wide range of mathematical functions.
Key Features
• Efficient storage and manipulation of numerical data.
• Functions for array operations, linear algebra, and random
number generation.

NumPy
• Stands for Numerical Python
• Is the fundamental package required for high performance
computing and data analysis
• NumPy is so important for numerical computations in Python
is because it is designed for efficiency on large arrays of data.
• It provides
• ndarray for creating multiple dimensional arrays
• Internally stores data in a contiguous block of memory, independent of other
built-in Python objects, use much less memory than built-in Python sequences.
• Standard math functions for fast operations on entire arrays of data without
having to write loops
• NumPy Arrays are important because they enable you to express batch
operations on data without writing any for loops. We call this vectorization.

NumPy ndarray vs list
• One of the key features of NumPy is its N-dimensional array
object, or ndarray, which is a fast, flexible container for large
datasets in Python.
• Whenever you see “array,” “NumPy array,” or “ndarray” in
the text, with few exceptions they all refer to the same thing:
the ndarray object.
• NumPy-based algorithms are generally 10 to 100 times faster
(or more) than their pure Python counterparts and use
significantly less memory.
import numpy as np
my_arr = np.arange(1000000)
my_list = list(range(1000000))

ndarray
• ndarray is used for storage of homogeneous data
• i.e., all elements the same type
• Every array must have a shape and a dtype
• Supports convenient slicing, indexing and efficient vectorized
computation
import numpy as np
data1 = [6, 7.5, 8, 0, 1]
arr1 = np.array(data1)
print(arr1)
print(arr1.dtype)
print(arr1.shape)
print(arr1.ndim)

Creating ndarrays
Using list of lists
import numpy as np
data2 = [[1, 2, 3, 4], [5, 6, 7, 8]] #list of lists
arr2 = np.array(data2)
print(arr2.ndim) #2
print(arr2.shape) # (2,4)

Create a 2d array from a list of list
• You can pass a list of lists to create a matrix-like a 2d array.
In:
Out:

The dtype argument
• You can specify the data-type by setting the dtype() argument.
• Some of the most commonly used NumPy dtypes are: float, int, bool,
str, and object.
In:
Out:

The astype argument
• You can also convert it to a different data-type using the astype method.
In: Out:
• Remember that, unlike lists, all items in an array have to be of the same
type.

dtype=‘object’
• However, if you are uncertain about what data type your
array will hold, or if you want to hold characters and
numbers in the same array, you can set the dtype as 'object'.
In: Out:

The tolist() function
• You can always convert an array into a list using the tolist() command.
In: Out:

Inspecting a NumPy array
• There are a range of functions built into NumPy that allow you to
inspect different aspects of an array:
In:
Out:

array = np.array([[0,1,2],[2,3,4]])
[[0 1 2]
[2 3 4]]
array = np.zeros((2,3))
[[0. 0. 0.]
[0. 0. 0.]]
array = np.ones((2,3))
[[1. 1. 1.]
[1. 1. 1.]]
array = np.eye(3)
[[1. 0. 0.]
[0. 1. 0.]
[0. 0. 1.]]
array = np.arange(0, 10, 2)
[0, 2, 4, 6, 8]
array = np.random.randint(0,
10, (3,3))
[[6 4 3]
[1 5 6]
[9 8 5]]
Creating ndarrays
arange is an array-valued version of the built-in Python range function

Arithmatic with NumPy Arrays
• Any arithmetic operations between equal-size arrays applies the
operation element-wise:
arr = np.array([[1., 2., 3.], [4., 5., 6.]])
print(arr)
[[1. 2. 3.]
[4. 5. 6.]]
print(arr * arr)
[[ 1. 4. 9.]
[16. 25. 36.]]
print(arr - arr)
[[0. 0. 0.]
[0. 0. 0.]]

Arithmatic with NumPy Arrays
• Arithmetic operations with scalars propagate the scalar argument
to each element in the array:
• Comparisons between arrays of the same size yield boolean
arrays:
arr = np.array([[1., 2., 3.], [4., 5., 6.]])
print(arr)
[[1. 2. 3.]
[4. 5. 6.]]
print(arr **2)
[[ 1. 4. 9.]
[16. 25. 36.]]
arr2 = np.array([[0., 4., 1.], [7., 2., 12.]])
print(arr2)
[[ 0. 4. 1.]
[ 7. 2. 12.]]
print(arr2 > arr)
[[False True False]
[ True False True]]

Extracting specific items from an array
• You can extract portions of the array using indices, much like when
you’re working with lists.
• Unlike lists, however, arrays can optionally accept as many
parameters in the square brackets as there are number of dimensions
In: Out:

Indexing and Slicing
• One-dimensional arrays are simple; on the surface they act
similarly to Python lists:
arr = np.arange(10)
print(arr) # [0 1 2 3 4 5 6 7 8 9]
print(arr[5]) #5
print(arr[5:8]) #[5 6 7]
arr[5:8] = 12
print(arr) #[ 0 1 2 3 4 12 12 12 8 9]

Indexing and Slicing
• As you can see, if you assign a scalar value to a slice, as in
arr[5:8] = 12, the value is propagated (or broadcasted) to the
entire selection.
• An important first distinction from Python’s built-in lists is that
array slices are views on the original array.
• This means that the data is not copied, and any modifications to the view will be
reflected in the source array.
arr = np.arange(10)
print(arr) # [0 1 2 3 4 5 6 7 8 9]
arr_slice = arr[5:8]
print(arr_slice) # [5 6 7]
arr_slice[1] = 12345
print(arr) # [ 0 1 2 3 4 5 12345 7 8 9]
arr_slice[:] = 64
print(arr) # [ 0 1 2 3 4 64 64 64 8 9]

Indexing
• In a two-dimensional array, the elements at each index are no
longer scalars but rather one-dimensional arrays:
• Thus, individual elements can be accessed recursively. But that is
a bit too much work, so you can pass a comma-separated list of
indices to select individual elements.
• So these are equivalent:
arr2d = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])
print(arr2d[2]) # [7 8 9]
print(arr2d[0][2]) # 3
print(arr2d[0, 2]) #3

Activity 3
• Consider the two-dimensional array, arr2d.
• Write a code to slice this array to display the last column,
[[3] [6] [9]]
• Write a code to slice this array to display the last 2 elements of
middle array,
[5 6]
arr2d = np.array([[1, 2, 3], [4, 5, 6], [7, 8, 9]])

Boolean indexing
• A boolean index array is of the same shape as the array-to-
be-filtered, but it only contains TRUE and FALSE values.
In: Out:

Pandas
• Pandas, like NumPy, is one of the most popular Python
libraries for data analysis.
• It is a high-level abstraction over low-level NumPy, which is
written in pure C.
• Pandas provides high-performance, easy-to-use data
structures and data analysis tools.
• There are two main structures used by pandas; data frames
and series.

Indices in a pandas series
• A pandas series is similar to a list, but differs in the fact that a series associates a label with
each element. This makes it look like a dictionary.
• If an index is not explicitly provided by the user, pandas creates a RangeIndex ranging from 0
to N-1.
• Each series object also has a data type.
In: Ou
t:

• As you may suspect by this point, a series has ways to extract all of
the values in the series, as well as individual elements by index.
In: Ou
t:
• You can also provide an index manually.
In:
Out:

• It is easy to retrieve several elements of a series by their indices or make
group assignments.
In:
Out:

Filtering and maths operations
• Filtering and maths operations are easy with Pandas as well.
In: Ou
t:

Pandas data frame
• Simplistically, a data frame is a table, with rows and columns.
• Each column in a data frame is a series object.
• Rows consist of elements inside series.
Case ID Variable one Variable two Variable 3
1 123 ABC 10
2 456 DEF 20
3 789 XYZ 30

Creating a Pandas data frame
• Pandas data frames can be constructed using Python dictionaries.
In:
Out:

• You can also create a data frame from a list.
In: Out:

• You can ascertain the type of a column with the type() function.
In:
Out:

• A Pandas data frame object as two indices; a column index and row
index.
• Again, if you do not provide one, Pandas will create a RangeIndex from
0 to N-1.
In:
Out:

• There are numerous ways to provide row indices explicitly.
• For example, you could provide an index when creating a data frame:
In: Out:
• or do it during runtime.
• Here, I also named the index ‘country
code’.
In:
Out:

• Row access using index can be performed in several ways.
• First, you could use .loc() and provide an index label.
• Second, you could use .iloc() and provide an index number
In: Out:
In: Out:

• A selection of particular rows and columns can be selected this way.
In: Out:
• You can feed .loc() two arguments, index list and column list, slicing operation
is supported as well:
In: Out:

Filtering
• Filtering is performed using so-called Boolean arrays.

Deleting columns
• You can delete a column using the drop() function.
In: Out:
In: Out:

Reading from and writing to a file
• Pandas supports many popular file formats including CSV, XML,
HTML, Excel, SQL, JSON, etc.
• Out of all of these, CSV is the file format that you will work with the
most.
• You can read in the data from a CSV file using the read_csv() function.
• Similarly, you can write a data frame to a csv file with the to_csv()
function.

• Pandas has the capacity to do much more than what we have
covered here, such as grouping data and even data
visualisation.
• However, as with NumPy, we don’t have enough time to cover
every aspect of pandas here.

Exploratory data analysis (EDA)
Exploring your data is a crucial step in data analysis. It involves:
• Organising the data set
• Plotting aspects of the data set
• Maybe producing some numerical summaries; central tendency
and spread, etc.
“Exploratory data analysis can never be the whole story, but
nothing else can serve as the foundation stone.”
- John Tukey.

Reading in the data
• First we import the Python packages we are going to use.
• Then we use Pandas to load in the dataset as a data frame.
NOTE: The argument index_col argument states that we'll treat the first
column of the dataset as the ID column.
NOTE: The encoding argument allows us to by pass an input error created
by special characters in the data set.

Chapter 5-Numpy-Pandas.pptx python programming

Recommended

More Related Content

Similar to Chapter 5-Numpy-Pandas.pptx python programming (20)

More from ssuser77162c (14)

Recently uploaded (20)

Chapter 5-Numpy-Pandas.pptx python programming

Editor's Notes