Merging DataFrames with Pandas

Fisseha Berhane, PhD

Senior Principal Data Scientist

Published Jun 11, 2017

Python's popularity in all the current trending technologies in IT is increasing from year to year. Therefore, mastering Python opens more options in the marketplace. Python is also one of the most popular data science tools. One of the reasons for Python's high popularity in data science is the Pandas package. In this blog post, we will see how to use Pandas to merge lots of data files.

Article available here

See more comments

To view or add a comment, sign in

More articles by Fisseha Berhane, PhD

Deep Learning with TensorFlow and Keras

Jul 5, 2019

Deep Learning with TensorFlow and Keras

I have been teaching deep learning (DNN and CNN) with TensorFlow and Keras to colleagues at work and I have shared them…

3 Comments
How using scikit-learn in Spark could save the day

Apr 5, 2019

How using scikit-learn in Spark could save the day

We may want to use scikit-learn with Spark when: 1- training a model in scikit-learn takes so long 2- the machine…

1 Comment
ROC Curve could be misleading with imbalanced data: Precision-Recall Curve is more informative

Apr 2, 2019

ROC Curve could be misleading with imbalanced data: Precision-Recall Curve is more informative

Even if ROC curve and area under the ROC curve are commonly used to evaluate model performance with balanced and…

2 Comments
Data distributions where K-means clustering fails; can DBSCAN be a solution? Examples with R, Python and Spark

Nov 4, 2018

Data distributions where K-means clustering fails; can DBSCAN be a solution? Examples with R, Python and Spark

For K-means clustering to work well the variance of the distribution of each attribute (variable) should be…

10 Comments
Sampling using truncated hash

Oct 25, 2018

Sampling using truncated hash

Let's suppose tens of millions of people visit your website everyday and you want to do ad hoc analysis. However, you…
Hive Partitioning with Spark

Oct 17, 2018

Hive Partitioning with Spark

I experimented with Hive partitioning and some of the things I discussed in this blog post are: Query response time…
Issues to pay attention to when performing PCA in Spark, Python and R

Oct 8, 2018

Issues to pay attention to when performing PCA in Spark, Python and R

Recently, I was using PCA with Spark with sparse matrix with millions of rows and to make sure everything was right I…

5 Comments
Simpson’s Paradox

Oct 3, 2018

Simpson’s Paradox

Today, I was listening to a data science podcast from DataCamp and they talked about Simpson's paradox: it is a…

1 Comment
Exploring the Pareto Distribution with R

Sep 20, 2018

Exploring the Pareto Distribution with R

We have learned various distributions in college. The most common one being the bell-curve, based on our area of study…

1 Comment
Benefits of and Tips on Hortonworks Apache Spark Certification

Mar 25, 2018

Benefits of and Tips on Hortonworks Apache Spark Certification

Recently, I took hands-on, performance-based certification for Spark on the Hortonworks Data Platform (HDPCD), and in…

See all articles

Merging DataFrames with Pandas

Fisseha Berhane, PhD

Senior Principal Data Scientist

More articles by Fisseha Berhane, PhD

Insights from the community

Others also viewed

Python Data Science Handbook (Jupyter Notebooks Format)

Friendly Reminder: Python Pandas Webinar on March 28th!

How To Import Text Files Into NumPy Arrays

Running our Python code

NumPy in Python - I started this Topic this week.

Emoji --> Text

Accelerated Python

Data Science: Python Tutorials

Use of Python and Jupyter Notebook for Blog Posts

Regression with Python

Explore topics

More articles by Fisseha Berhane, PhD

Deep Learning with TensorFlow and Keras

How using scikit-learn in Spark could save the day

ROC Curve could be misleading with imbalanced data: Precision-Recall Curve is more informative

Data distributions where K-means clustering fails; can DBSCAN be a solution? Examples with R, Python and Spark

Sampling using truncated hash

Hive Partitioning with Spark

Issues to pay attention to when performing PCA in Spark, Python and R

Simpson’s Paradox

Exploring the Pareto Distribution with R

Benefits of and Tips on Hortonworks Apache Spark Certification

Insights from the community

Others also viewed

Python Data Science Handbook (Jupyter Notebooks Format)

Friendly Reminder: Python Pandas Webinar on March 28th!

How To Import Text Files Into NumPy Arrays

Running our Python code

NumPy in Python - I started this Topic this week.

Emoji --> Text

Accelerated Python

Data Science: Python Tutorials

Use of Python and Jupyter Notebook for Blog Posts

Regression with Python

Explore topics