From the course: Apache Airflow Essential Training

Unlock this course with a free trial

Join today to access over 24,800 courses taught by industry experts.

Passing data using XCom with operators

Passing data using XCom with operators - Apache Airflow Tutorial

From the course: Apache Airflow Essential Training

Start my 1-month free trial Buy for my team

Passing data using XCom with operators

“

- [Instructor] In any real world pipeline, you're going to need to pass data between tasks. One task might generate some data, which is then needed by another task. In Apache Airflow, data is passed between tasks using XCom, which stands for cross communication. And we've already seen how to use XCom in the pre-req course. In this demo, I'll show you a quick example of how you can use XCom to pass data between tasks, using the Python operator API that we are familiar with. And we'll then convert the same DAG to use the Taskflow API. Here is my first Python function, get_order_prices, which takes in kwargs as an input argument. This just generates some prices for some orders placed on an e-commerce site. We access the task instance, ti, from the arguments dictionary passed in. I set up the order price data in the form of a dictionary, so we have orders oh one through oh five. I convert this order price data to a string…

Contents