7 Skills Every ETL Developer Should Have

ETL Developers can make life a lot easier for data teams, by Extracting, Transforming, and Loading your data in an efficient manner. Whether you're adding a ETL Developer to your team, or are aspiring to become one yourself, the Arkatechture ETL Developers, Adam Hersey and Jared Rohrdanz, came up with a list of 7 skills every ETL Developer should posess, and what hiring managers should be looking for.

1. ETL Tools/Software

ETL Developers obviously need a tool to develop on. An ETL Developer can create mappings that would take a team weeks to code from scratch in hours with one of these tools. These tools provide a graphical interface for the developer, usually drag and drop, so the devloper can see the entire workflow from source to target. Popular ETL tools include Talend, Informatica, and Pentaho. 

2. SQL

SQL, or Structured Query Language, is the lifeblood of ETL as it is the most popular database language. Every part of ETL can be done with SQL, and often is. There are other Query Languages that can be used, but SQL is the most popular for businesses. Often times, ETL Tools are really just SQL generators behind the scenes, so it's important to be able to use both interchangebly. 

3. Parameterization

Learning how to parameterize your ETL jobs can save tons of time and headaches. Using parameters allows you to dynamically change certain aspects of your ETL job with altering the job itself. For example, you could have a parameter that defines an environment that you want to run in. Instead of creating a seperate job for each environment, you would simply change the environment parameter, instead of repeating the same work again. This saves the ETL developers time, which saves the Client or Business money in turn. 

4. Scripting Language

There will be times where the ETL tools cannot do everything needed to complete the requirements, so ETL Developers often need to get their hands dirty in the systems they are working with. Learning a scripting language can aid developers with juggling files, directories, users, and permission issues that can complicate ETL. Popular scripting languages for ETL include Python, Perl, and Bash. 

5. Organization

As an ETL Developer you will have a lot of tasks to complete in a project or sprint. For any developer, being able to keep the work at hand organized and structured is always a huge plus. This is crucial for ETL Developers who not only have to orgnaize the tasks at hand, but also the ETL mappings and workflows that you create. To be able to keep things organized and structured will help with the debugging process, many businesses will already have some organization standards. For those that don't, establishing your own standards will be an integral part of maintenance and consistent development.

6. Creativity

Believe it or not, having creativity is a huge advantage as a developer. For example, as an ETL Developer, you will be sometimes handed a STTM (Source to Target Mapping) document, which outlines exactly what you'll need to do. However, you may not have this document and will need to create the mappings from scratch. Like an artist with a blank canvas, being able to use different ways requires quite a bit of creativity: thinking outside the box and having ideas that other developers may not think of makes the job easier, and more fun!

7. Debugging/Problem Solving

In ETL Development, nothing ever really goes according to plan the first time you run a job or make an update to an existing mapping. When you are on a production support role, the business/client will want the problem resolved in a certain timeframe as the business depends on these jobs to function properly. In order to be an effective problem solver, the rest of the skills on this list play a part: having an organized standard and familiarity with your ETL tool can streamline the problem solving process.