How to Build a Deep Learning Team?

How to Build a Deep Learning Team that is Capable of Delivering Real-World Products?

You cannot look or listen to any tech-focused media source nowadays without hearing about AI and deep learning (DL). However, when one looks at the number of products that are already available and utilizing this technology, you will find that the number is very small!

Whenever I meet with other entrepreneurs or engineering managers who are trying to build deep learning related products, I hear the same complaints over and over, there are not enough engineers to hire! The big guys (e.g. Google, Facebook, Apple, Amazon, Microsoft, etc.) have elevated the DL engineers compensation so much that it is impossible for anyone else to hire them. And, at the same time, the quality of DL resumes we get is very light given what is needed to build world-class products. On top of all of that, DL engineers have a high sense of entitlement and do not stick around for more than a year.  

Although most of these complaints are valid to a certain extent, the real question is how to deal with these circumstances in order to succeed in creating products that will enhance humans’ quality of life for generations to come. The answer is, as always, go back to basics and review your plan.  

Review the problem statement and decide what components are essential to build your desired product.

Most of the cases in which deep learning are supposed to be used effectively involve collecting a signal from sensors and then using deep learning to analyze the sensor stream of data. In almost all of these cases, there is a limited number of steps to be taken:

1.               The first step is typically to find a way to build your training data set.

2.               Devise a mechanism to augment the training data set to be sure the training process   is generalized.

3.               Devise a network architecture that is amenable to the problem at hand. This                                                                            architecture needs to take into account the nature in which data will be collected in the real device. Also, it needs to respect the power complexity budget bounds as well.   

4.               Train the network.

5.               Analyze the training results to better understand the weak points of the training    process.

6.               From the results analysis, define what changes should be done to the training set, augmentation mechanism, and maybe even network architecture.

7.               “Rinse and Repeat” steps 3 to 6. Go through as many iterations as necessary until desired performance is achieved or until you run out of time 😊

8.               Develop the necessary pre-processing and post-processing algorithms to bridge the gap between the achieved network results and consumer expectations.

9.               Perform complexity analysis on the network to be sure that you are within the final production power/complexity budget.

It is important to keep in mind that these steps cover only the algorithm development part of the process and do not deal with optimal deployment of the algorithm to target HW.

As you can see, for most of these steps, the key skill is strong experience and knowledge about the signal being processed. Only the network architecture design and exploration phase require deep experience in deep learning. Even so, as a dear friend of mine said, “In most cases, a ResNet or a DenseNet like architecture is good enough.” Today DL tools are very handy with a countless number of examples to get you started in few hours.  

The point to make, if you are trying to build a deep learning product, is to focus on hiring engineers with strong experience in dealing with the signal you are trying to analyze as long as they embrace and realize the value of using deep learning in solving the parts of the problems at hand. This applies even if their hands-on experience with using deep learning tools does not match that of deep learning focused engineers. By the same token, avoid two groups:

• DL Haters: brilliant engineers who has a deep resentment toward deep learning as a field because they are too good to use data-driven methods.
• DL tools’ operators (aka Kaggle Aficionados): very skilled SW engineers that know how to hack open-source components without any specific signal processing skills, and are trying to solve every problem using deep learning without even attempting to understand the problem at hand.

With these simple rules, you will be tapping into a talent pool that is somewhat different from what the big guys are pursuing. Moreover, these engineers will be easier to preserve as they are passionate about the problem you are trying to solve and have invested in a skill set that is matched to your product needs and hence will not change their color with the wind.  And, most importantly, you will be guaranteed to have a practical solution to your problem that is hard to replicate.