One of the most important parts of getting a good job in machine learning and artificial intelligence (AI) is to evaluate the quality of the company you want to join. This must be done during the interview process, and there are plenty of ways to evaluate companies and find out more about their culture and people.
The simplest way is to read as much as possible about the company to understand what kind of AI it uses and how deep a technical product it is. Read up on recent company news. You should also do at least some light Google searching. Obviously, if they’ve had some bad press recently, you might want to do additional due diligence about the business.
Second, look on Glassdoor to see if any employees have shared information about job interviews and/or the general nature of the business. Glassdoor is probably the best online repository of information about companies, including reviews from people interviewed with the company and current and former employees. If you see many bad reviews, it’s pretty simple: you should probably stay away.
However, the Glassdoor approach can be tricky, especially with a very small or very large company. With the former, there may not be any reviews at all. On the other hand, with a very large company, none of the reviews may pertain to the machine learning part of the business.
The third way is to search LinkedIn to find ML-related employees and consider how experienced they are. Check out people you would directly work with and employees at the director level and higher. You might want to check out their backgrounds to see the schools they attended, activities they are involved in, and of course, their most recent work history and how they describe their career.
Looking at profiles of current employees can help you get a sense of what that company typically likes, and then you can consider whether that works for you. For example, if they all have a Ph.D. in computer science or statistics, the company may have a complex, in-depth take on artificial intelligence.
From a technological perspective, you’ll want to consider what problems the company is trying to solve, its approach, its data, how it audits and monitors itself, and whether the company is thoughtfully applying machine learning.
1. What is the problem the company is trying to solve?
What does the company say it is trying to do, and is it worthy of machine learning? Let’s take Affectiva, for instance, which builds emotion recognition technology to accurately track and analyze people’s moods. Conceptually, this is a pattern recognition problem and thus would be one that machine learning could tackle. It would also be very challenging to approach through another means because it is too complex to program into a set of rules.
2. How is the company approaching that problem with machine learning?
Now that we understand the problem, we want to know how the company will tackle it. A company that specializes in emotion recognition could take a variety of approaches to develop its product. It could teach a computer vision system to recognize people’s facial expressions or an audio system to recognize people’s voice tones. We’re trying to figure out how the company has reframed its problem into a machine-learning problem and what data it’ll need to feed into its algorithms.
3. How does the company source its data?
We want to know how the company acquires data once we know what kind of data it requires. Most AI applications rely on supervised machine learning, which necessitates the use of clean, well-labeled data. Who is responsible for data labeling? Do the labels follow a scientific standard if the labels are subjective, such as emotions? In the case of Affectiva, you’d learn that the company collects audio and video data from users voluntarily then hires trained specialists to label the data in a strict, consistent manner. Knowing the specifics of this section of the pipeline can also help you spot any sources of data collection or labeling bias.
4. Does the company have processes for auditing products?
We should now look into whether or not the company tests its products. What is the accuracy of its algorithms? Are they subjected to a bias investigation? How frequently does it re-evaluate its algorithms to ensure that they are still performing well? What plans does the company have in place if it has algorithms that achieve the desired accuracy or fairness before deploying them?
5. Should the company use machine learning to solve this problem?
This is a more subjective decision. Even if machine learning can solve a problem, it’s important to consider whether it should. Just because you can build an emotion recognition platform that can recognize emotions with at least 80% accuracy across races and genders doesn’t mean it won’t be misused. Do the advantages of having this technology outweigh the potential for emotional surveillance to violate human rights? Is there any mechanism in place to mitigate any potential negative consequences?
A company with a high-quality ML product should be tackling a problem well-suited to machine learning, have a robust data acquisition pipeline and auditing processes, have high-accuracy algorithms or a plan to improve them, and be confronting ethical issues head-on.
Frequently, companies pass the first four tests but not the last. That’s a huge warning sign. It shows that the company isn’t thinking about how its technology can affect people’s lives holistically, and it has a good chance of pulling a Facebook later on. So, use this fantastic five-question framework to assess whether a company is really on the right track with its machine learning product.
