Running data science projects successfully is already quite challenging, and the first step is to hire some data scientists. Given the current market this is no easy task. I think it is even harder than hiring “traditional” software engineering roles, because the role is still new, there are different paths to become a data scientist, and there is more variability in the requirements for the role depending on where you are on your data science journey, and how you choose to run data science.
It is a pretty broad topic, but I’d like to address six important questions based on my experience, doing interviews, hiring data scientists into teams, consulting companies with data science teams, defining and shaping the job role, and teaching data science.
How much to pay for data scientists?
Expect to pay 10-20% more than software engineers at the same level. It is not uncommon that data scientists have a couple of offers if they are interviewing. Don’t loose out because you’re compensation is not on par.
What kind of experience is crucial?
Generally speaking, you’d want at least 1-2 years of industry experience and one project that they were part of from inception to production.
A core competency you should be looking for is knowing how to make the connection between a customer/business problem and ML.
Mastering ML methods and being able to run a data science project successfully are two different things. Learning ML focusses a lot on algorithms and technology, but to uncover the true potential for your business, they need to know how to go on the whole journey from understanding customer problems to a ML solution in production. Mapping customer problems to ML problems is a task that you best learn building an actual product.
A great way to test this skill is to take a new or hypothetical customer problem you have in your company and ask them how they would solve it with ML. It shouldn’t be too obvious how to do it and there also might not be only one correct answer. You can learn a lot from an open discussion about the approach a candidate takes.
I’m aware that this is a problem for people starting in the field. You need to get a position in a company to learn this skill, but you cannot get a position if you don’t know it yet. The solution is to hire juniors once you have seniors in your team (see below), and to look for any past projects (side projects, master theses, etc.) that go beyond simply applying a method to a problem.
How much software engineering should I look for?
The answer depends a lot on the product and the team structure.
Your application might be mostly small to medium scale data with more classical statistical and ML applications like forecasting, classification (probably mostly DS work in Python). Or it might be more large scale like recommendation, ad optimization (data needs some familiarity with scalable data processing like Spark or some SQL engine). Or it require state of the art ML methods like natural language processing and computer vision (requires familiarity with deep learning, GPUs).
It also depends how close the data scientists are to production, and whether you have strong software engineers to support them or not, and whether you have data engineers or ML engineers on the team.
And lastly it depends on whether you are starting out (more generalist profiles), or already scaled up (people who can refine an already existing solution further, potentially already more cross-functional teams).
What about take-home tests?
Don’t do take-home tests, but screening interviews may be ok.
I’m generally not a fan of take home tests. They take a lot of time for the candidate, and need to be designed very well to be of use. Also, the interviewer should really take the time to review the take home test, which also doesn’t happen always, unfortunately. In the current market, candidates might also just skip your company because others don’t require it.
What you can and should do are screening interviews that cover basic questions (for example, concepts like likelihood, algorithms like k-means clustering, etc.) before people enter the main interviews. The path to becoming data scientists are so diverse, that you need to make sure the basics which are important to you are there. You can also standardize this interview so that more people can do it.
Sometimes people are concerned about coding capabilities (see above). For that you can do a coding interview (e.g. for Python, at an intermediate level).
How well should people know ML? Should they be able to derive update equations?
Dive deep at least once in the interview.
I think it is important that people have good intuitive understanding of how a method works. If a ML method don’t work as expected, you need to have a mental model to understand failure modes and fix it. The most practical way to test this is to ask people about past projects and drill down on the main methods they have used there. It is a fair test because you are asking about something they should be highly familiar with. It is definitely better than asking random algorithmic questions. Unfortuantely, still many people fail this test.
Which roles to hire for a new team?
I generally recommend starting with one senior data scientist. They need to be able to work alone and make some impact. Next you need to hire another senior. It seems expensive, but data scientists often need someone to discuss and talk through things. After that you can start hiring junior data scientists. Look for seniors who are good mentors. Look for past evidence that they are keen to help people grow.
Where do I get data scientists?
That is probably the most important question, but unfortunately I don’t have an answer to this.
Have I missed something? What are the areas you are most struggling with?
Marginally Interesting by Mikio L. Braun 