It’s no surprise that the top trophy hire in data science is elusive: a “full-stack” data scientist is skilled in machine learning, statistics, and analytics. When teams are unable to obtain a three-in-one polymath, they focus their efforts on attracting the most impressive award among single-origin specialists. Which of the abilities is given the highest honour?
Today’s data science fashion mixes dazzling sophistication with a dash of science fiction, making AI and machine learning job market darlings. Statistics, which has a century-long reputation for discipline and mathematical supremacy, is an alternative challenger for the alpha place. What about the experts?
As a second-class citizen, analytics
If your major expertise is analytics (or data mining or business intelligence), your self-esteem has probably taken a hit as machine learning and statistics have gained traction in the workplace, in the media, and in academia.
But what the uninitiated don’t realise is that the three professions that go under the data science umbrella are vastly diverse. They may share some methodologies and formulae, but that’s about where the similarities end. Good analysts are a precondition for success in your data pursuits, not a weaker version of the other data science breeds. It’s risky to have them abandon you, but that’s exactly what will happen if you undervalue them.
Rather than urging an analyst to improve their statistics or machine learning skills, encourage them to first pursue the pinnacles of their own specialty. Excellence in one field trumps mediocrity in two in data science. So, let’s take a look at what it takes to be truly good in each of the data science disciplines, what value they add, and what personality attributes are required to succeed in each. This will make it easier to explain why analysts are important and how companies should use them.
In statistics, excellence means rigour.
Statisticians are experts at properly drawing inferences from data – they’re your best defence against misleading yourself in an uncertain world. To them, sloppily inferring something is a greater sin than leaving your mind blank, so expect a skilled statistician to put a damper on your euphoria. They are profoundly concerned about whether the methodologies used are appropriate for the task at hand, and they agonise over whether inferences are valid based on the available data.
What’s the end result? A viewpoint that aids leaders in making key decisions while minimising risk. In other words, they use statistics to reduce the chances of you making a poor decision.
Machine learning is at its best when it comes to performance.
If your response to “I bet you couldn’t design a model that passes testing at 99.99999 percent accuracy” is “Watch me,” you might be an applied machine learning/AI engineer. Machine learning experts realise that they won’t discover the ideal solution in a textbook. They have the coding skills to construct working prototypes and production systems, as well as the tenacity to fail every hour for multiple years if that’s what it takes. Having a good sense of how long it will take them to attempt each new choice is a major benefit, and it is more beneficial than having a deep understanding of how the algorithms work (though both are useful). Performance entails more than just passing a metric; it also entails models that are dependable, scalable, and simple to manage in production. Excellence in engineering is a prerequisite.
What’s the end result? A system that automates a difficult task successfully enough to pass your statistician’s stringent testing criteria and give the bold results a business leader wanted.
The difference between wide and deep
The preceding two professions have one thing in common: they both require a lot of effort to solve certain difficulties. You waste their time and your money if the problems they tackle aren’t worth tackling. “Our data science group is useless,” company bosses frequently grumble. And the issue is frequently due to a lack of analytics competence.
Because statisticians and machine learning engineers are narrow-and-deep workers — the shape of a rabbit hole, incidentally — it’s critical to direct them to challenges that are worthy of their time. Your data science investment will suffer low returns if your specialists are painstakingly solving the wrong challenges. To make effective use of narrow-and-deep expertise, you must either be certain that you have the right challenge or take a broad-and-shallow strategy to discovering one.
Analytical excellence: speed
The greatest analysts are lightning-fast coders who can quickly scan large databases for potential insights and surface them faster than other professionals can say “whiteboard.” The semi-sloppy coding style perplexed traditional software engineers, but they left in the dust. Their top attribute is speed, which is closely followed by the ability to spot possibly important gems. A mastery of visual information presentation is also beneficial: beautiful and effective plots allow the mind to extract information more quickly, which reduces the time it takes to reach possible breakthroughs.
As a result, the company has its finger on the pulse and its eyes on hitherto undiscovered unknowns. This provides the motivation that allows decision-makers to choose worthwhile objectives for statisticians and machine learning engineers to pursue, avoiding mathematically spectacular excavations of meaningless rabbit holes.
