In my career I have seen ample of instances where people ask why we shall engage a data scientist or a machine learning engineer, when we can solve the problem in hand via coding the rules.
Fair enough, for a lot of those problems we might be able to work with just coded rules and lookups etc, however as the size of data grows and here I am not talking about the volume of data but the attributes/features comprising that data, it becomes difficult and difficult to be able to read the hidden and not so obvious relations among attributes/features, and that is the time when instead of spending a lot of effort in trying to figure out the relations and trying to code them we should use (Linear Algebra, Calculus, Statistics and Probability) AKA data analytics and machine learning to find the patterns and automatically code them in form of a machine learning model.
Try to find the relation between A, B and C
Simple right, C = A + B // easy to code yay
Now, try to find the relation between A, B, C and D
Simple right, D = D-(A + B) // ok not easy as 2 variables but still ok
Let try one more time
Try to find the relation between A, B, C……….c and Z
How about now, and I am just talking about 30 parameters/ variables here. Googles BERT English language model has 345 million parameters.
Also, in real world we are not just looking at numerical data we are looking at text and all sort of data which makes it even more difficult to identify and code the patterns.
I think machine learning and data analytics using statistics should be an integral part of a SDE arsenal, which can help them to solve the problems faster and at a greater pace.
All the public clouds AWS, Azure and GCP all comes loaded with tools to build basics machine learning models and deploy them for inference.
The tools like Ludwig are trying to make deep learning as simple as creating a spreadsheet and supplying it to the tool to create a deep learning model.
So go-ahead and start using machine learning to solve your business problems, you can build the simple regressions models like in the problems stated above using AWS machine learning.