The Commoditization of Machine Learning

I recently saw a tweet from Ilya Sukhar that particularly resonated with me:

I've been interested in this space for a while. A broad prediction I have for the coming years is that, as a developer, you won't need to be proficient in machine learning to take advantage of its power. The technology is becoming increasingly democratized and opening up access to millions of new developers. Eventually, you won't even need to know how to program to perform data analysis with ML. In data warehousing, data analysts using old, traditional BI stacks will have access to a powerful new set of machine learning tools. In fact, in the future, using ML may be more about manipulating data, rather than hard mathematics or statistics (h/t Wiley for the comparison). We're moving away from obscure mathematical derivatives to teaching surface area to 4th graders.

A close comparison of this advancement is the proliferation of web development as we know it today. The development of a web application looks a whole hell of a lot different than earlier in the internet days. Before, you needed a strong knowledge of TCP/IP, Solaris servers, Oracle databases, etc. to build a web application. Eventually, these tools were abstracted into frameworks (Perl, Ruby on Rails, Bootstrap) and tools (AWS, Heroku, Parse), making the process of building, deploying, and scaling much easier. Taking it one step further, tools are even being built for non-developers to build apps (Treeline being a good example).

In the machine learning world, we're moving away from the TCP/IP days into the Ruby on Rails days. With limited ML background, it's now much easier to build ML applications than it was even a few years ago. With the rapid development of new open source toolkits, we're truly seeing a rapid commoditization of the technology:

This helps match the rapid growth of the field: 

Publication dates of almost 15000 Machine Learning conference papers scraped from IEEExplore [1]

The Ruby on Rails of ML is toolkits like Tensorflow, Caffe, Theano, and convnetjs. I recently worked with a friend on setting up a TF development environment on an AWS EC2 instance, and the process was a breeze. No need to build your own neural net from scratch anymore!

Recently, Makoto Koike, an embedded systems engineer in Japan, noticed that his parents spent a lot of time sorting and categorizing cucumbers on their farm. The process was just as complicated as growing the vegetable itself. He wanted to automate this process to save his parents from the added manual labor. Although he had limited computer vision background, he used Tensorflow, OpenCV tutorials, and a hardware + camera setup to automatically detect the quality and size of cucumbers grown on the farm - to a relatively high degree of accuracy. Fascinating case study. 

Obviously for more complex needs, you'll need a deep knowledge of the technology and will need to implement most special cases yourself, but such is the case with web applications as well. Tensorflow still, however, covers a wide variety of general use cases. Even experts in the field will use technology like Tensorflow for prototyping efforts. 

Back to Ilya's original tweet, I think there's an opportunity for a startup that liberates basic development with ML. Parse was a great product because it abstracted away the rough edges of building mobile backends. This precise model can be transferred to AI applications:

A good example of a company doing this is Clarifai. They make a dead simple image/video recognition API - and it works really well. I imagine something like this for a few more use cases - categorizing text, voice recognition, intent creation and fulfillment, etc. It's what Shivon Zillis likes to call "'Alchemists' Promising To Turn Your Data Into Gold". Possibilities are endless. Shoot me a message if you're working on this - I'd love to try it out.

______________

[1] https://www.reddit.com/r/dataisbeautiful/comments/4kjivw/publication_dates_of_almost_15000_machine/

Thanks to Ritwik for looking this over. I can be reached @niraj on twitter or by email.