Developers: Jump-Start Your Foray into ML with the Cortana Intelligence Gallery

This post is by Amy Nicholson, Technical Evangelist at Microsoft.

Here’s a task for you: Type “I want to get started with machine learning” into your favorite search engine. What do you see? I get back a whole list of options from Coursera, Azure, Amazon, kdnuggets, reddit,… and I could continue on and on. So where should one get started?

In this post, I share my experiences of using the Cortana Intelligence Gallery in conjunction with the Azure ML Studio, and how that combination helped me break down the otherwise high barriers into popular ML techniques and start building my own ML models as well as my knowledge about this space.

Where Did It All Start?

A little bit of background into how I got into this project. I’m a graduate from The University of Sheffield in the UK, where I studied Computer Science. Academia is where I started to find my passion for Data Science and ML. I was lucky enough to secure an internship year working at Microsoft Bing, on the Autosuggest Team. I met bright and passionate people there who taught me a lot about the power of ML and the amazing apps you can create when finding patterns in data. I took this insight and passion back to university where I was able to channel my ideas into my final year research and focused on all things ML, Data Processing and Adaptive Intelligence.

I applied to return to Microsoft as a graduate and landed a place as a Technical Evangelist in Microsoft UK. I work in a division called Developer Experience, which is all about supporting developers, start-ups, students and ISVs (Independent Software Vendors) with the latest tools and technologies to make them successful. I have been able to focus and share my passion around ML, Data Science and IoT since joining Microsoft.

Around the time I joined Microsoft, Azure ML was in an early preview release as a service on Azure. I worked with two other Technical Evangelists, Andrew Fryer and Bianca Furtuna to build models such as Twitter Sentiment Analysis and Predicting Flight Delays. I immediately saw the power of the service. Among other things, it suited Data Scientists who want to bring in their own R and Python code, and helped them run and publish their models using the Studio.

Azure ML Studio

Azure ML is a fully managed cloud service on Microsoft Azure that helps you easily build, deploy, and share predictive analytics solutions. It was designed for applied ML. Use best-in-class algorithms or the hundreds of built-in packages and support for custom code (R, Python). All of this in a simple drag-and-drop interface to go from an idea to the deployment of a ML API in a matter of clicks.

However, the Azure ML Studio can be a lonely place when you first sign in and create a workspace – your first 'blank' experiment looks a bit like below.

A data scientist would say:

"Look at the amazing choice of datasets from many sources and so many data processing/feature engineering options. Oh, oh… and don't forget the vast array of powerful ML algorithms at my fingertips, and then web service modules so that I can work in harmony with the app developers. But how do they all fit together?"

But if you are not a Data Scientist (noun): Person who is better at statistics than any software engineer and better at software engineering than any statistician [1], perhaps you are a developer, you might be thinking something else.

A data developer might say:

“Hmmm… where do I start, I need to understand what a feature and a label is and training/testing sets - this is quite a blank canvas!”

Over time, since Azure ML was released, the team has been adding new sample experiments to the Cortana Intelligence Gallery. Users can now leverage these as a starting point and for ideas as they continue to build their own experiments and solutions. So both Data Scientists and Data Developers have a resource to get started. For example, if you want to create a clustering experiment you might look at Clustering: Find Similar Companies or maybe a recommendation engine, Recommender: Movie Recommendation, as example cases.

What Did We Do?

The UK Developer Experience team also saw the Cortana Intelligence Gallery as a fantastic repository of learning resources and a great place to publish our own experiments. In the past 18 months, we’ve been running free developer events aimed at the 'Data Developer' audience – events with multiple in-depth, hands on workshop tracks in areas such as ML, Data Visualization, Big Data & IoT. To support the content we delivered in the ML track we created sample experiments in the gallery for attendees to use, one as a starting point and also the completed content. The completed content stage aimed to build an end-to-end ML model from dataset to publishing an API – meaning attendees got a chance to build using the Studio and felt confident going away that they could continue their ML journey using Azure ML.

Not only did the gallery help at those events but also with many other engagements we had with data developers such as in customer and community events. These events entailed different length sessions ranging from 1 or 2 hours to half a day or a full day. For attendees to get the full experience of feature engineering, ML algorithms and publishing modules, we used experiments completed to the pre-processing stage so that attendees could start building in the labs right away and see the features which differentiate this service from others.

To expand our reach with this content we also added the lab guide to the Experiment Description in the Cortana Intelligence Gallery, so you don't have to attend in-person to get started. If you want to start building, just go right ahead: use the lab document and learn about Azure ML today.

What Have We Observed?

After each event attendees have been excited about using Azure ML and left with a sense of accomplishment around building models in a fairly short space of time and with an understanding of ML processes.

The Cortana Intelligence Gallery experience is focused around community contribution: Anyone can add their experiments to the Gallery, and search for / consume others’ experiments. The Gallery acts is a springboard for developers to get started on learning about ML and lowers barriers to entry for those without PhDs in statistics, for instance. It encourages users to continue their journey and become experts in an area where there is a strong industry demand for such skills.

This theme continues in our Machine Learning online forums where Microsoft experts answer your queries. When you encounter issues or feature gaps you can always provide feedback and feature requests directly to the Azure ML team. The Cortana Intelligence Gallery is thus a service, community and learning destination all in one!

If you're looking at getting into ML, I highly recommend using Azure ML and starting with the Cortana Intelligence Gallery, based on my own personal experience.

Amy @AmyKateNicho

References: [1] From <https://twitter.com/josh_wills/status/198093512149958656>