Opening the eyes of the machine: Computer vision with AutoML (Part 1)
By Gavita Regunath and Dan Lantos
Automated Machine Learning, or AutoML, has gained a lot of popularity and traction in both the research and business sectors.
Why?
AutoML has proven to empower citizen data scientists and/or boost existing data scientists' productivity whilst enabling businesses to gain valuable insights. There is no doubt that AutoML has helped democratise and accelerate the development of machine learning models in businesses. Recently cloud providers have introduced the capability of AutoML for Computer Vision tasks.
This article consists of two parts, part one aims to set the scene by introducing the concept of computer vision, how it is used widely in many industries, and how AutoML for computer vision aims to democratise machine learning.
In part two, we will provide a step-by-step guide on using AutoML for computer vision to jumpstart your computer vision journey.
61% of data and analytics decision-makers whose firms are adopting Artificial Intelligence said they had implemented AutoML software or are in the process of implementing it”
Forrester Research
What is AutoML?
When you consider a typical machine learning lifecycle, many crucial steps need to be taken before developing a successful machine learning model. AutoML aims to do a lot of the heavy lifting by automating the most tedious parts of the machine learning lifecycle to produce the best model. Until recently, most AutoML tools were able to solve one of the three machine learning problems: regression, classification, and time-series forecasting problems.
However, more recently, cloud vendors have responded to AI and machine learning trends and have provided the capability to perform AutoML in computer vision. The computer vision market is expected to grow at a CAGR of 7.7% to reach USD 18.24 billion by 2025, according to a report by Grand View Research.
This is not surprising when you consider how tightly integrated computer vision is with our daily lives. Unlocking mobile devices through facial images, authorising payments through facial recognition when using mobile banking, using Google lens to translate content in real-time, and unlocking the front door using facial recognition are examples of computer vision in action. Having the ability to leverage AutoML in computer vision will be a game-changer for many businesses.
What is Computer Vision? 💻👓
Computer vision is a field of artificial intelligence that uses machine learning to analyse images captured by cameras or satellites to derive meaningful information. Essentially, computer vision aims to mimic how humans process information from images.
In part two, we will demonstrate how to perform image classification using Azure's new capability of AutoML for computer vision. At the time of writing this article, the AutoML for computer vision functionality (in preview) offers support for computer vision tasks such as multi-label image classification, image classification, object detection and image segmentation.
If you're curious what the distinctions are, the diagram below might help. Beginning on the right, multi-label image classification and image classification are where the model identifies what is in an image. Because there are four distinct animals in the first image, it is classified as a multi-label image classification issue, as opposed to the second image, which only has a dog. Object detection models generate a bounding box around specific objects, whereas image segmentation models provide the exact outline of the object within an image.
Why should we care about computer vision? 🤷🏽♀️🤷🏽♂️
The field of computer vision is progressing rapidly and is creating a lot of excitement. With ongoing research and technological advancement, computer vision is transforming industries globally, from autonomous vehicles to face recognition and diagnosing from x-ray images. Let's explore the most popular computer vision application use cases across a number of industries to provide more context. Note that the market for computer vision is continually expanding and is being adopted in more industries than what is listed below.
Finance
Auto-insurer Tokio Marine use computer vision system for examining damaged vehicles. Source: isurancejournal.com
Computer vision technology is beginning to significantly impact the financial services industry. Banks like the Spanish bank BBVA already use face recognition to onboard their customers remotely, reducing their onboarding times from hours to minutes. Facial recognition and retina scanning are also helping financial institutions to improve security procedures and therefore reducing fraud.
In the insurance business, Tokio Marine, a Japanese-based property and casualty insurer use computer vision to analyse and evaluate damaged cars, speeding up the evaluation process.
The use of computer vision applications for healthcare is often regarded as a turning point in medical image processing and diagnosis. It has already proven to be highly effective at saving hundreds of patient lives. Some examples of how computer vision is used in the healthcare industry is the detection of various illness such as cancerous cells, the accurate detection of blood loss during childbirth and generating accurate and precise reports based on medical imaging for example detecting lung disease from X-Ray imaging. More recently, one popular way in which computer vision was used in the healthcare industry was utilised in the COVID-Net device developed by Darwin AI, Canada. COVID-Net was used successfully for detecting COVID-19 cases using digital chest x-ray radiography (CXR).
HEalthcare
Machine Learning and Computer Vision play an important part here in detecting breast cancers well on time. Source: New York Times
Manufacturing
Computer vision used to detect hard hats on workers
Some of the common applications of computer vision in the manufacturing industry are for predictive maintenance where cameras and sensors are used to monitor and take corrective actions on machinery. Other popular examples include the use of computer vision to monitor the quality of goods on a production line, to detect if workers are wearing suitable protective equipment and for the identification of defective products.
Computer vision is being utilised in the retail business to analyse consumer behaviour and activity, which eventually delivers valuable insight back to the retailer. Retailers such as Sephora and Samsonite, for example, use computer vision in their stores to better understand their customers' behaviours, test new merchandising strategies, and experiment with more successful layouts. Amazon Go, a cashier-less checkout uses computer vision to provide their customer with a hassle-free shopping experience, where customers walk in, pick items they want and simply walk out of the store.
RETAIL
Amazon Go uses computer vision to detect when a customer taken an item from the shelf and automatically calculates the prices. Source: Amazon.com
Transportation
Tesla cars' Autopilot enables the driver to steer the car, accelerate and brake automatically within its lane. Source: Tesla Autopilot
Computer vision has been used in transportation for at least a decade now. Lane tracking, vehicle detection, traffic signs detection and pedestrian detection are core areas embedded within self-driving cars that utilise computer vision. Apart from self-driving cars, other examples of where computer vision is being used in the transportation sector are for parking occupancy detection, automated license plate recognition and road condition monitoring.
In agriculture, there are several examples of how computer vision is being used to improve the industry. RSIP Vision, has developed software to predict crop yield using computer vision and machine learning. Performing automatic weeding within large crops, which requires precision identification and classification of weeds, is another example of where computer vision is being used. Livestock monitoring, irrigation management, plant disease detection, and insect detection are a few other popular examples of how computer vision is being used in agriculture.
Agriculture
RSIP vision uses computer vision to predict agricultural yield. Source: rsipvision.com
Biodiversity conservation
Microsoft – AI for Earth and Gramener are helping save the penguins in Antarctica!🐧🐧. Source: Gramener
There are many ways in which computer vision is used for biodiversity conservation efforts. According to a recent Wildlabs.net analysis, computer vision is one of the top three promising conservation technologies. Camera trap imaging which is a way to automatically photograph animal species in the field is quickly becoming the gold standard in biodiversity conservation. Another example is where Gramener and Microsoft – AI for Earth have devised a solution using computer vision to monitor the Emperor penguin population that is at serious risk.
Security and Surveillance:
Facial detection and recognition are some of the most prominent computer vision technology used to detect and prevent crime. In fact, in most public places like car parks, bus stations, underground stations, and motorways, computer vision is used to monitor and prevent criminal activities. On social media platforms, face detection algorithms are used to detect and stop the spread of fake news.
Recently, computer vision has become an extremely popular application in the sports and fitness industry. Some examples of computer vision are tracking players in a match to offer insight, pose estimation of athletes to provide sports analytics, and using video assistant referees (VARs) to review decisions made.
In fact, Peloton recently released the Peloton Guide, where is uses machine learning and smart camera technology to track movements and offer recommendations. It is similar to having a personal trainer at home providing the right advice to gain maximum benefit.
Fitness
Whatever it is, the way you tell your story online can make all the difference.
Advertising
Artificial Intelligence Poster, Oxford Street London by M&C Saatchi created the first ever artificially intelligent poster campaign in the world, which evolves unique ads based on how people react to it.
When it comes to advertising, the success of an advert is measured by analysing customers' engagement. With computer vision, advertising agencies can now recognise customers' emotional reactions and use this information to deliver personalised and targeted ads. One of the examples of this is the AI poster by M&C Saatchi, London, which uses computer vision to capture people's emotions and provide targeted content.
Summary
So far, we've covered the basics of AutoML and computer vision, as well as numerous examples of its use in different industries.
If you want to get started with computer vision but lack the technical know-how, or if you wanted to increase the productivity of your data science team, utilising AutoML for computer vision is a great place to start. It uses minimal code to expedite the creation of computer vision models for tasks such as image classification, object detection and instance segmentation. In the following article, we will provide a step-by-step guide that will guide you through the process of using Azure AutoML in computer vision capability for object detection.