A new AI from Microsoft aims to automatically caption images in documents and emails so that software for visual impairments can read it out.

Researchers from Microsoft explained their machine learning model in a paper on preprint repository arXiv.

The model uses VIsual VOcabulary pre-training (VIVO) which leverages large amounts of paired image-tag data to learn a visual vocabulary.

A second dataset of properly captioned images is then used to help teach the AI how to best describe the pictures.

“Ideally, everyone would include alt text for all images in documents, on the web, in social media – as this enables people who are blind to access the content and participate in the conversation. But, alas, people don’t,” said Saqib Shaikh, a software engineering manager with Microsoft’s AI platform group. 

Overall, the researchers expect the AI to deliver twice the performance of Microsoft’s existing captioning system.

In order to benchmark the performance of their new AI, the researchers entered it into the ‘nocaps’ challenge. As of writing, Microsoft’s AI now ranks first on its leaderboard.

“The nocaps challenge is really how are you able to describe those novel objects that you haven’t seen in your training data? ” commented Lijuan Wang, a principal research manager in Microsoft’s research lab.

Developers wanting to get started with building apps using Microsoft’s auto-captioning AI can already do so as it’s available in Azure Cognitive Services’ Computer Vision package.

Microsoft’s impressive SeeingAI application – which uses computer vision to describe an individual’s surroundings for people suffering from vision loss – will be updated with features using the new AI. 

“Image captioning is one of the core computer vision capabilities that can enable a broad range of services,” said Xuedong Huang, Microsoft CTO of Azure AI Cognitive Services.

“We’re taking this AI breakthrough to Azure as a platform to serve a broader set of customers,” Huang continued. “It is not just a breakthrough on the research; the time it took to turn that breakthrough into production on Azure is also a breakthrough.”

The improved auto-captioning feature is also expected to be available in Outlook, Word, and PowerPoint later this year. 

Source: AI News

Do you like Abraham T Diress's articles? Follow on social!
Comments to: Microsoft’s new AI auto-captions images for the visually impaired

Your email address will not be published. Required fields are marked *

Attach images - Only PNG, JPG, JPEG and GIF are supported.

Good Reads

Google today revealed Google Maps updates aimed at warning users of pandemic-related threats. Soon, maps will display all-time COVID-19 cases identified in an area, along with fast links from local authorities to resources. Google will also start to demonstrate how bus, train, and subway lines are crowded in more locations across the globe. Maps also […]
Today, $60 million was raised by Hover, a startup creating AI-powered apps that build 3D models of homes from smartphone images. The 200-employee firm says the proceeds will be used as Hover expands its product offerings to strengthen established partnerships with insurance companies.  6.26% of insured homes experienced a claim in 2017, compared to just […]

Worlwide

Google today revealed Google Maps updates aimed at warning users of pandemic-related threats. Soon, maps will display all-time COVID-19 cases identified in an area, along with fast links from local authorities to resources. Google will also start to demonstrate how bus, train, and subway lines are crowded in more locations across the globe. Maps also […]
Today, $60 million was raised by Hover, a startup creating AI-powered apps that build 3D models of homes from smartphone images. The 200-employee firm says the proceeds will be used as Hover expands its product offerings to strengthen established partnerships with insurance companies.  6.26% of insured homes experienced a claim in 2017, compared to just […]
Motional, the joint autonomous driving alliance between Aptiv and Hyundai, announced today that the state of Nevada has obtained permission to test its autonomous vehicles without a driver behind the wheel. The firm claims this is part of the completion of a phase of self-imposed testing and evaluation.  In the U.S., relatively few businesses have […]

Trending

WHEN SARTRE SAID hell is other people, he wasn’t living through 2020. Right now, other people are the only thing between us and species collapse. Not just the people we occasionally encounter behind fugly masks—but the experts and innovators out in the world, leading the way. The 17-year-old hacker building his own coronavirus tracker. The […]
13 September marks six months since the first coronavirus announced in Ethiopia.In the half-year since then, reported cases are close to 64 Thousend, with more than 996 deaths. At the onset, COVID-19 mainly affected the capital city. However, the virus is now moving from high-density urban areas to informal settlements and then onward to rural […]
Present international artificial intelligence (AI) inventory and progression in self-driving vehicle research and development Complementary subjects in technology are also artificial intelligence ( AI) and self-driving vehicles. In brief, without someone involved, you just can’t debate one. While AI has been rapidly applied in different areas, a new hot topic has been the way you […]

Login

Welcome to Intech Analytica

AI news hub. It checks trusted sites and collects best pieces of AI info.
Join Intech Analytica