Google’s Android team today unveiled a prototype feature that enables developers to use Facebook’s PyTorch machine learning platform with hardware-accelerated inference. This helps more developers to exploit the ability of the Android Neural Network API (NNAPI) to run computer-intensive on-device AI models. This collaboration between the Android team and Facebook, Google says, will allow millions of Android users to benefit from experiences powered by real-time computer vision and audio enhancement models, such as 360-degree virtual backgrounds from Facebook Messenger.
Machine learning on the system will strengthen features that run locally without transmitting data to a remote server. On-device data processing results in lower latency and can enhance privacy, enabling apps to run without connectivity.
A single set of APIs to take advantage of accelerators such as graphics cards, digital signal processors, and neural processing units is available in the latest version of NNAPI, which is designed to run AI operations on Android devices. The launch of NNAPI 1.3 was marked by Android 11, which added support for service API consistency, memory domains, and more. This prototype builds on the robust support of partners across the Android ecosystem for over 100 processes, floating points, quantized data types, and hardware implementations.
Facebook says that speed has doubled, as has the reduction in power requirements using the PyTorch NNAPI prototype feature. This is in addition to offloading work from the processor, enabling other tasks to be done by the underlying models.
An Android C API or a higher-level application such as Google’s TensorFlow Lite will access the NNAPI. Support for well-known linear convolutional and multilayer perceptron models on Android 10 and above is included in the initial release. Compared to a single-threaded processor, performance testing using the popular MobileNetV2 computer vision model showed approximately 10 times the speed. Google says that future updates will integrate additional operators and model architectures, including Mask R-CNN, an object detection and instance segmentation model, as part of the work towards a stable release.
The PyTorch-focused Android enhancements follow the debut of PyTorch 1.4, which introduced a system based on the PyTorch Mobile for Android interface for distributed model parallel training and Java support for PyTorch inference. The experimental feature was only available for Linux and for inference when released. PyTorch Mobile for iOS and Android devices was released last fall as part of the PyTorch 1.3 rollout, with performance improvements coming from quantization, Google TPU support, and a JIT compiler update. (PyTorch currently supports Python and C++.)