Last year, Qualcomm stopped short of including an accelerator core called the neural processing unit its Snapdragon silicon. Instead, it announced that it would publish software to bend its existing smartphone chips to the whims of machine learning.
The strategy shift shows Qualcomm’s sense of urgency around machine learning, which has typically required powerful servers to understand speech and sensor readings. It also reflects the chipmaker’s plans, which it elaborated on Thursday when acquired Dutch machine learning start-up Scyfer.
Scyfer’s software has been used to classify manufacturing defects for Tata Steel and predict store revenues for Dutch supermarket chain Albert Heijh. It runs machine learning techniques on storehouses of data generated by factory sensors, online shoppers, cameras, bank transactions, and medical imaging machines.
Making sense of such information usually happens in data centers. Shelves of graphics chips train models on, for instance, the identifying features of images of cats. Then, algorithms apply that model to a particular cat in a smartphone video or photo. This is called inferencing, and it also typically occurs in the cloud.
Qualcomm wants to take inferencing out of the cloud, bringing it to individual gadgets. That would allow a smartphone to translate text written in a foreign language without calling back to the cloud, or a sensor to identify chemicals leaking from an oil refinery, instead of streaming raw data to the cloud for the final word.
Matt Grob, Qualcomm’s executive vice president of technology, wrote in a Thursday blog post that “in many cases, inference running entirely in the cloud will have issues for real-time applications that are latency-sensitive and mission-critical like autonomous driving.”
Grob added that such “applications cannot afford the roundtrip time or rely on critical functions to operate when in variable wireless coverage.” The benefits, he said, include more privacy and lower latency when chatting with the cloud over the internet. Messages sent to the cloud would also be condensed, saving battery life and network bandwidth.
“The cloud remains of course very important and will complement on-device processing,” Grob said. That strategy fits Qualcomm, which has been trying to plant roots in markets like robotics and sensors. Last year, it gave the clearest sign yet of its ambitions, agreeing to pay $47 billion for NXP Semiconductors, the world’s largest maker of automotive chips.
To bring intelligence to everyday gadgets, other companies are gunning for custom silicon. Last year, Intel bought vision chipmaker Movidius to enable intelligent security cameras and drones, while start-ups like ThinCI are cobbling together silicon engines for self-driving cars. Microsoft is customizing chips so that its HoloLens goggles can analyze what users have in their field of vision.
For years, Qualcomm teased plans of releasing silicon that resemble in a limited way how the human brain processes information. But last year, it appeared to shelve its neuromorphic chips in a pivot to software tools that make the most of conventional computing cores.
The software it released last year, called the Neural Processing Engine, carefully curates machine learning code that understands speech and translates text. It finds the right CPU, GPU, and DSP cores inside Qualcomm’s chips to execute tasks as fast and efficiently as possible.
“Qualcomm’s solutions already have the power, thermal, and processing efficiency to run powerful AI algorithms on the actual device,” Grob said in the Thursday blog post. “The diversity in architecture is essential and you can’t rely on just one type of engine for all workloads.”
The tool helps optimize algorithms for longer battery life and faster processing. Last month, Facebook announced plans to apply the software to augmented reality features in its smartphone app. The tool moves Facebook’s code into graphics cores, making augmented reality objects appear more realistic in photos or live video.
Buying Scyfer bolsters Qualcomm’s machine learning research ranks. The chipmaker adds Max Welling, one of Scyfer’s founders and a professor at University of Amsterdam. He is a former student of Geoffrey Hinton, who founded Google’s deep learning division and mentored scientists like Facebook’s Yann LeCun and Uber’s Zoubin Ghahramani.
Qualcomm didn’t disclose what it paid for the firm, which spun out of the University of Amsterdam in 2013.
