Latest News

Why I invested in Neural Magic — software that aims to replace high-cost, specialized AI hardware.

We’ve heard for decades that “software eats hardware,” and it continues to be true. The latest episode starts in a suburb of Boston with a little-known company called Neural Magic. A spin out of MIT by a brilliant professor and his multi-talented protege, this company is taking on the multi-billion dollar market of hardware accelerators by making machine learning algorithms run faster using software; not specialized hardware.

But I’m getting ahead of myself. Let’s step back and set the stage for the latest disruption. Over the last decade, we have seen the blossoming of machine learning algorithms due to their ability to be executed scalably on specialized hardware, namely Graphics Processor Units or GPUs. These processors are the same ones that render those amazing graphics pumped out by video games. The rise of machine learning and specifically neural networks is thanks to GPUs’ raw compute power, which can run neural networks that were previously impossible to execute at scale.

These GPUs unlocked the potential for neural networks to provide product recommendations, image recognition, object detection and other real world use cases. Now deep-learning based neural networks can drive cars, translate languages, recognize faces, generate realistic images of people, write convincing news articles, teach robots to grasp, and of course beat world champions in Go and Starcraft.

Thus the rush to apply ever more sophisticated hardware to accelerate the execution of neural networks. Large technology companies like Nvidia and Google and dozens of startups have invested hundreds of millions into specialized hardware. VCs invested $1.5B into AI hardware startups in 2017, justified by an expected market size of $66B by 2025. At least one of those startups, Graphcore, reached unicorn status with a $1.5B valuation based on a $200M Series D round in 2018.

However, there’s a little-discussed problem: simply applying more computing power to machine learning’s greatest challenges is inefficient, and still requires data science teams to compromise on the accuracy of their models to achieve incremental performance gains. These tradeoffs could mean, for example, the difference between discovering a tiny cancerous cell in an extraordinarily large medical image, or missing it altogether when the image size is reduced to fit within a GPU’s limited memory.

Enter the not so mild-mannered MIT Professor, Nir Shavit, and his truly mild-mannered post-doc, Alex Matveev, with a different strategy to accelerate neural networks: software. Software that is able to reformulate the neural network, retain its accuracy and deliver GPU-class performance and much more on commodity Intel CPUs. This is a first, and while still early, results are spectacular. Bryan House was soon brought on-board to help bring the magic to market.

I met this company and was immediately impressed. Their claim was as least as ambitious as it was contrarian. While other investors were chasing ever more expensive and specialized hardware to accelerate neural networks, I was asked to believe that you could match or even exceed GPU performance through software at lower cost, without the accuracy tradeoffs of specialized hardware. Needless to say, we spent months trying to validate this assertion. Everyone we spoke to said it couldn’t be done. We were finally able to test it ourselves and, to our surprise, indeed saw the performance improvement.

So we were left with a decision to buck the entire venture capital community who was investing in hardware. Many say venture firms travel in herds. Not Comcast Ventures. Not me. That’s not my M.O. While others were investing in quantum computing hardware, I invested in quantum software. So, for me, it was an easy decision. Here was an amazing team, with real technology, in a huge market and another case of software eating hardware. We at Comcast Ventures decided to lead Neural Magic’s seed round of funding and are eagerly looking forward to working with Nir, Alex, Bryan, and the rest of the team to build a great company together.

Software Eats Hardware: The Sequel was originally published in The Forecast on Medium, where people are continuing the conversation by highlighting and responding to this story.