We have been following the detailed efforts of Intel’s neuroumorphic efforts since it launched its first dedicated 14 nm silicon for neuromorphic computing, called Loihi, in early 2018. about hardware development and when we could see the second generation. Today is that day, and the group announces Loihi 2, a significant upgrade over the first generation that deals with many low fruits from the first design. What is perhaps equally interesting is the process node used: Intel announces that Loihi 2 is being built today, in silicone, using a pre-production version of Intel’s first EUV process node, Intel 4.
Neuromorphic computing for Intel
By creating an architecture that is at its core modeled like the brain, the idea is that owning millions of neurons and synapses will lead to computing tasks with unique strength / performance advantages in specific tasks for which the brain is designed. It is a long-term potential commercial product for Intel, however the team’s task was to develop technology and software to detect and accelerate tasks that match neuron-type computers.
The neuromorphic lab at Intel was actually backed up by the 2011 acquisition of Fulcrum Microsystems. At the time, the Fulcrum team was an asynchronous computer group working on network switches. That technology was moved to a network group within Intel, and the research department turned its attention to other uses of asynchronous computing and landed on Neuromorphic.
At the time, research on this type of neuromorphic computer architecture for actual workloads was still in its infancy — while the field had been present since the late 1980s, dedicated research hardware did not exist until the early 2010s. The Human Brain Project, a ten-year research project funded by the European Union to study the area, was founded only in 2013, of which the SpiNNaker system in 2019, with a million chips, a billion neurons, is 100 kW of active power.
By comparison, Intel’s first-generation Loihi supports 131,000 neurons at 60mm2 chip, and 768 chips can be assembled into a single Pohoiki Springs system with 100 million neurons in just 300 watts. In Intel marketing, they described this as the equivalent of a hamster. The new Loihi 2 chip, at a high level, uses 31 mm2 per chip per million neurons, effectively increasing the density 15x, however development exceeds raw numbers.
The high-level Loihi 2 chip might look similar: 128 neuromorphic nuclei, but now each nucleus has 8 times more neurons and synapses. Each of these 128 cores has 192 KB of flexible memory, compared to earlier where it was fixed per core at runtime, and each neuron can be assigned up to 4096 states depending on the model, while the previous limit was only 24. The neural model is now it can also be fully programmed, similar to an FPGA, which allows for greater flexibility.
Traditionally, neurons and spiked networks deliver data in a binary event, which Loihi v1 did. With Loihi 2, these events can be rated with a 32-bit payload, offering deeper flexibility for on-chip computing. These events can now be monitored in real time with new chip development / debugging capabilities, instead of pausing / reading / playing. Combined, this also allows for better control when dynamically changing computer loads, such as fan compression, weight scaling, convolution, and emission.
Perhaps one of the biggest improvements is connectivity. The first generation used a custom asynchronous protocol to create a large 2D network of neurons, while Loihi 2 can be configured to use different protocols as needed, but also in a 3D network. We are told that Loihi 2 is not just one chip, but it will be a family of chips with the same neural architecture, but a series of different connectivity options based on specific use cases. This can be used in conjunction with built-in message compression accelerators to effectively increase chip-to-chip throughput 10 times.
This extends to external Loihi connectivity to more conventional computing, which was previously mediated by FPGA – now Loihi 2 supports 10G Ethernet, GPIO and SPI. This should allow for easier integration without the need for custom systems, such as creating parsed Loihi 2 computational clusters.
Made on Intel 4
We were surprised to hear that Loihi 2 was built on a pre-production version of the Intel 4 process. We’re still a long way from Loihi 2 being part of Intel’s revenue, and the Neuromorfs team knows just as much, but the chip has proven to be perhaps the ideal candidate to launch a new process.
At 31 mm2, the size means that even if it is necessary to increase the yield, one board can offer more working chips than testing with a larger matrix size. While the team is doing post-silicon voltage / frequency / functionality testing, they can get back to Intel’s technology development team faster. We confirmed that there is silicone in the lab, and in fact the hardware will be available today via Intel’s DevCloud, directly on the metal, without any emulation.
Usually with new process nodes you need a customer with a small silicone matrix size to help you get through potential obstacles in bringing the process to full ramp and production. Intel’s competitors in the foundries usually do this with users who have chips the size of smartphones, and customer benefits usually mean hardware first or maybe some kind of initial discount (although, maybe not in today’s climate). Intel has previously struggled in that regard, as it only has its own silicon that it can use as a test vehicle.
The neuromorphic team said this fits really well, given that neuromorphic hardware requires high density and low static power provided by leading process nodes. The 128-core design also means it has a consistent, repeatable unit, allowing the process team to see regularity and consistency in production. Also, given that Loihi is still a research project for now, there are no serious expectations that the product will be launched in a particular window, which a large customer might need.
Does that mean Intel 4 is ready for production? Not exactly, but it indicates progress. A number of Loihi 2 metrics had a warning about “expected given simulated hardware results”, although several others were done on real silicon, and the company says it has real silicon for implementation in the cloud today. Intel 4 is Intel’s first process node with Extreme Ultra Violet (EUV) lithography, and Intel will be the last major semi-finished product manufacturer to launch the EUV manufacturing process. But we’re still a long way off – at Intel’s accelerated event, EUV and Intel 4 aren’t expected to really increase production until the second half of 2022.
In conclusion, from Intel’s announcement, we can look at the density of transistors. At 2.3 billion transistors in 31 mm2, this would lead to a density of 71.2 million per mm2, which is only a third of what we expect. Estimates based on Intel’s previous announcements would put Intel 4 at around 200 MTr / mm2. So why is Loihi 2 so low compared to that number?
The first may be that it is a neuromorphic chip and not a traditional logic design. The core has ~ 25 MB of SRAM along with its logic, which is 31 mm2 the chip could be a good part of the matrix area. Also, Intel’s main idea with neuromorphic chips is functionality first, second performance and third power. That’s why proper functioning is more important than fast work, so there’s not always a raw need for maximum density. There is also the fact that it is still a development chip and allows Intel to improve its EUV process and test accurate lithography without worrying about defects caused by dense transistor libraries. It will still come, I’m sure.
To add one more point, our briefing speculated that a neuromorphic IP address might be available in the future through Intel’s Foundry Service IP offering.
New Lava software framework
Regardless of the processing capabilities, one of the main building blocks for a neuromorphic system is the type of computation, and perhaps how difficult it is to write software to take advantage of such an architecture. In a discussion with Intel’s Mike Davies, director of Intel’s Neuromorphic Lab, we best described that modern computing is similar to a survey architecture – each cycle takes data and processes it. In contrast, neuromorphic computing is an interrupt-based architecture – it works when the data is ready. Neuromorphic computing depends more on the time domain than modern computing, so the concept of computing and the applications it can work on are almost orthogonal to traditional computing techniques. For example, while machine learning can be applied to neuromorphic computing in the form of Spiking Neural Networks (SNNs), traditional PyTorch and TensorFlow libraries are not built to enable SNNs.
Today, as part of the announcements, Intel is launching a new basic program framework for a neuromorphic community called Lava. This is an open source framework, not controlled by Intel, but by the community. Intel has framed a number of its early tools, and the idea is that over time, a complete software stack can be developed for use by anyone involved in neuromorphic computing, regardless of hardware (CPU, GPU, Neuromorphic Chip). Lava is designed to be modular, collapsible, extensible, hierarchical and open source. This includes a low-level interface for mapping neural networks to neuromorphic hardware, asynchronous channel-based message forwarding, and all libraries and functions are exposed through Python. The software will be available for free use under BSD-3 and LGPL-2.1 on GitHub.
The first version of Loihi 2 set up in Intel’s cloud services is the Oheo Gulch, which looks like an optional PCIe card that uses an FPGA to manage a large number of IOs, along with a connector on the motherboard if needed. 31 mm2 the chip is BGA, and here we see one of Intel’s internal connectors for holding BGA chips on the development board.
In the future, Intel will produce a 4 to 4 inch version called Kapoho Point, with eight chips on the board, designed for stacking and integration into a larger machine.
With such a small chip, I wonder if it’s not worth building it with a USB controller on silicon or having a USB-to-Ethernet interface and offering hardware on USB sticks, similar to what Intel Movidius once distributed. We asked Intel about extending the use of Loihi 2 to a wider non-research / non-commercial focused audience on tampering and home brewing, however since this is still an Intel Labs project, one of the key elements of the team is the dedicated collaboration they have with partners to advance the segment. . So, we will have to wait at least another generation or more to see if future Loihi systems will be offered on Amazon.
As of today, Loihi 2 should be available to research partners as part of Intel’s DevCloud. Local research / cooperation is expected in the next 12-24 months.
Friendly communicator. Music maven. Explorer. Pop culture trailblazer. Social media practitioner.