Huawei

Unlike most accelerators, the processor includes 16 Arm-compatible CPU cores, allowing it to run without an external host processor. The company reused the Taishan Arm v8.2 CPU from its Kunpeng 920 server processor. All 16 cores share a 16MB L3 cache, which connects with a pair of DDR4 SDRAM channels for external memory. The CPU subsystem enables the chip to run a full operating system. The 910 also has four video decoders.

In TSMC’s 7nm EUV (N7+) technology, the compute diemeasures 456mm² including on-die memories plus the Arm CPU complex. The Ascend 910 separates I/O functions into a 168mm² die called Nimbus, which is likely built in an older 12nm or 16nm process to reduce cost. Nimbus provides 24 lanes of PCIe Gen4 and two different cache-coherent interfaces: CCIX and a proprietary port called HCCS. Three HCCS ports, operating at 30GB/s each, enable a fully connected four-chip cluster. For larger clusters, the chip also has dual 100Gbps Ethernet ports that handle RDMA (RoCEv2), providing a low-latency interconnect for multiple chassis. Huawei rates the Ascend 910 chip at 310Wbut the complete accelerator card at 350W.

The company has announced several products using the Ascend 910: the Atlas 300AI training card, Atlas 800 AI server, and Atlas 900 AI cluster. The Atlas 800 server integrates eight Ascend 910 modules, four Kunpeng 920 processors, and 8x100GbE ports. The Atlas 300 PCIe accelerator card and Atlas 800 server began shipping in 1Q20. The Atlas 900 cluster, which we assume is primarily intended for the company’s public cloud services (Huawei Cloud), has up to 4,096 DaVinci nodes.

The Atlas 200module, which employs the Ascend 310 chip, targets edge systems that use less than 10W. To reduce power, this chip has only two DaVinci cores running at 500MHz, generating a peak performance of 16 TOPS (INT8). It also features eight Cortex-A55 CPU cores, a single Cortex-M3 CPU for real-time functions, and a video engine capable of decoding sixteen 1080p video channels at 30fps in H.264 or H.265 format. The module sports a full set of I/Os including four PCIe Gen3 lanes, USB, Gigabit Ethernet, and serial ports. Packing 4GB of LPDDR4X DRAM, the module has a typical power of 5.5W; the 8GB version requires 8W. These modules entered production in 3Q19.

To program its Atlas devices, Huawei developed a neural network framework called MindSpore, which it recently open-sourced. It also provides drivers for standard frameworks such as Pytorch and TensorFlow. It hopes to build a new ecosystem around MindSpore, but the standard frameworks allow customers to easily port existing neural networks. The company has expanded the software stack with algorithm libraries, pre-trained models, and other development aids.

Huawei has disclosed little benchmark data for the Ascend 910. It reports a speed of 1,787 IPS for a single chip when training ResNet-50—9% better than Nvidia’s V100. It also posted MLPerf Training results for a 512-chip cluster—again, only for ResNet-50—that was 32% faster than a similar cluster of V100 GPUs. On Bert-base, a single Ascend 910 achieves 210 sentences per second, about 25% ahead of the V100. All of the company’s tests employ its MindSpore software.

Conclusions

As Huawei’s first AI accelerator for the data center, the Ascend 910 is impressive. It comes within 20% of Nvidia’s A100 in peak performance on both FP16 (training) and INT8 (inference) data while using slightly less power. It’s also similar in on-chip-cache size and HBM bandwidth. More importantly, it achieves about 90% of the A100’s MLPerf score for training ResNet-50 when both are running in a 512-chip cluster. Despite being surpassed by Nvidia, the Ascend 910 gives Huawei customers a viable alternative for AI acceleration.

US sanctions, however, cut off Huawei’s supply of custom Ascend chips, and it can no longer supply systems using the accelerator, although customers can still rent it through the Huawei Cloud service. The sanctions also prevent the company from deploying design improvements, including architecture changes and a move to 5nm. In the meantime, Nvidia and other competitors will forge ahead. Until Huawei can resolve the US sanctions, realizing its AI dream will be impossible.