Gaudi2 Makes Impressive MLPerf Debut

In the latest round of MLPerf Training results, Graphcore’s Bow offers a modest improvement and Habana’s Gaudi2 triples performance over its predecessor, vaulting past Nvidia’s A100 on one benchmark.

Linley Gwennap

The latest round of MLPerf Training results includes a new object-detection neural network to better reflect real-world vision applications. It also includes two new AI accelerators: second-generation chips from Graphcore and Intel subsidiary Habana. Whereas Graphcore’s Bow offers a modest improvement, Habana’s Gaudi2 triples performance over its predecessor, vaulting into the industry lead on one benchmark.

Intel recently announced its Gaudi2 accelerator, which extends the original design with 3x more cores. Compared with the most recent Gaudi results, the new scores reveal a 3.05x throughput increase for ResNet-50 training, matching the company’s goal. On this test, Gaudi2 has a 1.5x advantage over Nvidia’s A100. But on Bert, the Habana accelerator delivers a bit less performance than the fastest A100 despite using more power.

The recent results are also interesting for what’s missing. Although Nvidia says shipments of its next-generation Hopper H100 chip are imminent, it submitted no results for the new GPU. Similarly, Intel declined to submit results for its Sapphire Rapids processor and its Ponte Vecchio (PVC) GPU, both of which are behind schedule but expected to ship in the next few months.

The MLPerf Training 2.0 release replaces the SSD model for lightweight object detection with a more modern and complex model called RetinaNet. This one-stage detection model employs a focal-loss function to ensure the training process emphasizes hard-to-predict cases, avoiding the need for a more complicated two-stage detector. The new benchmark also trains on Open Images, a larger and more diverse repository than the previous training set.