NeuPro-M Enhances BF16, FP8 Support

Ceva’s updated NeuPro-M artificial-intelligence accelerator can better handle floating-point and INT4 data, enabling developers to use different types at different neural-network layers to improve performance with less impact on accuracy.

Joseph Byrne

Ceva has updated its licensable NeuPro-M design to better handle Transformers—the neural networks underpinning ChatGPT and Dall-E AI software and now finding application at the edge in computer vision. Architectural changes to NeuPro-M improve its power efficiency and can increase its performance sevenfold on models that can take advantage of the new features. Critical among these is the addition of BF16 and FP8 support to more NeuPro-M function units and improved handling of sparse data.

Targeting AI inferencing tasks in embedded sensing applications like driver assistance, robotics, and surveillance cameras, the NeuPro-M architecture comprises a single “common” subsystem and one or more engines. The former interfaces to a host CPU and real-time peripherals such as an image signal processor (ISP), and it performs control, safety, and security functions as well as compressing/decompressing data and weights. The latter handles most of the processing, and Ceva scales the number of engines to address different performance levels.

The intellectual-property (IP) vendor offers NeuPro-M versions with one, two, four, or eight engines for up to 256 TOPS of raw performance. The two- and eight-engine models are new. The single-engine NPM11 is fully verified and has been delivered to its first customers. Ceva plans for the final RTL for the other configurations (NPM12, NPM14, and NPM18) to be available by the end of the year.