In High-Frequency Trading (HFT) speed and precision are paramount. Our recent collaboration with Napatech redefines these parameters by delivering ultra-low latency AI Inference solutions tailored for HFT environments.
The Challenge: Inference Latency in AI-Driven Trading
Financial institutions increasingly rely on AI and Machine Learning models to analyze vast datasets and execute trades. However, traditional server architectures often introduce latency and throughput bottlenecks that hinder real-time decision-making. In high-stakes trading, even nanoseconds can translate to significant financial gains or losses.
The Solution: Combining Napatech's Hardware with Xelera's Software
As we announced recently, Xelera has partnered with Napatech to provide an HFT inference solution comprised of Napatech SmartNICs and Xelera Silva inference acceleration software.
Napatech, renowned for its programmable FPGA-powered Network Interface Cards (NICs), has partnered with Xelera, a leader in AI acceleration software, to address these challenges. By integrating Xelera’s Silva™ software with Napatech’s programmable NICs, the collaboration offers a solution that significantly reduces AI inference latency. This integration ensures that AI inference processes occur within microseconds or even a single microsecond, enabling traders to make informed decisions faster than ever.
For a deeper dive into performance, please reference our whitepaper: LightGBM Inference Benchmark Report
For this blog, let’s focus on a Xelera Silva performance on LightGBM (Light Gradient Boosting Machine), a powerful, high-performance gradient boosting framework for machine learning tasks, running on a Napatech NT200A02 SmartNIC, with 128 features, 1000 trees and 8 levels of max depth per decision tree.
As you can see in the figure below, inference transactions, performed by Xelera Silva on Napatech SmartNICs run up to 30x faster than on Intel oneDAL, which has been optimized as a CPU based inference solution. Even at the 99th percentile, inference latency for Silva is close to 30x faster than the optimized Intel solution.
30x faster inference results. 30x faster decisions.
Comparing Xelera Silva performance to simple PCIe latency, the performance advantage becomes even clearer. Even with large models, Silva performs inference transactions at single micro-second scale.
Table 1: Xelera Silva performance relative to PCIe reads:
Implications for the Financial Sector
Our solution with Napatech is particularly impactful for the financial trading sector, where rapid data analysis and execution are critical. The combined solution not only accelerates AI inference but also ensures scalability and adaptability to evolving market demands. Financial institutions can now leverage this technology to gain a competitive edge, executing trades with unprecedented speed and accuracy.
Moving Forward
Xelera's collaboration between Napatech marks a significant milestone in the fusion of AI and financial trading. As markets continue to evolve, such innovations will be instrumental in shaping the future of trading, where milliseconds can determine success.
For more details on this collaboration, you can read the announcement here.
To learn more about the Xelera solution, please contact sales@xelera.io
To purchase from Napatech, please contact info@napatech.com
%201.svg){kind=icon}