tract - an efficient rust neural network inference engine

<p>Deploying neural networks in production environments presents unique challenges: models must run efficiently across diverse hardware, from powerful servers to resource-constrained embedded devices, while maintaining predictable performance without heavy runtime dependencies.</p> <p>This talk introduces <a href="https://github.com/sonos/tract"><strong>tract</strong></a>, <a href="https://sonos.com/">Sonos</a>'s open-source neural network inference toolkit started in 2018 and written in Rust. We'll explore how tract bridges the gap between training frameworks and production deployment by offering a no-nonsense, self-contained inference solution used today to deploy deep learning on millions of devices at Sonos.</p> <p>This toolkit has some unique strengths thanks to embedded graph optimization, automated streaming management, and symbolic abstraction for dynamic dimensions — plus support for multiple open exchange standards including <a href="https://onnx.ai/">ONNX</a>, <a href="https://www.khronos.org/nnef">NNEF</a>, and TensorFlow Lite.</p> <p><a href="https://github.com/sonos/tract"><strong>tract</strong></a> also has a companion project coined <a href="https://github.com/sonos/torch-to-nnef"><strong>torch-to-nnef</strong></a> that strive to export PyTorch models to an NNEF optimized for tract with maximum compatibility. It enables some unique features like quantization, better Fourier Transform support and easier extensibility: this will also be discussed shortly during this presentation.</p>