Bugs uncovered by the NNSmith project

January 14, 2025 · View on GitHub

Important

Summary of Bugs

System#Fixed#Confirmed#Pending#Total
PyTorch6091584
PyTorch-ONNX Converter120012
ONNX1001
ONNXRuntime34411
TVM340640
TensorRT62210
TensorFlow113014
Hidet130013
Sum1302827185

Note

  • Status: ✅ means fixed; 🔵 means confirmed; 🚨 means this bug has been marked with a high-priority label (PyTorch)
  • Symptom: 💥 Crash or exception; 🧮 Result inconsistency (silent semantic bug); 🧴 Sanitizers

PyTorch

PyTorch-ONNX Converter

ONNX

ONNXRuntime

TVM

TensorRT

TensorFlow

Hidet

Based on NNSmith, @soodoshll found a number of bugs for Hidet, including:

Note

Methodology

  • Though most bugs are identified via individual reports, there are cases where multiple similar-looking bugs are merged into one report to avoid potential duplication. Nonetheless, they might be counted for multiple times according to the actual required different fixes.
  • "won't fix" bugs are omitted.
  • Detected bugs come from the ASPLOS'23 (NNSmith) and FSE'23 (NeuRI) projects.