Table of Contents
1 Executive Summary
2 AI Processing for Camera Image Data
3 NVIDIA NGC on Supermicro Validated NGC
Ready Systems
4 Al/Ml Deployment in TensorRT
5 Converting ML Model to TensorRT
6 Inference Benchmark Results
7 Sizing for AI Inference
8 Supermicro GPU Servers Specifications
9 How to Run NGC
10 Guidelines for Model Development
11 Additional Training Results
12 Support & Services
13 Conclusion
14 References
Super Micro Computer, Inc.
980 Rock Avenue
San Jose, CA 95131 USA
www.supermicro.com
White Paper
Supermicro® Systems Powered
by NVIDIA GPUs for Best AI
Inference Performance Using
NVIDIA TensorRT
NVIDIA AI Software from the NGC Catalog for Training and Inference
Executive Summary
Deep learning inferencing to process camera image data is becoming
mainstream. With the availability of high-resolution network cameras,
accurate deep learning image processing software, and robust, cost-
effective GPU systems, businesses and governments are increasingly
adopting these technologies. The use cases include retail inventory
tracking, on-premise security, insurance claim damage assessment,
medical image diagnosis, and many other applications.
This document demonstrates the benefits of using NVIDIA NGC and
NVIDIA TensorRT to get the best inference performance using
Supermicro systems powered by NVIDIA GPUs. It also shows how to set
up NGC on a Supermicro server and how to use TensorRT for
inference. The primary focus of this paper is about the key capabilities
of Supermicro systems powered by NVIDIA GPUs for inference.
Benchmark data that Supermicro engineers collected, sizing
recommendations, and server selection for inference deployment are
also included in this paper.