Hardware requirements

This setup works on on-premises Nvidia GPUs, but also works with cloud providers such as AWS, Azure and GCP. Suggested GPU types include those from the RTX, Tesla, and Ampere family of products which have enough GPU memory and processing capability.

The main difference between these two types of GPUs is that the ones with virtualization usually have more GPU RAM and are offered by most cloud providers. Having more GPU RAM increases the maximum size of the image you can input to the model. In conclusion, virtualization GPUs are not significantly faster that the consumer GPUs.

You need a machine with the following hardware specifications:

Hardware specification	Requirements
Memory	minimum: 8 GB recommended: 16 GB
CPU	minimum: 2 cores with AVX2 enabled recommended: 4 cores with AVX2 enabled
GPU	minimum: 8 GB VRAM recommended: 16 GB VRAM
Storage	minimum: 30 GB free

Important: For optimal performance, we recommend upgrading the GPU, as the new model is optimized for the newer Turing architecture GPUs - out of which the T4 is best suited in terms of both cost and performance. These optimizations are not available on the Pascal family of GPUs, so there will be a slight performance gap when running the new model on one.

Note: For installations using Nvidia vGPU to work, make sure the vGPU is CUDA-enabled and the license is configured correctly.

Depending on the configuration you use (recommended GPU: Nvidia T4), you can expect the following performances (processing time is measured in seconds):

Resolution	Inference time	Total service time
1280x720	0.367	0.388
1440x900	0.487	0.515
1600x900	0.503	0.533
1920x1080	0.562	0.598
1920x1200	0.636	0.675
2560x1440	0.832	0.884
3840x2160	1.484	1.581