Despite there is an explanation "The NCv3-series and NC T4_v3-series sizes are optimized for compute-intensive GPU-accelerated applications. Some examples are CUDA and OpenCL-based applications and simulations, AI, and Deep Learning. The NC T4 v3-series is focused on inference workloads featuring NVIDIA's Tesla T4 GPU and AMD EPYC2 Rome processor. The NCv3-series is focused on high-performance computing and AI workloads featuring NVIDIA’s Tesla V100 GPU.", I cannot understand the difference between inference workloads and high-performance computing and AI workloads.
https://learn.microsoft.com/en-us/azure/virtual-machines/sizes-gpu
Can anyone explain what they are?
With "NC T3 v3" you get an Nvidia Tesla T4 GPU (you also get one for free in Google Colab). The T4 is basically identical with a downclocked RTX 2070 Super gaming GPU but has 15GB memory at 320GB/s. It's ok for FP32 compute and has Tensor cores for FP16 matrix operations, which are commonly used in training neural networks (inference).
With the "NCv3" you get a much more powerful Nvidia Tesla V100 with 16GB memory at 900GB/s. This can do everything the T4 can, but a lot faster since it's memory is about 3x as fast, and it can do FP64 compute as well. FP64 is required by some compute workloads (for example physics simulations), and only very few GPUs can do it.