GPU accelerator not matching with desired memory.

[jaytimbadia]

Hello, 

We have opted for Standard_NC8as_T4_v3 which claims to have 56GB memory. 

But, when I am doing nvidia-smi in the notebook, its showing only ~16 GB, 

Why?

Please let me know what is happening here?  

Jay

[Alberto_Umana]

Hello @jaytimbadia,

When you run the nvidia-smi command, it reports the GPU memory, which is why you are seeing approximately 16GB. The 56GB refers to the overall system memory available for the instance, not the memory available on the GPU itself.

The discrepancy you are observing is due to the difference between the total system memory and the GPU memory. The Standard_NC8as_T4_v3 instance type has a total of 56GB of system memory (RAM), but the NVIDIA T4 GPU within this instance has only 16GB of GPU memory

[jaytimbadia]

Ok, so where is this mentioned? Can you please provide some transparency?

[Alberto_Umana]

Please refer to: https://learn.microsoft.com/en-us/azure/databricks/compute/gpu