GPU accelerator not matching with desired memory.

jaytimbadia
Databricks Partner

Hello, 

We have opted for Standard_NC8as_T4_v3 which claims to have 56GB memory. 

But, when I am doing nvidia-smi in the notebook, its showing only ~16 GB, 

Why?

Please let me know what is happening here?  

Jay

Alberto_Umana
Databricks Employee
Databricks Employee

Hello @jaytimbadia,

When you run the nvidia-smi command, it reports the GPU memory, which is why you are seeing approximately 16GB. The 56GB refers to the overall system memory available for the instance, not the memory available on the GPU itself.

The discrepancy you are observing is due to the difference between the total system memory and the GPU memory. The Standard_NC8as_T4_v3 instance type has a total of 56GB of system memory (RAM), but the NVIDIA T4 GPU within this instance has only 16GB of GPU memory

jaytimbadia
Databricks Partner

Ok, so where is this mentioned? Can you please provide some transparency?

Alberto_Umana
Databricks Employee
Databricks Employee