topic Re: GPU accelerator not matching with desired memory. in Administration & Architecture

GPU accelerator not matching with desired memory.

jaytimbadia — Wed, 22 Jan 2025 10:27:52 GMT

Hello,

We have opted for Standard_NC8as_T4_v3 which claims to have 56GB memory.

But, when I am doing nvidia-smi in the notebook, its showing only ~16 GB,

Why?

Please let me know what is happening here?

Jay

Re: GPU accelerator not matching with desired memory.

Alberto_Umana — Wed, 22 Jan 2025 12:54:48 GMT

Hello @jaytimbadia,

When you run the nvidia-smi command, it reports the GPU memory, which is why you are seeing approximately 16GB. The 56GB refers to the overall system memory available for the instance, not the memory available on the GPU itself.

The discrepancy you are observing is due to the difference between the total system memory and the GPU memory. The Standard_NC8as_T4_v3 instance type has a total of 56GB of system memory (RAM), but the NVIDIA T4 GPU within this instance has only 16GB of GPU memory

Re: GPU accelerator not matching with desired memory.

jaytimbadia — Thu, 23 Jan 2025 04:42:05 GMT

Ok, so where is this mentioned? Can you please provide some transparency?

Re: GPU accelerator not matching with desired memory.

Alberto_Umana — Thu, 23 Jan 2025 12:52:17 GMT

Please refer to: https://learn.microsoft.com/en-us/azure/databricks/compute/gpu