Databricks Community

RiyuLite · ‎10-05-2023

I need to retrieve the accountBillage usage from Audit logs

I have enabled Diagnostic logs, and it's been 36 hours. While enabling the logs , I selected every possible logs in this image.

But still i am not able to see the containers for account level logs

I checked every container, present here, couldn't find the AcountBilling logs. How long would it take for logs to get updated? Or did i miss any step to load and retrieve the account usage logs?

Kaniz_Fatma · ‎10-05-2023

Hi @RiyuLite, it seems like you have enabled the Diagnostic logs correctly. However, it might take some time for the logs to appear. The initial setup of audit log delivery takes up to one hour for log delivery to begin. After log delivery begins, auditable events are typically logged within 15 minutes. Additional configuration changes usually take an hour to take effect.

Account-level audit events that are not associated with any single workspace are delivered to the workspaceId=0 partition if you configured audit logs delivery for the entire account.

If you still can't find the logs, please make sure you have checked the correct location. The delivery location is <bucket-name>/<delivery-path-prefix>/workspaceId=<workspaceId>/date=<yyyy-mm-dd>/auditlogs_<internal-id>.json.

If you're still unable to find the logs, it's possible that there might be an issue with the configuration, or there might be a delay in the delivery of the logs.

If the problem persists, I recommend contacting Databricks support for further assistance.

RiyuLite · ‎10-06-2023

Hi @Kaniz_Fatma ,

Aren't these the steps to enable billable logs in AWS, Do Azure also follow the same steps to enable the account-level logs?

Can you please tell the step by step procedure to enable account level audit logs for Azure databricks, I followed all the steps present in the documentation, but was only able to enable it for a single workspace, Please do let me know how it's done for a complete account.

Kaniz_Fatma · ‎10-06-2023

Hi @RiyuLite , These steps are specific to AWS and cannot be directly applied to Azure. Azure has its own set of services, APIs and mechanisms for log delivery and management. I would get back to you on it.

RiyuLite · ‎10-10-2023

Hi @Kaniz_Fatma ,
Do I need owner/root account access to enable account level (billing-usage-logs) in Azure?

Kaniz_Fatma · ‎11-17-2023

Hi @RiyuLite , To enable account level (billing-usage-logs) in Azure, you do not need owner/root account access. You can provide others access to the billing information for your account in the Azure portal by ass....

The Owner role gives the user full access to all resources in the subscription, including the right .... The Account Administrator is the only owner for a Microsoft Online Service Program billing account. The role is assigned to a person who signed up for Azure. Account Administrators are authorized to perform various billing tasks like create subscriptions, vi....

I hope this helps!

Kaniz_Fatma · ‎11-17-2023

Hi @RiyuLite , Regarding your first question,

To retrieve the account usage logs from audit logs, you need to enable diagnostic settings for your Azure resources. Based on the error message you’ve shared, it seems that you have enabled diagnostic logs correctly, but you are not able to see the containers for account level logs.

It might take some time for the logs to appear. The initial setup of audit log delivery takes up to one hour for log delivery to begin. After log delivery begins, auditable events are typically logged within 15 minutes. Additional configuration changes usually take an hour to take effect.

If you still can’t find the logs, please make sure you have checked the correct location. The delivery location is <bucket-name>/<delivery-path-prefix>/workspaceId=<workspaceId>/date=<yyyy-m...2.

If you’re still unable to find the logs, it’s possible that there might be an issue with the configuration, or there might be a delay in the delivery of the logs. If the problem persists, I recommend contacting Databricks support for further assistance.

I hope this helps!