Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
Hi Support TeamWe want to connect to tables in Azure Databricks via Power BI. We are able to connect this via Power BI Desktop but when we try to Publish the same, we can see the dataset associated does not refresh and throws error from Powerbi.comIt...
Hi NoorThank you soo much for your response. Please see the below details for the error message. I just got to know that Power BI are Azure Databricks are in different tenants. Do you think it causes any issues? Do we need VNet peering to be configur...
Hi, what is the custom alert about? Also, could you please raise an Azure support case on the same to triage this further? Please tag @Debayan with your next response which will notify me. Thank you!
Before running a script which would create an agent on a cluster, you have to provide SPARK_LOCAL_IP variable. How can I find it? Does it change over time or its a constant?
Hi, Could you please refer to https://www.datadoghq.com/blog/databricks-monitoring-datadog/ and let us know if this helps. SPARK_LOCAL_IP is the environment variable, FYI, https://spark.apache.org/docs/latest/configuration.html
Hi All,Vnet peering settings is not enabled in Azure databricks , even though its deployed inside my VNET?Here i not mentioned my vnet and subnet details , but filled this and created databricks (without private endpoint - allow public access)virtual...
Hi, VNET peering is not supported or possible on VNET-injected workspaces. Please refer: https://learn.microsoft.com/en-us/azure/databricks/administration-guide/cloud-configurations/azure/vnet-peering#requirements
Hi @maddy v ,I recommend that you use the Databricks SQL module for this type of reports and email alerts. It is a very interesting module with multiple options for your use case.https://learn.microsoft.com/en-us/azure/databricks/sql/user/dashboards...
Using the Azure Databricks R Shiny example, I am easily able to instantiate a R Shiny web front session. The URL that is provided by Databricks provides the proxy session that allows my targeted audience to directly access my `R Shiny` app using tha...
Dear Team,I am unable to create a SQL Warehouse using AZURE Databricks subscription , below are the detailsi am able to create a cluster with single node ,but i am not able to create SQL Warehouse , i am using Cluster size - 2X Small , as part of the...
In portal.azure.com, write quotas and ask for an increase. https://portal.azure.com/#view/Microsoft_Azure_Capacity/QuotaMenuBlade/~/myQuotasAdditionally, in SQL warehouse "Advanced options," you can change "Spot instance policy" from "cost-optimized"...
HelloWe are looking to move a PAYG subscription to a new tenant. We've got Azure Databricks deployed. There are a few questions:Will Databricks continue to function properly (aside from the RBAC list which we will re-apply to all resources)Since Data...
Is there a way to create a generic user account and personal access token to connect to databricks. I have Azure build pipeline and VSCode test that is using my personal access token for running builds and tests.
You can create a service account (principle) for jobs, applications etc. Here's a link to the docs:https://docs.databricks.com/administration-guide/users-groups/service-principals.html
Azure Databricks provisions a Managed Resource Group when you create your Workspace. Is there a way to specify the name of the Managed Resource Group and its resources during creation?The defaults created by the Workspace violate my company's standar...
Hey there @Robert Biddle Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too.Cheers!
I'm using Azure Databricks Python notebooks. We are preparing a front end to display the Databricks tables via API to query the tables. Is there a solution from Databricks to host callable APIs for querying its table and sending it as response to fro...
@Prabakar Ammeappin Thanks for the linkAlso was wondering for web page front end will it be more effective to query from SQL Database or from Azure Databricks tables. If from Azure SQL database, is there any efficient way to sync the tables from Az...
