- 892 Views
- 5 replies
- 5 kudos
PowerApps + Delta Lake
Is there any native way to Insert data from PowerApps directly into Delta Lake?I can of course create a workflow - and push the data via API, but is there any alternative ?
- 892 Views
- 5 replies
- 5 kudos
- 5 kudos
Haven't thought of that actually, yes that will work! But still you need a seperate azure function for it, which is a bit cumbersome still I would say.
- 5 kudos
- 816 Views
- 5 replies
- 1 kudos
ADF - Fails to run job in workspace in another subscription
We are trying to run a Databricks job from ADF, but we keep getting the following error:Operation on target Run dbt job failed: {"error_code":"PERMISSION_DENIED","message":"User <adf-mi-prod-id> does not have Manage Run or Owner or Admin permissions...
- 816 Views
- 5 replies
- 1 kudos
- 1 kudos
I am seeing the error message above whenever job 123 is triggered by ADF, but I can't find any instances of this job running in Databricks.
- 1 kudos
- 901 Views
- 3 replies
- 2 kudos
Unable to login to databricks community edition
Hi Team, I was not able to login to Databricks community edition It’s working fine for me yesterday when I used but today while logging in it’s directing to same login page after entering user name and password . Can someone please look into this iss...
- 901 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi I have the same problem. I used CE for logging MLFlow experiments and now can't login into my account. Can you please look into it @Sujitha @Kaniz_Fatma?
- 2 kudos
- 263 Views
- 1 replies
- 0 kudos
Hi I am trying to change datatype of DOB column from stringtype to DateType but getting issue
Hi I am trying to change datatype of DOB column from String type to DateType but getting issue"SparkUpgradeException: [] You may get a different result due to the upgrading to Spark >= 3.0: Caused by: DateTimeParseException: Text '7/8/1972' could not...
- 263 Views
- 1 replies
- 0 kudos
- 0 kudos
It's because the format you're using it wrong. Use `'M/d/yyyy'` instead.
- 0 kudos
- 768 Views
- 2 replies
- 1 kudos
Resolved! Cluster Termination Issue
Hi, I am new to ADB and started recently using this community edition. I created a cluster and it is terminating immediately when i use to connect.What I am missing or what i need do? kindly help me.
- 768 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi Walter_C, thanks for your response. Your assumption is correct. Now I tried to create new Cluster whenever I start practice. Then no issues.
- 1 kudos
- 401 Views
- 2 replies
- 1 kudos
Winston-Salem/Greensboro, NC User Group
Hi All! I'm looking to start at Databricks Usergroup in the Greensboro/Winston-Salem, NC area. Wanted to gauge interest to find others who would like to join.
- 401 Views
- 2 replies
- 1 kudos
- 854 Views
- 4 replies
- 0 kudos
Where the "Driver logs" are stored by default, and How much default space to store it.
Recently, when I was using databricks for deep learning I ran into an issue, i.e., after a certain amount of time of execution the cluster would break and restart. The logs are as below:The Event logs displayed "Metastore is down; Driver is up but is...
- 854 Views
- 4 replies
- 0 kudos
- 0 kudos
Hello Jaron, is it not possible for you to redirect the login to an ABFSS or S3 bucket?
- 0 kudos
- 527 Views
- 1 replies
- 0 kudos
Databricks vs. Competitors: Key Features
Hi everyone,Can anyone share insights on the key features that differentiate Databricks from its competitors? Looking forward to your thoughts!Thanks!
- 527 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Sushant-db, Databricks stands out in several key areas compared to its competitors: Performance Optimization: Databricks optimizes Apache Spark™’s performance, offering faster data processing speeds1.Enhanced Data Reliability: Integration with...
- 0 kudos
- 630 Views
- 2 replies
- 0 kudos
Databricks JDBC driver attempting to pass access token (JWT) from an external IDP (OKTA)
Configured a Databricks workspace for SSO to an IDP (OKTA).Databricks JDBC driver 02.06.38.1068Attempting to connect to Databricks using a URL similar to following where access token is obtained from the IDP (OKTA). Using tool such as SQLSquirrel wit...
- 630 Views
- 2 replies
- 0 kudos
- 0 kudos
Yes, latest version 02.06.38.1068Would like to know if other persons have successfully passed access tokens from an external IDP via the driver.
- 0 kudos
- 3091 Views
- 4 replies
- 1 kudos
run datarbicks worflow as service pricipal (managed identity) reads from azure dev ops repo Failed
Hello,we are running a workflow as a service principal, that is a aad managed identity. This does result in the issue: run databricks workflow as service principal the reads from azure dev ops repo Failed to checkout Git repository: PERMISSION_DENIED...
- 3091 Views
- 4 replies
- 1 kudos
- 1 kudos
We managed to solve this problem, however it is not an elegant solution. Databricks should simplify this.The steps that have to be done are listed below. We are using user assigned managed identity (MI), but I assume this should work for Azure Servic...
- 1 kudos
- 1029 Views
- 1 replies
- 0 kudos
Cluster Auto Termination Best Practices
Are there any recommended practices to set cluster auto termination for cost optimization?
- 1029 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @dataailearner , Greetings of the day! Here are a few steps that you can follow for cost optimizations: 1. Choose the most efficient compute size: Databricks runs one executor per worker node. The total number of cores across all executors is a...
- 0 kudos
- 3017 Views
- 2 replies
- 0 kudos
Databricks Custom model Serving endpoint Failing
Hello all, I have created a custom model serving endpoint in Azure databricks. This endpoint connects with the AzureopenAI model and Azure postgres connection.All of these Azure services are with Private endpoints. When I run this notebook,I am able ...
- 3017 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @vaidhaicha, It sounds like you’re encountering issues with your custom model serving endpoint in Azure Databricks, specifically when querying through the serving endpoint using your Personal Access Token (PAT). A private endpoint is a network ...
- 0 kudos
- 442 Views
- 2 replies
- 0 kudos
VSCode Databricks Extension
Hi all,I've been trying to sync my VSCode IDE with our Databricks GCP workspace using the Databricks extension. I am able to connect authenticate my account and workspace and find our clusters. However, when I try to sync a destination it throws a st...
- 442 Views
- 2 replies
- 0 kudos
- 0 kudos
@Kaniz_Fatma thanks for you response.I am not running through a proxy. At least, not on purpose. How do I know if I am running through a proxy? And where can I find the values of <proxy_url> and <port> so that I can try restarting my VSCode.I have tr...
- 0 kudos
- 2973 Views
- 4 replies
- 2 kudos
Cluster compute metrics
I want to fetch compute metrics(hardware, gpu and spark) and use them in certain dashboard on Databricks, however i'm not able to fetch them. i have tried GET API request and system tables. The system tables only have CPU utilization and memory utili...
- 2973 Views
- 4 replies
- 2 kudos
- 342 Views
- 1 replies
- 0 kudos
Fetching CPU and memory data using REST APIs
Hi,I am trying to fetch CPU and memory details from Databricks. Are there any APIs present to which I can connect using postman and fetch these details?
- 342 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Neelofer, To fetch CPU and memory details from Databricks, you can utilize the Databricks REST APIs. While it doesn’t provide real-time data, the Spark UI offers information related to Spark-specific performance, such as job and task breakdo...
- 0 kudos
