I am currently taking the new Advanced Data Engineering with Databricks training. I am stuck in the Demo: Auto Load to Bronze section of Streaming ETL Patterns with DLT. Looks to be an access issue to AWS as the pipeline errors out with the following...
Jobs within the all purpose DB Cluster are failing with "the spark driver has stopped unexpectedly and is restarting. Your notebook will be automatically reattached"In the event log it says "Event_type=DRIVER_NOT_RESPONDING & MESSAGE= "Driver is up b...
just now there is one cluster issuecluster error: Driver is unresponsive likely due to GCcluster conf:worker: Standard_D8ads_v5Driver: standard_E16d_v4What do you suggest here ??
Hello Team, we are trying to refreshing a dataset which has near about 1 billion rows and we have partiotioned it to run periodicaly and in parralel-distrubted mechansism but the refrshing is failing after hours stating inactivity time out error are ...
Hi @Zahid-CSA ,You can take a look at below thread, kind of similar problem. @SethParker suggested one possible solution, maybe worth a try:Re: Power BI Import Model Refresh from Databricks ... - Databricks Community - 51661
I’m using Databricks on GCP and currently facing an issue where I want the RunName of a job to be displayed as the source in the Job Compute Grid.Here’s what I’ve already done:I’ve included the RunName in the custom_tags section of my cluster JSON:Wh...
What is the preferred way of accessing a UC enabled SqlWarehouse table from Databricks Spark Cluster . My requirement is to fetch the data from a sqlwarehouse table using complex queries, transform it using Pyspark notebook and save the results.But t...
Hi @phanisaisrir ,Use spark sql. This is the native and most integrated way to interact with data within Databricks.
Hi, "Data" is not showing up in the left hand side menu. I have attached the cluster to the notebook, pfa snapshot for the reference.Any idea, how can it be resolved ?
Thanks for the response @szymon_dybczak .Actually, I am having community edition. It seems data pane is unavailable for the community edition.
I am seeking how can I serve a model from Databricks. My question is once I get the endpoint of the model, will the endpoint be accessible outside Databricks?
I tried to curl the following url in a notebook %sh curl https://staging-api.newrelic.com/graphql -v But I got the following error message { [5 bytes data] * TLSv1.2 (OUT), TLS header, Supplemental data (23): } [5 bytes data] * TLSv1.2 (IN), TLS he...
Hi im trying to connect to Databricks using the mulesoft database connector, using databricks JDBC, I already connect using personal access token, but i need to use OAuth2 M2M, and only return a error 500151 Invalid local Address:org.mule.runtime.api...
Assuming you followed the guide (https://learn.microsoft.com/en-us/azure/databricks/integrations/jdbc/authentication#--oauth-machine-to-machine-m2m-authentication), is it possible your workspace uses private link?because that won't work.Also: only Da...
When running a Databricks notebook,an error occurs stating that SOME_TABLE is not a Delta table.However, after executing the describe detail command and checking the format,the table is shown as Delta.Without taking any specific actions, re-running ...
I am using Databricks Connect v2 to connect to a UC enabled cluster. I have a package I have made and installed in a wheel file on the cluster. When using vscode to import the package and use it I get a module not found error when running cell by ce...
Hi,Last year I created a customer-academy account for learning purpose.New I need to move/merge my account to partner-academy using my company mail. The goal is to only have a partner-academy account where my primary mail will be my private email and...
Hello Databricks Community,I’m currently in the process of migrating our codebase to Unity Catalog using UCX and would appreciate some advice. Our environment has a mix of jobs and tables running on both Unity Catalog and hive_metastore.After running...
Hi @Reply_Domenico, How are you dong today?To filter jobs, try adjusting the UCX assessment query or use a script to exclude jobs already on Unity Catalog. Unfortunately, UCX doesn't yet have specific commands for automating code migration to Unity C...
Issues with UTF-8 in DLTI am having issues with UTF-8 in DLT:I have tried to set the spark config on the cluster running the DLT pipeline: I have fixed this with normal compute under advanced settings like this:spark.conf.set("spark.driver.extraJava...
@Retired_mod Hi, DLT has updated its runtime but I get a different error now:this is my code:
Hi,I hope this message finds you well.I am reaching out regarding a concern with Databricks Administrator privileges. I have an Azure subscription and I use Azure Databricks for my tutorials, but I currently do not have Global Administrator access, w...
Hi @Meghana_Vasavad ,During initial setup of unity catalog you need to find a person with global administrator role of Entra ID tenant. It's one time action, because then he can give necessary permission to manage catalog to your account, or even bet...
