- 1059 Views
- 1 replies
- 1 kudos
- 1059 Views
- 1 replies
- 1 kudos
- 1 kudos
please write more details and your code
- 1 kudos
- 1226 Views
- 1 replies
- 3 kudos
Resolved! How to specify directory when reading table?
Hi, Assume that I have created several tables in Databricks workspace. If two tables have similar names but stored in different folders, how I can read them by "spark.read.table". I ask this question because input to the "spark.read.table" is only ta...
- 1226 Views
- 1 replies
- 3 kudos
- 3 kudos
spark.read.table reads table registered in catalog/metastore. If you have multiple databases in metastore, just prefix table name with database name db.table
- 3 kudos
- 2740 Views
- 2 replies
- 2 kudos
Data source V2 streaming is not supported on table acl or credential passthrough clusters
Using:( hostname is hidden )kafka = spark.readStream\ .format("kafka")\ .option("kafka.sasl.mechanism", "SCRAM-SHA-512")\ .option("kafka.security.protocol", "SASL_SSL")\ .option("kafka.sasl.jaas.config", f'org.apache.kafka.common.security...
- 2740 Views
- 2 replies
- 2 kudos
- 2 kudos
With TACL enabled cluster, you got many restrictions, so streaming will not work. Generally, you can read only things registered in metastore; please disable it for your use case,Additionally, remember that the unity catalog doesn't support streaming...
- 2 kudos
- 968 Views
- 1 replies
- 0 kudos
Azure Extension to get Databricks Cluster ID
Is there custom azure extension to call databricks cluster ID on azure Pipeline
- 968 Views
- 1 replies
- 0 kudos
- 0 kudos
I do not believe there is. You will likely need to use the Databricks REST APIs.
- 0 kudos
- 626 Views
- 0 replies
- 2 kudos
www.linkedin.com
Calling all Data Engineers! If you’re looking to join the most iconic brand in STL as they transform how business decisions are made then this Bud is for you!https://www.linkedin.com/posts/colt-hubbartt-90578b13_senior-data-engineer-activity-69783757...
- 626 Views
- 0 replies
- 2 kudos
- 4601 Views
- 4 replies
- 1 kudos
Where did the settings button move to - where is the admin console?
I'm new to databricks and the gui just changed and I am trying to find answers in the release notes but can't find what I'm looking for
- 4601 Views
- 4 replies
- 1 kudos
- 1 kudos
I have found the Administrator Console. It is in the top right under my name. My ui is different.Thanks community for your support. I am a junior on this product.
- 1 kudos
- 10296 Views
- 3 replies
- 4 kudos
Performance Tuning of Databricks Notebook
Hi Everyone ,I am trying to run a databricks notebook in parallel using ThreadPoolExecutor .Can anyone suggest how to reduce the time taken based on the below findings so far.Current Performance:Time taken - 25 minutes ThreadPoolExecutor max_workers ...
- 10296 Views
- 3 replies
- 4 kudos
- 4 kudos
ThreadPoolExecutor will not help as Databricks/Spark will process job by job.So please analyze in Spark UI what is consuming the most time.There are a lot of tips on how to optimize they depend on the dataset (size etc. transformations)Look for data ...
- 4 kudos
- 3696 Views
- 2 replies
- 4 kudos
Resolved! Adjust label size in SQL visualization
Hi,Anyone knows if there's a way to adjust the font size of label in the SQL visualization? Our dashboard users complain that it's too small.
- 3696 Views
- 2 replies
- 4 kudos
- 4 kudos
It's on our radar to make this customisable in the future. For now, browser zoom as suggested by @debayan is the suggested workaround.
- 4 kudos
- 4531 Views
- 2 replies
- 4 kudos
Error while writing Data Frame from Azure Databricks to Azure synapse Dedicated Pool
I am reading delta tables from my Databricks workspace as DF and then I am trying to write this DF into Azure Synapse Dedicated SQL Pool, but I am getting error like:-Py4JJavaError: An error occurred while calling o1509.save. : com.databricks.spark.s...
- 4531 Views
- 2 replies
- 4 kudos
- 4 kudos
The issue was getting created because I was passing SAS Token for setting the configuration for Storage Account instead of Storage Access Key. It's working now without any issue..spark.conf.set("fs.azure.account.key.datalakestorage.dfs.core.windows.n...
- 4 kudos
- 697 Views
- 0 replies
- 0 kudos
The Nova City Islamabad, is an upcoming and emerging housing project in the federal capital. This project aims to target all budget genres of people o...
The Nova City Islamabad, is an upcoming and emerging housing project in the federal capital. This project aims to target all budget genres of people of Pakistan. In the future, this project is going to compete with the mega projects of Islamabad, Pak...
- 697 Views
- 0 replies
- 0 kudos
- 718 Views
- 0 replies
- 0 kudos
 The New Metro City Gujar khan Payment Plan has not yet been made public, but the developers are anticipated to do so shortly. All of the plots will b...
The New Metro City Gujar khan Payment Plan has not yet been made public, but the developers are anticipated to do so shortly. All of the plots will be within the budget and at reasonable pricing. The investment in this project will be simple and hass...
- 718 Views
- 0 replies
- 0 kudos
- 3316 Views
- 6 replies
- 0 kudos
Does Databricks support proxy for BigQuery?
Hi team, we tried to use the proxy options for BigQuery Spark connector as mentioned in this documentation. However, we keep getting "connect timed out" error. The proxy host is working on our end. This made us wonder if by chance Databricks does not...
- 3316 Views
- 6 replies
- 0 kudos
- 0 kudos
Hi @Ayushi Pandey​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 0 kudos
- 2157 Views
- 2 replies
- 0 kudos
Order By disabled in sql dashboard
Hello, I have an issue, when I created my query, with an Order By, and my vizualization, everything was in the right order. Then, when I put it in my dashboard the order changed. As you can see with the pictures, when I clik on "edit visualization", ...
- 2157 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @Laure Decaudin​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 0 kudos
- 8255 Views
- 4 replies
- 1 kudos
ADF - Failed To Send Request to Databricks Cluster
I have noticed that my orchestrated pipelines (in ADF) sometimes fail due to this error:ErrorCode=FailedToReadFromAzureDatabricksDeltaLake,Failed to read from azure databricks delta lake. Error message : Failed to send request to Azure Databricks Clu...
- 8255 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Oscar Dyremyhr​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 1 kudos
- 2876 Views
- 5 replies
- 2 kudos
Resolved! Databricks Premium for Personal
Hi Databricks Community,I want to setup premium cluster with multiple nodes to run some Spark workload for my personal project. Do Databricks offer​s premium upgrade for personal? How to do upgrade?​Thanks.​
- 2876 Views
- 5 replies
- 2 kudos
- 2 kudos
Hi @fritzwijaya​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks...
- 2 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
