Administration & Architecture
Explore discussions on Databricks administration, deployment strategies, and architectural best prac...
1320 Posts
Activity in Databricks Platform Discussions
- 17 Views
- 1 replies
- 0 kudos
Restrict certain queries on SQL Warehouse
We manage a databricks workspace where majority of the users are working, we haven't switched over to Serverless yet because there is lot of observability and control that is lacking. So we are still on one classic SQL warehouse which is shared among...
- 17 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
You can implement a monitoring workflow that periodically scans query history for users issuing SET STATEMENT_TIMEOUT, SELECT Alive, Duplicate Query & Rate Limit statements. This can help identify users who are bypassing the globally configured defau...
- 0 kudos
- 17 Views
- 0 replies
- 0 kudos
Telg.ram@Easyphase Miami USA SNOW COKE PLUG
Discretion 100%, same day drops all over USA. Miami snow plug USA
- 17 Views
- 0 replies
- 0 kudos
- 36 Views
- 0 replies
- 0 kudos
Tel..gram@Easyphase Paris France SNOW COKE PLUG
Looking for a reliable Paris France snow plug
- 36 Views
- 0 replies
- 0 kudos
- 39 Views
- 0 replies
- 0 kudos
Telgra..am@Easyphase Vancouver Canada WEED THC CARTS PLUG
Discrete delivery Vancouver 420 weed coke ket snow plug in Canada
- 39 Views
- 0 replies
- 0 kudos
- 35 Views
- 0 replies
- 0 kudos
Telgra..am@Easyphase Toronto Canada SNOW COKE PLUG
Fast and Reliable Toronto snow coke ket plug in Canada
- 35 Views
- 0 replies
- 0 kudos
- 60 Views
- 0 replies
- 0 kudos
Telgr..am@Easyphase LONDON UK SNOW COKE PLUG
London snow plug UK
- 60 Views
- 0 replies
- 0 kudos
- 47 Views
- 0 replies
- 0 kudos
Models failing in tutorial
Hello,I am following the "Get started: Build your first machine learning model on Databricks" tutorial, and am getting stuck on "Parallel training using Optuna".When I Search runs to retrieve the best model, the following code fails as there are no m...
- 47 Views
- 0 replies
- 0 kudos
- 56 Views
- 0 replies
- 0 kudos
Does combo chart have group by color function ?
Does combo chart have group by color function by specific field (type)? Like Line Chart: the line will perfectly split into multiple lines of different colors, with each line representing a distinct Type.Bar Chart: The original single bar will also ...
- 56 Views
- 0 replies
- 0 kudos
- 82 Views
- 1 replies
- 0 kudos
Can multiple questions be added to the same sql query in genie space
Hi, Can we add multiple sample questions to one SQL query in the sql queries instructions so Genie learns to handle similar variations?
- 82 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @IM_01, The public guidance points to one natural-language question or title per example SQL query, rather than multiple sample questions attached to a single query. In the Tune Genie Space quality docs, Databricks says that for each example SQL q...
- 0 kudos
- 169 Views
- 5 replies
- 2 kudos
Resolved! Databricks unable to list ADLS folder and files
Hi Databricks Community,I am able to list the container from my databricks workspace but unable to list the folder and files further.If I try to access the same files and folder from the Databricks UI, external location path, I am able to see all fil...
- 169 Views
- 5 replies
- 2 kudos
Latest Reply
- 2 kudos
Following are may be the Causes1. Different authentication methods- The UI's external location uses Unity Catalog credentials- Your dbutils.fs.ls() command uses the compute's Spark configurations- These may be using different credentials with differe...
- 2 kudos
- 61 Views
- 1 replies
- 0 kudos
DESCRIBE HISTORY Performance Issue for Large Scale Tables (22K Tables)
Hi everyone, I’m working with around 22,000 Unity Catalog external Delta tables, and my requirement is to execute DESCRIBE HISTORY table_name LIMIT 1 for each table and append the latest record into a single consolidated table. I’ve already tried mul...
- 61 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi,The reason your performance degrades so badly (4 mins for 2k tables, but 50 mins for 12k) is because of the Spark Driver. When you run spark.sql("DESCRIBE HISTORY...") inside a ThreadPoolExecutor, every single one of those 22,000 queries has to be...
- 0 kudos
- 360 Views
- 3 replies
- 0 kudos
foreachPartition
Is there any difference between pyspark.RDD.foreachPartition vs pyspark.sql.DataFrame.foreachPartition under the hood? The PySpark documentation describes pyspark.sql.DataFrame.foreachPartition as "a shorthand for df.rdd.foreachPartition()"If DataFra...
- 360 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Although the PySpark documentation states that DataFrame.foreachPartition() is a shorthand for df.rdd. foreachPartition(), there is an important difference when running on Databricks shared clusters (especially with Unity Catalog and Spark Connect).D...
- 0 kudos
- 788 Views
- 5 replies
- 0 kudos
Databricks Access Issue with UC
Hi All ,We are facing issues while reading Storage account where stream data from data verse in Unity catalog through External table but not every time . It was running fine with hiveAn error occurred while calling o393.sql.: org.apache.spark.SparkEx...
- 788 Views
- 5 replies
- 0 kudos
Latest Reply
- 0 kudos
This issue appears to be related to Azure Storage access through Unity Catalog rather than the data itself, especially since the same workload was working fine with Hive and the failure is intermittent.A few areas worth checking:1. Storage Credential...
- 0 kudos
- 138 Views
- 3 replies
- 0 kudos
Sizing Tables and delt logs/CDF
Hi,I need to compare the sizes of my delta tables , what's the correct approach ?Table size reported by analyze command ? , but how do I check the delta log size , if I enable CDF .. how do I know the CDF log size(the overhead it adds) ? , kind of l...
- 138 Views
- 3 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi , While DESCRIBE DETAIL gives you the total sizeInBytes, it lumps everything together. Neither ANALYZE nor the Information Schema will break down the exact physical storage overhead of your Change Data Feed (CDF) or your Delta logs.Because a Delta...
- 0 kudos
- 842 Views
- 7 replies
- 1 kudos
Databricks excel plugin error
Hi, i have an error with the excel pluging.It was working fine until today. I can login and browse the catalog but I get an error when trying to query any data. It seems like its sending in a blank warehouse id. I have choosen a warehouse and its run...
- 842 Views
- 7 replies
- 1 kudos
Latest Reply
- 1 kudos
Hey there,I also have seen this type of error. But it automatically vanished the next day. I have not done any change in set up. Do you guys also faced the same behaviour of excel add-ins from databricks? In general, I get error and the next day or a...
- 1 kudos
| User | Count |
|---|---|
| 1837 | |
| 885 | |
| 788 | |
| 471 | |
| 312 |
