- 2472 Views
- 1 replies
- 0 kudos
DIAS 2023 -- recommend the training!
Did the SparkUI training yesterday with Mark Ott, and I highly recommend it. It was super helpful and provided a lot of clarity around some of the vaguer terms and metrics, and some surprise penalties.In-memory partition size is the the main thing to...
- 2472 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, I can't find this course... can you please share the full name of this course? Thanks in advance
- 0 kudos
- 2751 Views
- 3 replies
- 4 kudos
Resolved! How to read Databricks UniForm format tables present in ADLS
We have Databricks UniForm format (iceberg) tables are present in azure data lake storage (ADLS) which has already integrated with Databricks unity catalog. How to read Uniform format tables using Databricks as a query engine?
- 2751 Views
- 3 replies
- 4 kudos
- 4 kudos
Query Using Unity Catalog:SQL:sqlCopiar códigoSELECT * FROM catalog_name.schema_name.table_name;PySpark:pythonCopiar códigodf = spark.sql("SELECT * FROM catalog_name.schema_name.table_name") df.display()Direct Access by Path: If not using Unity Catal...
- 4 kudos
- 11318 Views
- 2 replies
- 0 kudos
Cluster Memory Issue (Termination)
Hi,I have a single-node personal cluster with 56GB memory(Node type: Standard_DS5_v2, runtime: 14.3 LTS ML). The same configuration is done for the job cluster as well and the following problem applies to both clusters:To start with: once I start my ...
- 11318 Views
- 2 replies
- 0 kudos
- 2962 Views
- 2 replies
- 1 kudos
Creating table in Unity Catalog with file scheme dbfs is not supported
code:# Define the path for the staging Delta tablestaging_table_path = "dbfs:/user/hive/warehouse/staging_order_tracking"spark.sql( f"CREATE TABLE IF NOT EXISTS staging_order_tracking USING DELTA LOCATION '{staging_table_path}'" )Creating table in U...
- 2962 Views
- 2 replies
- 1 kudos
- 1 kudos
I believe, using mount point only we can able to connect to our storage account - containers. If this is anti pattern by data bricks what is the way? Can you please explain what is external location of UC is it our local system folders or something n...
- 1 kudos
- 3880 Views
- 2 replies
- 3 kudos
Resolved! Cannot find "Databricks Apps"
Hi, I saw a demo about "Databricks Apps" 2 months ago. I haven't used Databricks for about 3 months, and I recently recreated a Premium Workspace to try something out (I use Azure), however I can't find "Apps" when I click "New". How can I enable and...
- 3880 Views
- 2 replies
- 3 kudos
- 2327 Views
- 4 replies
- 0 kudos
Issue Querying Registered Tables on Glue Catalog via Data bricks
Im having an issue to query registered tables on glue catalog thru databricks with the following error: AnalysisException: [TABLE_OR_VIEW_NOT_FOUND] The table or view looker.ccc_data cannot be found.Verify the spelling and correctness of the schema a...
- 2327 Views
- 4 replies
- 0 kudos
- 0 kudos
Can you specify the full catalog.schema.table? and also check the current schema SELECT current_schema();
- 0 kudos
- 1451 Views
- 2 replies
- 1 kudos
Delete non-community Databricks account
Hi everyone!I have mistakenly created a non-community account using my personal email address.I would like to delete it in order to create a new account using my business email.How should I proceed? I tried to find this option on the console, with no...
- 1451 Views
- 2 replies
- 1 kudos
- 1 kudos
Hello @gabrielsantana!Could you please try raising a ticket with the Databricks support team?
- 1 kudos
- 6987 Views
- 1 replies
- 1 kudos
How to overwrite the existing file using databricks cli
If i use databricks fs cp then it does not overwrite the existing file, it just skip copying the file. Any suggestion how to overwrite the file using databricks cli?
- 6987 Views
- 1 replies
- 1 kudos
- 1 kudos
You can use the --overwrite option to overwrite your file.https://docs.databricks.com/en/dev-tools/cli/fs-commands.html
- 1 kudos
- 1905 Views
- 1 replies
- 1 kudos
Resolved! The databricks jdbc driver has a memory leak
https://community.databricks.com/t5/community-platform-discussions/memory-leak/td-p/80756 My question is the same as above Unable to upload pictures, I had to dictateQuestion from ResultFileDownloadMonitor. M_requestList parametersBecause is ResultFi...
- 1905 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello @gf thanks for your question, it seems that this has been reported with Simba, but no fix has been provided yet, as a temporary workaround, you can consider using reflection to periodically clean up the m_requestList by removing KV pairs whose ...
- 1 kudos
- 1605 Views
- 3 replies
- 0 kudos
Databricks Champions Sign-Up Page Failing or Throwing Error
Following link is throwing error when tried to Sign-Uphttps://advocates.databricks.com/users/sign_up?join-code=community
- 1605 Views
- 3 replies
- 0 kudos
- 0 kudos
Let me check on this internally, I am not sure if this actually requires invitation.
- 0 kudos
- 2865 Views
- 5 replies
- 0 kudos
Issue with Adding New Members to Existing Groups During Migration in User group Service Principle
Hi all,I have implemented a migration process to move groups from a source workspace to a target workspace using the following code. The code successfully migrates groups and their members to the target system, but I am facing an issue when it comes...
- 2865 Views
- 5 replies
- 0 kudos
- 0 kudos
I have provided response in https://community.databricks.com/t5/get-started-discussions/migrating-service-principals-from-non-unity-to-unity-enabled/m-p/103017#M4679
- 0 kudos
- 9724 Views
- 1 replies
- 1 kudos
Resolved! Why does a join on (df1.id == df2.id) result in duplicate columns while on="id" does not?
Why does a join with on (df1.id == df2.id) result in duplicate columns, but on="id" does not?I encountered an interesting behavior while performing a join on two Data frames. Here's the scenario: df1 = spark.createDataFrame([(1, "Alice"), (2, "Bob"),...
- 9724 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Tanay , Your intuition is correct here. In Apache Spark, the difference in behavior between on (df1.id == df2.id) and on="id" in a join stems from how Spark resolves and handles column naming during the join operation.When you use the first synta...
- 1 kudos
- 1549 Views
- 2 replies
- 0 kudos
Create multiple dashboard subscription with filters
Hi Databricks community, We developed a dashboard that surfaces up several important KPIs for each project we have.In the top filter, we select the project name and the time frame and the dashboard will present the relevant KPIs and charts. I can eas...
- 1549 Views
- 2 replies
- 0 kudos
- 0 kudos
You can achieve this by setting up different schedules for each project and specifying the default filter values accordingly Create the Dashboard: Ensure your dashboard is set up with the necessary filters, including the project filter. Set Defau...
- 0 kudos
- 8898 Views
- 2 replies
- 0 kudos
PPT material or document from Databricks Learning
Hello Databricks Community,I am a beginner with Databricks. I am wondering if we can download power point slides or learning documents from the Databricks Learning Platform. I like to read after taking the online course. Could you let me know? Curren...
- 8898 Views
- 2 replies
- 0 kudos
- 867 Views
- 1 replies
- 0 kudos
Databricks, Cloud Services Pricing
i am unable to find reason of not getting Databricks, Cloud Services Pricing why?
- 867 Views
- 1 replies
- 0 kudos
- 0 kudos
Can you please provide some more context on the issue you are facing to be able to properly assist you?
- 0 kudos
