- 5746 Views
- 7 replies
- 0 kudos
Cannot reset password, no support
I cannot log in to my Databricks community account. I have already tried to receive support and no real support has been given. I attempt to reset my password, the link gets sent, but once I enter the new password it gets stuck permanently loading. I...
- 5746 Views
- 7 replies
- 0 kudos
- 0 kudos
Hi @Ahmet Korkmaz​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 0 kudos
- 27012 Views
- 14 replies
- 15 kudos
Resolved! How to convert records in Azure Databricks delta table to a nested JSON structure?
Let's say I have a delta table in Azure databricks that stores the staff details (denormalized). I wanted to export the data in the JSON format and save it as a single file on a storage location. I need help with the databricks sql query to group/co...
- 27012 Views
- 14 replies
- 15 kudos
- 2338 Views
- 3 replies
- 3 kudos
Where is the learning material to get Fundamentals of the Databricks Lakehouse Platform Accreditation?
Please provide me some information about how to get the martial to pass Fundamentals of the Databricks Lakehouse Platform Accreditation?
- 2338 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Shanthala Baleer​,Just a friendly follow-up. Are you still looking for help? adding @Vidula Khanna​ for visibility
- 3 kudos
- 1659 Views
- 1 replies
- 0 kudos
How to restart snowflake connector?
After using spark.read.format("snowflake").options(**options).option("dbtable", "table_name").load() to read a table from Snowflake, when I then change the table from Snowflake and read it again, it gives me the first version of the table. I have wor...
- 1659 Views
- 1 replies
- 0 kudos
- 0 kudos
Yes, that would work. However, it is a longish Snowflake query producing a number of tables that are all called by the Databricks notebook, so it requires quite a few changes. I'll use this alternative if I automate the process. However, I think this...
- 0 kudos
- 8303 Views
- 5 replies
- 7 kudos
Resolved! Why was SQL formatting removed inside spark.sql functions? Now it looks like a plain string.
Previously we were able to see SQL queries inside spark.sql() like this:But now it just looks like a plain string: I know it's not a big issue, but it's still annoying to have to code in SQL while having it all be blue, it makes debugging more cumber...
- 8303 Views
- 5 replies
- 7 kudos
- 7 kudos
Hi @Emilio Garza​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 7 kudos
- 6325 Views
- 4 replies
- 0 kudos
Creating a spot only single-node job compute cluster policy
Hi there,I need some help creating a new cluster policy that utilizes a single spot-instnace server to complete a job. I want to set this up as a job-compute to reduce costs and also utilize 1 spot instance.The jobs I need to ETL are very short and c...
- 6325 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Avkash Kana​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 0 kudos
- 2423 Views
- 4 replies
- 0 kudos
how to optimize the runtime in 10.4 cluster
i am loading the 1billion data from spark dataframe into target table, but in the 7.3 cluster it takes 3 hours to complete but after migrated to 10.4 cluster its taking 8 hours to complete , how can i reduce the time duration​
- 2423 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Mohammed sadamusean​,Could you provide more details on what are you doing? What type of transformations/actions are you doing? whats your source and sink? batch or streaming? all that information will help.
- 0 kudos
- 5100 Views
- 3 replies
- 0 kudos
Query Databricks from Power BI with Row Level Security
I am trying to apply RLS to the solution but Power BI only connects to Databricks(DB) using a token which cant be used in DB groups. Is there no other way to apply Row Level security using Power BI?
- 5100 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Daniel Gomes​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 0 kudos
- 3189 Views
- 3 replies
- 2 kudos
Unable to connect/read files from ADLS Gen2 using account key
It gives error[RequestId=5e57b66f-b69f-4e8b-8706-3fe5baeb77a0 ErrorClass=METASTORE_DOES_NOT_EXIST] No metastore assigned for the current workspace.using the following codespark.conf.set( "fs.azure.account.key.mystorageaccount.dfs.core.windows.net", ...
- 3189 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Farooq ur rehman​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.
- 2 kudos
- 6869 Views
- 5 replies
- 0 kudos
Delta Live Tables data quality rules application.
I have a requirement, where I need to apply inverse DQ rule on a table to track the invalid data. For which I can use the following approach:import dltrules = {}quarantine_rules = {}rules["valid_website"] = "(Website IS NOT NULL)"rules["valid_locatio...
- 6869 Views
- 5 replies
- 0 kudos
- 0 kudos
You can get additional info from DLT event log which is in delta so you can load it as table https://docs.databricks.com/workflows/delta-live-tables/delta-live-tables-event-log.html#data-quality
- 0 kudos
- 1603 Views
- 1 replies
- 10 kudos
Since databricks runtime 12.1 "WHEN NOT MATCHED BY SOURCE" was added to MERGE syntax. For example, using that option, we can quickly delete ...
Since databricks runtime 12.1 "WHEN NOT MATCHED BY SOURCE" was added to MERGE syntax. For example, using that option, we can quickly delete all target rows which doesn't match any source.
- 1603 Views
- 1 replies
- 10 kudos
- 10 kudos
Thank you for sharing @Hubert Dudek​
- 10 kudos
- 8140 Views
- 3 replies
- 4 kudos
DLT streaming table and LEFT JOIN
I'm trying to build gold level streaming live table based on two streaming silver live tables with left join.This attempt fails with the next error:"Append mode error: Stream-stream LeftOuter join between two streaming DataFrame/Datasets is not suppo...
- 8140 Views
- 3 replies
- 4 kudos
- 4 kudos
Thanks Fatma,I do understand the need for watermarks, but I'm just wondering if this supported by SQL syntax?
- 4 kudos
- 4826 Views
- 1 replies
- 1 kudos
add custom metadata to avro file with pyspark
Hello, i need to add a custom metadata into a avro file. The avro file containt data. we have tried to use "option" within the write function but it's not taken without generated any error.df.write.format("avro").option("avro.codec", "snappy").option...
- 4826 Views
- 1 replies
- 1 kudos
- 2642 Views
- 1 replies
- 1 kudos
What is the maximum number of workspaces per account using Databricks on AWS?
I've been looking through the documentation and I swear this used to be listed somewhere, but for the life of me I can't find it anymore.
- 2642 Views
- 1 replies
- 1 kudos
- 1 kudos
Thanks @Kaniz Fatma​
- 1 kudos
- 1309 Views
- 0 replies
- 0 kudos
- 1309 Views
- 0 replies
- 0 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 485 | |
| 349 | |
| 287 |
