- 900 Views
- 1 replies
- 0 kudos
Can I load the files based on the data in my table as variable without iterating through each row?
Hi,I have created this table which contains the data that I need for my source path and target table. source_path: /data/customer/sid={sid}/abc=1/attr_provider={attr_prov}/source_data_provider_code={src_prov}/So basically, the value of each row are c...
- 900 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @zll_0091, To efficiently load only the necessary files without manually iterating through each row of your table, you can use Spark's DataFrame operations. First, read your table into a DataFrame and determine the maximum key value. Then, filter ...
- 0 kudos
- 1699 Views
- 2 replies
- 0 kudos
Databricks Certification exam got Suspended - Need Support
Hello Team, @Cert-Team , @Cert-TeamOPS I faced a very bad experience while attempting my 1st DataBricks certification.I was asked to exit the exam multiple times by the support team saying technical issues. My test got rescheduled multiple times with...
- 1699 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi @ozbieG, I'm sorry to hear your exam was suspended. Thank you for filing a ticket with our support team. Please allow the support team 24-48 hours to resolve. In the meantime, you can review the following documentation: Room requirements Behaviour...
- 0 kudos
- 1348 Views
- 1 replies
- 0 kudos
What API Testing Tool Do You Use?
Hi Databricks!I am a relatively new developer that's looking for a solid API testing tool. I am interested in hearing about other developers, new or experienced, about their experiences with API testing tools, regardless if they are good or bad. I've...
- 1348 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @bytetogo,In my daily work I use Postman. It has user-friendly interface, supports automated testing and has support for popular patterns and libraries. It is also compatible with Linux, MacOs, Windows.
- 0 kudos
- 6840 Views
- 1 replies
- 1 kudos
Databricks book recommendations
Hi all,I am very new to databricks. I am looking for any good book recommendations that can help me get started. I know there is a vast resource available online but I feel a book will give me a structured approach to get startedAny book recommendati...
- 6840 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @uniqueusername ,I would start with books that teach you spark.Learning Spark, 2nd Edition by Jules S. Damji, Brooke Wenig, Tathagata Das, Denny LeeData Analysis with Python and PySpark by Jonathan Rioux (Author)After you learn spark foundation, o...
- 1 kudos
- 1610 Views
- 3 replies
- 0 kudos
Unable to create a workspace
Noel Nosse <nnosse@my.wgu.edu> 9:03 PM (0 minutes ago) to Databricks To complete a tutorial requires a workspace. The directions for the quickstart are outdated and do not match AWS. AWS has their own guide but cloudformation requires email ...
- 1610 Views
- 3 replies
- 0 kudos
- 0 kudos
Now I get: Redirecting to: https://accounts.cloud.databricks.com/login/password?next_url=%2Fapi%2F2.
- 0 kudos
- 1567 Views
- 1 replies
- 0 kudos
Job Cluster best practices for production workloads
Hi All,Can you please share the best practices for job clusters configurations for production workloadsand which is good when compared to serverless and job cluster in production in terms of cost and performance?Regards,Phani
- 1567 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Phani1, For configuring job clusters for production workloads in Databricks, follow these best practices: match cluster size to workload needs, enable autoscaling for dynamic adjustment of worker nodes, use spot instances with a fallback to on-de...
- 0 kudos
- 1423 Views
- 0 replies
- 0 kudos
Databricks visualization Data labels sssssss
I have been using visualization for a lot of different usecases and has been working for instead of using 3rd party libraries. Recently I had a need to customize the data labels but I haven't seen anything in the documentation that how to do that. If...
- 1423 Views
- 0 replies
- 0 kudos
- 2311 Views
- 3 replies
- 1 kudos
How to return the function result instead of the function syntax of a variable?
Hi,I'm trying to get the certain value of my variable in the for loop but it's returning the syntax instead of the value. Also, is it possible to covert this value to an integer? Thanks
- 2311 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @zll_0091 ,Could you provide more code? What's inside dsfd variable? What's your expected outcome?
- 1 kudos
- 1271 Views
- 1 replies
- 1 kudos
Azure Databricks add repo
While adding arepo in databricks workspace I am getting error as 'Error creating repoThe Azure container does not exist'Please see the attached screenshot.Anyone please suggest the fix.
- 1271 Views
- 1 replies
- 1 kudos
- 1 kudos
There are three possible causes. The Azure container might not have been properly created when the workspace was provisioned.The Azure container might have been deleted or moved after it was created.There might be a problem with the permissions or r...
- 1 kudos
- 3678 Views
- 6 replies
- 4 kudos
Cannot Create databricks account
I am trying to create databricks community account but after providing all the information and completing the puzzle it is showing me an error occurred: I also recorded the the network request for the error:Header:Request URL:https://www.databricks.c...
- 3678 Views
- 6 replies
- 4 kudos
- 4 kudos
@AllI am trying it from Bangladesh. Is there any country wise restrictions??
- 4 kudos
- 2609 Views
- 4 replies
- 0 kudos
Cannot sign-in at accounts.cloud.databricks.com
Hi,I have registered for Community Edition and can access it with no problems trough: https://community.cloud.databricks.com/login.htmlNow, I'm interested in completing the free "lakehouse fundamentals" training here and taking the quiz to get the ba...
- 2609 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @slechtd , @qiuqiu You can't log in because this is login page for databricks customers. You should use login at community edition, like on the bottom left side of the below screen Furthermore, to get databricks fundamentals accreditation you need...
- 0 kudos
- 3089 Views
- 1 replies
- 1 kudos
Liquid clustering on and dynamic overwrites
I use the following option to write from multiple tasks to the same table with overwrite (in Pyspark).option("partitionOverwriteMode", "dynamic")The table was created with partition by so it works as expected.I read about liquid clustering and it's b...
- 3089 Views
- 1 replies
- 1 kudos
- 1 kudos
No it doesn't support partition overwrite.
- 1 kudos
- 1729 Views
- 1 replies
- 0 kudos
ConcurrentAppendException
Hi Team,We're dealing with a concurrency issue when we attempt to run multiple jobs at the same time, and we're still having the same problem even after using partition and liquid clustering features. Now we're making sure to have the right where con...
- 1729 Views
- 1 replies
- 0 kudos
- 0 kudos
Option 3 Bit complicated process but works well: Considering you will do Update, Insert and Delete operation on the same table at the same time by using multiple job.1. Create a Dummy table with the target table schema with additional column called O...
- 0 kudos
- 987 Views
- 0 replies
- 0 kudos
How to test dashboards for free?
Does the community edition include the ability to create dashbiards? I’d not, is there a free option to test dashboards?
- 987 Views
- 0 replies
- 0 kudos
- 1548 Views
- 0 replies
- 1 kudos
FAQ for Databricks Learning Festival (Virtual): 10 July - 24 July 2024
IObit Smart Defrag Pro 9.3.0.341 Serial Key Smart defragmentation works quickly, mechanically, and silently in history and is suitable for large hard drives. It facilitates its hard work with greater success than any other product on the market, free...
- 1548 Views
- 0 replies
- 1 kudos
