- 6312 Views
- 1 replies
- 3 kudos
How to resolve our of memory error?
Hi, I am working as azure support engineerI found this error while I am checking the pipeline failure, and showing below error"org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 72403.0 failed 4 times, most recent fail...
- 6312 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @mahesh bmk​ ,It would be nice to see the sql_query.is there some window function used? You might try to run this on bigger cluster.
- 3 kudos
- 7852 Views
- 0 replies
- 4 kudos
Export commands and output of the Databricks Notebook to MS Excel
I have a Databricks notebook and I have several headers, SQL commands and their output. I am currently copying the output and SQL commands manually to excel for a report. How can I reduce the manual work of copy pasting from Notebook to excel and au...
- 7852 Views
- 0 replies
- 4 kudos
- 3620 Views
- 2 replies
- 5 kudos
Hello, I'm trying to use Databricks on Azure with a Spark structured streaming job and an having very mysterious issue. I boiled the job down it i...
Hello,I'm trying to use Databricks on Azure with a Spark structured streaming job and an having very mysterious issue.I boiled the job down it it's basics for testing, reading from a Kafka topic and writing to console in a forEachBatch.On local, ever...
- 3620 Views
- 2 replies
- 5 kudos
- 4549 Views
- 2 replies
- 4 kudos
Resolved! How to check, who did changes in databrics notebook and what are changes done in databrics notebook.
How to check, who did changes in databrics notebook and what are changes done in databrics notebook.
- 4549 Views
- 2 replies
- 4 kudos
- 4 kudos
@Hubert Dudek​ Thanks for the information. It is working.I have one more question:Is notebook saves changes automatically ?? If yes, I can see 'save now' option on top right corner(where user details shows/Version details shows) what is use of that o...
- 4 kudos
- 2428 Views
- 3 replies
- 2 kudos
Resolved! When and where would I receive my Certificate after passing the exam?
I have passed the Databricks Certified Data Analyst Associate exam on 29 October 2022. I have not received the corticate yet.
- 2428 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Sufyan Shafique​ We are really sorry for the delays.Just a friendly- follow-up, have you got your certification and badge? If yes, please mark the answer as best.Thanks and Regards
- 2 kudos
- 3722 Views
- 1 replies
- 1 kudos
Databricks integration with DocumentDB
Hi Everyone,I was wondering if anyone here has any experience or tips reading data from AWS DocumentDB. I am working on this using the MongoDB connector. For DocumentDB we also need to work with the required creds issued as a .pem file by AWS. Th...
- 3722 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Kaniz Fatma​ ,Thank you so much for your response. Your suggestions were helpful. As per the AWS documentation, DocumentDB is MongoDB compatible. "With Amazon DocumentDB, you can run the same application code and use the same drivers and tools th...
- 1 kudos
- 5152 Views
- 9 replies
- 2 kudos
SQL Warehouse connections in us-east-2
I have 2 very similarly configured workspaces, one in us-west-2 and one in us-east-2.Both got configured by default with a "Starter Warehouse".The one in us-west-2 I can reach via the internet using python databricks-sql-connector, but the one in us-...
- 5152 Views
- 9 replies
- 2 kudos
- 2 kudos
Hi @Marcus Simonsen​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 2 kudos
- 1948 Views
- 1 replies
- 0 kudos
unable to find LoginModule class: org.apache.kafka.common.security.plain.PlainLoginModule
In databricks I am trying to read data from a protected kafka topic using pyspark.I am getting an error "unable to find LoginModule class: org.apache.kafka.common.security.plain.PlainLoginModule".
- 1948 Views
- 1 replies
- 0 kudos
- 0 kudos
Did you able to resolve it?
- 0 kudos
- 4491 Views
- 4 replies
- 0 kudos
Resolved! authentication is not configured for provider
Hi team, I started getting this message lately when trying add some new config or change my workspace with terraform :Error: cannot create global init script: authentication is not configured for provider. Please check https://registry.terraform.io/p...
- 4491 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Avi Edri​ looks like you are using a provider that is authenticated to the Accounts console (https://accounts.cloud.databricks.com) to create a global init script within the workspace. Can you try authentication with host and PAT token? Follow th...
- 0 kudos
- 2049 Views
- 0 replies
- 3 kudos
How to specify which columns to use when using DLT APPLY CHANGES INTO
I have a notebook with the code below, where I try to do an upsert into a dimension table and only include one column from the source table. I get an error even though I think the syntax matches what I see in the docs. How can I write this in the cor...
- 2049 Views
- 0 replies
- 3 kudos
- 11344 Views
- 6 replies
- 2 kudos
Resolved! Failed to fetch archive.ubuntu
I am trying to use selenium webdriver to do a scraping project in Databricks. The notebook used to run properly but now has an issue with the Get:1 http://archive.ubuntu.com/ubuntu focal/main amd64 fonts-liberation all 1:1.07.4-11 [822 kB]command .In...
- 11344 Views
- 6 replies
- 2 kudos
- 2 kudos
Hi, @Dagart Allison​ . I've created a new version of the selenium with the databricks manual. Please look here https://community.databricks.com/s/feed/0D58Y00009SWgVuSAL
- 2 kudos
- 5215 Views
- 2 replies
- 0 kudos
Using Selenium Chrome Driver in Databricks, runs the first time but fails after that
I have a notebook that uses a Selenium Web Driver for Chrome and it works the first time I run the notebook. If I run the notebook again, it will not work and gives the error message: WebDriverException: Message: unknown error: unable to discover op...
- 5215 Views
- 2 replies
- 0 kudos
- 0 kudos
Hi, @Dagart Allison​ . I've created a new version of the selenium with the databricks manual. Please look here https://community.databricks.com/s/feed/0D58Y00009SWgVuSAL
- 0 kudos
- 10048 Views
- 6 replies
- 3 kudos
How to do bucketing in Databricks?
We are migrating a job from onprem to databricks. We are trying to optimize the jobs but couldn't use bucketing because by default databricks stores all tables as delta table and it shows error that bucketing is not supported for delta. Is there anyw...
- 10048 Views
- 6 replies
- 3 kudos
- 3 kudos
Hi @Arun Balaji​ ,bucketing is not supported for the delta tables as you have noticed.For the optimization and best practices with delta tables check this:https://docs.databricks.com/optimizations/index.htmlhttps://docs.databricks.com/delta/best-prac...
- 3 kudos
- 16073 Views
- 13 replies
- 23 kudos
Resolved! Users are failing query data from S3 bucket
Hi team, Users are unable run select on data located on S3 buckets, S3 permission are ok.The only way they manage do it by granted the databricks workspace admin permission.Attached the error.Thanks!
- 16073 Views
- 13 replies
- 23 kudos
- 23 kudos
@Avi Edri​ adding some more info to @Pat Sienkiewicz​ suggestion, @Avi Edri​ are you using cluster with instance profile, if you are using instance profile configured, please validate read permissions are there on that bucket and instance profile ass...
- 23 kudos
- 25736 Views
- 7 replies
- 10 kudos
Resolved! Migrate tables from one azure databricks workspace to another
Hello all,I'm currently trying to move the tables contained in one azure workspace to another, because of a change in the way we use our resources groups. I have not been able to move more than metadata with the databrickslabs/migrate repo. I was won...
- 25736 Views
- 7 replies
- 10 kudos
- 10 kudos
Hi @Quentin Maire​ ,We need a bit more details.where is your data stored ?are you using external or managed tables?the migrate tool allows you to export DDL statements not the data itself.I can think about few scenarios on Top of my head.if you had p...
- 10 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 485 | |
| 349 | |
| 287 |
