- 3880 Views
- 3 replies
- 5 kudos
Resolved! Identity column definition lost using save as table
I found an issue:For a table with an identity column defined.when the table column is renamed using this method, the identity definition will be removed. That means using an identity column in a table requires extra attention to check whether the ide...
- 3880 Views
- 3 replies
- 5 kudos
- 5 kudos
try to avoid reload table, I found we can upgrade table version, and use rename column commandALTER TABLE test_id2 SET TBLPROPERTIES ( 'delta.columnMapping.mode' = 'name', 'delta.minReaderVersion' = '2', 'delta.minWriterVersion' = '6')ALTER TABLE ...
- 5 kudos
- 4174 Views
- 3 replies
- 1 kudos
Resolved! unable to add new instance profile
Hi team, I want to start adding more instance profile per team . when adding it via TF , im getting the below error:Im able BTW to add and change other databricks resources via TF .This is my new code block:The new role created by the TF but wasnt ad...
- 4174 Views
- 3 replies
- 1 kudos
- 1 kudos
Thank you @Vivian Wilfred​ all ok now, with the databricks console and the TFHave a graet day!
- 1 kudos
- 651 Views
- 0 replies
- 1 kudos
Hi,I am a Solutions Architect at Databricks Northern Europe. I had a deep dive demo session at Data + AI Work Tour in London on Nov 2, 2022. The title...
Hi,I am a Solutions Architect at Databricks Northern Europe. I had a deep dive demo session at Data + AI Work Tour in London on Nov 2, 2022. The title was How to Build Your Modern Data Stack on Databricks to Solve Modern Problems.In the above-mention...
- 651 Views
- 0 replies
- 1 kudos
- 1924 Views
- 4 replies
- 1 kudos
Certificate not received
I have passed Databricks Certified Data Engineer Associate on 29 October 2022 but still didn't receive the certificate, kindly help me on that.
- 1924 Views
- 4 replies
- 1 kudos
- 1 kudos
Hi @Anandhakumar R​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue.If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please visit the ...
- 1 kudos
- 1860 Views
- 4 replies
- 0 kudos
I have passed my data analyst associate exam on 30th October , but still not received my certificate, its been more than than 4 days now , however Dat...
I have passed my data analyst associate exam on 30th October , but still not received my certificate, its been more than than 4 days now , however Databricks mentioned in an email that will deliver within 24 hours. Can anyone help on this ?
- 1860 Views
- 4 replies
- 0 kudos
- 9852 Views
- 8 replies
- 7 kudos
Resolved! data frame takes unusually long time to write for small data sets
We have configured workspace with own vpc. We need to extract data from DB2 and write as delta format. we tried to for 550k records with 230 columns, it took 50mins to complete the task. 15mn records takes more than 18hrs. Not sure why this takes suc...
- 9852 Views
- 8 replies
- 7 kudos
- 7 kudos
Hello. We face exactly the same issue. Reading is quick but writing takes long time. Just to clarify that it is about a table with only 700k rows. Any suggestions please? Thank youremote_table = spark.read.format ( "jdbc" ) \.option ( "driver" , "com...
- 7 kudos
- 7127 Views
- 3 replies
- 7 kudos
Resolved! Converting a transformation written in Spark Scala to PySpark
Hello all,I've been tasked to convert a Scala Spark code to PySpark code with minimal changes (kinda literal translation).I've come across some code that claims to be a list comprehension. Look below for code snippet:%scala val desiredColumn = Seq("f...
- 7127 Views
- 3 replies
- 7 kudos
- 7 kudos
Another follow-up question, if you don't mind. @Pat Sienkiewicz​ As I was trying to parse the name column into multiple columns. I came across the data below:("James,\"A,B\", Smith", "2018", "M", 3000)In order to parse these comma-included middle na...
- 7 kudos
- 2287 Views
- 4 replies
- 19 kudos
Resolved! How to prepare for Databricks Data Engineer Professional
Hi all,Could you please help suggest me some resource to prepare for " Databricks Data Engineer Professional" exam?I have also take the course in Databricks Accademy but seems not enough for this exam?Thank you so much!!!Best Regards,Nhan Nguyen
- 2287 Views
- 4 replies
- 19 kudos
- 19 kudos
waitting some one post more details and experience road map to take exam
- 19 kudos
- 5554 Views
- 1 replies
- 3 kudos
How to resolve our of memory error?
Hi, I am working as azure support engineerI found this error while I am checking the pipeline failure, and showing below error"org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 72403.0 failed 4 times, most recent fail...
- 5554 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @mahesh bmk​ ,It would be nice to see the sql_query.is there some window function used? You might try to run this on bigger cluster.
- 3 kudos
- 7362 Views
- 0 replies
- 4 kudos
Export commands and output of the Databricks Notebook to MS Excel
I have a Databricks notebook and I have several headers, SQL commands and their output. I am currently copying the output and SQL commands manually to excel for a report. How can I reduce the manual work of copy pasting from Notebook to excel and au...
- 7362 Views
- 0 replies
- 4 kudos
- 3197 Views
- 2 replies
- 5 kudos
Hello, I'm trying to use Databricks on Azure with a Spark structured streaming job and an having very mysterious issue. I boiled the job down it i...
Hello,I'm trying to use Databricks on Azure with a Spark structured streaming job and an having very mysterious issue.I boiled the job down it it's basics for testing, reading from a Kafka topic and writing to console in a forEachBatch.On local, ever...
- 3197 Views
- 2 replies
- 5 kudos
- 3761 Views
- 2 replies
- 4 kudos
Resolved! How to check, who did changes in databrics notebook and what are changes done in databrics notebook.
How to check, who did changes in databrics notebook and what are changes done in databrics notebook.
- 3761 Views
- 2 replies
- 4 kudos
- 4 kudos
@Hubert Dudek​ Thanks for the information. It is working.I have one more question:Is notebook saves changes automatically ?? If yes, I can see 'save now' option on top right corner(where user details shows/Version details shows) what is use of that o...
- 4 kudos
- 1870 Views
- 3 replies
- 2 kudos
Resolved! When and where would I receive my Certificate after passing the exam?
I have passed the Databricks Certified Data Analyst Associate exam on 29 October 2022. I have not received the corticate yet.
- 1870 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @Sufyan Shafique​ We are really sorry for the delays.Just a friendly- follow-up, have you got your certification and badge? If yes, please mark the answer as best.Thanks and Regards
- 2 kudos
- 2902 Views
- 1 replies
- 1 kudos
Databricks integration with DocumentDB
Hi Everyone,I was wondering if anyone here has any experience or tips reading data from AWS DocumentDB. I am working on this using the MongoDB connector. For DocumentDB we also need to work with the required creds issued as a .pem file by AWS. Th...
- 2902 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Kaniz Fatma​ ,Thank you so much for your response. Your suggestions were helpful. As per the AWS documentation, DocumentDB is MongoDB compatible. "With Amazon DocumentDB, you can run the same application code and use the same drivers and tools th...
- 1 kudos
- 4135 Views
- 9 replies
- 2 kudos
SQL Warehouse connections in us-east-2
I have 2 very similarly configured workspaces, one in us-west-2 and one in us-east-2.Both got configured by default with a "Starter Warehouse".The one in us-west-2 I can reach via the internet using python databricks-sql-connector, but the one in us-...
- 4135 Views
- 9 replies
- 2 kudos
- 2 kudos
Hi @Marcus Simonsen​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...
- 2 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
