- 3858 Views
- 1 replies
- 1 kudos
Resolved! CloudFormation failure while creating workspace
The cloud formation script is fails to create workspace due to following error. The solution to this error is not available in the standard troubleshooting guide. Following is the error message from cloud watch logs. It seems like a bug in the api, b...
- 3858 Views
- 1 replies
- 1 kudos
- 1 kudos
I found a solution to this by creating a new role following these articles -https://docs.databricks.com/administration-guide/account-api/iam-role.htmlhttps://docs.databricks.com/administration-guide/account-api/aws-storage.html
- 1 kudos
- 4066 Views
- 7 replies
- 0 kudos
Hi Databricks Team,​ I passed the associate data engineer exam day before but still haven't received on accredible or on db academy. My registere...
Hi Databricks Team,​ I passed the associate data engineer exam day before but still haven't received on accredible or on db academy. My registered email id for exam is sharath.koushik@gmail.com. Could you please help ?​
- 4066 Views
- 7 replies
- 0 kudos
- 0 kudos
Hi @Sharath K​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 0 kudos
- 7033 Views
- 3 replies
- 2 kudos
How to remove extra ENTER line in csv UTF-16 while reading
Dear Friends,I have a csv and it looks like this‡‡Id‡‡,‡‡Version‡‡,‡‡Questionnaire‡‡,‡‡Date‡‡‡‡123456‡‡,‡‡Version2‡‡,‡‡All questions have been answered accurately and the guidance in the questionnaire was understood and followed‡‡,‡‡2010-12-16 00:01:...
- 7033 Views
- 3 replies
- 2 kudos
- 2 kudos
This is working fine, from pyspark.sql.functions import regexp_replace path="dbfs:/FileStore/df/test.csv" dff = spark.read.option("header", "true").option("inferSchema", "true").option('multiline', 'true').option('encoding', 'UTF-8').option("delimi...
- 2 kudos
- 7703 Views
- 6 replies
- 8 kudos
Resolved! How can we move the excel file from adls to share point
I have one excel file in the adls, i want to move that file into sharepoint, but i tried this method in data factory, but in sink sharepoint is not available, is there any possible way to do this?​
- 7703 Views
- 6 replies
- 8 kudos
- 8 kudos
If you want to upload raw Excel file (not Sharepoint List) then IMO it would be easier to simply mount the Sharepoint Site as a network drive (https://support.microsoft.com/en-us/office/map-a-network-drive-to-a-sharepoint-library-751148de-f579-42f9-b...
- 8 kudos
- 3160 Views
- 4 replies
- 3 kudos
- 3160 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @A DB​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 3 kudos
- 9076 Views
- 5 replies
- 4 kudos
"Detected schema change" error while reading from delta table in streaming after applying "ALTER COLUMN DROP NOT NULL" to more than one columns.
Hi!I have a delta table and a process that reading a stream from this table.I need to drop the NOT NULL constraint from some of the columns of this table.The first drop command does not affect the reading stream.But the second command results in erro...
- 9076 Views
- 5 replies
- 4 kudos
- 4 kudos
Hi @Anatoly Tikhonov​ Hope everything is going great.Does @Kaniz Fatma​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 4 kudos
- 5747 Views
- 7 replies
- 4 kudos
Certificate not received -
Hi ThereI successfully passed the exam on 27th Oct 2022 - but I haven't received the certificate yet.
- 5747 Views
- 7 replies
- 4 kudos
- 4 kudos
Hi @Gaurav Jhamb​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
- 4 kudos
- 5200 Views
- 6 replies
- 0 kudos
I've successfully passed Databricks Data Engineer Associate Certified exam but still have not received the certificate. Could you help on it pleas...
I've successfully passed Databricks Data Engineer Associate Certified exam but still have not received the certificate. Could you help on it please.
- 5200 Views
- 6 replies
- 0 kudos
- 0 kudos
Hi @Venkata Sai Anuroop Samudrala​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to he...
- 0 kudos
- 3488 Views
- 2 replies
- 1 kudos
Configure DLT Pipeline to use existing running cluster
How can I configure a DLT pipeline to use an existing running cluster? I don't see where in the settings to set the pipeline to use an existing cluster. Instead it wants to always standup a new cluster.
- 3488 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Justin Stuparitz​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
- 1 kudos
- 4871 Views
- 3 replies
- 0 kudos
Cluster not following the zone: auto configuration
Hi, I'm trying some new instance types, so to be sure I will get one I set the Availability zone to 'auto'.But this is happening, the image of the error and JSON are attached.So, the cluster is trying three times to upscale and the three times is usi...
- 4871 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Alejandro Martinez​ Hope everything is going great.Does @Sivaprasad C S​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 0 kudos
- 2564 Views
- 2 replies
- 2 kudos
Time Parameter Returns Incorrect Time
For some reason, the parameter keeps showing me my local time in UTC. When converting this parameter from EST, the offset is wrong. In the screenshot, you can see that my local session TZ is set to ETC/UTC (for whatever reason). It looks like the par...
- 2564 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Zachary Higgins​ Hope everything is going great.Does @Prasad Wagh​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!
- 2 kudos
- 7234 Views
- 6 replies
- 10 kudos
databricks learning Full screen disabled in selfpaced course
databricks learning self paced course Databricks Certified Associate Developer for Apache Spark 3 has below issues 1) Full screen button is disabled, difficult to see small font and stress on eyes.PFA2) courses do not have captions. Sometimes it is d...
- 7234 Views
- 6 replies
- 10 kudos
- 10 kudos
Hi @Smitha Nelapati​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us ...
- 10 kudos
- 6869 Views
- 3 replies
- 0 kudos
bug: add csv data UI: missing leading zero
use add data UI, add csv manually, even set data type as string, the leading zero will be missingexample csvval1,val20012345, abcafter load data, 123,abc is stored in table
- 6869 Views
- 3 replies
- 0 kudos
- 0 kudos
there are no issues using spark.read in notebooksthe issue is specific to using Add Data User interface and adding a csv file manually.
- 0 kudos
- 2716 Views
- 0 replies
- 0 kudos
Connection from on-premise R sparklyr session to Databricks, " invalid method toDF for object 17/org.apache.spark.sql.DataFrameReader fields 0 selected 0"
Hello,I'm working with an on-premise R session and would like to connect to Databricks using sparklyr. RStudio server in this case is not an option.I would like to use jdbc. I tested RJDBC + DBI and can conect locally and perform operations. However,...
- 2716 Views
- 0 replies
- 0 kudos
- 4191 Views
- 2 replies
- 1 kudos
10.4LTS has outdated snowflake spark connector, how to force latest snowflake spark connector
Hi,I am trying to run my code from a scala fatjar on azure-databricks, which connects to snowflake for the data.I usually run my jar on 9.1 LTS.However when I run on 10.4 LTS the performace was 4x degraded and in the log it says WARN SnowflakeConnect...
- 4191 Views
- 2 replies
- 1 kudos
- 1 kudos
I also encountered the similar problem. This is a snippet from my log file:22/12/18 09:36:28 WARN SnowflakeConnectorUtils$: Query pushdown is not supported because you are using Spark 3.2.0 with a connector designed to support Spark 3.1. Either use t...
- 1 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 576 | |
| 349 | |
| 287 |
