- 9530 Views
- 4 replies
- 3 kudos
Resolved! Can't read large multiline json,
Hey! So I'm struggling to read a multiline json. Some details:It's gzipped from the API I get it fromjust a single file in the folder currrentlystored in ADLS Gen2 storage. 95 MB zipped, approximately 1.2 GB unzippedI can read it just fine using the...
- 9530 Views
- 4 replies
- 3 kudos
- 3 kudos
Thanks a lot for the help! Removing colon fixed it. Now I need to fix the Data Factory instance that writes to my storage container. Hope it's easy, Data Factory is such a hassle.
- 3 kudos
- 3052 Views
- 3 replies
- 3 kudos
500550
[Simba][SimbaSparkJDBCDriver] (500550) The next rowset buffer is already marked as consumed. The fetch thread might have terminated unexpectedly. Foreground thread ID
- 3052 Views
- 3 replies
- 3 kudos
- 3672 Views
- 5 replies
- 3 kudos
Databricks administrator accreditation exam
Hi,I have recently passed the administrator accreditation exam in the partner academy. However, I have recieved no certificate. How can I get my certificate?
- 3672 Views
- 5 replies
- 3 kudos
- 3 kudos
Hi @Daan Cremers Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 3 kudos
- 5304 Views
- 4 replies
- 2 kudos
My Databricks Lakehouse badge doesn't appear in Databricks Academy
Hello, I passed the Databricks Lakehouse accreditation exam at the Databricks academy, but the badge does not appear on the platform and it has been about an hour, I hope you can help me. What it does let me do is download a certificate.
- 5304 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Luis Sanchez Otiniano Thank you for reaching out! Please submit a ticket to our Training Team here: https://help.databricks.com/s/contact-us?ReqType=training and our team will get back to you shortly.
- 2 kudos
- 2294 Views
- 3 replies
- 3 kudos
Previous data is getting wiped off for delta tables
I am using only insert sql query to insert the hist. load but previous data getting deleted.Tried with python query also but same issue persists.Reading the data from gcp bucket(parquet file)writing the data into gcp bucket(delta file)..the deleted f...
- 2294 Views
- 3 replies
- 3 kudos
- 3 kudos
Share your query and also look for any error messages in the driver logs. This might help to undertand better what is happening.
- 3 kudos
- 2370 Views
- 2 replies
- 2 kudos
Not seeing rewards on Canary
Hi @Christy Seto I received the fundamental certificate and have joined the community group but still not showing any rewards on Canary, it's been a week I did those. Could you please have a look on it. Thanks in advance!
- 2370 Views
- 2 replies
- 2 kudos
- 1931 Views
- 1 replies
- 0 kudos
Connect to PostgreSQL to Databricks community edition error
Dear Team,I am trying to establish a connectivity to PostgreSQL to Databricks community edition using sql notebookhowever I am encountering the below errorError in SQL statement: IllegalArgumentException: requirement failed: Host name should not cont...
- 1931 Views
- 1 replies
- 0 kudos
- 0 kudos
@Teamany suggestions ?
- 0 kudos
- 1571 Views
- 1 replies
- 5 kudos
Lakehosue table structure design
We are in process of implementing a lakehouse using Azure Databricks. We already have a datalake in placeAzure Storage Datalake – Contains containers which has data in its native format.How we are planningBuild Bronze layer by create bronze tables by...
- 1571 Views
- 1 replies
- 5 kudos
- 5 kudos
hey @Kuldeep Chitrakar as you are saying that you do not have a partitioning in bronze tables , so according to that statement that is okay . but in silver as you are going to implement partitioning so , what i will recommend to you is that for bett...
- 5 kudos
- 4072 Views
- 0 replies
- 6 kudos
To connect Delta Lake with Microsoft Excel, you can use the Microsoft Power Query for Excel add-in. Power Query is a data connection tool that allows ...
To connect Delta Lake with Microsoft Excel, you can use the Microsoft Power Query for Excel add-in. Power Query is a data connection tool that allows you to connect to various data sources, including Delta Lake. Here's how to do it:Install the Micros...
- 4072 Views
- 0 replies
- 6 kudos
- 3931 Views
- 1 replies
- 4 kudos
What is the purpose of the USAGE privilege?
I watched a couple of courses on Databricks Academy, none of which clearly explains or demonstrates the purpose of the USAGE privilege.USAGE: does not give any abilities, but is an additional requirement to perform any action on a schema object.I hav...
- 3931 Views
- 1 replies
- 4 kudos
- 4 kudos
hey @S L I also have these questions , and what i get to know that usage is the minimum and mandot requirement which individual should have to perform any actions , that does not mean that you can do any actions by only usage permission , usage is...
- 4 kudos
- 7689 Views
- 3 replies
- 2 kudos
Resolved! New spark cluster being configured in local mode
Hi,We have two workspaces on Databricks, prod and dev. On prod, if we create a new all-purpose cluster through the web interface and go to Environment in the the spark UI, the spark.master setting is correctly set to be the host IP. This results in a...
- 7689 Views
- 3 replies
- 2 kudos
- 2 kudos
I found the same issue when choosing the default cluster setup on first setup that when I went to edit the cluster to add an instance profile, I was not able to save without fixing this. Thanks for the tip
- 2 kudos
- 1235 Views
- 2 replies
- 2 kudos
sharikrishna26.medium.com
Spark Dataframe MetadataSpark Dataframe is structurally the same as the table. However, it does not store any schema information in the metadata store. Instead, we have a runtime metadata catalog to store the Dataframe schema information. It is simil...
- 1235 Views
- 2 replies
- 2 kudos
- 3053 Views
- 3 replies
- 5 kudos
Availability Zone in Azure
Please Find the content Herehttps://medium.com/@sharikrishna26/availability-zone-in-azure-52e7764357b6
- 3053 Views
- 3 replies
- 5 kudos
- 5 kudos
yeah this is awesome thanks, but ever you think that what will happen if in that region no instance will left , how our jobs will start , any idea here?
- 5 kudos
- 2541 Views
- 3 replies
- 2 kudos
Spark Notebook to import data into Excel
Is there a way to create a notebook that will take the SQL that I want to put into the Notebook and populate Excel daily and send it to a particular person?
- 2541 Views
- 3 replies
- 2 kudos
- 4240 Views
- 3 replies
- 3 kudos
Resolved! Use of badRecordsPath in COPY INTO SQL command
I'm trying to use the badRecordsPath to catch improperly formed records in a CSV file and continue loading the remainder of the file. I can get the option to work using python like thisdf = spark.read\ .format("csv")\ .option("header","true")\ .op...
- 4240 Views
- 3 replies
- 3 kudos
- 3 kudos
Thanks. It was the inferSchema setting. I tried it with and without the SELECT and it worked both ways when I added inferSchemaBoth of these workeddrop table my_db.t2; create table my_db.t2 (col1 int,col2 int); copy into my_db.t2 from (SELECT cast(...
- 3 kudos
| User | Count |
|---|---|
| 1616 | |
| 790 | |
| 466 | |
| 349 | |
| 287 |
