- 13493 Views
- 2 replies
- 3 kudos
Resolved! Error : MALFORMED_REQUEST
Hello,I get error for below json. Can you please advice what am I missing here?{ "error_code": "MALFORMED_REQUEST", "message": "Invalid JSON given in the body of the request - failed to parse given JSON"_________________________________________...
- 13493 Views
- 2 replies
- 3 kudos
- 3 kudos
Thank you Pat. I just realised I had copy pasted the code and it didnt quite copy 1:1. In copy paste double quotes were appeared to be copied but behind the scene, it was some other character. Nothing wrong with the code. It works
- 3 kudos
- 1153 Views
- 0 replies
- 1 kudos
Sergio Copy Data World tour Orchestration_v02.pptx (4)
Databricks Workflows: Reliable orchestration for data, analytics, and AIHello, I am a Solutions Architect at Databricks; and I recently presented at the Data and AI World Tour in London Databricks Workflows. You can see some of the presented slides a...
- 1153 Views
- 0 replies
- 1 kudos
- 6549 Views
- 5 replies
- 1 kudos
Resolved! Databricks Academy LMS - shortcomings
I am using Databricks Academy extensively and see a few shortcomings in the (new) LMS -- Feedback:- Videos do not have captions/transcript to make them accessible to all audience including non-native English speakers.- "FullScreen" icon does not work...
- 6549 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @al_joe! My name is Astrid, and I’m on the curriculum team here at Databricks. I was able to review your original post in this thread and wanted to update you on some of the input and questions you provided: Videos do not have captions/transcript ...
- 1 kudos
- 6310 Views
- 3 replies
- 5 kudos
The transaction log has failed integrity checks. We recommend you contact Databricks support for assistance. Failed verification at version 48 of:
I am trying to write data frame data into delta table. Previously it was working fine but now it is throwing "Log has failed integrity checks"
- 6310 Views
- 3 replies
- 5 kudos
- 5 kudos
@Shanmuganathan Jothikumar​ I've the same exception after upgrading into unity catalog. Need to investigate a little more but adding the following setting, it works:spark.conf.set("spark.databricks.delta.state.corruptionIsFatal", False)
- 5 kudos
- 4723 Views
- 3 replies
- 4 kudos
Resolved! pyspark streaming failed for now reason
Hi everyone, I have a pyspark streaming reading from an aws kinesis that suddenly failed for no reason (I mean, we did not make any changes in the last time).It is giving the following error: ERROR MicroBatchExecution: Query kinesis_events_prod_bronz...
- 4723 Views
- 3 replies
- 4 kudos
- 4 kudos
@patricio tojo​ I've the same problem, however in my case is after migrating into unity catalog. Need to investigate a little more but adding this to my spark job, it works:spark.conf.set("spark.databricks.delta.state.corruptionIsFatal", False)
- 4 kudos
- 13182 Views
- 5 replies
- 2 kudos
Resolved! Databricks Terraform Provider Issues Passing Providers to Child Modules
I have been following the documentation on the terraform databricks documentation in order to provision account level resources on AWS. I can create the workspace fine, add users, etc... However, when I go to use the provider in non-mws mode, I am re...
- 13182 Views
- 5 replies
- 2 kudos
- 2 kudos
So the answer to this was that you need to explicitly pass the provider argument to each of the data resources blocks. The docs should be updated to accommodate that. ​i.e. data "databricks_spark_version" "latest" { provider = databricks.workspace ...
- 2 kudos
- 2678 Views
- 1 replies
- 7 kudos
Delta Live tables support for UNITY CATALOG
Is there any upcoming update in which UC will support DLT? if yes any expected ETA?
- 2678 Views
- 1 replies
- 7 kudos
- 7 kudos
Hi @Rahul Mishra​,I think you need to contact your comapny databricks representative for this.The last time I heard about the ETA It was end of the November I believe.You might try to join the Databricks Office Hours tomorrow and ask the question or ...
- 7 kudos
- 2242 Views
- 1 replies
- 2 kudos
- 2242 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Ranjeeth Rikkala​ ,you can check this:https://docs.databricks.com/optimizations/index.html#optimization-recommendations-on-databricksTBH, It’s nit enough info. Make sure you have partioned data, Files are nit too small, try to use bigger cluster,...
- 2 kudos
- 1219 Views
- 0 replies
- 5 kudos
DAIWT22_RadicalSpeenInLakehouse_Photon
Topic: Radical Speed on the Lakehouse: Photon under the hoodI am Hari and I works as a Specialist Solutions Architect at Databricks. I specialise in Data engineering and Cloud platforms problems helping client in EMEA.Purpose:I recently presented a t...
- 1219 Views
- 0 replies
- 5 kudos
- 1474 Views
- 0 replies
- 7 kudos
This post is regarding Data Streaming on the Lakehouse' session in Data + AI World Tour 2022 in London. I am Resident Solutions Architect at Datab...
This post is regarding Data Streaming on the Lakehouse' session in Data + AI World Tour 2022 in London. I am Resident Solutions Architect at Databricks. I specialise in data engineering.In this session, I talked about how to leverage real time data t...
- 1474 Views
- 0 replies
- 7 kudos
- 19099 Views
- 2 replies
- 3 kudos
Resolved! Default Query Limit 1000
By default, we return back up to 1000 query results when a user runs a cell in Databricks. E.g., if you run display(storeData) and you have ten million customers, the UI will show the first 1000 results. If you graph that by age of customer, similarl...
- 19099 Views
- 2 replies
- 3 kudos
- 3 kudos
This is simple in Databricks SQL, just uncheck LIMIT 1000 in the drop down.
- 3 kudos
- 9187 Views
- 2 replies
- 2 kudos
Resolved! Default Query Limit 1000
Is there any way to change the 1000 for the display row limit at workspace, cluster and notebook level?
- 9187 Views
- 2 replies
- 2 kudos
- 2 kudos
This is simple in Databricks SQL, just uncheck LIMIT 1000 in the drop down.
- 2 kudos
- 10175 Views
- 4 replies
- 8 kudos
Resolved! Create Databricks tables dynamically
Hi, I would like to be able to do something like this...create table if not exists table1using parquetlocation = '/mnt/somelocationsome location needs to be a concatenation of static and code generated string. Documentation suggests that location onl...
- 10175 Views
- 4 replies
- 8 kudos
- 8 kudos
FString Python can be used. example > spark.sql(f"CREATE TABLE {table_name} (id INT, name STRING, value DOUBLE, state STRING)")
- 8 kudos
- 3662 Views
- 2 replies
- 5 kudos
Resolved! Databricks set up in Prod environment
Hi! can anyone please help me with a documentation which can help me set up integration between data bricks with AWS without a QuickStart default cloud formation template. I would want to use my own CFT rather than using the default due to security ...
- 3662 Views
- 2 replies
- 5 kudos
- 5 kudos
Hi @Atul S​ ,I think that terraform is recommended way to go with Databricks deployment. I mean it's also supported now by the Databricks support.I haven't look much on the CloudFormation setup, because we decided to go with the Terraform in the comp...
- 5 kudos
- 11606 Views
- 1 replies
- 4 kudos
Resolved! Accept widget value during runtime from user
list1 = ('alpha', 'beta', 'gamma', 'eta', 'Theta')list2 = ('alpha', 'beta')df1 = spark.createDataFrame(list1, 'String').withColumnRenamed('value', 'dataset')df2 = spark.createDataFrame(list2, 'String').withColumnRenamed('value', 'dataset')df = df1.ex...
- 11606 Views
- 1 replies
- 4 kudos
- 4 kudos
Every time the user change selection in the widget, you get that selection—behavior after the dropdown change you can define in widget settings.
- 4 kudos
| User | Count |
|---|---|
| 1632 | |
| 791 | |
| 513 | |
| 349 | |
| 287 |
