- 2405 Views
- 1 replies
- 0 kudos
Resolved! Lineage graph now working.
Hey everyone,I've run the following code successfully:CREATE CATALOG IF NOT EXISTS lineage_data;CREATE SCHEMA IF NOT EXISTS lineage_data.lineagedemo;CREATE TABLE IF NOT EXISTS lineage_data.lineagedemo.menu ( recipe_id INT, app string, main ...
- 2405 Views
- 1 replies
- 0 kudos
- 0 kudos
I recommend you open a ticket to the support.
- 0 kudos
- 2060 Views
- 0 replies
- 0 kudos
How to update table location with wasb to abfss
I created a table including location such as: wasb://<container>@<storageaccount>.blob.core.windows.net/foldername We have updated access to storage accounts to use abfssI am trying to execute the following command: alter table mydatabase.mytable set...
- 2060 Views
- 0 replies
- 0 kudos
- 21051 Views
- 8 replies
- 3 kudos
Dynamic Partition Overwrite for Delta Tables
Spark supports dynamic partition overwrite for parquet tables by setting the config: spark.conf.set("spark.sql.sources.partitionOverwriteMode","dynamic") before writing to a partitioned table. With delta tables is appears you need to manually specif...
- 21051 Views
- 8 replies
- 3 kudos
- 3 kudos
@SamCallister wrote: Spark supports dynamic partition overwrite for parquet tables by setting the config:spark.conf.set("spark.sql.sources.partitionOverwriteMode","dynamic")before writing to a partitioned table. With delta tables is appears you need ...
- 3 kudos
- 1435 Views
- 0 replies
- 0 kudos
Web scraping with Databricks
Hello,What is the easiest way to do web scraping in Databricks? Let's imagine that from this link: http://automated.pythonanywhere.com , I need to grab this element "/html/body/div[1]/div/h1[1]" and return a text, how can I do it? Can somebody write ...
- 1435 Views
- 0 replies
- 0 kudos
- 2074 Views
- 1 replies
- 1 kudos
Structured Streaming from TimescaleDB?
I realize that the best practice would be to integrate our service with Kafka as a streaming source for Databricks, but given that the service already stores data into TimescaleDB, how can I stream data from TimescaleDB into DBX? Debezium doesn't wor...
- 2074 Views
- 1 replies
- 1 kudos
- 1 kudos
https://github.com/noctarius/timescaledb-event-streamer/Might help.
- 1 kudos
- 10176 Views
- 2 replies
- 2 kudos
Resolved! Get metadata of tables in hive metastore
Hi team,I have a requirement to get the metadata of tables available in databricks hive metastore.Is there any way to get the metadata of all the tables inspite of looping through tables using Describe table_name.As hive metastore doesnot support inf...
- 10176 Views
- 2 replies
- 2 kudos
- 3340 Views
- 2 replies
- 0 kudos
Resolved! The driver is temporarily unavailable
My job fails with Driver is temporarily unavailable. Apparently, it's permanently unavailable, because the job is not pausing but failing.
- 3340 Views
- 2 replies
- 0 kudos
- 0 kudos
I am facing the same issues . I am writing in batches using a simple for loop. I don't have any collect statements inside the loop. I am rewriting the partitions with partition overwrite dynamic mode in a huge wide delta table - several tb. The incr...
- 0 kudos
- 9119 Views
- 10 replies
- 1 kudos
How to run sql query/notebook present on S3 on databricks cluster?
I have code files (.hql) in s3 which was running on Hive. But now, I need to run them on the Databricks cluster. I can rename the files to .sql and add comment "-- Databricks notebook source" on top of the files to treat them as single cell notebook....
- 9119 Views
- 10 replies
- 1 kudos
- 2473 Views
- 1 replies
- 1 kudos
try_to_timestamp not working as expected
try_to_timestamp should return NULL for following expression but it is throwing error: try_to_timestamp( '2019-02-28 23:59:59.000', 'yyyy-MM-dd HH:mm:ss' ) I understand above expression is wrong as the date format does no...
- 2473 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @PriyanshuKumar,Thank you for your question in the Databricks community.The try_to_timestamp() function is designed to attempt to convert a string to timestamp format based on a specified pattern and return null if it fails to do so. However, in t...
- 1 kudos
- 8194 Views
- 5 replies
- 8 kudos
Resolved! Change Data Feed Databricks
Hello,I am doing some testing with this feature Change Data Feed using Databricks and Pyspark, of course the Delta format and I don't understand something:I created a tableSaved some data insideEnabled Change Data Feed featureApply a merge with a dat...
- 8194 Views
- 5 replies
- 8 kudos
- 8 kudos
@Mihai_Cog You have to split your merge statement into 2 parts. (Update and Insert/Delete).MERGE INTO test t USING src s ON s.Id = t.Id and s.date_field = t.date_field and s.fields <> t.fields WHEN MATCHED THEN UPDATE SET * MERGE INTO test t USING sr...
- 8 kudos
- 7823 Views
- 10 replies
- 9 kudos
Resolved! Request for reattempt voucher. Databricks Certified Associate Developer for Apache Spark 3.0 exam
Hi,I gave Databricks Certified Associate Developer for Apache Spark 3.0 exam today but missed by one percent. I got 68.33% and pass is 70%.I am planning to reattempt the exam, could you kindly give me another opportunity and provide reattempt voucher...
- 7823 Views
- 10 replies
- 9 kudos
- 9 kudos
Hi,I gave Databricks Certified Associate Developer for Apache Spark 3.0 Python exam yesterday but missed by three percent. I got 66.66% and pass is 70%.I am planning to reattempt the exam, could you kindly give me another opportunity and provide reat...
- 9 kudos
- 7206 Views
- 6 replies
- 12 kudos
Resolved! SSRS (on-prem) on Databricks SQL
Has anybody succeeded in querying Databricks SQL with an on-prem SSRS (so an on-prem Report Server and Report Builder)?I manage to create a connection that works (according to the connection test anyway), but the moment I try to create a dataset on t...
- 7206 Views
- 6 replies
- 12 kudos
- 12 kudos
Hi @-werners- , I am able to build connection between Microsoft visual studio and data bricks using Simba Spark ODBC driver. I can query delta tables sitting in Databricks from Microsoft Visual studio (SSRS). However, when I am deploying the report t...
- 12 kudos
- 7301 Views
- 4 replies
- 1 kudos
SSRS Connect to Databricks
Hi ,I need to connect databricks query from microsoft SSRS.is it possible ? How do you make the connection?
- 7301 Views
- 4 replies
- 1 kudos
- 1 kudos
@Haider93 https://www.databricks.com/company/partners/technology This is the list of supported databricks technology partners. SSRS is not yet part of it. We do have official support for Power BI. You can use this document to mimic it for SSRS - http...
- 1 kudos
- 814 Views
- 0 replies
- 0 kudos
Why no Practice Test for Databricks Professional Exam ?
Hi Databricks Team/Community,I'm planning for Databricks Data Engineering Professional Certification but what I have seen is that Databricks don't give any practice test for this professional/advanced certification. I don't know why, If anyone is rea...
- 814 Views
- 0 replies
- 0 kudos
- 28603 Views
- 2 replies
- 0 kudos
Can you delete a widget, or force a value to it?
Hello everyone! Basically, I have a dropdown widget, and at some point I want to change the options available, but still maintain the same selected option. The problems that I find are: - If I want to delete the widget and create a new one, it seems...
- 28603 Views
- 2 replies
- 0 kudos
- 0 kudos
I know this is 5+ years late but I would like to add an answer (or work around) to this in case anyone else was looking for the same solution.What I did was to essentially 'reset' the widget whenever I had to assign (or force) a value to it.#Cell #1 ...
- 0 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
