Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Good post
I can see on Databricks SQL warehouse Data tab that clusters, catalogs and schemas have a unique ID. User created tables, views and functions must have and unique ID too, but it is not exposed to the user as far as I can tell. I need to retrieve the ...
I am pulling data from Google BigQuery and writing it to a bronze table on an interval. I do this in a separate continuous job because DLT did not like the BigQuery connector calling collect on a dataframe inside of DLT. In Python, I would like to ...
The below code is a solution. I was missing that I could read from a table with `spark.readStream.format("delta").table("...")`. Simple. Just missed it. This is different than `dlt.read_stream()` which appears in the examples a lot.This is referenced...
We've been using the DQBSQL API to perform CRUD on queries and alerts.Part of that process added a slack channel as alert destination using the /subscriptions element on an alert post as below.As of today I am getting a 410 'Gone' error from the API ...
Hi @Suteja Kanuri​ , @Vidula Khanna​ thanks for getting back with a solution.The suggested solution looks fine, but for a number of reasons I went with another option to use the Jobs API. This allowed me to preserve more of the automation I had alrea...
IPL Prediction is very challenging task. Due to the nature of the game, it is challenging to correctly predict the winner because even one player's performance can significantly affect the outcome. It is also challenging to forecast which club woul...
Hi FerArribas,I struggle with PrivateLink connection between Databricks account and my AWS account. I have seen that you had a similar problem. I can create a workspace, but when I try to create a cluster I get an error: "Cluster terminated. Reason: ...
Hi @Marcin Sieradzan​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answ...
Hi , I am trying to load power bi report in databricks,but i am getting..empty while printing report.. i have already installed pip install powerbiclient,and device_auth is also successfull,but report = Report(group_id=group_id, report_id=report_id, ...
Hi @Adarsh Singh​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers ...
Hi everyone, can you please give some inputs on this please...
We are going to be a databricks customer and did some PoC tests. Our one test contains dataset in one partitioned table (15colums) is roughly 250M rows, each partition is ~50K-150K rows. Occasionally we have hundreds (up to one thousand) concurrent u...
Hi @Marian Kovac​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...
Hi, Recently, I am having some problems viewing the query plans in the Spark UI SQL/Dataframe tab. I would expect to see large query plans in the SQL tab where we can observe the details of the query such as the rows read/written/shuffled. Howeve...
@Koray Beyaz​ :This issue may be related to a change in the default behavior of the Spark UI in recent versions of Databricks Runtime. In earlier versions, the Spark UI would display the full query plan for SQL and DataFrame operations in the SQL/Dat...
Hello Everyone, We are trying to connect DataBricks SQL warehouse using ODBC url but we are not able to do it. We can see only JDBC url in connection details which works fine. Was anyone able to connect using ODBC url? Can someone please help?
Hi,Can you please try to connect once after disabling VPN in your local system (if its enabled)?Thanks.
I was setting a password for a new account in the databricks management page, and when I attempted to add a password of 50 characters, the prompt said "password must be no more than 50 characters" (bold added by me for emphasis). It would only accept...
@Vidula Khanna​ This question isn't related to Training -- none of the prompts in that ticket page are relevant. It's related to the Databricks Account Management Page password-setting prompt.
We have a team of multiple people, and they need to do Databricks Terraform config from the CLI. We'd like to do this without having all people be account admins in Databricks.What is the best way to do that?Right now, we have Terraform set up so tha...
I think the Service Principal definitely works for workspaces created from scratch with the Service Principal. I've tested that in my test environment, and have not encountered these kinds of authentication problem.The issue I am seeing is with works...
Hi all,My databricks "recents" (recent notebooks) are only saving for a given 'session'. If I close databricks, and come back in, all of my recents are cleared both on the login screen and the one I get when I click the "recents" icon on the left me...
Did you ever find a solution to this? My recents aren't working either.
I have a number of datasets that are published to Power BI service. When I published these from Power BI Desktop, I was able to enable period refresh. This morning however, the periodic refresh failed with a 403 Authentication Failed error. I can't e...
Hi @Michael Anckaert​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
