Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
Problem: Databricks Apps timeout when connecting to SQL Warehouses, but same connection works locally. First Databricks App I've attempted to create. Symptoms:Local Streamlit app works fine with Databricks SQL connectorDeployed Databricks App hangs ...
Hi @johnv, Do you have any error messages you can share, or is it a silent connection timeout? I assume you confirmed that the SQL Warehouse had started up? (they can take several minutes to start up sometimes) If that's not the issue, you could al...
Is there a way to calculate the percentage of total using the Databricks dashboard's custom calculations and also the cumulative percentage? What function would be equivalent to the DAX All and AllSelect functions, but using Databricks' aggregation f...
Hi @genebaldorios ,I don't use databricks dashboards on my project (we are PBI shop), but I guess you need to use AGGREGATE OVER clause with cumulative frame:
Hello Everyone,I recently started using AI/BI dashboard and I'm finding it difficult to cross-filter two visuals from two different dataset. Visual 1 displays Sales by region (month to date) sourced from dataset 1 and Visual 2 displays Open orders by...
@lennox1 I've just gave this a try and didn't have much luck for cross-filter across multiple datasets. The documentation seems to indicate the same thing: https://learn.microsoft.com/en-us/azure/databricks/dashboards/#cross-filter I scanned the rele...
I have a Databricks Dashboard that contains several choropleth maps. They all stopped working today. I can see a background image, but no data is represented on them. The dataset seems ok and the visualizations don't show any errors. I don't see ...
Glad to hear you figured it out. Appreciate you updating the thread with the fix, @Unimog!
I followed the official Databricks documentation("https://docs.databricks.com/en/_extras/notebooks/source/mongodb.html")to integrate MongoDB Atlas with Spark by setting up the MongoDB Spark Connector and configuring the connection string in my Datab...
Hi @vidya_kothavale ,Could you try to change "spark.mongodb.input.uri" to following?spark.read.format("mongodb").option("spark.mongodb.read.connection.uri"
Hi everyone,I'm trying to query data from our adls2 delta lake using a serverless sql warehouse. We've already set up private connectivity via NCC, but hitting a snag when running queries like:SELECT * FROM delta.`abfss://container@xxx.dfs.core.windo...
Hi, thanks for the detailed explanation.Unfortunately, configuring fs.azure.account.key in the Serverless advanced options didn’t help (I’m sure I wrote it correctly) - still receiving the same error.I saw in some sources around the net that I should...
I am using streaming tables to read from multiple parquet files and create a table in my raw layer.Tech Stack -- dbt-databricksWhile loading I have a column called "ets" in my source which is nothing "milliseconds since Jan 1st 1970." I want this to ...
Hi @singh_tushar_14 ,Since your ets columns already contains integer that represents milliseconds since Jan 1st 1970, can't you just use Spark SQL fuction? You don't need to add anything to 1970-01-01, since that information is already "encoded" in y...
Hi all,Lakebase is currently advertised as a database system that can scale down to zero:https://www.databricks.com/blog/what-is-a-lakebaseDoes anyone know if this scale-to-zero behavior is handled automatically by Databricks when the database is idl...
I think that it's a feature they will add eventually, but it's not actually available now.I've had a Lakebase active for a month, unused for about half the time, and it use 24 DBUs per day pretty much constantly.
serverless compute:cannot find column index for attribute error We are seeing this error if there are a complex view, the same view runs fine in sql warehousebut break in serverlesss compute onlylooks like serverless compute has issue parse large co...
ok, I will test on latest version 4 and will let you know if any more issues, thanks
Hi All,I've encountered what seems like a very specific bug. For some reason, running a query with the following where clause results in an execution error.where lower(organization_name) like '%trac%'There's no execution error when trying "tra" and i...
@JPan I noticed you had the new SQL editor mode on in your video. For what it's worth, I flicked between both when testing and both were successful. Given that you're finding it's only happening on some fields, do you think this is to do with special...
What are the new features we are going to see in the Dashboard this year?
@szymon_dybczak thanks for those suggestions. I didn't even know about the Product Roadmap Webinar every quarter. I'll definitely be tuning in for those.All the best,BS
Hi All,I've been using notebooks to run patch requests to an external API using the Python requests library. Often times certain notebooks will randomly start to fail throughout the day and will raise a ImportHookFinder.find_spec() error when attempt...
Hey, Did you manage to solve this issue? I am experiencing the same error but under a different context of using pytest in a notebook.My scenario:import pytest import os import sys os.chdir(f'/Workspace/Shared/DataTechnology-Databricks-Core') # Skip...
Photon ran out of memory while executing this query. Photon failed to reserve 32.0 KiB for BufferPool, in BroadcastBufferedRelation(spark_plan_id=13407). Memory usage: Total task memory (including non-Photon): 1829.7 MiB BroadcastBufferedRelation(spa...
Hello @vijamit Good day!!It was very hard to analyise the error but : The causes for the error shared are : Your query has run out of memory during execution, specifically when using the BuildHashedRelation and PartitionedRelation functions.Running o...
Using the old SQL editor, there was an option to add tags to saved queries under the "edit query info" option shown in the screenshots. Since starting to use the new editor, I noticed that option has disappeared from its previous location and I haven...
Hello @jtbing!At the moment, adding or editing tags for saved queries isn’t available in the new SQL editor UI. There’s also an upcoming Bricktalks session with the Product Manager for Databricks SQL, introducing the new SQL editor; you may find it u...
Hi All,I am getting below message when hovering on the Create SQL warehouse. I can only see the Serverless Starter Warehouse compute.You have reached the maximum number of SQL warehouses. Delete an existing warehouse to create new one.Regards,RN
