cancel
Showing results for 
Search instead for 
Did you mean: 
Warehousing & Analytics
Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

MadelynM
by Databricks Employee
  • 1340 Views
  • 0 replies
  • 0 kudos

[Recap] Data + AI Summit 2024 - Warehousing & Analytics | Improve performance and increase insights

Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights.  Keynote: Data Warehouse presente...

Screenshot 2024-07-03 at 10.15.26 AM.png
Warehousing & Analytics
AI BI Dashboards
AI BI Genie
Databricks SQL
  • 1340 Views
  • 0 replies
  • 0 kudos
BamBam
by New Contributor III
  • 2272 Views
  • 1 replies
  • 0 kudos

Where are driver logs for SQL Pro Warehouse?

In an All-Purpose Cluster, it is pretty easy to get at the Driver logs.  Where do I find the Driver Logs for a SQL Pro Warehouse?  The reason I ask is because sometimes in a SQL Editor we get generic error messages like "Task failed while writing row...

Warehousing & Analytics
SQLProWarehouse
  • 2272 Views
  • 1 replies
  • 0 kudos
Kaz
by New Contributor II
  • 3488 Views
  • 4 replies
  • 1 kudos

Automatically importing packages in notebooks

Within our team, there are certain (custom) python packages we always use and import in the same way. When starting a new notebook or analysis, we have to import these packages every time. Is it possible to automatically make these imports available ...

  • 3488 Views
  • 4 replies
  • 1 kudos
Latest Reply
Tharun-Kumar
Databricks Employee
  • 1 kudos

@Kaz  You can install these libraries using the Libraries section in the Compute.  All of the libraries mentioned here would be installed whenever the cluster is spun up.

  • 1 kudos
3 More Replies
SimonMcCor
by New Contributor
  • 2986 Views
  • 1 replies
  • 1 kudos

Calculated Field in Dashboards

Is there a way to create a calculated field in a dashboard from the data that has been put into it?I have an aggregated dataset that goes into a dashboard, but using an average in the calculation will only work if I display the average by the grouped...

  • 2986 Views
  • 1 replies
  • 1 kudos
Erik
by Valued Contributor III
  • 8570 Views
  • 0 replies
  • 0 kudos

Hot path event processing and serving in databricks

We have a setup where we process sensor data in databricks using pyspark structured streaming from kafka streams, and continuisly write these to delta tables. These delta tables are served through a SQL warehouse endpoint to the users. We also store ...

  • 8570 Views
  • 0 replies
  • 0 kudos
colinsorensen
by New Contributor III
  • 2374 Views
  • 0 replies
  • 0 kudos

Unhandled error while executing ['DatabricksSQLCursorWrapper' object has no attribute 'fetchmany'

Getting this error in dbt when trying to run a query. Not happening in the actual SQL warehouse in Databricks. Is this a bug? Can only find source code when I search 'DatabricksSQLCursorWrapper' but no documentation or information otherwise.

  • 2374 Views
  • 0 replies
  • 0 kudos
hehuan-yu-zen
by New Contributor II
  • 1721 Views
  • 1 replies
  • 0 kudos

customise the dates showing in the calendar selection in sql editor/dashboard

Does anybody know whether we could customise the dates showing in the calendar selection in sql editor/dashboard?My query has a time frame in a particular period, however when I use DateRange parameter in sql editor, it could allow users to choose th...

  • 1721 Views
  • 1 replies
  • 0 kudos
mortenhaga
by Contributor
  • 6484 Views
  • 3 replies
  • 3 kudos

Resolved! Databricks SQL and Engineer Notebooks yields different outputs from same script

Hi allWe are having some alarming issues regarding a script that yields different output when running on SQL vs Notebook. The correct output should be 8625 rows which it is in the notebook, but the output in Databricks SQL is 156 rows. The script use...

serverless cluster.JPG notebook compute info.JPG corrupt script wrong output notebook with serverless cluster.JPG corrupt script wrong output sql.JPG
Warehousing & Analytics
Databricks SQL
Notebook
Wrong output
  • 6484 Views
  • 3 replies
  • 3 kudos
Latest Reply
mortenhaga
Contributor
  • 3 kudos

UPDATE:I think we have identefied and solved the issue. It seems like using LAST with Databricks SQL requires to excplicitly be careful about setting the "ignoreNull" argument and also be careful about the correct datatype. I guess this is because of...

  • 3 kudos
2 More Replies
uberweiss
by New Contributor II
  • 4440 Views
  • 1 replies
  • 0 kudos

Unable to access Databricks cluster through ODBC in R

We have previously been able to access our Databricks cluster in R using ODBC but it stopped working a couple of months ago and now i can't get it to connect.I've downloaded the latest drivers and added the right information in odbc/odbcinst files bu...

Warehousing & Analytics
cluster
Databricks
ODBC
R
  • 4440 Views
  • 1 replies
  • 0 kudos
peterlandis
by New Contributor II
  • 3090 Views
  • 1 replies
  • 0 kudos

Calculate the total size in bytes for a column

I wanted to calculate the total size in bytes for a given column for a table.  I saw that you can use the bit_length function and did something like this giving you the total bits of the column but not sure if this is correct.SELECT sum(bit_length(to...

  • 3090 Views
  • 1 replies
  • 0 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 0 kudos

I looked at the docs of bit_length and it does not state if it is before or after compression.However since spark decompresses data on read, it is very likely it is the size before compression.The table size is read from metadata and is compressed.To...

  • 0 kudos
MonikaSamant
by New Contributor II
  • 3053 Views
  • 3 replies
  • 2 kudos

Need to connect Looker studio with Databricks tables

Hi Team, I am creating my data warehouse on AWS s3 and corresponding tables on databricks warehouse.I want to connect looker studio (which is different from looker) to these databricks tables and be able to create reports.Could you please help us on ...

  • 3053 Views
  • 3 replies
  • 2 kudos
Latest Reply
807326
New Contributor II
  • 2 kudos

We are also interested in this functionality. But there are no databricks connectors for Looker Studio. But as Databricks SQL Statement Execution REST API is already available in public preview, it is now possible to Build a Community Connector for d...

  • 2 kudos
2 More Replies