Engage in discussions on data warehousing, analytics, and BI solutions within the Databricks Community. Share insights, tips, and best practices for leveraging data for informed decision-making.
Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights.
Keynote: Data Warehouse presente...
Did you know the default timeout setting for SQL #databricks Warehouse is two days?The default timeout can be too long for most use cases. You can easily change this for your session or in the general SQL warehouse configuration.
Hi @Hubert-Dudek we have a java service which uses JDBC Template for connecting to databricks warehouse. We recently set a high connection idle timeout and maxLifeTime for our hikari pool connection. We are now seeing error related to invalid session...
Hi @Martin_Pham ,At the current moment SQL Warehouses don't have this option. But according to one of the employees at databricks, this feature is coming soon: https://community.databricks.com/t5/warehousing-analytics/sqlwarehouse-case-insensitive/td...
Application connects to Databricks serverless SQL warehouse via Databricks JDBC driver. It executes SQL select statements only. We see small number of statements failed each day with the following error detail: java.sql.SQLException: [Databricks][JDB...
Hi there,I'm building a Sink that uses the Databricks JDBC Driver version 2.6.27. It appears that regardless of using a try-with-resources block and explicitly closing the Connection, the driver's internals are not releasing IdleConnectionEvictor thr...
Still not fixed got 14k instances of com.databricks.client.jdbc42.internal.apache.http.impl.client.IdleConnectionEvictor 14,037 (0%) 673,776 B (0%) n/a And due to that application got to crashed. can you please help out as this is blocking to release...
Hi Team,There is a requirement to load a backup file into a database inside a SQL Warehouse. However, I don't see any option to directly load abackup file.I have tried reading the backup file using a notebook, but I’m unable to interpret the contents...
Hi community,we have several databricks workspaces per different teams and in each workspace several use case covered.How can we trace the costs per individual use cost for SQL Warehosue / serverless in case we use one SQL warehouse per workspace to ...
Is it possible to migrate Databricks Legacy Dashboards along with associated components such as queries and datasets from one workspace to another? I have attempted to export the Legacy Dashboards along with the queries and datasets, but upon importi...
Hello, to migrate between workspaces you'll need a different strategy based on the asset type
Legacy dashboards you have completed, but for anyone reading you can use the 'export' functionalityQueries: you can either put them together in a folder and...
When trying incorporate an R package into my Spark workflow using the spark_apply() funciton in Sparklyr, I get the error:Error: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.RowEncoder$.apply(Lorg/apache/spark/sql/types/StructT...
I trying to connect databricks workspace to Redshift with security groups. Before I set up security groups it worked fine but now I can't connect to it because of inbound rules. What databricks IP's I should add to inbound rules to make it work?
Hi all,Just started using Db Dashboards on Azure. While generating a line chart with a x-axis value based on a date data type I've applied a filter which removes some of the data. The above is without filtering, the below is with filtering. You can o...
I'm encountering an issue while importing a .gz file containing JSON data into a Spark DataFrame in Databricks. The error indicates a column name conflict. Could you please advise on how to resolve this issue and handle duplicate column names during ...
We are having Databricks tables and we have an API which query that data, the query will be dynamic and API would allow user to query anything. However users can query lot of data and consume lot of DBU but generic rate limiting wont help, as any sin...
Hi everyone,We are at the moment stumbeling upon a big challenge with loading data into PowerBI. I need some advice!To give a bit of conext: we introduced Databricks instead of Azure Synapse for a client of ours. We are currently busy with moving all...
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.