Here's your Data + AI Summit 2024 - Warehousing & Analytics recap as you use intelligent data warehousing to improve performance and increase your organization’s productivity with analytics, dashboards and insights. Keynote: Data Warehouse presente...
How to tuning databricks query performance from Power BI Dosktop
Hi Joeshph,How are you doing today?Give a try with below inputs and let me know if works well.Filter and aggregate data in Databricks to reduce load before it reaches Power BI. Use DirectQuery carefully, simplify measures, and reduce the number of vi...
I have a query using LCA. When referencing another table that has a column with the same name as the column used as LCA, the behavior of the query changes and it starts referencing the table column instead of the column that is already in the select ...
Hi @Kaniz_Fatma,we had the same problem as @paulocorrea.That's why it would be correct for to me to throw an error on ambiguous columns and the LCA could/must be addressed with a default identifier.Thanks
When trying to execute a query via sql warehouse, I get the following error:INVALID_PARAMETER_MARKER_VALUE.DUPLICATE_NAMEthe sql statement uses ? placeholders and the correct number of arguments are being passed.I am not able to use named placeholder...
Hi @RickB ,Which API are you using to invoke this? Parameter markers can be provided by:Python using its pyspark.sql.SparkSession.sql() API.Scala using its org.apache.spark.sql.SparkSession.sql() API.Java using its org.apache.spark.sql.SparkSession.s...
I'm planning to connect SQL Server Management Studio (SSMS) with Databricks using Lakehouse Federation. I understand that there are some differences in the SQL dialects between SSMS and Databricks SQL. For instance, in SSMS, we use TOP 10 to limit th...
To add on this:if you really have to use T-SQL (the MS dialect of SQL), you can define the SQL warehouse from databricks as a linked server on your SQL server.As said: SSMS is merely a sql client, the SQL dialect to be used is defined by the database...
Hello Community . I am a newbie here having an experience in tableau and power bi . I wanted to explore Dashboard creation in Lakeview . I have created a free trial databricks account . Although there are plenty of articles and videos on how to crea...
Hi @AZHAR-QUADRI ,You probably created workspace in a standard tier, that's why you can't see side bar. Recreate your workspace as a premium tier.
Did you know the default timeout setting for SQL #databricks Warehouse is two days?The default timeout can be too long for most use cases. You can easily change this for your session or in the general SQL warehouse configuration.
Hi @Hubert-Dudek we have a java service which uses JDBC Template for connecting to databricks warehouse. We recently set a high connection idle timeout and maxLifeTime for our hikari pool connection. We are now seeing error related to invalid session...
Does SQL Warehouse now have a feature that allows me to query all data without distinguishing between uppercase and lowercase?
Hi @Martin_Pham ,At the current moment SQL Warehouses don't have this option. But according to one of the employees at databricks, this feature is coming soon: https://community.databricks.com/t5/warehousing-analytics/sqlwarehouse-case-insensitive/td...
Application connects to Databricks serverless SQL warehouse via Databricks JDBC driver. It executes SQL select statements only. We see small number of statements failed each day with the following error detail: java.sql.SQLException: [Databricks][JDB...
Hi there,I'm building a Sink that uses the Databricks JDBC Driver version 2.6.27. It appears that regardless of using a try-with-resources block and explicitly closing the Connection, the driver's internals are not releasing IdleConnectionEvictor thr...
Still not fixed got 14k instances of com.databricks.client.jdbc42.internal.apache.http.impl.client.IdleConnectionEvictor 14,037 (0%) 673,776 B (0%) n/a And due to that application got to crashed. can you please help out as this is blocking to release...
Hi Team,There is a requirement to load a backup file into a database inside a SQL Warehouse. However, I don't see any option to directly load abackup file.I have tried reading the backup file using a notebook, but I’m unable to interpret the contents...
@szymon_dybczak could you please suggest some other options for how I can proceed to load data from the database?
Hi community,we have several databricks workspaces per different teams and in each workspace several use case covered.How can we trace the costs per individual use cost for SQL Warehosue / serverless in case we use one SQL warehouse per workspace to ...
I trying to connect databricks workspace to Redshift with security groups. Before I set up security groups it worked fine but now I can't connect to it because of inbound rules. What databricks IP's I should add to inbound rules to make it work?
Hi @ilya22239, To connect your Databricks workspace to Redshift, you need to whitelist Databricks cluster IP addresses in your Redshift security group. Start by identifying Databricks IPs, which are dynamic and may require a range or dynamic solution...
Is it possible to migrate Databricks Legacy Dashboards along with associated components such as queries and datasets from one workspace to another? I have attempted to export the Legacy Dashboards along with the queries and datasets, but upon importi...
Hello, to migrate between workspaces you'll need a different strategy based on the asset type Legacy dashboards you have completed, but for anyone reading you can use the 'export' functionalityQueries: you can either put them together in a folder and...
When trying incorporate an R package into my Spark workflow using the spark_apply() funciton in Sparklyr, I get the error:Error: java.lang.NoSuchMethodError: org.apache.spark.sql.catalyst.encoders.RowEncoder$.apply(Lorg/apache/spark/sql/types/StructT...
