cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

robert37201
by New Contributor II
  • 3030 Views
  • 3 replies
  • 4 kudos

Job aborted due to stage failure: Input buffer size 0 for bloom filter is not power of 2

Query works great in a notebook, fails in Classic SQL Warehouse (photon enabled) with that error. Tables are relatively small. Just don't know where to begin understanding that error, google wasn't much help and Query History doesn't give me anything...

  • 3030 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Robert McCartney​ We haven't heard from you since the last response from @Lakshay Goel​ â€‹, and I was checking back to see if his suggestions helped you.Or else, If you have any solution, please share it with the community, as it can be helpful to...

  • 4 kudos
2 More Replies
js54123875
by New Contributor III
  • 3397 Views
  • 2 replies
  • 2 kudos

Failure to initialize configuration' on SQL Warehouse Tables

Yesterday I had a basic DLT pipeline up and running, and was able to query the hive_metastore tables successfully. The pipeline uses autloader to ingest a few csv files from cloud storage to streaming live bronze and silver tables. Today after star...

image of error image
  • 3397 Views
  • 2 replies
  • 2 kudos
Latest Reply
Anonymous
Not applicable
  • 2 kudos

Hi @Jennette Shepard​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feed...

  • 2 kudos
1 More Replies
Sujitha
by Databricks Employee
  • 7946 Views
  • 0 replies
  • 2 kudos

Weekly Release Notes RecapHere’s a quick recap of the latest release notes updates from the past one week. Databricks platform release notesMarch 13 -...

Weekly Release Notes RecapHere’s a quick recap of the latest release notes updates from the past one week.Databricks platform release notesMarch 13 - 17, 2023Execute SQL cells in the notebook in parallelYou can now run SQL cells in Databricks noteboo...

  • 7946 Views
  • 0 replies
  • 2 kudos
vinaykumar
by New Contributor III
  • 4044 Views
  • 1 replies
  • 0 kudos

Resolved! Pros and cons - running SQL query in databricks notebook and serverless warehouse sql editor

Can you please help technical pros and cons for running SQL query in databricks notebook (data engineering workspace) and serverless warehouse SQL editor

  • 4044 Views
  • 1 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

NotebookPROS: More traditional cluster, git integration, choose DBR versionCONS: Cluster startup time, photon not automatically part of the clusterServerlessPROS: Faster, almost immediate startup time, less expensive for a single query, photon enable...

  • 0 kudos
Cano
by New Contributor III
  • 5405 Views
  • 4 replies
  • 2 kudos

SQL warehouse failing to start ( Please check network connectivity from the data plane to the control plane )

Hi, My SQL warehouse is failing to start with the following error message:Details for the latest failure: Error: [id: InstanceId(i-01b84b6705ff09104), status: INSTANCE_INITIALIZING, workerEnvId:WorkerEnvId(workerenv-3023557811934763-c8cef827-a038-455...

  • 5405 Views
  • 4 replies
  • 2 kudos
Latest Reply
Debayan
Databricks Employee
  • 2 kudos

Hi, There is a line in the attached logs as below:[Bootstrap Event] Can reach ohio.cloud.databricks.com: [FAILED][Bootstrap Event] DNS output for databricks-prod-artifacts-us-east-2.s3.us-east-2.amazonaws.com: Server: 10.187.0.2Address: 10.187.0.2#5...

  • 2 kudos
3 More Replies
Taha_Hussain
by Databricks Employee
  • 9419 Views
  • 5 replies
  • 5 kudos

Connect a BI Tool: How do I access my lakehouse data from my BI tool?

You can find a rich ecosystem of tools that allow you to work with all your data in-place and deliver real-time business insights faster.This post will help you connect your existing tools like dbt, Fivetran, PowerBI, Tableau or SAP to ingest, transf...

Screen Shot 2022-08-09 at 11.54.56 PM
  • 9419 Views
  • 5 replies
  • 5 kudos
Latest Reply
Axserv
New Contributor II
  • 5 kudos

Hello Taha, here is a fairly recent video provided by Databricks on conncecting Power BI : Demo Video: Connect to Power BI Desktop from Databricks - YouTube

  • 5 kudos
4 More Replies
abizid
by New Contributor
  • 1127 Views
  • 0 replies
  • 0 kudos

.Net thrift client for sql warehouse

I'm trying to port python-sql thrift client to .net and I receive a 500 error when trying to open a session.Is there a way to have an sql warehouse server mock in order to investigate the error.

  • 1127 Views
  • 0 replies
  • 0 kudos
mickniz
by Contributor
  • 6619 Views
  • 3 replies
  • 6 kudos

Unable to create materialized view in Unity enabled Catalogues.

Hi Team,I was going through one of the videos of Databricks Sql Serverless and it say there is materialized view support . We can create materialized view .I tried same on my cluster of Sql Warehouse gives below error: 

image
  • 6619 Views
  • 3 replies
  • 6 kudos
Latest Reply
Felix
Databricks Employee
  • 6 kudos

Materialized views is in private preview right now afaik. Please talk to your account or customer success team at Databricks in order to sign up and enable it for your workspace. Thanks!

  • 6 kudos
2 More Replies
dimandfacts
by New Contributor III
  • 2920 Views
  • 2 replies
  • 6 kudos

Community Edition SQL Warehouse is not starting up, is it not free to even trail ?

When i start the sql warehosue , i get this error. Is there a way around to start up, I just want to try some features. Clusters are failing to launch. Cluster launch will be retried.Details for the latest failure: Error: Error code: PublicIPCountLim...

  • 2920 Views
  • 2 replies
  • 6 kudos
Latest Reply
Anonymous
Not applicable
  • 6 kudos

Hi @Anbarasan Dhanushkodi​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from ...

  • 6 kudos
1 More Replies
db-avengers2rul
by Contributor II
  • 4439 Views
  • 3 replies
  • 1 kudos

Resolved! Unable to create SQL Warehouse using AZURE DATABRICKS Subscription

Dear Team,I am unable to create a SQL Warehouse using AZURE Databricks subscription , below are the detailsi am able to create a cluster with single node ,but i am not able to create SQL Warehouse , i am using Cluster size - 2X Small , as part of the...

response from microsoft support
  • 4439 Views
  • 3 replies
  • 1 kudos
Latest Reply
Hubert-Dudek
Databricks MVP
  • 1 kudos

In portal.azure.com, write quotas and ask for an increase. https://portal.azure.com/#view/Microsoft_Azure_Capacity/QuotaMenuBlade/~/myQuotasAdditionally, in SQL warehouse "Advanced options," you can change "Spot instance policy" from "cost-optimized"...

  • 1 kudos
2 More Replies
bozhu
by Contributor
  • 8135 Views
  • 4 replies
  • 3 kudos

Resolved! SQL Warehouse Serverless Endpoint Error

Our SQL Warehouse Serverless Endpoint started failing from this morning (2022-08-23 18:00:00 UTC):“org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Unable to build AWSGlueClient: com.amazonaws.SdkClientException: Unable to find...

  • 8135 Views
  • 4 replies
  • 3 kudos
Latest Reply
Vidula
Honored Contributor
  • 3 kudos

Hi @Bo Zhu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 3 kudos
3 More Replies
lizou
by Contributor III
  • 3807 Views
  • 4 replies
  • 2 kudos

call saved query in sql warehouse

in python cursor.executecan you call a saved query with a parameter? like call a stored procedure in relational db?https://docs.microsoft.com/en-us/azure/databricks/dev-tools/python-sql-connector#cursor-method

  • 3807 Views
  • 4 replies
  • 2 kudos
Latest Reply
Vidula
Honored Contributor
  • 2 kudos

Hi @lizou​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 2 kudos
3 More Replies
Cano
by New Contributor III
  • 1198 Views
  • 1 replies
  • 0 kudos

Hi,I'll like to know if it's possible to connect to Postgresql RDS from the Databricks SQL Warehouse.

Hi,I'll like to know if it's possible to connect to Postgresql RDS from the Databricks SQL Warehouse.

  • 1198 Views
  • 1 replies
  • 0 kudos
Latest Reply
Cano
New Contributor III
  • 0 kudos

I should have posted this as a question and not a post. Please forgive me, I'm a newbie.

  • 0 kudos
Labels