cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
noimeta
by • Contributor III
  • 4867 Views
  • 9 replies
  • 4 kudos

Resolved! Databricks SQL: catalog of each query

Currently, we are migrating from hive metastore to UC. We have several dashboards and a huge number of queries whose catalogs have been set to hive_metastore and using <db>.<table> access pattern.I'm just wondering if there's a way to switch catalogs...

Data Engineering
Reply
  • 4867 Views
  • 9 replies
  • 4 kudos
View Replies
Latest Reply
h_h_ak
Contributor
  • 4 kudos

May you can also have a look here, if you need hot fix  https://github.com/databrickslabs/ucx 

  • 4 kudos
8 More Replies
Vijesh
by • New Contributor II
  • 3754 Views
  • 5 replies
  • 1 kudos

parsing error in Databricks SQL endpoint

I have two tables EMPLOYEE & EMPLOYEE_ROLE. I'm trying to Update a column with a value from another column. I'm using SQL server join but i get an error - [parse_syntax_error] Syntax error at or near 'FROM' line 3. UPDATE CSET C.title = B.title FROM ...

Data Engineering
Reply
  • 3754 Views
  • 5 replies
  • 1 kudos
View Replies
Latest Reply
sensanjoy
Contributor
  • 1 kudos

Hi @Vijesh V​ Try to use merge into to perform cdc between tables :MERGE INTO target aUSING source bON {merge_condition}WHEN MATCHED THEN {matched_action}WHEN NOT MATCHED THEN {not_matched_action}

  • 1 kudos
4 More Replies
cgrant
by • Databricks Employee
  • 3591 Views
  • 4 replies
  • 6 kudos

How do I know how much of a query/job used Photon?

I'm trying to use the native execution engine, Photon. How can I tell if a query is using Photon or is falling back to the non-native Spark engine?

Data Engineering
Reply
  • 3591 Views
  • 4 replies
  • 6 kudos
View Replies
Latest Reply
venkat09
New Contributor III
  • 6 kudos

Typo error in my second point of the previous post. Click the execution plan of your task[this is available under SQL/Dataframe tab in Spark UI]. It explains what operations run in the photon engine and what didn't execute by photon.

  • 6 kudos
3 More Replies
bernardocouto
by • New Contributor II
  • 1428 Views
  • 1 replies
  • 4 kudos

Resolved! Databricks SQL Connector Abstraction for Python

Databricks SQL framework, easy to learn, fast to code, ready for production.I built an abstraction of the databricks-sql-connector in order to follow a pattern closer to the concepts of ORM tools, in addition to facilitating the adoption of the data ...

Data Engineering
Reply
  • 1428 Views
  • 1 replies
  • 4 kudos
View Replies
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 4 kudos

Sure, will try and provide feedback for same

  • 4 kudos
Yuliya
by • New Contributor II
  • 1811 Views
  • 2 replies
  • 3 kudos

Azure Databricks SQL Warehouse connection issue

When trying to start SQL Warehouse from my Azure pay-as-you-go subscription, I'm getting error about not enough vCPUs provisioned. Documentation says to increase quota at Azure portal - but it requires knowing type of vCPUs to provision. What type of...

Data Engineering
Reply
  • 1811 Views
  • 2 replies
  • 3 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 3 kudos

Hi @Yuliya Quintela​,Just a friendly follow-up. Did Rostislaw's response help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 3 kudos
1 More Replies
Chandana
by • New Contributor II
  • 918 Views
  • 1 replies
  • 3 kudos

What’s the USP? DB SQL serverless? When is it coming to Azure?

What’s the USP? DB SQL serverless? When is it coming to Azure?

Data Engineering
Reply
  • 918 Views
  • 1 replies
  • 3 kudos
View Replies
Latest Reply
User16752242622
Valued Contributor
  • 3 kudos

Hi @Chandana Basani​ DB SQL Serverless in Azure is planned for GA on the Release quarter FY23 Q1 in the release month 2023-01Best,Akash

  • 3 kudos
StephanieAlba
by • Databricks Employee
  • 4498 Views
  • 2 replies
  • 5 kudos

Resolved! How to add a select all option in a Databricks SQL parameter? I would like to use a query-based drop-down list.

So I want to create a select all button in a parameter. The actual parameter has around 200 options because of the size of the database. However, if I want a general summary where you can see all the options I would have to select one by one and that...

Data Engineering
Reply
  • 4498 Views
  • 2 replies
  • 5 kudos
View Replies
Latest Reply
StephanieAlba
Databricks Employee
  • 5 kudos

You could add '--- All Stores ---' to your list. Here is the query I would use to populate the drop-down. S.O. answer hereSELECT store as store_name FROM ( Select Distinct store From Table   UNION ALL   SELECT ...

  • 5 kudos
1 More Replies
KrishnaWahi
by • New Contributor II
  • 2909 Views
  • 3 replies
  • 2 kudos

Resolved! Query Databricks SQL Endpoint Database from NodeJs

I have my Databricks SQL Endpoint running, I have created many tables there using AWS S3 Delta. Now I want to query the Databricks SQL Endpoint from NodeJs. So It is possible ?I tried to find and researched a lot but didn't get any useful tutorial fo...

Data Engineering
Reply
  • 2909 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
BilalAslamDbrx
Databricks Employee
  • 2 kudos

We are very interested in supporting two capabilities -- in fact these are being worked on already:The ability to issue a SQL query over REST and track its status, retrieve results etc.A NodeJS SDK for Databricks SQLKeep an eye on our release notes

  • 2 kudos
2 More Replies
User16826992666
by • Valued Contributor
  • 2766 Views
  • 2 replies
  • 1 kudos

Resolved! As an admin of a Databricks SQL environment, can I cancel long running queries?

I don't want one long or poorly written query to block my entire SQL endpoint for everyone else. Do I have the ability to kill specific queries?

Data Engineering
Reply
  • 2766 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
DevB
New Contributor II
  • 1 kudos

Is there a way to stop the session programmatically? like "kill session_id" or something similar in API?

  • 1 kudos
1 More Replies
User16826994223
by • Honored Contributor III
  • 679 Views
  • 0 replies
  • 0 kudos

Cluster Sizees on DB sql Cluster size Driver size Worker count 2X-Small i3.2xlarge 1 X-Small i3.2xlarge 2 Small i3.4xlarge 4 Med...

Cluster Sizees on DB sql Cluster size Driver size Worker count 2X-Small i3.2xlarge 1 X-Small i3.2xlarge 2 Small i3.4xlarge 4 Medium i3.8xlarge 8 Large i3.8xlarge 16 X-Large i3.16xlarge 32 2X-Large i3.16xlarge...

Data Engineering
Reply
  • 679 Views
  • 0 replies
  • 0 kudos
Labels