- 34511 Views
- 15 replies
- 7 kudos
Resolved! What's the equivalent of "DECLARE..." in Databricks SQL ?
Hello everyone,I'm new in Databricks SQL, and I'm comming from SQL Server.I would like to know what's the equivalent of :DECLARE @P_Name varchar(50) = 'BackOffice'It's for use it like this : CREATE DATABASE @P_NameThanks.
- 34511 Views
- 15 replies
- 7 kudos
- 7 kudos
Hi @Salah K.​ you can go through this code block%python P_Name = 'BackOffice'spark.sql(f""" create database {P_name} """)
- 7 kudos
- 4558 Views
- 2 replies
- 7 kudos
what formatter is used to format SQL cell in databricks
Databricks launched formatter Black which formats python cells, I can also see SQL cell getting formatted, but not sure which formatter is being used for SQL cell formatting. No clarity given on docs.
- 4558 Views
- 2 replies
- 7 kudos
- 7 kudos
Hi @SANJAY qq​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 7 kudos
- 1474 Views
- 2 replies
- 2 kudos
I have been getting a False Alert on Data bricks SQL.
When ever a legitimate alert is triggered, I get a False Alert with 0.00 triggered at 12.00AM the next day. I tried Altering the Query but its still the same. Not posting examples as data is not shareable but I can give an example. if the alert is se...
- 1474 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @justin moorthy​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...
- 2 kudos
- 17694 Views
- 5 replies
- 8 kudos
Resolved! How to get all the tables name with a specific column or columns in a database?
let's say there is a database db in which 700 tables are there, and we need to find all the tables name in which column "project_id" is present.just an example for ubderstanding the questions.
- 17694 Views
- 5 replies
- 8 kudos
- 8 kudos
databaseName = "db" desiredColumn = "project_id" database = spark.sql(f"show tables in {databaseName} ").collect() tablenames = [] for row in database: cols = spark.table(row.tableName).columns if desiredColumn in cols: tablenames.append(row....
- 8 kudos
- 25092 Views
- 12 replies
- 10 kudos
Resolved! Fetching top 1 L rows in Databricks SQL
how to fetch the top 1L rows from a long SQL query as the query is returning 5gb data, I want to check only the first 1L ROWS.
- 25092 Views
- 12 replies
- 10 kudos
- 2523 Views
- 2 replies
- 5 kudos
Resolved! Databricks SQL External Connections
Lakehouse architectures seem enticing, especially from the standpoint of querying the data lake directly as it sits (as opposed to first migrating the data to an external data warehouse). While documentation and support seems pretty clear regarding ...
- 2523 Views
- 2 replies
- 5 kudos
- 1063 Views
- 1 replies
- 1 kudos
Weekly Release Notes RecapHere’s a quick recap of the latest release notes updates from the past one week. Databricks platform release notesFebruary 2...
Weekly Release Notes RecapHere’s a quick recap of the latest release notes updates from the past one week.Databricks platform release notesFebruary 21 - 28, 2023Ray on Databricks (Public Preview)With Databricks Runtime 12.0 and above, you can create ...
- 1063 Views
- 1 replies
- 1 kudos
- 1 kudos
Thank you for sharing!!!
- 1 kudos
- 17714 Views
- 14 replies
- 15 kudos
Resolved! How to convert records in Azure Databricks delta table to a nested JSON structure?
Let's say I have a delta table in Azure databricks that stores the staff details (denormalized). I wanted to export the data in the JSON format and save it as a single file on a storage location. I need help with the databricks sql query to group/co...
- 17714 Views
- 14 replies
- 15 kudos
- 7679 Views
- 7 replies
- 4 kudos
Resolved! Unsupported cloud file system schema 'wasbs' error in custom catalog, but not in hive_metastore
When trying to run a create table USING CSV that pulls data from an azure blob storage path in a custom catalog I created, I get an error stating Unsupported cloud file system schema 'wasbs'.However, when I run this code in the hive_metastore catalog...
- 7679 Views
- 7 replies
- 4 kudos
- 12349 Views
- 4 replies
- 3 kudos
Resolved! Databricks Audit Logs, What is "dataSourceId"?
Hi,I want to access the Databricks Audit Logs to check user activity.I created a Databricks workspace on the premium pricing tier.I configured Audit logs to be sent to Azure Diagnostic log delivery. What I got in the "Log Analytics Workspace": I hav...
- 12349 Views
- 4 replies
- 3 kudos
- 3 kudos
The data_source_id field specifies the id of the SQL warehouse against which this query will run. You can use the Data Sources API to see a complete list of available SQL warehouses.
- 3 kudos
- 3233 Views
- 2 replies
- 0 kudos
How to make backreferences in regexp_replace repl string work correctly in Databricks SQL?
Both of these work in Spark SQL:regexp_replace('1234567890abc', '^(?<one>\\w)(?<two>\\w)(?<three>\\w)', '$1') regexp_replace('1234567890abc', '^(?<one>\\w)(?<two>\\w)(?<three>\\w)', '${one}')However, neither work in Databricks SQL. I found that this ...
- 3233 Views
- 2 replies
- 0 kudos
- 0 kudos
Hello @Stephen Wilcoxon​ Could you please share the expected output in Spark SQL?
- 0 kudos
- 2857 Views
- 1 replies
- 0 kudos
Resolved! Pros and cons - running SQL query in databricks notebook and serverless warehouse sql editor
Can you please help technical pros and cons for running SQL query in databricks notebook (data engineering workspace) and serverless warehouse SQL editor
- 2857 Views
- 1 replies
- 0 kudos
- 0 kudos
NotebookPROS: More traditional cluster, git integration, choose DBR versionCONS: Cluster startup time, photon not automatically part of the clusterServerlessPROS: Faster, almost immediate startup time, less expensive for a single query, photon enable...
- 0 kudos
- 7621 Views
- 6 replies
- 11 kudos
Resolved! When to increase maximum bound vs when to increase cluster size?
Hello experts,For the below question, I am trying to understand why option C was selected instead of B? As B would also have resolved the issueQuestion 40A data analyst has noticed that their Databricks SQL queries are running too slowly. They claim ...
- 7621 Views
- 6 replies
- 11 kudos
- 11 kudos
On a sql server, there are wait states. Wait states occur when several processors (vCPUs) are processing and several threads are working through the processors. A longer running thread that has dependencies, can cause the thread that may have begun o...
- 11 kudos
- 18604 Views
- 5 replies
- 8 kudos
Resolved! Databricks to Oracle
How to write data from Databricks SQL to Oracle DB
- 18604 Views
- 5 replies
- 8 kudos
- 8 kudos
we can use JDBC driver to write dataframe to Oracle tables. Every database will use jdbc connect to connect & access database. You can follow same process for connecting to any database.Download Oracle ojdbc6.jar JDBC DriverYou need an Oracle jdbc dr...
- 8 kudos
- 3798 Views
- 3 replies
- 0 kudos
Query fails with 'Error occurred while deserializing arrow data' on Databricks SQL with Channel set to Preview
Noticed with a query based on inline select and joins fails to the client with 'Error occurred while deserializing arrow data' I.e the query succeeds on Databricks but client (DBeaver, AtScale) receives an errorThe error is only noticed with Databri...
- 3798 Views
- 3 replies
- 0 kudos
- 0 kudos
Opened an ES on this, looks like an issue with the Preview channel. Thanks for your help!
- 0 kudos