cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
joao_vnb
by • New Contributor III
  • 49320 Views
  • 7 replies
  • 11 kudos

Resolved! Automate the Databricks workflow deployment

Hi everyone,Do you guys know if it's possible to automate the Databricks workflow deployment through azure devops (like what we do with the deployment of notebooks)?

Data Engineering
Reply
  • 49320 Views
  • 7 replies
  • 11 kudos
View Replies
Latest Reply
asingamaneni
New Contributor II
  • 11 kudos

Did you get a chance to try Brickflows - https://github.com/Nike-Inc/brickflowYou can find the documentation here - https://engineering.nike.com/brickflow/v0.11.2/Brickflow uses - Databricks Asset Bundles(DAB) under the hood but provides a Pythonic w...

  • 11 kudos
6 More Replies
yatharth29
by • New Contributor II
  • 2397 Views
  • 3 replies
  • 1 kudos

Resolved! Is it possible to use SAS (Statistical Analysis System) tool within Databricks?

Can we link or directly use SAS within Databricks environment. If yes how can we achieve this?

Data Engineering
Reply
  • 2397 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Baskar
New Contributor III
  • 1 kudos

No it has to be rewritten in spark

  • 1 kudos
2 More Replies
Moriondo
by • New Contributor III
  • 2429 Views
  • 3 replies
  • 2 kudos

Resolved! How to filter a dashboard by the current user email?

Hello,I would like to know if it is possible to filter a dashboard by the current user email?For example, I have a table result of a group of people with the following columns: user_id, user_email, date, productivity. So with this table I create som...

Data Engineering
Reply
  • 2429 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
Moriondo
New Contributor III
  • 2 kudos

Hey guys, After some research on the documentation, I found out that if a filter the query using the current_user() function, I will get the result that I was looking for.If anyone need look at this:https://docs.databricks.com/sql/language-manual/fun...

  • 2 kudos
2 More Replies
testname1
by • New Contributor II
  • 2010 Views
  • 1 replies
  • 1 kudos

Is it possible to use the databricks-sql-nodejs driver in a create-react-app app?

I'm using the typescript example for the databricks sql driver but I'm getting errors when compiling:

image.png
Data Engineering
Reply
  • 2010 Views
  • 1 replies
  • 1 kudos
View Replies
Latest Reply
User16502773013
Databricks Employee
  • 1 kudos

Hello @asdf fdsa​ ,The NodeJS connector is built for NodeJS environment it will not integrate ReactJSFor cases where a web execution is needed we advise to use SQL Exec APIPlease check documentation here for the same:https://docs.databricks.com/sql/a...

  • 1 kudos
gfar
by • New Contributor II
  • 12013 Views
  • 2 replies
  • 0 kudos

Is it possible to connect QGIS to Databricks using ODBC?

I can connect ArcGIS to Databricks using ODBC, but using the same ODBC DSN for QGIS I get an error - Unable to initialize ODBC connection to DSNHas anyone got this working?

Data Engineering
Reply
  • 12013 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Anonymous
Not applicable
  • 0 kudos

@Grainne Farrant​ :It is possible to connect QGIS to Databricks using ODBC, but it requires additional configuration. Here are the general steps to follow:Install the ODBC driver for Databricks on your machine where QGIS is installed. You can downloa...

  • 0 kudos
1 More Replies
same213
by • New Contributor III
  • 4771 Views
  • 4 replies
  • 8 kudos

Is it possible to create a sqlite database and export it?

I am trying to create a sqlite database in databricks and add a few tables to it. Ultimately, I want to export this using Azure. Is this possible?

Data Engineering
Reply
  • 4771 Views
  • 4 replies
  • 8 kudos
View Replies
Latest Reply
same213
New Contributor III
  • 8 kudos

@Hubert Dudek​  We currently have a process in place that reads in a SQLite file. We recently transitioned to using Databricks. We were hoping to be able to create a SQLite file so we didn't have to alter the current process we have in place.

  • 8 kudos
3 More Replies
Herkimer
by • New Contributor II
  • 1817 Views
  • 3 replies
  • 0 kudos

Is it possible to install databricks-cli on a shared laptop.

I have a government furnished laptop (GFE). My normal user is not an admin on the laptop. I have a separate admin login on the laptop. I was able to install databricks-cli as the admin user but it installed under that users appdata python path which ...

Data Engineering
Reply
  • 1817 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Vidula
Honored Contributor
  • 0 kudos

Hey there @John Zajic​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you....

  • 0 kudos
2 More Replies
wyzer
by • Contributor II
  • 7943 Views
  • 2 replies
  • 2 kudos

Why database/table names are in lower case ?

Hello,When I run this code :CREATE DATABASE BackOfficeI see the database like this :backofficeWhy everything is in lower case ?Is it possible to configure Databricks in order to keep the real name ?Thanks.

Data Engineering
Reply
  • 7943 Views
  • 2 replies
  • 2 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 2 kudos

It is managed by hive metastore as you can put it in different databases is saver this way as some database are Case Sensitive and some not (you can easily test it with standard WHERE syntax).Probably you could change it with some hive settings but i...

  • 2 kudos
1 More Replies
User16826992666
by • Valued Contributor
  • 1988 Views
  • 3 replies
  • 0 kudos

Is it possible to enable encryption in between worker nodes?

I have a security requirement to encrypt all data when it is in transit. I am wondering if there is a setting I can use to enable encryption of the data during shuffles between the worker nodes.

Data Engineering
Reply
  • 1988 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
amr
Databricks Employee
  • 0 kudos

Inter-node encryption is a requirement for HIPPA compliance, reach out to your account management team and ask them for HIPPA compliant shards.

  • 0 kudos
2 More Replies
Anonymous
by • Not applicable
  • 1603 Views
  • 1 replies
  • 0 kudos

Resolved! Is it possible to write single Spark stream to 2 different Delta tables? Recommendations around that?

Data Engineering
Reply
  • 1603 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
Ryan_Chynoweth
Esteemed Contributor
  • 0 kudos

In this scenario, the best option would be to have a single readStream reading a source delta table. Since checkpoint logs are controlled when writing to delta tables you would be able to maintain separate logs for each of your writeStreams. I would...

  • 0 kudos
Labels