- 3834 Views
- 3 replies
- 1 kudos
Resolved! Is there a way to protect the secrets on databricks?
Hi all, I’ve a lot of secrets on databricks, however my users are able to see it when they make a simple loop over the secret.is there a way to prevent that?regards
- 3834 Views
- 3 replies
- 1 kudos
- 1 kudos
Do use the Databricks CLI or API 2.0 to manage secrets. Don't leave them in your notebooks for everyone to see, same applies for salting or hash strings.Secret management - Azure Databricks | Microsoft Docs
- 1 kudos
- 26864 Views
- 12 replies
- 11 kudos
Resolved! Is there any difference between performance of Python and SQL ?
I read somewhere that Python code is converted to SQL at the end. So is it true or there is any difference in performance while working with Scala, Python or SQL ?
- 26864 Views
- 12 replies
- 11 kudos
- 11 kudos
To add on the consideration of UDFs, try to consider using HOFs (Higher Order Functions) whenever possible first as there is a signifcant performance benefit as seen here.
- 11 kudos
- 1658 Views
- 0 replies
- 1 kudos
Structured streaming vs Confluent Kstream
For Ultra low latency customer facing App, I am curious on cost efficiency between Structured streaming and Kstream; which work better in terms of cost ? Though still achieving the ultra low latency and quality outcome. Appreciate any thoughts from p...
- 1658 Views
- 0 replies
- 1 kudos
- 1959 Views
- 1 replies
- 1 kudos
Resolved! How to improve the performance of small delta tables?
How to improve the performance of small delta tables?
- 1959 Views
- 1 replies
- 1 kudos
- 1 kudos
Use databricks optimize command on delta tables. It will regroup al the files and provides better performance​
- 1 kudos
- 4635 Views
- 1 replies
- 2 kudos
Resolved! Workflow dependencies
Is it possible to make one workflow job dependent on successful completion of another job?​
- 4635 Views
- 1 replies
- 2 kudos
- 2 kudos
I believe you can set workflow dependencies between other workflows.
- 2 kudos
- 2160 Views
- 0 replies
- 1 kudos
Project lightspeed
It’s just a breeze for all the streaming users. What’s the best venue to learn more about it. Is there a Jira ticket that tracks all the progresses? also wonder which Spark version it will come with.
- 2160 Views
- 0 replies
- 1 kudos
- 3232 Views
- 1 replies
- 3 kudos
Resolved! Does databricks run on GPU?
Does databricks run on GPU?
- 3232 Views
- 1 replies
- 3 kudos
- 3 kudos
There is support for running on GPU which will be beneficial to certain ML workloads.​Cluster​s are configured to run on CPU by default, but you can choose GPU based nodes during creation.
- 3 kudos
- 2194 Views
- 1 replies
- 2 kudos
Resolved! Project Lightspeed
Is Spark Structured Streaming now comparable with Flink on streaming workloads?
- 2194 Views
- 1 replies
- 2 kudos
- 2 kudos
Hard to say. Project Lightspeed ​is a work in progress and has not yet been released.
- 2 kudos
- 3839 Views
- 2 replies
- 1 kudos
Resolved! How to use multi character encodings in Delta Tables?
Can we mix different encodings in Delta tables? Downstream needs different character encodings like UTF-8 and Shift JIS.
- 3839 Views
- 2 replies
- 1 kudos
- 2111 Views
- 1 replies
- 0 kudos
Resolved! Delta Tables
I’m beginning my journey into Delta Tables and one thing that is still confusing me is where is the best place to save your delta tables if you need to query them later.For example I'm migrating several tables from on-prem to azure databricks into in...
- 2111 Views
- 1 replies
- 0 kudos
- 0 kudos
I usually recommend people to store data in a separate storage account (either mounted, or used directly), and don't use the internal storage of workspace for that tasks. Primary reason - it's easier to share this data with other workspaces, or other...
- 0 kudos
- 6322 Views
- 4 replies
- 2 kudos
How to change email on databricks community account?
I want to change the email associated with my databricks community edition account to a different email. How do I do that?
- 6322 Views
- 4 replies
- 2 kudos
- 2 kudos
You can add email in the account console and add them as an admin. One other solution is to close the current account and start a new community edition account with a different email.
- 2 kudos
- 7282 Views
- 3 replies
- 0 kudos
Resolved! Can python futures utilise all cluster nodes ?
I used python futures to call a function multiple times concurrently, however I am not sure if all nodes is utilised or how to make sure it use all cluster nodes.Can you confirm if I create a cluster with 5 works each with 8 memory cores for example....
- 7282 Views
- 3 replies
- 0 kudos
- 0 kudos
You can create a init script and then add it during cluster start up ​
- 0 kudos
- 2982 Views
- 1 replies
- 0 kudos
Resolved! How to do deterministic encryption on databricks?
How can I do deterministic encryption on databricks? And protect PI columns.
- 2982 Views
- 1 replies
- 0 kudos
- 0 kudos
In order to deterministic encryption, we need to use aes encryption. Using AES the encrypted text will always remain the same for same input. Databricks recently implemented aes_encrypt and aes_decrypt functions and is the recommended way to perform ...
- 0 kudos
- 1113 Views
- 0 replies
- 0 kudos
Has anyone done any performance tests for Serverless and cluster SQL endpoints?
Has anyone done any performance tests for Serverless and cluster SQL endpoints?
- 1113 Views
- 0 replies
- 0 kudos
- 1645 Views
- 1 replies
- 1 kudos
Resolved! Where can I learn more about Live tables
I keep hearing about Live tables. Where can I read more about it?​
- 1645 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello,​These are some good places to start!​Getting started docs: https://databricks.com/discover/pages/getting-started-with-delta-live-tables​​Notebook examples to walk through: https://github.com/databricks/delta-live-tables-notebooks
- 1 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 586 | |
| 349 | |
| 287 |
