- 1389 Views
- 1 replies
- 0 kudos
Resolved! Delta Tables
I’m beginning my journey into Delta Tables and one thing that is still confusing me is where is the best place to save your delta tables if you need to query them later.For example I'm migrating several tables from on-prem to azure databricks into in...
- 1389 Views
- 1 replies
- 0 kudos
- 0 kudos
I usually recommend people to store data in a separate storage account (either mounted, or used directly), and don't use the internal storage of workspace for that tasks. Primary reason - it's easier to share this data with other workspaces, or other...
- 0 kudos
- 4414 Views
- 4 replies
- 2 kudos
How to change email on databricks community account?
I want to change the email associated with my databricks community edition account to a different email. How do I do that?
- 4414 Views
- 4 replies
- 2 kudos
- 2 kudos
You can add email in the account console and add them as an admin. One other solution is to close the current account and start a new community edition account with a different email.
- 2 kudos
- 6045 Views
- 3 replies
- 0 kudos
Resolved! Can python futures utilise all cluster nodes ?
I used python futures to call a function multiple times concurrently, however I am not sure if all nodes is utilised or how to make sure it use all cluster nodes.Can you confirm if I create a cluster with 5 works each with 8 memory cores for example....
- 6045 Views
- 3 replies
- 0 kudos
- 0 kudos
You can create a init script and then add it during cluster start up ​
- 0 kudos
- 1970 Views
- 1 replies
- 0 kudos
Resolved! How to do deterministic encryption on databricks?
How can I do deterministic encryption on databricks? And protect PI columns.
- 1970 Views
- 1 replies
- 0 kudos
- 0 kudos
In order to deterministic encryption, we need to use aes encryption. Using AES the encrypted text will always remain the same for same input. Databricks recently implemented aes_encrypt and aes_decrypt functions and is the recommended way to perform ...
- 0 kudos
- 817 Views
- 0 replies
- 0 kudos
Has anyone done any performance tests for Serverless and cluster SQL endpoints?
Has anyone done any performance tests for Serverless and cluster SQL endpoints?
- 817 Views
- 0 replies
- 0 kudos
- 986 Views
- 1 replies
- 1 kudos
Resolved! Where can I learn more about Live tables
I keep hearing about Live tables. Where can I read more about it?​
- 986 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello,​These are some good places to start!​Getting started docs: https://databricks.com/discover/pages/getting-started-with-delta-live-tables​​Notebook examples to walk through: https://github.com/databricks/delta-live-tables-notebooks
- 1 kudos
- 3188 Views
- 2 replies
- 5 kudos
Resolved! Databricks Job is slower.
Hello, I have a data bricks question. A Dataframe job that writes in an s3 bucket usually takes 8 minutes to finish, but now it takes from 8 to 9 hours to complete. Does anybody have some clues about this behavior?the data frame size is about 300 or ...
- 3188 Views
- 2 replies
- 5 kudos
- 3973 Views
- 1 replies
- 8 kudos
- 3973 Views
- 1 replies
- 8 kudos
- 8 kudos
Yes, it is supporting window functions https://docs.databricks.com/sql/language-manual/sql-ref-window-functions.html; one of them is row_number()
- 8 kudos
- 903 Views
- 1 replies
- 2 kudos
Excited about delta 2.0#​
Excited about delta 2.0#​
- 903 Views
- 1 replies
- 2 kudos
- 2 kudos
Since 2.0, everything related to the delta has been open-source. Excellent!
- 2 kudos
- 1784 Views
- 1 replies
- 2 kudos
Resolved! Autoloader checkpoint issue
Hello guys, anyone issuing problems with autoloader checkpoints on azure?​
- 1784 Views
- 1 replies
- 2 kudos
- 2 kudos
Me not, but it is good to specify both the autoloader checkpoint and streaming (for write) checkpoint.And it happened to me during some experiments that I had to delete everything from the autoloader directory.
- 2 kudos
- 1278 Views
- 2 replies
- 2 kudos
Brilliant idea of the concept.! I want to learn much more about DataBricks!!
Brilliant idea of the concept.! I want to learn much more about DataBricks!!
- 1278 Views
- 2 replies
- 2 kudos
- 1400 Views
- 2 replies
- 4 kudos
Become a Community Champion! As you know, Data + AI Summit 2022 is June 27-30. During this time we want to see as many of our virtual and in-person at...
Become a Community Champion! As you know, Data + AI Summit 2022 is June 27-30. During this time we want to see as many of our virtual and in-person attendees becoming the best communtiy version of themselves: a Community Champion. Because the qualifi...
- 1400 Views
- 2 replies
- 4 kudos
- 13023 Views
- 9 replies
- 2 kudos
What is the best way to read in a ms access .accdb database into Databricks from a mounted drive?
I am currently trying to read in .accdb files from a mounted drive. Based on my research it looks like I would have to use a package like JayDeBeApi with ucanaccess drivers or pyodbc with ms access drivers.Will this work?Thanks for any help.
- 13023 Views
- 9 replies
- 2 kudos
- 2 kudos
Hi @Leland Adams​ Hope you are doing well. Thank you for posting your question and giving us additional information. Do you think you were able to solve the query?We'd love to hear from you.
- 2 kudos
- 7596 Views
- 2 replies
- 2 kudos
Resolved! How spark will handles 1TB data if it has cluster of 1GB memory ?
If my cluster memory is 1GB for example and my data is 1TB how Spark will handle it?If it is in memory computing how does it handles the data that is greater than the memory size ?
- 7596 Views
- 2 replies
- 2 kudos
- 1959 Views
- 1 replies
- 0 kudos
Disable managed tables on Azure Databricks
When a user creates a table without a path, it writes it as a managed table in the root bucket. Can this functionality be disabled so users are forced to provide a storage path and follow our organization best practices?
- 1959 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi,I couldn't find the option to turn it off. It might be worth banning dbfs. However, you can configure the default location using a cluster policy by adding the following configuration:"spark_conf.spark.sql.warehouse.dir": { "type": "fixed", "val...
- 0 kudos
| User | Count |
|---|---|
| 1615 | |
| 779 | |
| 349 | |
| 287 | |
| 267 |
