- 953 Views
- 1 replies
- 1 kudos
Resolved! Where can I learn more about Live tables
I keep hearing about Live tables. Where can I read more about it?​
- 953 Views
- 1 replies
- 1 kudos
- 1 kudos
Hello,​These are some good places to start!​Getting started docs: https://databricks.com/discover/pages/getting-started-with-delta-live-tables​​Notebook examples to walk through: https://github.com/databricks/delta-live-tables-notebooks
- 1 kudos
- 3092 Views
- 2 replies
- 5 kudos
Resolved! Databricks Job is slower.
Hello, I have a data bricks question. A Dataframe job that writes in an s3 bucket usually takes 8 minutes to finish, but now it takes from 8 to 9 hours to complete. Does anybody have some clues about this behavior?the data frame size is about 300 or ...
- 3092 Views
- 2 replies
- 5 kudos
- 3825 Views
- 1 replies
- 8 kudos
- 3825 Views
- 1 replies
- 8 kudos
- 8 kudos
Yes, it is supporting window functions https://docs.databricks.com/sql/language-manual/sql-ref-window-functions.html; one of them is row_number()
- 8 kudos
- 868 Views
- 1 replies
- 2 kudos
Excited about delta 2.0#​
Excited about delta 2.0#​
- 868 Views
- 1 replies
- 2 kudos
- 2 kudos
Since 2.0, everything related to the delta has been open-source. Excellent!
- 2 kudos
- 1724 Views
- 1 replies
- 2 kudos
Resolved! Autoloader checkpoint issue
Hello guys, anyone issuing problems with autoloader checkpoints on azure?​
- 1724 Views
- 1 replies
- 2 kudos
- 2 kudos
Me not, but it is good to specify both the autoloader checkpoint and streaming (for write) checkpoint.And it happened to me during some experiments that I had to delete everything from the autoloader directory.
- 2 kudos
- 1225 Views
- 2 replies
- 2 kudos
Brilliant idea of the concept.! I want to learn much more about DataBricks!!
Brilliant idea of the concept.! I want to learn much more about DataBricks!!
- 1225 Views
- 2 replies
- 2 kudos
- 1353 Views
- 2 replies
- 4 kudos
Become a Community Champion! As you know, Data + AI Summit 2022 is June 27-30. During this time we want to see as many of our virtual and in-person at...
Become a Community Champion! As you know, Data + AI Summit 2022 is June 27-30. During this time we want to see as many of our virtual and in-person attendees becoming the best communtiy version of themselves: a Community Champion. Because the qualifi...
- 1353 Views
- 2 replies
- 4 kudos
- 12650 Views
- 9 replies
- 2 kudos
What is the best way to read in a ms access .accdb database into Databricks from a mounted drive?
I am currently trying to read in .accdb files from a mounted drive. Based on my research it looks like I would have to use a package like JayDeBeApi with ucanaccess drivers or pyodbc with ms access drivers.Will this work?Thanks for any help.
- 12650 Views
- 9 replies
- 2 kudos
- 2 kudos
Hi @Leland Adams​ Hope you are doing well. Thank you for posting your question and giving us additional information. Do you think you were able to solve the query?We'd love to hear from you.
- 2 kudos
- 7321 Views
- 2 replies
- 2 kudos
Resolved! How spark will handles 1TB data if it has cluster of 1GB memory ?
If my cluster memory is 1GB for example and my data is 1TB how Spark will handle it?If it is in memory computing how does it handles the data that is greater than the memory size ?
- 7321 Views
- 2 replies
- 2 kudos
- 1872 Views
- 1 replies
- 0 kudos
Disable managed tables on Azure Databricks
When a user creates a table without a path, it writes it as a managed table in the root bucket. Can this functionality be disabled so users are forced to provide a storage path and follow our organization best practices?
- 1872 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi,I couldn't find the option to turn it off. It might be worth banning dbfs. However, you can configure the default location using a cluster policy by adding the following configuration:"spark_conf.spark.sql.warehouse.dir": { "type": "fixed", "val...
- 0 kudos
- 2862 Views
- 1 replies
- 5 kudos
Resolved! Pyspark environment and python packages on executors
At my company we use conda-pack to make certain packages available on the spark executors. Is there a better a better alternative to get away from creating a new environment and pack it every time I need a new python lib to be available for the execu...
- 2862 Views
- 1 replies
- 5 kudos
- 5 kudos
databricks provides library installation in the form of PyPi packages, or wheel/egg.If you install the packages like that on the cluster, they are automatically sent to all executors.
- 5 kudos
- 964 Views
- 0 replies
- 2 kudos
- 964 Views
- 0 replies
- 2 kudos
- 971 Views
- 0 replies
- 1 kudos
Can I use Spark to stream data from Kafka?
Can I use Spark to stream data from Kafka?
- 971 Views
- 0 replies
- 1 kudos
- 1266 Views
- 0 replies
- 1 kudos
Primer vez en este evento, la verdad muy buen evento, aunque este foro no tiene contenido en español, la comunidad hispano parlante está creciendo y e...
Primer vez en este evento, la verdad muy buen evento, aunque este foro no tiene contenido en español, la comunidad hispano parlante está creciendo y esperamos poder aportar en el desarrollo de nuestros paises a traves de la tecnologia!.First time on ...
- 1266 Views
- 0 replies
- 1 kudos
- 942 Views
- 1 replies
- 2 kudos
The Data + AI summit is a blast so far. There are so many new technologies being released such as Delta Lake ​2.0 being open source.
The Data + AI summit is a blast so far. There are so many new technologies being released such as Delta Lake ​2.0 being open source.
- 942 Views
- 1 replies
- 2 kudos
- 2 kudos
Agreed! You should check out the Azure booth if you haven't already they have a really cool demo.​
- 2 kudos
| User | Count |
|---|---|
| 1614 | |
| 773 | |
| 349 | |
| 286 | |
| 253 |
