- 431 Views
- 0 replies
- 1 kudos
State of stateful streaming
For stateful streaming in #databricks, you can now easily read what is in the state.
- 431 Views
- 0 replies
- 1 kudos
Forum Posts
- 790 Views
- 4 replies
- 0 kudos
Unable to mount GCS bucket with underscores in the name
I have two buckets with the same configurations and labels.One is named my-bucket and the other is my_bucket. I am able to mount my-bucket but get an opaque error message when trying to mount my_bucket. Is this known/expected behavior? Are underscore...
- 790 Views
- 4 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @legobricks , Curious on the error that you are getting. However, for GCS - https://cloud.google.com/storage/docs/buckets#naming I do see underscores are allowed but there is also a note below: You can use a bucket name in a DNS record as part of ...
- 0 kudos
- 1583 Views
- 2 replies
- 1 kudos
Debug stream
One of the easiest ways to debug a stream is to stream it to memory and query it with SQL #databricks
- 1583 Views
- 2 replies
- 1 kudos
- 579 Views
- 0 replies
- 0 kudos
Financial Crime detection with the help of Apache Spark, Data Mesh and Data Lake
For those interested in Data Mesh and Data Lakes for FinCrime detection:Data mesh is a relatively new architectural concept for data management that emphasizes domain-driven data ownership and self-service data availability. It promotes the decentral...
- 579 Views
- 0 replies
- 0 kudos
- 1184 Views
- 1 replies
- 2 kudos
Databricks Notebook Workflow
Exciting news for Databricks users! The ability to view job details within the notebook workflow section, particularly for multithreaded jobs, is available now. Instead of manually inspecting each job for failures, this feature enables us to swiftly ...
- 1184 Views
- 1 replies
- 2 kudos
Latest Reply
- 2 kudos
Thank you for sharing
- 2 kudos
- 1503 Views
- 3 replies
- 5 kudos
🌟 Welcome to the Knowledge Sharing Hub! 🌟
Are you passionate about sharing your discoveries and insights with the world? Look no further! Our Knowledge Sharing Hub is the perfect space for you to showcase your research and connect with like-minded individuals across the globe. Here's why you...
- 1503 Views
- 3 replies
- 5 kudos
- 566 Views
- 1 replies
- 0 kudos
Spot databricks VMs - eviction rates
Before using Spot machines in #databricks, it's a good idea to check their eviction rates in your region. Azure Resource Graph Explorer and that simple query will help. SpotResources | where type =~ 'microsoft.compute/skuspotevictionrate/location' ...
- 566 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Thank you for sharing this @Hubert-Dudek
- 0 kudos
- 681 Views
- 0 replies
- 0 kudos
Installing CrowdStrike Falcon Sensor on Databricks Workers
Greetings,Does anyone here have experience deploying the CrowdStrike Falcon sensor on Databricks worker instances? For context, the cluster is deployed in AWS and we use a Databricks Ubuntu 20.04 AMI. Databricks allows adding a bootstrap/startup scri...
- 681 Views
- 0 replies
- 0 kudos
- 1389 Views
- 0 replies
- 1 kudos
Post: Lakehouse Federation - Databricks
Lakehouse Federation - Databricks In the world of data, innovation is constant. And the most recent revolution comes with Lakehouse Federation, a fusion between data lakes and data warehouses, taking data manipulation to a new level. This advancement...
- 1389 Views
- 0 replies
- 1 kudos
- 1500 Views
- 1 replies
- 1 kudos
Jonathan Frankel at Sigma talk
Hi @Sujitha Just to follow up on your suggestion to share my takeaways from Jonathan Frankel's talk at Sigma in NYC. The key ideas I came away with is:Building in-house custom models is more than just possible, there's advantages to itThere's danger...
- 1500 Views
- 1 replies
- 1 kudos
Latest Reply
- 1 kudos
@Danny_Lee This is super insightful! Really appreciate your time to share your key takeaways with us.
- 1 kudos
- 778 Views
- 0 replies
- 0 kudos
Databricks AI Security Framework
Today Databricks announced the release of the Databricks AI Security Framework (LinkedIn Post)You can download the paper (PDF) from blog post. Anyone else download this and have thoughts? My first thought is its a great start and has an excellent G...
- 778 Views
- 0 replies
- 0 kudos
- 1098 Views
- 1 replies
- 0 kudos
Predictive optimization log
After you enable predictive optimization, it is good to look at the system table and see what is going on with your tables #databricks
- 1098 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi @Hubert-Dudek , Thank you for sharing your valuable insights on the Knowledge Sharing Hub. Your expertise is greatly appreciated and contributes significantly to our community's knowledge base.
- 0 kudos
- 908 Views
- 0 replies
- 0 kudos
GCP - Initial External Location to GCP Bucket is wrong
When creating a new Workspace in GCP the default GCP External Location is wrong.Its easily fixed by Catalog (on the left) > External Data (on the bottom) > External Locations > choose the connection and edit the URL by deleting the second BucketId af...
- 908 Views
- 0 replies
- 0 kudos
- 2840 Views
- 3 replies
- 2 kudos
Materials to pass Databricks Data Engineering Associate Exam
Hi Guys, I have passed it already some time ago, but just recently have summarized all the materials which helped me to do it. Pay special attention to GitHub repository, which contains many great exercises prepared by Databricks teamhttps://youtu.be...
- 2840 Views
- 3 replies
- 2 kudos
- 938 Views
- 0 replies
- 0 kudos
Feature article: Leveraging Generative AI with Apache Spark: Transforming Data Engineering
I created this article in Linkedlin to allow both this community and Apache Spark user community to have access to it.It is particularly useful for data engineers who want to have a basic understanding of what Generative AI with Spark can do.Leverag...
- 938 Views
- 0 replies
- 0 kudos
