cancel
Showing results for 
Search instead for 
Did you mean: 
Knowledge Sharing Hub
Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practices. Join the conversation today and unlock a wealth of collective wisdom to enhance your experience and drive success.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

RobinK
by New Contributor III
  • 43 Views
  • 0 replies
  • 0 kudos

How to switch Workspaces via menue

Hello,In various webinars and videos featuring Databricks instructors, I have noticed that it is possible to switch between different workspaces using the top menu within a workspace. However, in our organization, we have three separate workspaces wi...

  • 43 Views
  • 0 replies
  • 0 kudos
Danny_Lee
by Contributor
  • 656 Views
  • 1 replies
  • 1 kudos

Jonathan Frankel at Sigma talk

Hi @Sujitha Just to follow up on your suggestion to share my takeaways from Jonathan Frankel's talk at Sigma in NYC.  The key ideas I came away with is:Building in-house custom models is more than just possible, there's advantages to itThere's danger...

  • 656 Views
  • 1 replies
  • 1 kudos
Latest Reply
Sujitha
Community Manager
  • 1 kudos

@Danny_Lee This is super insightful! Really appreciate your time to share your key takeaways with us. 

  • 1 kudos
Danny_Lee
by Contributor
  • 162 Views
  • 0 replies
  • 0 kudos

Databricks AI Security Framework

Today Databricks announced the release of the Databricks AI Security Framework (LinkedIn Post)You can download the paper (PDF) from blog post. Anyone else download this and have thoughts?   My first thought is its a great start and has an excellent G...

  • 162 Views
  • 0 replies
  • 0 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 304 Views
  • 1 replies
  • 0 kudos

Predictive optimization log

After you enable predictive optimization, it is good to look at the system table and see what is going on with your tables #databricks

pl.png
  • 304 Views
  • 1 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Hubert-Dudek , Thank you for sharing your valuable insights on the Knowledge Sharing Hub. Your expertise is greatly appreciated and contributes significantly to our community's knowledge base.

  • 0 kudos
avrm91
by New Contributor II
  • 323 Views
  • 0 replies
  • 0 kudos

GCP - Initial External Location to GCP Bucket is wrong

When creating a new Workspace in GCP the default GCP External Location is wrong.Its easily fixed by Catalog (on the left) > External Data (on the bottom) > External Locations > choose the connection and edit the URL by deleting the second BucketId af...

avrm91_1-1711447524318.png avrm91_0-1711447511611.png
  • 323 Views
  • 0 replies
  • 0 kudos
Emil_Kaminski
by Contributor
  • 1769 Views
  • 3 replies
  • 2 kudos

Materials to pass Databricks Data Engineering Associate Exam

Hi Guys, I have passed it already some time ago, but just recently have summarized all the materials which helped me to do it. Pay special attention to GitHub repository, which contains many great exercises prepared by Databricks teamhttps://youtu.be...

  • 1769 Views
  • 3 replies
  • 2 kudos
Latest Reply
PurtiSharma
New Contributor II
  • 2 kudos

Thanks for sharing. It is indeed very useful.

  • 2 kudos
2 More Replies
Hubert-Dudek
by Esteemed Contributor III
  • 444 Views
  • 0 replies
  • 0 kudos

Debug stream

One of the easiest ways to debug a stream is to stream it to memory and query it with SQL #databricks

debug.png
  • 444 Views
  • 0 replies
  • 0 kudos
MichTalebzadeh
by New Contributor III
  • 338 Views
  • 0 replies
  • 0 kudos

Feature article: Leveraging Generative AI with Apache Spark: Transforming Data Engineering

I created this article in Linkedlin to allow both this community and Apache Spark user community to have access to it.It is particularly useful for data engineers who want to have a basic understanding of what  Generative AI with Spark can do.Leverag...

Knowledge Sharing Hub
Generative AI
spark
  • 338 Views
  • 0 replies
  • 0 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 605 Views
  • 1 replies
  • 3 kudos

DBR 15.0 beta

databricks runtime 15 is out there!Some breaking changes. More info here https://docs.databricks.com/en/release-notes/runtime/15.0.html

15.png
  • 605 Views
  • 1 replies
  • 3 kudos
Latest Reply
jose_gonzalez
Moderator
  • 3 kudos

Thanks for sharing this information @Hubert-Dudek!!!

  • 3 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 1290 Views
  • 1 replies
  • 1 kudos

Notebook IDE

This is an excellent step for #databricks notebooks. Integrated debugger and CLI in notebook terminal is a big step towards a fully functional cloud IDE.

ide.png
  • 1290 Views
  • 1 replies
  • 1 kudos
Latest Reply
jose_gonzalez
Moderator
  • 1 kudos

Thank you for sharing this @Hubert-Dudek!!!

  • 1 kudos
MichTalebzadeh
by New Contributor III
  • 619 Views
  • 2 replies
  • 0 kudos

Build a machine learning model to detect fraudulent transactions using PySpark's MLlib library

IntroductionFinancial fraud is a significant concern for businesses and consumers alike. I have written about this concern a few times in Linkedlin articles. Machine learning offers powerful tools to combat this issue by automatically identifying sus...

Knowledge Sharing Hub
Financial Fraud
PySpark MLlib
spark
  • 619 Views
  • 2 replies
  • 0 kudos
Latest Reply
deborah621
New Contributor II
  • 0 kudos

Looking to build a machine learning model for detecting fraudulent transactions using PySpark’s MLlib. Generate synthetic transaction data. Provides a dataset for model training without using sensitive real-world data. Enables the creation of diverse...

  • 0 kudos
1 More Replies
alexgv12
by New Contributor III
  • 391 Views
  • 1 replies
  • 1 kudos

is it possible to have a class level separation in databricks or implement a design pattern in datab

if you have thought about making your code inside databricks and notebooks more reusable and organized and you have thought about implementing a design pattern or class level separation in databricks the answer is yes, I am going to tell you the deta...

  • 391 Views
  • 1 replies
  • 1 kudos
Latest Reply
-werners-
Esteemed Contributor III
  • 1 kudos

tnx! I have spent quite some time on figuring out what the best way is. Your approach is certainly a valid one.Myself I prefer to package reused classes in a jar (we mainly code in scala). Works fine too.

  • 1 kudos
Sujitha
by Community Manager
  • 652 Views
  • 2 replies
  • 5 kudos

🌟 Welcome to the Knowledge Sharing Hub! 🌟

Are you passionate about sharing your discoveries and insights with the world? Look no further! Our Knowledge Sharing Hub is the perfect space for you to showcase your research and connect with like-minded individuals across the globe. Here's why you...

  • 652 Views
  • 2 replies
  • 5 kudos
Latest Reply
neerajbhadani
New Contributor II
  • 5 kudos

Hi Sujitha,    Thanks for sharing this information with us. I tried to register for the event with the link mentioned in right panel. However, the registration link is not working for me.Also, could you please share more details about the event as we...

  • 5 kudos
1 More Replies
MichTalebzadeh
by New Contributor III
  • 1341 Views
  • 3 replies
  • 2 kudos

Resolved! Building Event-Driven Real-Time Data Processor with Spark Structured Streaming and API Integration

 I recently saw an article from Databricks titled "Scalable Spark Structured Streaming for REST API Destinations". A great article focusing on continuous Spark Structured Streaming (SSS). About a year old. I then decided, given customer demands to wo...

Knowledge Sharing Hub
Event-driven architecture
Flask
spark
Spark Structure Streaming
Spark Structured Streaming
  • 1341 Views
  • 3 replies
  • 2 kudos
Latest Reply
Kaniz
Community Manager
  • 2 kudos

Hi @MichTalebzadeh, Thank you for sharing your insightful article, "Building an Event-Driven Real-Time Data Processor with Spark Structured Streaming and API Integration." Your approach to integrating Apache Spark™, REST API, and Flask to create a ro...

  • 2 kudos
2 More Replies