cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 
Data + AI Summit 2024 - Data Engineering & Streaming

Forum Posts

JohanBringsdal
by New Contributor
  • 485 Views
  • 0 replies
  • 0 kudos

Migrating old solution to new optimal delta lake setup

Hi Databricks community!I have previsouly worked on a project that easily could be optimized with Databricks. It is currently running on Azure Synapse, but the premise is the same.Ill describe the scenario here:1. Data owners send a constant flow of ...

  • 485 Views
  • 0 replies
  • 0 kudos
christo_M
by New Contributor
  • 668 Views
  • 1 replies
  • 0 kudos

Data Governance

How can I propagate a deletion to all tables where a customer requests to be removed from the database as part of the GDPR compliance ?  

  • 668 Views
  • 1 replies
  • 0 kudos
Latest Reply
Datajoe
Contributor
  • 0 kudos

We use a python script that enables and removes access to tables based on role-group, but can be user as well. Also have a script that removes all access- can be executes in seconds.

  • 0 kudos
matmat13
by New Contributor II
  • 10556 Views
  • 20 replies
  • 11 kudos

Resolved! Lakehouse Fundamentals Certificate/Badge not appearing

Hello! I just passed the Lakehouse Fundamentals Accreditation and I haven't received any badge or certificate for it. I understand that I need to go to credentials.databricks.com but it is not there. How long before it appears? Need help

  • 10556 Views
  • 20 replies
  • 11 kudos
Latest Reply
andy25
New Contributor II
  • 11 kudos

Hello can you help me, I have the same problem. I don't have the certificate available on credentials.databricks.com.

  • 11 kudos
19 More Replies
teja_7
by New Contributor
  • 415 Views
  • 0 replies
  • 0 kudos

DataSummit 2023

Happy to be part of the data summit 2023. Wondering if DBSQL is enabled for DLT tables

  • 415 Views
  • 0 replies
  • 0 kudos
Bollam
by New Contributor II
  • 431 Views
  • 0 replies
  • 0 kudos

Utility catalog and data governance

Attended the Data and AI Summit 2023 and gained insights into the utility catalog and services that it has to offer, definitely  going to try the data governance as it's a game changer. 

  • 431 Views
  • 0 replies
  • 0 kudos
Ankith
by New Contributor
  • 2360 Views
  • 1 replies
  • 1 kudos

Converting column of XML strings to column of Jsons

Hi,I want to convert column of XML strings to column of Json in PySpark., using withcolumn and xmltodict method as UDF, is giving Json with '=' instead of ':' in the dictionary. Please let me know if there is any alternative for this.

  • 2360 Views
  • 1 replies
  • 1 kudos
Latest Reply
DessertKid
New Contributor II
  • 1 kudos

To convert a column of XML strings to a column of JSON in PySpark, you can use the `from_json` function along with the `xmltodict` library. However, instead of using a UDF with `withColumn`, you can use the `select` function to transform the column.

  • 1 kudos
AlanF
by New Contributor II
  • 301 Views
  • 0 replies
  • 0 kudos

Great Summit

Having a great time at the community hub at the Summit. Highly recommend!

  • 301 Views
  • 0 replies
  • 0 kudos
Lakehouse
by New Contributor
  • 1476 Views
  • 2 replies
  • 0 kudos

Is Rust the future if analytics?

Today I walked into a session that talked about a fairly new language - Rust. The name can mislead you, I believe taking a look at the roots of how to best use CPU cycles is a game changer and Rust is traversing new areas that others might have ignor...

  • 1476 Views
  • 2 replies
  • 0 kudos
Latest Reply
Tom-Coffin
New Contributor II
  • 0 kudos

Yes, Rust is definitely part of the future.  It brings performance and simplicity to us.  I think it will add to the community, rather than replacing.  Scala and R will never go away, Python will always be strong, but Rust gives us one other tool in ...

  • 0 kudos
1 More Replies
kingmobil
by New Contributor
  • 272 Views
  • 0 replies
  • 0 kudos

Summit

Really enjoying my time at the summit!

  • 272 Views
  • 0 replies
  • 0 kudos
Orthoscope
by New Contributor
  • 338 Views
  • 0 replies
  • 0 kudos

Epic Keynote

Wow what was cool, so many of my workarounds have materialized into solutions!!

  • 338 Views
  • 0 replies
  • 0 kudos
abetogi
by New Contributor III
  • 369 Views
  • 0 replies
  • 0 kudos

AI

At Chevron we actively use Databricks to provide answers to business users. It was extremely interesting to see the use LakeHouseIQ initiatives as it can expedite how fast our users can receive their answers/reports. Is there any documentation that I...

  • 369 Views
  • 0 replies
  • 0 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Labels