Databricks Community

ThomazRossito · 04-14-2024

Lakehouse Federation - Databricks In the world of data, innovation is constant. And the most recent revolution comes with Lakehouse Federation, a fusion between data lakes and data warehouses, taking data manipulation to a new level. This advancemen...

ThomazRossito

Mine took quite a while too; I didn't get an email for that badge, I only found out it had arrived when I checked the page -> https://credentials.databricks.com/profile

ThomazRossito · 12-11-2024

Hello,With this new structure on the Databricks Academy platform, I am also unable to find the files containing the notebooks.

ThomazRossito · 04-16-2024

Hello,You will need to use some CSV configuration parametersfollow the documentation belowhttps://spark.apache.org/docs/latest/sql-data-sources-csv.html

ThomazRossito · 04-14-2024

Hello,I managed to parse using the "json_tuple" functionThere are other functions that can helpschema_of_jsonget_json_objectfrom_json dd = (df.select(json_tuple(col("AdditionalRequestParameters"), "Locale", "Setversion", "Flags", "ScalingID", "Versio...

ThomazRossito · 04-14-2024

Hi,I noticed that in the scheme you are creating, there are more columns than in your csv file, I was able to understand that the final result needs to include the 6 columnsI would use withColumn, for the 3 columns that do not exist in the fileBelow ...

Databricks Community

User Stats

User Activity

Post: Lakehouse Federation - Databricks

Re: Request for Databricks Partner Solutions Architect Essentials (Champion Series) badge

Re: How to find the source code for the data engineering learning path?

Re: DataFrame to CSV write has issues due to multiple commas inside an row value

Re: DataFrame to CSV write has issues due to multiple commas inside an row value

Re: When reading a csv file with Spark.read, the data is not loading in the appropriate column while