Databricks Community

ThomazRossito · 04-14-2024

Lakehouse Federation - Databricks In the world of data, innovation is constant. And the most recent revolution comes with Lakehouse Federation, a fusion between data lakes and data warehouses, taking data manipulation to a new level. This advancemen...

ThomazRossito

Hello,With this new structure on the Databricks Academy platform, I am also unable to find the files containing the notebooks.

ThomazRossito · 04-16-2024

Hello,You will need to use some CSV configuration parametersfollow the documentation belowhttps://spark.apache.org/docs/latest/sql-data-sources-csv.html

ThomazRossito · 04-14-2024

Hello,I managed to parse using the "json_tuple" functionThere are other functions that can helpschema_of_jsonget_json_objectfrom_json dd = (df.select(json_tuple(col("AdditionalRequestParameters"), "Locale", "Setversion", "Flags", "ScalingID", "Versio...

ThomazRossito · 04-14-2024

Hi,I noticed that in the scheme you are creating, there are more columns than in your csv file, I was able to understand that the final result needs to include the 6 columnsI would use withColumn, for the 3 columns that do not exist in the fileBelow ...

ThomazRossito · 04-14-2024

Hello,It is possible to have the expected result with dbutils.fs.cp("Origin location", "Destination location", True) and then create the table with the LOCATION of the Destination locationHope this helps

Databricks Community

User Stats

User Activity

Post: Lakehouse Federation - Databricks

Re: How to find the source code for the data engineering learning path?

Re: DataFrame to CSV write has issues due to multiple commas inside an row value

Re: DataFrame to CSV write has issues due to multiple commas inside an row value

Re: When reading a csv file with Spark.read, the data is not loading in the appropriate column while

Re: Move a delta table from a non UC metastore to a UC metastore preserving history