Databricks Community

raduq · 02-12-2024

I just saw the Databricks announcement on Twitter about the launch of the GenAI Engineer Pathway certification and I wanted to start learning.https://twitter.com/databricks/status/1756721509224108353But when logging into the Academy, they are all loc...

raduq · 04-04-2022

Hi, I'm a fairly new user and I am using Azure Databricks to process a ~50Gb JSON file containing real estate data. I uploaded the JSON file to Azure Data Lake Gen2 storage and read the JSON file into a dataframe.df = spark.read.option('multiline', '...

raduq · 02-12-2024

I'm a bit of a noob with the Academy interface, found the way to enroll. We actually have to search the catalog for the course and then if you find it from there, you can select it from the list and an "Enroll" button will appear and allow you to sta...

raduq · 04-08-2022

Hi Kaniz, I'm actually trying them right now. I have recently found a schema definition for my large JSON and I'm building it in Pyspark right now. I will most likely post an update to his feedback in a few hours.

raduq · 04-05-2022

@Hubert Dudek , I have also tried your suggested first step, sometime after posting the original question. Unfortunately, that doesn't work either, which confuses me, since I expected it to work. I did think to write it to delta to benefit from the ...

Databricks Community

User Stats

User Activity

Why does the Generative AI Engineering Pathway not available?

How to efficiently process a 50Gb JSON file and store it in Delta?

Re: Why does the Generative AI Engineering Pathway not available?

Re: How to efficiently process a 50Gb JSON file and store it in Delta?

Re: How to efficiently process a 50Gb JSON file and store it in Delta?