Databricks Community

You can get the default language of a notebook using dbutils.notebook.get_notebook_language()  try this example: %pythonimport dbutilsdefault_language = dbutils.notebook.get_notebook_language()print(default_language)

Hi @Retired_mod I am having the same issue where i made a inner join on two spark dataframes they are running only a single node not sure how to modify to run on many nodes and same thing with when i write a 30 gb data to a delta table it is almost 3...

Are you getting any error messages? what happens when you do a "ls /dbfs/"? are you able to list all the parquet files?

@Retired_mod thank you for quick response, we are able to generate for streaming and materialized views. but only confusion that i am seeing is, in terms of limitations that are mentioned in DLT Identity columns are not supported with tables that are...

Using Spark readStream with SQS will be very similar to using CloudFiles with FileNotification mode (which also uses SQS on the backend).CloudFiles comes with some additional options compared to native Spark readStream:Common Auto Loader OptionsOne b...

If you are ok for users to have only SQL syntax available ( no mounts ) , you can provision SQL warehouse for users , not clusters

Hello Karene, You can do the transformation in following manner from string to struct and refer to the example below: data =[('001','{"name":"bhupendra","zipcode":"260100"}')]schema = ['id','propertytype']df = spark.createDataFrame(data,schema)df.sho...

Hi Faisal, APPLY CHANGES INTO does not support a materialized view as a source, this must be a streaming table. Ideally, your bronze tables are append-only with the source providing data incrementally. If you do get revisions on previous records in y...