Showing results for 
Search instead for 
Did you mean: 
Valued Contributor II
since ‎09-22-2022

User Stats

  • 84 Posts
  • 0 Solutions
  • 30 Kudos given
  • 40 Kudos received

User Activity

I have a delta table in Databricks named "prod.silver.control_table". It has a few columns including "table_name" with string data type and "transform_options" with the below structure:  |-- transform_options: map (nullable = true) | |-- key: str...
I have a task to revise CSV ingestion in Azure Databricks. The current implementation uses the below settings: source_query = ( spark.readStream.format("cloudFiles") .option("cloudFiles.format", "csv") .schema(defined_schema) .option(...
Hi,When caching a DataFrame, I always use "df.cache().count()".However, in this reference, it is suggested to save the cached DataFrame into a new variable:When you cache a DataFrame create a new variable for it cachedDF = df.cache(). This will allow...
Hi,I have a DataFrame and different transformations are applied on the DataFrame. I want to display DataFrame after several transformations to check the results.  However, according to the Reference, every time I try to display results, it runs the e...
I get the following error when getting a list of files stored in an Azure Storage account using "" command in Databrciks.Failure to initialize configuration for storage account Invalid configuration value dete...