Databricks Community

Hello @Abhishek10745 It was just like you said! We have a completely private instance of Databricks and the DevOps Pipeline that I was using didin't have access to the private vnet. Switching pools solved the problem. Thanks for all the help!

Hi @B_J_Innov, To resolve the "Unauthorized" error when triggering a Databricks job via the API, ensure you authenticate using a personal access token (PAT) and include it as the `Bearer` token in your request headers. Verify that your API endpoint i...

Hi @anushajalesh28, To create a catalog in Azure Databricks, you need to specify the Azure storage account and storage container in the MANAGED LOCATION clause.    Let’s break down the query:   CREATE CATALOG IF NOT EXISTS Databricks_Anu_Jal_270220...

Hi @nan,  Let’s dive into the intricacies of timestamp interpretation in Databricks on AWS. Timestamp Type in Databricks: A timestamp in Databricks represents an absolute point in time, comprising values for year, month, day, hour, minute, and se...

Hi @Surajv, To pass configuration values to a Spark session at runtime in PySpark without relying on the Databricks Connect configuration, you can access and set Spark configuration parameters programmatically. First, retrieve the current Spark conte...

It happens with any code; even something as simple as...x <- 2 + 2

I need to delete my old post which contain the details which should not be shared.https://community.databricks.com/t5/data-engineering/need-to-define-the-struct-and-array-of-struct-field-colum-in-the/m-p/58131#M31022 

Hi @Peter_Jones,  1. Update Command in Databricks SQL: The UPDATE statement in Databricks SQL allows you to modify column values for rows that match a specified predicate. Here’s the syntax for the UPDATE statement: UPDATE table_name [table_alias]...

I just hope Databricks will pay attention to it.

Hi @vigneshp , Good Day!  In Databricks, bitmap_count function returns the number of bits set in a BINARY string representing a bitmap. This function is typically used to count distinct values in combination with the bitmap_bucket_number() and the bi...

Hi @farazanwar, Jira Software offers a wide range of integrations to enhance its functionality. While there isn’t a native connector specifically developed by the Jira team, there are several apps (also known as add-ons or plugins) available in the A...

Hi @Phani1, To efficiently copy around 3 TB of Parquet data from HDFS to Databricks Delta format and create external tables, you can follow these best practices: Use the COPY INTO SQL Command: The COPY INTO SQL command allows you to load data fr...