Databricks Community

You can increase compute resources for your Streamlit Databricks app, but this requires explicitly configuring the compute size in the Databricks app management UI or via deployment configuration—environment variables like DATABRICKS_CLUSTER_ID alone...

The "INTERNAL_ERROR: AI service did not respond with a valid answer" in Databricks AI/BI Genie typically means the Genie service failed to process your query, often due to one of a few common issues. This can include problems with the table existence...

The most efficient way to list all file paths in an Azure Blob Storage container from Databricks, especially when Hierarchical Namespace (HNS) is not enabled, is to use Azure SDKs targeting the blob flat namespace directly rather than filesystem prot...

The issue you're experiencing—where your script freezes in VS Code when running spark.sql locally using Databricks Connect, but works correctly when deployed—can result from several common causes related to Databricks Connect configuration, networkin...

Yes, you can define environment-specific constants at the bundle level in Databricks Asset Bundles and make them accessible inside Databricks notebooks, without relying on task-level parameters. This can be done using environment variables, bundle co...

You are experiencing an issue in Azure Data Factory (ADF) where the Lookup activity does not show your Unity Catalog databases in the configuration dropdown, even though connectivity from ADF to Databricks is successful and you have followed all reco...

GeoParquet 2.0’s formalization within the Apache Parquet specification is a significant step for native geospatial data storage across the modern data ecosystem, particularly for platforms like Databricks and Delta Lake. In summary, Delta Lake's reli...

No, there is currently no simple, direct way to add new schema access to an existing Serverless Databricks SQL warehouse connection through Partner Connect for Preset—neither through Databricks UI, BI tool configuration, nor the Databricks service pr...

Your error is due to how Databricks on AWS is trying to access GCS: it's defaulting to using the GCP metadata server (which only exists on Google Cloud VMs), not the service account key you provided. This is a common issue when connecting GCS from no...

The main issue is that Databricks jobs typically run in environments where the file system may be read-only or restricted—especially for files packaged within the asset bundle or inside locations like /databricks/driver, /databricks/conda, or other s...

There appears to be a recurring issue with Delta Live Table (DLT) pipelines in Databricks where the checkpoint is unexpectedly stored in the dbfs:/ path, rather than in the intended external storage location (such as Azure Blob Storage or ADLS). This...

Thansk for the update @thelogicplus  I will have look and try to connect with team of Travinto.

Hi Hasan. Great to see your motivation! Here’s a good way to start your journey into data engineering:Master SQL, it’s the foundation of everything in data.Enroll in the Databricks Academy (free) and take the beginner courses like “Get Started with D...