Databricks Community is an open-source platform for data enthusiasts and professionals to discuss, share insights, and collaborate on everything related to Databricks. Members can ask questions, share knowledge, and support each other in an environment that ensures respectful interactions.
I created a simple Databricks procedure that should return a single value."SELECT 1 AS result;" When I call this procedure from my .NET API using ExecuteReader, ExecuteAdapter, or ExecuteScalar, the...
Hello all, I am trying to enable file events on my Azure Workspace for the File Arrival Trigger trigger mode for Databricks Workflows. I'm following this documentation exactly (I think) but I'm not ...
Trying to load a csv file from a private S3 bucket please clarify requirements to do this- Can I do it in community edition (if yes then how)? How to do it in premium version? I have IAM role and I...
I am trying to load data from MongoDB into Spark. I am using the Community/Free version of DataBricks so my Jupiter Notebook is in a Chrome browser. Here is my code: from pyspark.sql import SparkSe...
In DBR 164.LTS, I am trying to add the following Spark config: spark.scheduler.allocation.file: file:/Workspace/init/fairscheduler.xml But the all purpose cluster is throwing this error &nb...
Hello, When evaluating data quality frameworks for PySpark pipelines, I came across DQX. I noticed it's available on PyPI (databricks-labs-dqx) and GitHub, which is great for accessibility. H...
Hello Community, We have been trying to migrate our jobs from Classic Compute to Serverless Compute. As part of this process, we face several challenges, and this is one of them. When we try to exe...