Databricks Community

While your account Solution Architect (SA) will be able to guide you, if you still want to check what peers did here https://community.databricks.com/t5/warehousing-analytics/migrate-azure-synapse-analytics-data-to-databricks/td-p/90663 and here http...

I am interested to participate in this group if it is available 

Hi @DebIT2011,You're facing a classic architectural decision between orchestration with ADF versus direct transformation using Databricks PySpark notebooks. Both tools are powerful but serve different purposes depending on your project needs. Below i...

Databricks enables the creation of scalable, end-to-end machine learning (ML) pipelines by providing a comprehensive and collaborative platform that integrates key components for data handling, experimentation, and model deployment. Here’s how Databr...

I am also facing the same issue now one approach tomorrow i will try I will create a job that using serverless job cluster. Then whenever user will click on download button from UI. This should trigger the job now this job. Will read the table as dat...

Thanks, but no thanks, spam resides in JUNK folder

Please refer to some of the other community articles with the no module error https://community.databricks.com/t5/data-engineering/udf-importing-from-other-modules/td-p/58988

You might want to verify the file path and permissions within your CI/CD process—sometimes the context in which the pipeline runs lacks proper DBFS mount visibility. We've encountered similar visibility inconsistencies while working on data aggregati...

For your consideration:   To migrate legacy dashboards from the Hive Metastore to Unity Catalog in Databricks programmatically while retaining SQL queries, data visualizations, and ensuring compatibility with Unity Catalog schemas and tables using Py...

I am trying to set up an ODBC or JDBC direct connection from ArcGIS Pro  and ArcGIS Enterprise Server; has anyone done this successfully? I was able to make DSN successful connection, but no tables are being delivered; I did not use a connection stri...

ust to add, you might also want to check the cluster logs (driver and init script logs) for any hidden errors or timeouts during startup. Sometimes dependencies silently fail to install,even if the cluster appears to be running. If possible, try brea...

Hello @oricaruso! It looks like this post duplicates the one you posted earlier. A response has already been provided in the original thread. I recommend continuing the discussion there to keep the conversation focused and organized.

Hi @OmarSEKkOUTI , you can leverage DLT feature to do so.  Please check: https://docs.databricks.com/aws/en/dlt/transform https://docs.databricks.com/aws/en/dlt/stateful-processing Here is the step-by-step tutorial: https://docs.databricks.com/aws/en...