Databricks Community

Here are some things to consider:   The best practice for designing a feature store in your scenario depends on balancing scalability, maintainability, and the dynamic nature of some dimensions like doctor names. Here's an outlined recommendation bas...

Hi @VigneshJaisanka The issue likely comes from a permissions or configuration mismatch. Here are a few things worth checking:Make sure the SPN is set as the pipeline owner and has the necessary permissions on the ADLS resource.If you’re using Unity ...

The initial /query request during a Delta Sharing operation with Unity Catalog serves a critical purpose in the query lifecycle. It is intended to retrieve the schema and basic metadata of the table, which helps in query planning and optimization. Th...

While your account Solution Architect (SA) will be able to guide you, if you still want to check what peers did here https://community.databricks.com/t5/warehousing-analytics/migrate-azure-synapse-analytics-data-to-databricks/td-p/90663 and here http...

I am interested to participate in this group if it is available 

Hi @DebIT2011,You're facing a classic architectural decision between orchestration with ADF versus direct transformation using Databricks PySpark notebooks. Both tools are powerful but serve different purposes depending on your project needs. Below i...

Databricks enables the creation of scalable, end-to-end machine learning (ML) pipelines by providing a comprehensive and collaborative platform that integrates key components for data handling, experimentation, and model deployment. Here’s how Databr...

I am also facing the same issue now one approach tomorrow i will try I will create a job that using serverless job cluster. Then whenever user will click on download button from UI. This should trigger the job now this job. Will read the table as dat...

Thanks, but no thanks, spam resides in JUNK folder

Please refer to some of the other community articles with the no module error https://community.databricks.com/t5/data-engineering/udf-importing-from-other-modules/td-p/58988

You might want to verify the file path and permissions within your CI/CD process—sometimes the context in which the pipeline runs lacks proper DBFS mount visibility. We've encountered similar visibility inconsistencies while working on data aggregati...

For your consideration:   To migrate legacy dashboards from the Hive Metastore to Unity Catalog in Databricks programmatically while retaining SQL queries, data visualizations, and ensuring compatibility with Unity Catalog schemas and tables using Py...

I am trying to set up an ODBC or JDBC direct connection from ArcGIS Pro  and ArcGIS Enterprise Server; has anyone done this successfully? I was able to make DSN successful connection, but no tables are being delivered; I did not use a connection stri...

ust to add, you might also want to check the cluster logs (driver and init script logs) for any hidden errors or timeouts during startup. Sometimes dependencies silently fail to install,even if the cluster appears to be running. If possible, try brea...