Databricks Community

Hi @erigaud, Here’s how you can achieve this:   Define Variables in the Bundle Configuration: You can define custom variables in your databricks.yml file to hold the catalog names for different environments. For example:   variables:    catalog_name...

@devagya wrote:I'm working on this project which involves moving data from Infor to Databricks.Infor is somewhat of an enterprise solution. I could not find much resources on this. I could not even find any free trial option on their site.If anyone h...

Can you also try this validate on the privileges please Grant CREATE EXTERNAL TABLE on the external location to create a delta table.  on UNITY_CATALOG_EXTERNAL_GENERATE_PATH_CREDENTIALS_DENIED because that privilege is blocked behind.

Hi @schluca ,I’ve encountered an issue where an error occurred when trying to reference a table after deleting and recreating the source table for a Shallow Clone, and then performing the Shallow Clone again. As a solution, try deleting the destinati...

This will be extremely beneficial. We have certain use cases where we do not leverage Spark in our pipelines and lose the lineage. I would prefer to set an extra parameter when writing a table to specify the lineage. 

Hi @vinitkhandelwal, Looks like the token could be missing required permissions for the operation. Please refer to: You can clone public remote repositories without Git credentials (a personal access token and a username). To modify a public remote r...

I'm having a similar problem, my aim is from an Azure DataBricks notebook to invoke an AzureDataDactory pipeline I created an Access Connector for Azure Databricks to which I gave Data Factory Contributor permissions. Using these lines pythonfrom azu...

Hi @parasupadhyay ,I don't think there is much you can do. For security reasons, you can't use shared access mode in UC to access sparkContext. Your only option to make it work is to use single mode cluster.

with the databricks cli you can get all infos about a job run with this command:databricks jobs get-run <run-id> replace <run-id> with your actual run-id

Hi @Dev ,Generally, What happens spark reader APIs point to the DBFS by default. And, to read the file from User workspace, we need to append 'file:/' in the prefix.Thanks

@AH my loyola wrote:Hi Team Could Please you answer these three questions to onboard Genie Space for Analytics?Do you know if we can use GenieSpace in our web application through API or SDK?Is there any way to manage access control other than the Uni...

For Structured Streaming Applications, this would be a nice feature. Delta Live Tables manages checkpoints for you out of the box - you don't even have to reason about checkpoints at all, would recommend checking it out!