Databricks Community

Hi @eheinlein, You can obtain the workspace ID from within a Databricks Notebook by running the following command in a Python or Scala cell: spark.conf.get("spark.databricks.clusterUsageTags.clusterOwnerOrgId") This command will return the worksp...

Hi @samsonite, The error message you’re encountering seems related to a lock request.  Ensure that the credentials and permissions used for deploying to the production environment are correct. Differences in credentials between local and CI/CD enviro...

You can also use “CASCADE” to drop schema and tables as well. It is recursive. 

 Solved understanding the databricks.yml configuration!

Hi @pranay,  Since Databricks Notebooks allow you to run Python code, you can leverage Python libraries to manipulate Excel files.Instead of using pywin32, consider using libraries like pandas or openpyxl to read, modify, and save Excel files.You can...

Hi @Awoke101,  Shared access clusters in Databricks have certain restrictions due to Unity Catalog limitations.I recommend trying the single-user Unity Catalog cluster or configuring data persistence outside the container. Let me know if you need fur...

Hi @kellenpink, Databricks and Snowflake are both powerful platforms, but they excel in different areas:   Databricks shines in big data processing, machine learning, and AI workloads. It provides an integrated environment for data enginee...

Hi @kellenpink, Assessing the value created by implementing Databricks (DBRX) involves several considerations. Here are some steps you can take: Business Objectives and KPIs: Start by defining clear business objectives for using Databricks. What ...

You have changed the numpy version, and presumably that is not compatible with other libraries in the runtime. If flair requires later numpy, then use a later DBR runtime for best results, which already has later numpy versions

@nicole_lu_PM : Do you have any suggestions or feedback for the above question ? It will be really helpful if we can get some insights. 

Hi @pdemeulenaer, When working with Databricks Asset Bundles, you can specify library dependencies for local development in the requirements*.txt file at the root of the bundle project. However, job task library dependencies are declared in your bund...

Hi @PragyaS,  References: Creating Zip file on Azure storage explorer - Databricks CommunityHow to zip files (on Azure Blob Storage) with shutil in DatabricksWorking with Unity Catalog in Azure DatabricksAnnouncing the General Availability of Unity C...

Hi @nileshtiwaari , The error message you’re encountering indicates that the .exceptAll() operation is not supported on streaming DataFrames. In structured streaming, certain operations have limitations due to the nature of streaming data. To addr...

Hi @bamvallar, Welcome to the Databricks Community! We're thrilled to have you here and thank you for attending the DAIS and visiting our community booth. Your participation means a lot to us, and we're excited to see the contributions you'll bring ...