- 2018 Views
- 1 replies
- 0 kudos
Automating the Migration of ML Models and Metadata to Unity Catalog
Hello everyone,I am currently working on a project where I need to migrate ML models from the Model Registry of one Databricks workspace to the Unity Catalog (schema/catalog) of another workspace. This migration needs to include not only the models t...
- 2018 Views
- 1 replies
- 0 kudos
- 0 kudos
Deploy as Code is recommended. https://learn.microsoft.com/en-us/azure/databricks/machine-learning/mlops/deployment-patterns#deploy-code If you need to copy the model as a Pickle file then use https://github.com/mlflow/mlflow-export-import CICD imple...
- 0 kudos
- 2326 Views
- 1 replies
- 0 kudos
Unable to clone delta live table
Hello Team,In order to avoid any cost of running workflow for historical data, i am trying DEEP clone in order to copy data of streaming table from PROD workspace to QA workspace but it is giving below error. Please suggest some efficient data copy m...
- 2326 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anish_2 ,This is well known limitation of DLT. You can read about it at limitation section in documentation. Conclusion is following. If you want to have ability to use all feature of delta protocol, it's better not to use dlt framework. It's gre...
- 0 kudos
- 1617 Views
- 1 replies
- 0 kudos
Price of UCX assessment and migration
Hi,I am planning to migrate few tables and workflows from Hive metastore to Unity catalog( For example 8 tables and 2 small) workflows. What is the cost of using UCX (Unity Catalog Upgrade eXperience) for assessment and migration of artifacts from a ...
- 1617 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @satya1206 ,There is no place involved when it comes to using ucx. It's free tool, open source tool created by databricks labs. More info you can find on below: https://github.com/databrickslabs/ucx
- 0 kudos
- 920 Views
- 0 replies
- 0 kudos
Scheduled Dashboard Email
Hi community! I'm wondering if there is an option to personalize the email body of an scheduled dashboard. I'm specially interested in any way that allows me to change the attached dashboard snapshot to an excel file or make it available a table with...
- 920 Views
- 0 replies
- 0 kudos
- 5930 Views
- 6 replies
- 5 kudos
PowerApps + Delta Lake
Is there any native way to Insert data from PowerApps directly into Delta Lake?I can of course create a workflow - and push the data via API, but is there any alternative ?
- 5930 Views
- 6 replies
- 5 kudos
- 5 kudos
I have started creating solution for that: Here is initial version with PAT https://github.com/lakime/powerappstodelta
- 5 kudos
- 1459 Views
- 0 replies
- 0 kudos
Best Tools for API Design? SwaggerHub Alternatives?
Curious if anyone has moved from SwaggerHub to other platforms for API design, especially within data projects. Tools like Apidog, Postman, or Insomnia have been mentioned to me – does anyone have thoughts on how they compare for efficiency and colla...
- 1459 Views
- 0 replies
- 0 kudos
- 4389 Views
- 7 replies
- 1 kudos
Coursera Apache SQL Course
Hi, I have enrolled into Databricks Apache Spark SQL for Data Analysts course on Coursera.I am in Module 3, I imported the file through given url in course material. In 3.2Basic Queries I ran the first block which contains following command "%run ....
- 4389 Views
- 7 replies
- 1 kudos
- 1 kudos
Hi, I enrolled in the same coursera "databricks" course, and met the same issue. I unmounted and re-run the classroom set up as following However, when I move forward to list the mounted dirctory, the same issue happened.
- 1 kudos
- 661 Views
- 0 replies
- 0 kudos
Enterprise chatbot ml model, from datalake parquet data.
I want to create a chatbot ml model, to answer questions about the data, my organization have in the datalake. It should answer comprehension questions about the data.example: what do you understand from the data received from yesterday and today.
- 661 Views
- 0 replies
- 0 kudos
- 5077 Views
- 2 replies
- 0 kudos
File size upload limit through CLI
Does anyone know the size limit for uploading files through the CLI? I'm not finding it in the documentation.
- 5077 Views
- 2 replies
- 0 kudos
- 1533 Views
- 0 replies
- 0 kudos
Compatibility Issue Between Databricks Runtime and Delta Live Tables: Request for v15.2 Release Time
We are experiencing a compatibility issue between the Databricks Runtime version and the Delta Live Tables (DLT) Runtime. We need to upgrade to Databricks Runtime 15.2 to enable the auto schema evolution feature. However, Databricks Runtime 15.2 requ...
- 1533 Views
- 0 replies
- 0 kudos
- 9273 Views
- 4 replies
- 0 kudos
Resolved! Have code stay hidden even when the notebook is copied
When I save a certain Python notebook where I have selected Hide Code and Hide Results on certain cells, those conditions persist. For example, when I come back the next day in a new session, the hidden material is still hidden.When the notebook is ...
- 9273 Views
- 4 replies
- 0 kudos
- 0 kudos
In my situation we cannot split this notebook as ADF pipeline is already in PROD ,I have tried to use the option %%capture .It helps to ran the notebook within size limits but somehow it is corrupting the output. Also checked in the Databricks AI and...
- 0 kudos
- 1716 Views
- 0 replies
- 0 kudos
Databricks Asset Bundles - Failed to install provider
Hello All,When I try to deploy my bundle, I get the following error.I can't edit the bundle.tf.json, I suppose it is created automatically. Does anyone have a solution for the same problem?Many Thanks,Can$ databricks bundle deploy -t devBuilding my_p...
- 1716 Views
- 0 replies
- 0 kudos
- 2238 Views
- 2 replies
- 0 kudos
CLI is not helpful in exporting Error: expected to have the absolute path of the object or directory
I try to export a job as a DBA in order to create an Asset Bundle according to thishttps://community.databricks.com/t5/data-engineering/databricks-asset-bundle-dab-from-existing-workspace/td-p/49309I am on Windows 10 Pro x64 withDatabricks CLI v0.223...
- 2238 Views
- 2 replies
- 0 kudos
- 0 kudos
To export an existing folder under /Workspace/... export-dir command could be used : databricks workspace export-dir /Workspace/Applications/ucx/logs/migrate-tables/run-123-0/ /Users/artem.sheiko/logs
- 0 kudos
- 787 Views
- 0 replies
- 0 kudos
Loading parquet files with earlier timestamp looking for newer files
I have a setup where i am replicating Delta live tables parquet, and checkpoint files using azure RAGZRS to peer region for disaster recovery. When i load the the replicated files in peer region using delta format, i get an error that _delta_log/0000...
- 787 Views
- 0 replies
- 0 kudos
- 3683 Views
- 1 replies
- 0 kudos
Databricks serverless Vs snowflake
Hi All,We want to switch from Snowflake to Databricks SQL Warehouse/serverless to simplify our data layers and reduce data copies before the reporting layer. Please share the benefits of using serverless over Snowflake and any limitations you see. We...
- 3683 Views
- 1 replies
- 0 kudos
- 0 kudos
one big pro is that you do not need to copy data to the dwh. also your transformations and analytics queries reside on the same platform (databricks).If databricks can cover all the requirementsm compared to snowflakem is hard to tell. Probably ther...
- 0 kudos
