- 521 Views
- 1 replies
- 0 kudos
Unable to clone delta live table
Hello Team,In order to avoid any cost of running workflow for historical data, i am trying DEEP clone in order to copy data of streaming table from PROD workspace to QA workspace but it is giving below error. Please suggest some efficient data copy m...
- 521 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Anish_2 ,This is well known limitation of DLT. You can read about it at limitation section in documentation. Conclusion is following. If you want to have ability to use all feature of delta protocol, it's better not to use dlt framework. It's gre...
- 0 kudos
- 3287 Views
- 2 replies
- 0 kudos
File size upload limit through CLI
Does anyone know the size limit for uploading files through the CLI? I'm not finding it in the documentation.
- 3287 Views
- 2 replies
- 0 kudos
- 0 kudos
Does anyone know the size limit for uploading files through the CLI? I'm not finding it in the documentation.
- 0 kudos
- 655 Views
- 2 replies
- 0 kudos
CLI is not helpful in exporting Error: expected to have the absolute path of the object or directory
I try to export a job as a DBA in order to create an Asset Bundle according to thishttps://community.databricks.com/t5/data-engineering/databricks-asset-bundle-dab-from-existing-workspace/td-p/49309I am on Windows 10 Pro x64 withDatabricks CLI v0.223...
- 655 Views
- 2 replies
- 0 kudos
- 0 kudos
To export an existing folder under /Workspace/... export-dir command could be used : databricks workspace export-dir /Workspace/Applications/ucx/logs/migrate-tables/run-123-0/ /Users/artem.sheiko/logs
- 0 kudos
- 468 Views
- 1 replies
- 0 kudos
Databricks serverless Vs snowflake
Hi All,We want to switch from Snowflake to Databricks SQL Warehouse/serverless to simplify our data layers and reduce data copies before the reporting layer. Please share the benefits of using serverless over Snowflake and any limitations you see. We...
- 468 Views
- 1 replies
- 0 kudos
- 0 kudos
one big pro is that you do not need to copy data to the dwh. also your transformations and analytics queries reside on the same platform (databricks).If databricks can cover all the requirementsm compared to snowflakem is hard to tell. Probably ther...
- 0 kudos
- 327 Views
- 0 replies
- 0 kudos
Azure Databricks to GCP Databricks Migration
Hi Team, Can you provide your thoughts on moving Databricks from Azure to GCP? What services are required for the migration, and are there any limitations on GCP compared to Azure? Also, are there any tools that can assist with the migration? Please ...
- 327 Views
- 0 replies
- 0 kudos
- 452 Views
- 0 replies
- 0 kudos
Testing and Issues Related to Admin Role Changes
Hello,I would like to ask a question regarding user permissions.Currently, all team members are admins. Recently, we plan to change the admin roles so that only I and another user, A, will be admins. The other members will retain general usage permis...
- 452 Views
- 0 replies
- 0 kudos
- 626 Views
- 1 replies
- 1 kudos
Restricting Notebook Access in a Team of Admins
Hello,I would like to ask a question regarding user permissions:In a situation where all team members are admins, is it possible to restrict access to a specific notebook in Databricks so that only I can view and write it?Thank you
- 626 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @weilin0323 ,If Worskspace Access Control is enabled, you are able to manage access to directories and individual notebooks, so you will be able to remove the access, but since all the other users are admins, they will be able to grant access to t...
- 1 kudos
- 5777 Views
- 5 replies
- 0 kudos
Issue with Visual Studio Code Databricks extension
Hello,I successfully installed the extension and connected it to my databricks account. But when I try to select the repo (which already exists under repos in my databricks repo account) for syncing , I don't see it.I use Azure Devops (Git repo) as s...
- 5777 Views
- 5 replies
- 0 kudos
- 0 kudos
I think the building spinner can be confusing - we are looking at improving that experience. Did the cell-by-cell execution work for you now?
- 0 kudos
- 355 Views
- 1 replies
- 1 kudos
Best Practices for Optimizing Delta Lake Performance
I’m currently working on a project using Delta Lake on Databricks and would like to optimize performance for both batch and streaming workloads.
- 355 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @joshbuttler ,It's a big topic and it really depends on the use case. You can follow some guides/recommendations from databricks documentation:- Optimization recommendations on Databricks | Databricks on AWS- Use liquid clustering for Delta tables...
- 1 kudos
- 1056 Views
- 4 replies
- 0 kudos
Has anyone being able to connect to Databricks using ODBC?
I'm trying to connect to the Databricks cluster from the Excel using ODBC and following this official guide:https://docs.databricks.com/en/integrations/excel.html#connect-from-microsoft-excelBut the authentication is always failing, returning "500 In...
- 1056 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @Vadimalk ,Could you check if you have following options enabled in simba driver configuration:
- 0 kudos
- 153 Views
- 0 replies
- 0 kudos
Right course for ML engineer
Hi I would like to learn databricks so that I could look for job opportunities as a ML engineer. I have background with python programming, computer vision (OpenCV) .not having much of experience with azure , aws so on.which course here is good with ...
- 153 Views
- 0 replies
- 0 kudos
- 611 Views
- 1 replies
- 1 kudos
Troubleshoot in Databricks
Hi everyone,I’m having trouble with a query in Databricks. My code seems correct, but I’m getting unexpected results. Can anyone help me troubleshoot this or provide tips for debugging in Databricks?Thanks in advance! locksmith
- 611 Views
- 1 replies
- 1 kudos
- 1 kudos
It's hard to advice when you didn't provide any code. But you can try to use debugger inside notebook:
- 1 kudos
- 7288 Views
- 7 replies
- 1 kudos
How to move a metastore to a new Storage Account in unity catalog?
Hello, I would like to change the Metastore location in Databricks Account Console. I have one metastore created that is in an undesired container/storage account. I could see that it's not possible to edit a metastore that is already created. I coul...
- 7288 Views
- 7 replies
- 1 kudos
- 1 kudos
We ended up 1) deleting the metastore (which only contained catalogs/schemas/tables), 2) creating a new one in the desired storage account and 3) re-populating it by running all Delta Live Tables pipeline. All our underlying raw data is stored in ano...
- 1 kudos
- 1627 Views
- 5 replies
- 5 kudos
Resolved! Looking for Databricks Study Materials/Resources
Hello,I’m looking for study materials or resources to help me learn Databricks more effectively. Any recommendations would be greatly appreciated!
- 1627 Views
- 5 replies
- 5 kudos
- 5 kudos
@igor779, @Rishabh-Pandey and @szymon_dybczak .Thank you all for the suggestions! Unfortunately, I don’t have access to content from partner companies and mainly rely on free resources for my studies.
- 5 kudos
- 1568 Views
- 4 replies
- 0 kudos
Resolved! Data Masking Techniques and Issues with Creating Tables
Hello Databricks Team,I understand that the mask function can be used to mask columns, but I have a few questions:When users with access use a masked TABLE to create a downstream TABLE, the downstream TABLE does not inherit the mask function directly...
- 1568 Views
- 4 replies
- 0 kudos
- 0 kudos
Hi @weilin0323, How are you doing today?As per my understanding, When creating a downstream table from a masked table, you’ll need to reapply the mask function to the necessary columns in the new table if you want to maintain the same level of data p...
- 0 kudos
