cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
saniafatimi
by • New Contributor II
  • 3895 Views
  • 3 replies
  • 1 kudos

How to migrate power bi reports to databricks

I have a sample set of power bi(.pbix) reports with all dropdowns, tables, filters etc. Now I would like to migrate this reports to data bricks. whatever visuals are created in power bi, I would like to create same in data bricks from scratch. I wou...

Data Engineering
Reply
  • 3895 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Neeljy
New Contributor II
  • 1 kudos

We must ensure you are sure that the Databricks cluster is operational. These are the steps needed for integration between Azure Databricks into Power BI Desktop.1. Constructing the URL for the connectionConnect to the cluster, and click the Advanced...

  • 1 kudos
2 More Replies
Dataengineer_mm
by • New Contributor
  • 1840 Views
  • 2 replies
  • 0 kudos

Databricks workflow migration to higher environments

How do we migrate the databricks workflows to higher environment ? I do see an option for calling the tasks (notebooks,python) from the github repositories. But as such how do we migrate the entire workflow jobs to other environment ?

Data Engineering
Reply
  • 1840 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
jose_gonzalez
Databricks Employee
  • 0 kudos

Hi @Menaka Murugesan​,Just a friendly follow-up. Did any of the responses help you to resolve your question? if it did, please mark it as best. Otherwise, please let us know if you still need help.

  • 0 kudos
1 More Replies
MachuPichu
by • New Contributor III
  • 7169 Views
  • 9 replies
  • 4 kudos

Copying Delta table from QA to Prod

I am looking for a way to copy large managed Delta table like 4TB from one environment (QA) to other(Prod) . QA and Prod are in different subscription and in different region. I understand Databricks provides a way to clone table. But I am not sure i...

Data Engineering
Reply
  • 7169 Views
  • 9 replies
  • 4 kudos
View Replies
Latest Reply
Hubert-Dudek
Esteemed Contributor III
  • 4 kudos

I would use a data factory to copy 4TB files as it has gigantic throughput. After completing a copy of everything, I would register as a table in the new metastore.

  • 4 kudos
8 More Replies
rgb
by • New Contributor
  • 899 Views
  • 0 replies
  • 0 kudos

Migration_pipeline.py failing to get default credentials

cat ~/.databrickscfg looks like this (with the correct token/host values in place of xxxxxx)[DEFAULT]host = xxxxxxtoken = xxxxxxjobs-api-version = 2.0The command I run to start the pipeline with default configured credentials is :sudo python3 migrati...

databrickserror
Data Engineering
Reply
  • 899 Views
  • 0 replies
  • 0 kudos
sawya
by • New Contributor II
  • 2425 Views
  • 3 replies
  • 0 kudos

Migrate workspaces to another AWS account

Hi everyone,I have a Databricks workspace in an AWS account that I have to migrate to a new AWS accountDo you know how I can do it ? Or it's better to recreate a new one and move all the workbooks and if I choose to create one new how can you export ...

Data Engineering
Reply
  • 2425 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
Abishek
Databricks Employee
  • 0 kudos

@AMADOU THIOUNE​ Can you check the below link to export the run jobs? https://docs.databricks.com/jobs.html#export-job-runs. Try to reuse the same job_id with the /update and /reset endpoints, it should allow you much better access to previous run re...

  • 0 kudos
2 More Replies
vk217
by • Contributor
  • 867 Views
  • 0 replies
  • 0 kudos

Databricks migration from 7.3 LTS to X.x

We are currently on 7.3 LTS version with Python 3.7. I see that we are several versions off the latest 11.1 release https://docs.databricks.com/release-notes/runtime/releases.html.I see that the End of support for 11.1 is earlier than 10.4 LTS. What ...

Data Engineering
Reply
  • 867 Views
  • 0 replies
  • 0 kudos
Direo
by • Contributor
  • 9375 Views
  • 3 replies
  • 2 kudos

Default indentation for Python has changed after migration to the new workspace

In our old workspace default identation was 2 spaces. In our new one it has changed to 4 spaces. Of course you can manually change it back to 2 spaces as we used to have, but it does not work. Does anyone know how to solve this issue?

Data Engineering
Reply
  • 9375 Views
  • 3 replies
  • 2 kudos
View Replies
Latest Reply
ranged_coop
Valued Contributor II
  • 2 kudos

You do have that option of Settings --> User Settings (Admin Settings ? not sure - I don't have admin access) --> Notebook Settings --> Default indentation for Python cells (in spaces)This will change the indentation for newer cells, but existing one...

  • 2 kudos
2 More Replies
Lonnie
by • New Contributor
  • 2089 Views
  • 0 replies
  • 0 kudos

Recommended Redshift-2-Delta Migration Path

Hello All!My team is previewing Databricks and are contemplating the steps to take to perform one-time migrations of datasets from Redshift to Delta. Based on our understandings of the tool, here are our initial thoughts:Export data from Redshift-2-S...

Data Engineering
Reply
  • 2089 Views
  • 0 replies
  • 0 kudos
Soma
by • Valued Contributor
  • 2877 Views
  • 4 replies
  • 0 kudos

Queries regarding workspace Migration to Premium

We are planning to migrate from standard to premium workspaceWe need to know if below artifacts will be maintainedneed to check on streaming Job DowntimeAccess token DBFS Access Production Cluster /JobsCluster ID Job ID and other properties like URL ...

Data Engineering
Reply
  • 2877 Views
  • 4 replies
  • 0 kudos
View Replies
Latest Reply
Soma
Valued Contributor
  • 0 kudos

hi @Kaniz Fatma​  then I can assume there wont be any impact on metastore and all the metadata(table definition,schema ) will be available post upgradation

  • 0 kudos
3 More Replies
User16789201666
by • Databricks Employee
  • 1524 Views
  • 2 replies
  • 0 kudos

What are some guidelines for migrating to DBR 7/Spark 3?

What are some guidelines for migrating to DBR 7/Spark 3?

Data Engineering
Reply
  • 1524 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
shan_chandra
Databricks Employee
  • 0 kudos

Please refer to the below reference for switching to DBR 7.xWe have extended our DBR 6.4 support until December 2021, DBR 6.4 extended support - Release notes: https://docs.databricks.com/release-notes/runtime/6.4x.htmlMigration guide to DBR 7.x: htt...

  • 0 kudos
1 More Replies
User16826994223
by • Honored Contributor III
  • 1378 Views
  • 1 replies
  • 0 kudos

What is databricks Sync

I am trying to migrate my workload to another workspace ( from ST to E2), I am planning to use data bricks sync, but still I am not sure, will it migrate everything like , currents, user , groups, job, notebook etc or has some limitations which I s...

Data Engineering
Reply
  • 1378 Views
  • 1 replies
  • 0 kudos
View Replies
Latest Reply
sajith_appukutt
Honored Contributor II
  • 0 kudos

Here is the support matrix for import/export operations for databricks-syncAlso checkout https://github.com/databrickslabs/migrate

  • 0 kudos
Labels