cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
cancel
Turn on suggestions
Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
MunikrishnaS
by • New Contributor II
  • 1156 Views
  • 7 replies
  • 0 kudos

What are optimized solutions for moving on-premise IBM DB2 CDC data to Databricks Delta table

Hi Team,My requirement is to move build a solution to move zos(db2) CDC data to Delta table on Realtime bases(at least near realtime) , data volume and number of tables are little huge (100 tables) I have researched I dont find any inbuild options in...

Data Engineering
Reply
  • 1156 Views
  • 7 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
6 More Replies
Pratibha
by • New Contributor II
  • 581 Views
  • 2 replies
  • 0 kudos

how max_retry_interval_millis works with retry_on_timeout in Data bricks.

 my project I want if job take longer time then it will terminate and again it will try even if there is timeout error and in databricks launched status should show retry by scheduler and it should follow min_retry_interval_millis before  start retry...

Data Engineering
min_retry_interval_millis
Reply
  • 581 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
1 More Replies
DH_Fable
by • New Contributor II
  • 434 Views
  • 2 replies
  • 0 kudos

Downloading multiple excel files at once from repo

I have a notebook that produces lots of excel files which I want downloading on my local machine.I can only currently download one by one which takes a long time when there are a lot of them.Is there a way without using Azure CLI to download all of t...

Data Engineering
Reply
  • 434 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
1 More Replies
Databricks-acn
by • New Contributor II
  • 885 Views
  • 5 replies
  • 0 kudos

Unable to load data in DLT tables from Federated data sources

I tried to run this query and failing to load the data .What do I need to do load from federated data sources using DLT if this is not correct CREATE OR REPLACE LIVE TABLE bulkuploadhistory COMMENT 'Table generated for bulkuploadhistory.' TBLPROPERTI...

Data Engineering
dlt
UC
Reply
  • 885 Views
  • 5 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
4 More Replies
pawelzak
by • New Contributor III
  • 1636 Views
  • 3 replies
  • 1 kudos

Dashboard update through API

Hi,I would like to create / update dashboard definition based on the json file. How can one do it? I tried the following:databricks api post /api/2.0/preview/sql/dashboards/$dashboard_id --json @file.json  But it does not update the widgets...How can...

Data Engineering
Reply
  • 1636 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 1 kudos
2 More Replies
Kaushik2
by • New Contributor
  • 365 Views
  • 2 replies
  • 0 kudos

Reports on list of users and roles that have access to Databricks workspace

Are there any in-built reports available in Databricks UI that list the users and roles with access to the workspace?

Data Engineering
Reply
  • 365 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
1 More Replies
Ruby8376
by • Valued Contributor
  • 636 Views
  • 3 replies
  • 1 kudos

Query endpoint on Azure sql or databricks?

Hi Currently all data reauired resides in Az sql database. We have a project in which we need to query on demand this data in Salesforce data cloud to be further used for reporting in CRMA dashboard.do we need to move this data from az sql to delta l...

Data Engineering
Reply
  • 636 Views
  • 3 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 1 kudos
2 More Replies
dataman17
by • New Contributor
  • 661 Views
  • 2 replies
  • 0 kudos

Stop a interactive cluster running a continuous job to allow for a restart

Hello,I am wondering if there is a way in Databricks to run a job continuously except for 1 or 2 hours every night in which the cluster could restart. We are using interactive clusters for our jobs and development in Dev and UAT. In Prod we are still...

Data Engineering
Reply
  • 661 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
1 More Replies
Murthy1
by • Contributor II
  • 3185 Views
  • 2 replies
  • 0 kudos

How can we use existing all purpose cluster for a DLT pipeline?

I understand that DLT is a separate job compute but I would like to use an existing all purpose cluster for the DLT pipeline. Is there a way I can achieve this?

Data Engineering
Reply
  • 3185 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
1 More Replies
feed
by • New Contributor III
  • 3809 Views
  • 7 replies
  • 3 kudos

TesseractNotFoundError

TesseractNotFoundError: tesseract is not installed or it's not in your PATH. See README file for more information. in databricks

Data Engineering
Reply
  • 3809 Views
  • 7 replies
  • 3 kudos
View Replies
Latest Reply
neha_ayodhya
New Contributor II
  • 3 kudos

%sh apt-get install -y tesseract-ocr this command is not working in my new Databricks free trail account, earlier it worked fine in my old Databricks instance. I get below error: E: Could not open lock file /var/lib/dpkg/lock-frontend - open (13: Per...

  • 3 kudos
6 More Replies
sandeephenkel23
by • New Contributor II
  • 1070 Views
  • 2 replies
  • 1 kudos

How to run Powershell file script.ps1 using the databricks notebook

Hello All,Following command on running through databricks notebook is not working Command%sh# Bash code to print 'Hello, PowerShell!'echo 'Hello, PowerShell!'# powershell.exe -ExecutionPolicy Restricted -File /dbfs:/FileStore/Read_Vault_Inventory.ps1...

Data Engineering
Reply
  • 1070 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 1 kudos
1 More Replies
hv129
by • New Contributor
  • 857 Views
  • 2 replies
  • 1 kudos

java.lang.OutOfMemoryError on Data Ingestion and Storage Pipeline

I have around 25GBs of data in my Azure storage. I am performing data ingestion using Autoloader in databricks. Below are the steps I am performing:Setting the enableChangeDataFeed as true.Reading the complete raw data using readStream.Writing as del...

Data Engineering
Reply
  • 857 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 1 kudos
1 More Replies
AndyKeel
by • New Contributor II
  • 442 Views
  • 3 replies
  • 0 kudos

Creating an ADLS storage credential for an AWS Workspace

I'd like to create a storage credential for an Azure Storage Account in an AWS workspace. I then plan to use this storage credential to create an external volume.Is this possible, and if so what are the steps? Thanks for any help!

Data Engineering
Reply
  • 442 Views
  • 3 replies
  • 0 kudos
View Replies
Latest Reply
AndyKeel
New Contributor II
  • 0 kudos

Thanks for your help.I'm struggling to create the Storage Credential. I have created a managed identity via an Azure Databricks Access Connector and am making an API call based on what I'm reading in the API docs: Create a storage credential | Storag...

  • 0 kudos
2 More Replies
NirmalaSathiya
by • New Contributor
  • 439 Views
  • 2 replies
  • 0 kudos

Not able to use _metadata to retrieve file name while reading xml files

We are trying to retrieve xml file name using _metadata but not working. we are not able to use input _file_name() also as we are using shared cluster.we are reading the xml files using com.datadricks.spark.xml library  

Data Engineering
filename
read
XML
Reply
  • 439 Views
  • 2 replies
  • 0 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 0 kudos
1 More Replies
ElaPG
by • New Contributor III
  • 1516 Views
  • 7 replies
  • 1 kudos

Command restrictions

Is there any possibility to restrict usage of specified commands (like mount/unmount or SQL grant) based on group assignment? I do not want everybody to be able to execute these commands.

Data Engineering
Reply
  • 1516 Views
  • 7 replies
  • 1 kudos
View Replies
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers your question?This...

  • 1 kudos
6 More Replies
Labels
Top Kudoed Authors
View all