cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Kash
by Contributor III
  • 4904 Views
  • 5 replies
  • 0 kudos

Cannot install TA-LIB via cluster libraries

Hi there,I can't seem to find a way to install Ta-lib on a databricks server. I can manually install it in the notebook using the code below but if we detach the notebook then I have to install it again. Please let me know if you've found a fix for t...

  • 4904 Views
  • 5 replies
  • 0 kudos
Latest Reply
Kash
Contributor III
  • 0 kudos

Hi Vivian,Thanks for the note. I tried that and it shows the package was installed but when I try to import talib in the notebook it says no module names 'talib'Any thoughts on what might be happening?Thanks,K

  • 0 kudos
4 More Replies
huy_le
by New Contributor II
  • 1244 Views
  • 0 replies
  • 1 kudos

How to Update openapi specs ?

I found a few errors/differences between https://docs.databricks.com/_static/api-refs/permissions-2.0-aws.yaml and the actual api. Where can we update this specs for everyone because engineers are using this specs to generate their clients

  • 1244 Views
  • 0 replies
  • 1 kudos
mickniz
by Contributor
  • 4410 Views
  • 3 replies
  • 3 kudos

Dbx installation for local development on Vscode

Hi Folks,Since databricks is now asking to use DBX instead of databricks-connect ,We are trying to set up our local environment following the guide.dbx by Databricks Labs - Azure Databricks | Microsoft LearnHave create conf/deployment.yml and dbx/pro...

  • 4410 Views
  • 3 replies
  • 3 kudos
Latest Reply
mickniz
Contributor
  • 3 kudos

fixed this issue. but I am getting another issue while syncing local repo with Workspace in Databricks UI.When I run commanddbx sync repo -d workspace name --source.Command runs fine . I can see that dbfs but not under workspace in Databricks page.An...

  • 3 kudos
2 More Replies
rishabh4312
by Contributor II
  • 11659 Views
  • 15 replies
  • 47 kudos

Voucher code error

Hi,I received a voucher in Nov 2020 for 'Databricks Certified Associate Developer for Apache Spark 3.0 exam' with an expiry date on 10th Nov 2022. However I receive an error stating the promotion code has been used. I have never used the code. Please...

  • 11659 Views
  • 15 replies
  • 47 kudos
Latest Reply
Anonymous
Not applicable
  • 47 kudos

Hi @Rishabh Jain​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thank...

  • 47 kudos
14 More Replies
Anonymous
by Not applicable
  • 1717 Views
  • 1 replies
  • 11 kudos

Explain about "Activate Gift Certificate" section in Databricks Community Rewards.

Hi every,Could some one explain about the Active Gift Certificate in  in Databricks Community Rewards. And how to use it?Thanks vere much 

Image
  • 1717 Views
  • 1 replies
  • 11 kudos
Latest Reply
Unforgiven
Valued Contributor III
  • 11 kudos

@Jensen Ackles​ give ur info i had try to this function does not working. I used cert id lakehouse input into box active gift certificate not help for claim 100points to rewards point in my account

  • 11 kudos
B_Seibert
by New Contributor III
  • 3198 Views
  • 2 replies
  • 8 kudos

Restore Delta table after adding columns.

At version 3 of our Delta Lake table we added a column. We later restored from version 11 back to version 10, which is now the most current version. But now when we run the table build from Azure Data Factory (ADF) on the full history of the data, we...

  • 3198 Views
  • 2 replies
  • 8 kudos
Latest Reply
B_Seibert
New Contributor III
  • 8 kudos

ignoreDeletes works. But I recommend to other developers that you have a think aboout all of the schema change scenarios and solve this problem above as part of a complete solution to every schema change scenario, instead of dealing with it as a one...

  • 8 kudos
1 More Replies
bharani
by New Contributor II
  • 976 Views
  • 0 replies
  • 0 kudos

coverage in databricks

Hi Prabakar, I would like to do test coverage in databricks notebooks so is there any way that I can do and also can I do testing with pytest in databricks? Are there any other tests like nutter library and doc test supports coverage? 

  • 976 Views
  • 0 replies
  • 0 kudos
Nilave
by New Contributor III
  • 8361 Views
  • 4 replies
  • 8 kudos

Resolved! Azure Databricks Python Test Coverage report

Hi,I'm trying to enable coverage report for Azure Databricks python code. Found a solution using pytest with databricks connect but faced below challenges Pytest framework using Databricks Connect:- This works for simple notebooks. But doesnt work fo...

  • 8361 Views
  • 4 replies
  • 8 kudos
Latest Reply
Prabakar
Databricks Employee
  • 8 kudos

@Nilave Chakraborty​ , There is no support for code coverage functionality yet. I don't think it will be available in the near future. However, it will be considered later. I do not have any timeline for now.

  • 8 kudos
3 More Replies
PhoenBounce
by New Contributor III
  • 7391 Views
  • 3 replies
  • 11 kudos

Resolved! Unable to see the Delta Live Tables tab in Workflows on Azure Databricks

I have looked everywhere but I am unable to find the option but doing the training indicates I need to do this to create a pipeline.I must be missing something obvious as I am new to thisThanksSteve

  • 7391 Views
  • 3 replies
  • 11 kudos
Latest Reply
Unforgiven
Valued Contributor III
  • 11 kudos

Delta Live Tables (DLT) is the first ETL framework that uses a simple declarative approach to building reliable data pipelines and automatically managing your infrastructure at scale so data analysts and engineers can spend less time on tooling and f...

  • 11 kudos
2 More Replies
AB_MN
by New Contributor III
  • 6548 Views
  • 5 replies
  • 1 kudos

Error when using DLT with SCD type 1

I'm trying to load data using DLT and SCD 1 and am running into the error message "Detected a data update in the source table at version x. This is currently not supported. If you'd like to ignore updates, set the option 'ignoreChanges' to 'true'.I h...

image
  • 6548 Views
  • 5 replies
  • 1 kudos
Latest Reply
Anonymous
Not applicable
  • 1 kudos

Hi @Andy Bolk​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 1 kudos
4 More Replies
KVNARK
by Honored Contributor II
  • 3113 Views
  • 3 replies
  • 8 kudos

Resolved! Badge for Data Analyst associate course completion was not sent. But I got the course completion certificate.

Badge for Data Analyst associate course completion was not sent

  • 3113 Views
  • 3 replies
  • 8 kudos
Latest Reply
Anonymous
Not applicable
  • 8 kudos

Hi @KVNARK .​ Hope all is well!Just wanted to check in that did you receive your badge? If yes then please mark it as the best answer. Else please let us know if you need more help. We'd love to hear from you.Thanks!

  • 8 kudos
2 More Replies
PabloSeoaneF
by New Contributor III
  • 7502 Views
  • 5 replies
  • 15 kudos

Databricks cluster's problems at start

Hello,I've spent some time trying to initialize a couple of clusters into an Azure Databricks private environment.Apparently, without reason they've been causing these two failures: With the advice of retrying in some minutes. Looking for information...

image.png image image
  • 7502 Views
  • 5 replies
  • 15 kudos
Latest Reply
PabloSeoaneF
New Contributor III
  • 15 kudos

Hello both! I attach here the 2 error details:Bootstrap timeout:HelpBootstrap Timeout. Please try again later.Instance bootstrap failed command: BootstrapTimeoutFailure message: Bootstrap script took too long and timeout. Please try again later.VM ex...

  • 15 kudos
4 More Replies
carlosancassani
by New Contributor III
  • 3229 Views
  • 3 replies
  • 5 kudos

Error: Credential size is more than configured size limit. As a result credential passthrough won't work for this notebook run.

I get this error when trying to execute parallel slave notebook from a Pyspark "master notebook".note 1: I use same class, functions, cluster, credential for another use case of parallel notebook in the same databricks instance and it works fine.note...

image
  • 3229 Views
  • 3 replies
  • 5 kudos
Latest Reply
Anonymous
Not applicable
  • 5 kudos

Hi @carlosancassani​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Th...

  • 5 kudos
2 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels