cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

yogu
by Databricks Partner
  • 2241 Views
  • 3 replies
  • 25 kudos

Can anyone plz help for mine below issues

Can anyone plz help for mine below issues

  • 2241 Views
  • 3 replies
  • 25 kudos
Latest Reply
Anonymous
Not applicable
  • 25 kudos

Hi @Yogita Chavan​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks

  • 25 kudos
2 More Replies
Siebert_Looije
by Contributor
  • 7319 Views
  • 4 replies
  • 11 kudos

How to setup an all-purpose cluster pool for all my jobs?

Today, we start working on setting up an all-purpose cluster pool for all the jobs that we are running on databricks. We used the documentation for this but we got some issues when running our jobs.The errors in the jobs are the following: The jobs a...

Error message Jobs pool configuration cluster configuration
  • 7319 Views
  • 4 replies
  • 11 kudos
Latest Reply
Anonymous
Not applicable
  • 11 kudos

Hi @Siebert Looije​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Tha...

  • 11 kudos
3 More Replies
clant
by New Contributor II
  • 1941 Views
  • 1 replies
  • 4 kudos

Structured Streaming from SFTP

Hello,Is it possible to use a SFTP location to load from for structured streaming.At the moment we are going from SFTP->S3->databricks via structured streaming. I would like to cut out the S3 part.CheersChris

  • 1941 Views
  • 1 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Chris Lant​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks.

  • 4 kudos
manuelt
by New Contributor III
  • 3881 Views
  • 4 replies
  • 9 kudos

How long does it take to enable SQL Serverless for Azure Databricks?

I have filled the form to enable SQL Serverless twice in our Azure Databricks workspace, but I have not received any response through email nor I have serverless enabled in our workspace.

  • 3881 Views
  • 4 replies
  • 9 kudos
Latest Reply
Anonymous
Not applicable
  • 9 kudos

Hi @Manuel Torrez​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 9 kudos
3 More Replies
Ancil
by Contributor II
  • 10391 Views
  • 1 replies
  • 4 kudos

Resolved! TypeError: 'JavaPackage' object is not callable - DocumentAssembler() - Spark NLP

Hi,I am doing a grammar check using spark NLP using azure databricks. But am getting TypeError: 'JavaPackage' object is not callable - in DocumentAssembler() intialization line.document_assembler = DocumentAssembler()\ .setInputCol("text")\ .setOutpu...

image.png
  • 10391 Views
  • 1 replies
  • 4 kudos
Latest Reply
Ancil
Contributor II
  • 4 kudos

Hi its working after adding below configs in cluster. Please check URL for more info :- https://nlp.johnsnowlabs.com/docs/en/install#databricks-support

  • 4 kudos
Srikrishna
by New Contributor II
  • 2359 Views
  • 3 replies
  • 4 kudos

Lakehouse Fundamentals Accreditation badge not received

Hi ive cleared Databricks Lakehouse Fundamental on 10/26/2022 and the certificate i received after clicking "download your certificate here" is not accurate as it only shows my name,i've also checked with accredible site. Please help me on this matte...

  • 2359 Views
  • 3 replies
  • 4 kudos
Latest Reply
Anonymous
Not applicable
  • 4 kudos

Hi @Srikrishna Parthasarathy​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear fr...

  • 4 kudos
2 More Replies
Harish14
by New Contributor III
  • 838 Views
  • 0 replies
  • 3 kudos

Hi @Jose Gonzalez​  ​i have taken databricks data engineer associate exam on nov 27th . in result mail it is mentioned i have obtained below 70% in as...

Hi @Jose Gonzalez​ â€‹i have taken databricks data engineer associate exam on nov 27th . in result mail it is mentioned i have obtained below 70% in assessment but as per the section wise results i have gained more than 70% . Can you please check and u...

  • 838 Views
  • 0 replies
  • 3 kudos
Marco9898
by New Contributor II
  • 9217 Views
  • 2 replies
  • 3 kudos

Running SQL Data File through Notebook Python

I am attempting to run larger sql scripts through Databricks Notbook and export data to a file. For the most part the Notebook works when the sql script is a single SELECT statement. However, if the sql file is more complicated such as involving the ...

  • 9217 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Marco Perez​ Does @Jose Gonzalez​ response answer your question? If yes, would you be happy to mark it as best so that other members can find the solution more quickly?We'd love to hear from you.Thanks!

  • 3 kudos
1 More Replies
Bevin
by Databricks Partner
  • 3677 Views
  • 5 replies
  • 9 kudos

Resolved! What is the best way to manage the growth of the default storage location?

What is the best way to manage the growth of the default storage location in a shared workspace? I have a number of users logging in and working on different clusters in a single workspace. I understand that all the results and notebooks are saved t...

  • 3677 Views
  • 5 replies
  • 9 kudos
Latest Reply
Anonymous
Not applicable
  • 9 kudos

Hi @Bevin Maragh​ Glad to hear that! It's a request that mark an answer as best. It will be highly appreciable.Thanks and Regards

  • 9 kudos
4 More Replies
Mado
by Valued Contributor II
  • 3227 Views
  • 2 replies
  • 3 kudos

When should I use ".start()" with writeStream?

Hi,I am practicing with Databricks. In sample notebooks,I have seen different use of writeStream with or without ".start()" method. Samples are below:Without .start() spark.readStream   .format("cloudFiles")   .option("cloudFiles.f...

  • 3227 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Mohammad Saber​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks

  • 3 kudos
1 More Replies
Mado
by Valued Contributor II
  • 3853 Views
  • 2 replies
  • 3 kudos

Question about "foreachBatch" to remove duplicate records when streaming data

Hi,I am practicing with Databricks sample notebook published here:https://github.com/databricks-academy/advanced-data-engineering-with-databricksIn one of the notebooks (ADE 3.1 - Streaming Deduplication) (URL), there is a sample code to remove dupli...

  • 3853 Views
  • 2 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

Hi @Mohammad Saber​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question first. Or else bricksters will get back to you soon. Thanks

  • 3 kudos
1 More Replies
Lalli
by Databricks Partner
  • 1462 Views
  • 1 replies
  • 2 kudos

Databricks notebook is not printing image in python

When I run a python command that should display an image. The image is not showing, and I do not get any error message.

  • 1462 Views
  • 1 replies
  • 2 kudos
Latest Reply
User16753725469
Databricks Employee
  • 2 kudos

Could you please share the code snippet you are using to print the image?

  • 2 kudos
data_explorer
by New Contributor II
  • 1928 Views
  • 1 replies
  • 2 kudos
  • 1928 Views
  • 1 replies
  • 2 kudos
Latest Reply
User16753725469
Databricks Employee
  • 2 kudos

Please refer: https://www.databricks.com/blog/2021/05/26/introducing-databricks-unity-catalog-fine-grained-governance-for-data-and-ai-on-the-lakehouse.html

  • 2 kudos
Nisarg_Khamar
by New Contributor
  • 2103 Views
  • 1 replies
  • 1 kudos

Did not get bedge and 100 points for Databricks Lakehouse Platform Accreditation exam

I have completed databricks Fundamentals of the Databricks Lakehouse Platform Accreditation exam but i did not get bedge and 100 points in my account.Could someone please help.

  • 2103 Views
  • 1 replies
  • 1 kudos
Latest Reply
pkgltn
New Contributor III
  • 1 kudos

Same question here too. I had the accreditation done, but I do not know how to redeem those 100 points

  • 1 kudos
stupendousenzio
by New Contributor III
  • 3751 Views
  • 4 replies
  • 7 kudos

Unable to access workspace after the trial period in databricks in Google cloud provider.

I was using the trial period in databricks for 14 days and had some important notebooks where I had made all the changes. Now I have extended the service and have subscribed for databricks in GCP. When I enter the workspace section I cannot see the w...

  • 3751 Views
  • 4 replies
  • 7 kudos
Latest Reply
Anonymous
Not applicable
  • 7 kudos

Hi @Aditya Aranya​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...

  • 7 kudos
3 More Replies
Labels