cancel
Showing results for 
Search instead for 
Did you mean: 
Community Platform Discussions
Connect with fellow community members to discuss general topics related to the Databricks platform, industry trends, and best practices. Share experiences, ask questions, and foster collaboration within the community.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

priyakant1
by New Contributor II
  • 724 Views
  • 1 replies
  • 0 kudos

Suspension of Data Engineer Professional exam

Hi Databricks TeamI had scheduled my exam on 6th sep 2023, during exam same pop up came up, stating that I am looking in some other direction. I told them that my laptop mouse is not working properly, so I was looking at it. But still they suspended ...

  • 724 Views
  • 1 replies
  • 0 kudos
Latest Reply
sirishavemula20
New Contributor III
  • 0 kudos

Hi @priyakant1 ,Have you got any response from the team, like did they reschedule your exam?

  • 0 kudos
sirishavemula20
by New Contributor III
  • 2026 Views
  • 1 replies
  • 0 kudos

My exam has suspended , Need help Urgently (21/08/2023)

Hello Team,I encountered Pathetic experience while attempting my 1st DataBricks certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam.I want to file a complain...

  • 2026 Views
  • 1 replies
  • 0 kudos
Latest Reply
sirishavemula20
New Contributor III
  • 0 kudos

Sub: My exam Datbricks Data Engineer Associate got suspended_need immediate help please (10/09/2023)I encountered Pathetic experience while attempting my DataBricks Data engineer certification. Abruptly, Proctor asked me to show my desk, after showin...

  • 0 kudos
Policepatil
by New Contributor III
  • 2382 Views
  • 1 replies
  • 1 kudos

Resolved! Records are missing while filtering the dataframe in multithreading

 Hi, I need to process nearly 30 files from different locations and insert records to RDS. I am using multi-threading to process these files parallelly like below.   Test data:               I have configuration like below based on column 4: If colum...

Policepatil_0-1694077661899.png
  • 2382 Views
  • 1 replies
  • 1 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 1 kudos

Looks like you are comparing to strings like "1", not values like 1 in your filter condition. It's hard to say, there are some details missing like the rest of the code and the DF schema, and what output you are observing.

  • 1 kudos
VMeghraj
by New Contributor II
  • 1261 Views
  • 1 replies
  • 0 kudos

Increase cores for Spark History Server

By default SHS uses spark.history.fs.numReplayThreads = 25% of avaliable cores (Number of threads that will be used by history server to process event logs)How can we increase the number of cores for Spark History Server ?

  • 1261 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
By default SHS uses spark.history.fs.numReplayThreads = 25% of avaliable cores (Number of threads that will be used by history server to process event logs)How can we increase the number of cores for Spark History Server ?

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
Policepatil
by New Contributor III
  • 5767 Views
  • 0 replies
  • 0 kudos

Is it good to process files in multithreading?

Hi,I need to process nearly 30 files from different locations and insert records to RDS.I am using multi-threading to process these files parallelly like below. def process_files(file_path):    <process files here>    1. Find bad records based on fie...

  • 5767 Views
  • 0 replies
  • 0 kudos
bachan
by New Contributor II
  • 1369 Views
  • 1 replies
  • 0 kudos

Data Insertion

Scenario: Data from blob storage to SQL db once a week.I have 15(from current date to next 15 days) days data into the blob storage, stored date wise in parquet format, and after seven days the next 15 days data will be inserted. Means till 7th day t...

  • 1369 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Scenario: Data from blob storage to SQL db once a week.I have 15(from current date to next 15 days) days data into the blob storage, stored date wise in parquet format, and after seven days the next 15 days data will be inserted. Means till 7th day t...

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
Gilg
by Contributor II
  • 2941 Views
  • 1 replies
  • 0 kudos

Server error: OK - Notebook

Hi I am currently having a weird notebook behavior. Every time I write, I am getting the following error. My gut feeling is that it causes by the Auto-save feature.Cheers,Gil

GilGonong_0-1694040831818.png
  • 2941 Views
  • 1 replies
  • 0 kudos
Latest Reply
" src="" />
This widget could not be displayed.
This widget could not be displayed.
This widget could not be displayed.
  • 0 kudos

This widget could not be displayed.
Hi I am currently having a weird notebook behavior. Every time I write, I am getting the following error. My gut feeling is that it causes by the Auto-save feature.Cheers,Gil

This widget could not be displayed.
  • 0 kudos
This widget could not be displayed.
Sivaji
by New Contributor
  • 714 Views
  • 1 replies
  • 0 kudos

Databricks data engineer associate Exam got suspended.

Hello Team, I encountered Pathetic experience while attempting my 1st DataBricks certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam. I want to file a compla...

Community Platform Discussions
Data engineer Associate
Exam.
  • 714 Views
  • 1 replies
  • 0 kudos
Latest Reply
Cert-Team
Esteemed Contributor
  • 0 kudos

Hi @Sivaji Sorry to hear you had a bad experience, and that you got a slow response here in the community. I see that you have taken and passed the exam, Congratulations!For the future, our support team handles cases from here first so it tends to be...

  • 0 kudos
JRL
by New Contributor II
  • 1064 Views
  • 1 replies
  • 0 kudos

Github "Danger Zone"

There is a "Danger zone" appearing in the Github indicating that the repositories I share on Databricks should be Suspended and possibly that Databricks should be uninstalled.   This may be something standard in Github.  Has anyone run across it?

DangerZone.PNG
  • 1064 Views
  • 1 replies
  • 0 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 0 kudos

It's not telling you that you should do these things. It's telling you that you may break stuff by doing these things. Yes the "Danger Zone" is a thing on Github, it tries to warn you before you do things like click to delete a repo.

  • 0 kudos
jermaineharsh
by New Contributor II
  • 481 Views
  • 0 replies
  • 0 kudos

How to switch from free trial to Community Edition of Databricks in my Azure workspace?

hello,I am trying to switch into Databricks Community Edition after a 14 day trial. I was able to register, but when I try to start my new cluster, I get an error message, "Cluster start feature is currently disabled, and the cluster does not run".In...

  • 481 Views
  • 0 replies
  • 0 kudos
Picci
by New Contributor III
  • 3023 Views
  • 3 replies
  • 3 kudos

Resolved! Databricks community edition still available?

Is Databricks platform still available in its Community edition (outside Azure, AWS or GCP)? Can someone share the updated link?Thanks,Elisa

  • 3023 Views
  • 3 replies
  • 3 kudos
Latest Reply
jamescw
New Contributor II
  • 3 kudos

Look : it is still available but afaik always linked to azure/gcp/aws

  • 3 kudos
2 More Replies
bento
by New Contributor
  • 12250 Views
  • 1 replies
  • 1 kudos

Resolved! Notebook Langchain ModuleNotFoundError: No module named 'langchain.retrievers.merger_retriever'

Hi,As mentioned in the title, receiving this error despite%pip install --upgrade langchainSpecific line of code:from langchain.retrievers.merger_retriever import MergerRetriever All other langchain import works when this is commented out. Same line w...

  • 12250 Views
  • 1 replies
  • 1 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 1 kudos

More specifically: langchain releases a new update every few days, and it is likely that you are using code or a library that needs a later version of langchain than you have (or, perhaps, a later version that removed whatever part of langchain you r...

  • 1 kudos
ankit_batra1
by New Contributor
  • 1268 Views
  • 2 replies
  • 1 kudos

Databricks notebook execution using only one task

I am running a databricks notebook. While running, i only see one task on one worker getting started. My cluster has min 6 workers but seems like they are not getting used.I am performing a read operation from Cosmos DB.Can someone please help me her...

  • 1268 Views
  • 2 replies
  • 1 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 1 kudos

If your code does not use Spark, it will not use any machines except the driver. If you're using Spark but your source data that you operate on has 1 partition, there will be only 1 task. Hard to say more without knowing what you are doing in more de...

  • 1 kudos
1 More Replies
FutureLegend
by New Contributor III
  • 2865 Views
  • 2 replies
  • 1 kudos

MIT License and Fine-tuning

Some questions related to fine-tuning and the MIT License, I read the MIT license but still confusing about some points.If I fine-tune the Dolly-v2 model, say using LoRA and my own dataset,Do I "own" the fine-tuned model?Am I allow to change the name...

  • 2865 Views
  • 2 replies
  • 1 kudos
Latest Reply
sean_owen
Honored Contributor II
  • 1 kudos

I am not sure I agree with the discussion so far. While none of here are lawyers, I think it's fairly straightforward to reason about the licensing.You have created a combined, derivative work from the Dolly weights in this case. You have copyright i...

  • 1 kudos
1 More Replies

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.

If there isn’t a group near you, start one and help create a community that brings people together.

Request a New Group
Top Kudoed Authors