- 11768 Views
- 1 replies
- 4 kudos
Resolved! Accept widget value during runtime from user
list1 = ('alpha', 'beta', 'gamma', 'eta', 'Theta')list2 = ('alpha', 'beta')df1 = spark.createDataFrame(list1, 'String').withColumnRenamed('value', 'dataset')df2 = spark.createDataFrame(list2, 'String').withColumnRenamed('value', 'dataset')df = df1.ex...
- 11768 Views
- 1 replies
- 4 kudos
- 4 kudos
Every time the user change selection in the widget, you get that selection—behavior after the dropdown change you can define in widget settings.
- 4 kudos
- 7844 Views
- 0 replies
- 5 kudos
Unable to create a cluster in Databricks with a new subscription
I have upgraded my expired Student subscription to 'Azure subscription 1' in Azure portal today. I want to use Databricks for personal projects as pay-as-you-go.When I go to my Databricks workspace and to my notebook and try to create a cluster,Comp...
- 7844 Views
- 0 replies
- 5 kudos
- 1405 Views
- 1 replies
- 1 kudos
Attended 18Oct22 webinar but didnt recieve voucher
Hi @Kaniz Fatma​ Attended the webinar on 18th Oct uploaded the datalakehouse cert but didnt recieve voucher as of now,also didnt recieve data engg associate certificate.​Please help!!
- 1405 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Arindam Bose​ Just a friendly- follow-up, have you got your certification and badge? If yes, please mark the answer as best.Thanks and Regards
- 1 kudos
- 2640 Views
- 0 replies
- 2 kudos
I will appreciate if there is a black friday deal on the Databricks Data Engineering Associate course or if I can get a personal coupon.
I will appreciate if there is a black friday deal on the Databricks Data Engineering Associate course or if I can get a personal coupon.
- 2640 Views
- 0 replies
- 2 kudos
- 3479 Views
- 0 replies
- 6 kudos
Catch when a notebook fails and terminate command in threaded parallel notebook run
I have a command that is running notebooks in parallel using threading. I want the command to fail whenever one of the notebooks that is running fails. Right now it is just continuing to run the command.Below is the command line that I'm currently ru...
- 3479 Views
- 0 replies
- 6 kudos
- 6601 Views
- 5 replies
- 8 kudos
Autoloader cluster
I wanted to setup Autoloader to process files from Azure Data Lake (Blob) automatically whenever new files arrive. For this to work, I wanted to know if AutoLoader requires that the cluster is on all the time.
- 6601 Views
- 5 replies
- 8 kudos
- 8 kudos
@Kaniz Fatma​ , If my cluster is not active, and I have uploaded 50 files in storage location, then where this Auto Loader will list out these 50 files. Will it use any checkpoint location, if yes, then how can I set the checkpoint location in Cloud ...
- 8 kudos
- 1590 Views
- 0 replies
- 1 kudos
I have created account but forgot to check community edition option while signing up. How this can be solved now.
I have created account but forgot to check community edition option while signing up. How this can be solved now.
- 1590 Views
- 0 replies
- 1 kudos
- 3546 Views
- 6 replies
- 4 kudos
Resolved! Databricks Certified Data Engineer Associate Certificate not received
Hello,I passed Databricks Certified Data Engineer Associate on 30 October 2022, but still didn't receive my certificate/badge. Could you please help me to obtain it?Regards,Ali
- 3546 Views
- 6 replies
- 4 kudos
- 4 kudos
Hi @ali.ganbarov ali.ganbarov​ We are really sorry for the delays.The certification has been issued but due to the lag in the system, it is taking time. Our team is working on it. Please visit the credible site once.Thanks and Regards
- 4 kudos
- 7416 Views
- 4 replies
- 7 kudos
Schedule job runs with different parameters?
Is it possible to schedule different runs for job with parameters? I have a notebook that generates data based on the supplied parameter but would like to schedule runs instead of manually starting them. I assume that this would be possible using the...
- 7416 Views
- 4 replies
- 7 kudos
- 7 kudos
You can pass parameters for your task. Each task type has different requirements for formatting and passing the parameters. https://docs.databricks.com/workflows/jobs/jobs.html#create-a-jobREST API can also pass parameters fro jobs. Tokens replace pa...
- 7 kudos
- 3986 Views
- 1 replies
- 2 kudos
Resolved! Cluster Configuration Best Practices
I have a cluster with the configuration of 400 GB RAM, 160 Cores.Which of the following would be the ideal configuration to use in case of one or more VM failures?Cluster A: Total RAM 400 GB Total Cores 160 Total VMs: 1 400 GB/Exec & 160 c...
- 3986 Views
- 1 replies
- 2 kudos
- 2 kudos
@Santhosh Raj​ can you please confirm cluster sizes you are taking are related to driver and worker node. how much you want to allocate to Driver and Worker? once we are sure about type of driver and worker we would like to pick, we need to enable au...
- 2 kudos
- 8869 Views
- 3 replies
- 3 kudos
Resolved! ipywidgets stopped displaying today
everything was working yesterday, but today it stopped working as below: The example from the DB website does not work either with the same error. The page source says This is affecting my work~~~a bit annoying, is DB people going to look into this ...
- 8869 Views
- 3 replies
- 3 kudos
- 3 kudos
Today its back to work! I got a pop up window sayingthis should be the reason it was broken
- 3 kudos
- 3598 Views
- 0 replies
- 3 kudos
Cassandra connection from ADB
Hi , Could anyone please help to know the steps for connecting Cassandra from Azure Databricks? I have followed the steps in https://learn.microsoft.com/en-us/azure/databricks/_static/notebooks/azure/cassandra-azure.html But I am getting below error....
- 3598 Views
- 0 replies
- 3 kudos
- 1543 Views
- 0 replies
- 0 kudos
Databricks Jobs
Consider User A has deployed the job to prod. User B has scheduled the job through an external orchestration tool.User C has got the owner privileges from User A. Whose email id would be displayed while running the databricks job?
- 1543 Views
- 0 replies
- 0 kudos
- 3476 Views
- 2 replies
- 4 kudos
CMK for managed services automatic rotation
The docs for the CMK for workspace storage states:After you add a customer-managed key for storage, you cannot later rotate the key by setting a different key ARN for the workspace. However, AWS provides automatic CMK master key rotation, which rotat...
- 3476 Views
- 2 replies
- 4 kudos
- 4 kudos
Hi @Constantino Schillebeeckx​ , You can update/rotate CMK at a later time (on a running workspace). Please refer: https://docs.databricks.com/security/keys/customer-managed-keys-managed-services-aws.html?_ga=2.214562071.1895504292.1667411694-6435253...
- 4 kudos
- 40522 Views
- 33 replies
- 14 kudos
- 40522 Views
- 33 replies
- 14 kudos
- 14 kudos
I think plain python code will run with databricks connect (if it is a python program you are writing), and spark sql can be done by spark.sql(...).
- 14 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 554 | |
| 349 | |
| 287 |
