- 14707 Views
- 2 replies
- 3 kudos
Resolved! Default Query Limit 1000
By default, we return back up to 1000 query results when a user runs a cell in Databricks. E.g., if you run display(storeData) and you have ten million customers, the UI will show the first 1000 results. If you graph that by age of customer, similarl...
- 14707 Views
- 2 replies
- 3 kudos
- 3 kudos
This is simple in Databricks SQL, just uncheck LIMIT 1000 in the drop down.
- 3 kudos
- 7474 Views
- 2 replies
- 2 kudos
Resolved! Default Query Limit 1000
Is there any way to change the 1000 for the display row limit at workspace, cluster and notebook level?
- 7474 Views
- 2 replies
- 2 kudos
- 2 kudos
This is simple in Databricks SQL, just uncheck LIMIT 1000 in the drop down.
- 2 kudos
- 8554 Views
- 4 replies
- 8 kudos
Resolved! Create Databricks tables dynamically
Hi, I would like to be able to do something like this...create table if not exists table1using parquetlocation = '/mnt/somelocationsome location needs to be a concatenation of static and code generated string. Documentation suggests that location onl...
- 8554 Views
- 4 replies
- 8 kudos
- 8 kudos
FString Python can be used. example > spark.sql(f"CREATE TABLE {table_name} (id INT, name STRING, value DOUBLE, state STRING)")
- 8 kudos
- 2816 Views
- 2 replies
- 5 kudos
Resolved! Databricks set up in Prod environment
Hi! can anyone please help me with a documentation which can help me set up integration between data bricks with AWS without a QuickStart default cloud formation template. I would want to use my own CFT rather than using the default due to security ...
- 2816 Views
- 2 replies
- 5 kudos
- 5 kudos
Hi @Atul S​ ,I think that terraform is recommended way to go with Databricks deployment. I mean it's also supported now by the Databricks support.I haven't look much on the CloudFormation setup, because we decided to go with the Terraform in the comp...
- 5 kudos
- 11052 Views
- 1 replies
- 4 kudos
Resolved! Accept widget value during runtime from user
list1 = ('alpha', 'beta', 'gamma', 'eta', 'Theta')list2 = ('alpha', 'beta')df1 = spark.createDataFrame(list1, 'String').withColumnRenamed('value', 'dataset')df2 = spark.createDataFrame(list2, 'String').withColumnRenamed('value', 'dataset')df = df1.ex...
- 11052 Views
- 1 replies
- 4 kudos
- 4 kudos
Every time the user change selection in the widget, you get that selection—behavior after the dropdown change you can define in widget settings.
- 4 kudos
- 6470 Views
- 0 replies
- 5 kudos
Unable to create a cluster in Databricks with a new subscription
I have upgraded my expired Student subscription to 'Azure subscription 1' in Azure portal today. I want to use Databricks for personal projects as pay-as-you-go.When I go to my Databricks workspace and to my notebook and try to create a cluster,Comp...
- 6470 Views
- 0 replies
- 5 kudos
- 1146 Views
- 1 replies
- 1 kudos
Attended 18Oct22 webinar but didnt recieve voucher
Hi @Kaniz Fatma​ Attended the webinar on 18th Oct uploaded the datalakehouse cert but didnt recieve voucher as of now,also didnt recieve data engg associate certificate.​Please help!!
- 1146 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Arindam Bose​ Just a friendly- follow-up, have you got your certification and badge? If yes, please mark the answer as best.Thanks and Regards
- 1 kudos
- 2211 Views
- 0 replies
- 2 kudos
I will appreciate if there is a black friday deal on the Databricks Data Engineering Associate course or if I can get a personal coupon.
I will appreciate if there is a black friday deal on the Databricks Data Engineering Associate course or if I can get a personal coupon.
- 2211 Views
- 0 replies
- 2 kudos
- 3095 Views
- 0 replies
- 6 kudos
Catch when a notebook fails and terminate command in threaded parallel notebook run
I have a command that is running notebooks in parallel using threading. I want the command to fail whenever one of the notebooks that is running fails. Right now it is just continuing to run the command.Below is the command line that I'm currently ru...
- 3095 Views
- 0 replies
- 6 kudos
- 5090 Views
- 5 replies
- 8 kudos
Autoloader cluster
I wanted to setup Autoloader to process files from Azure Data Lake (Blob) automatically whenever new files arrive. For this to work, I wanted to know if AutoLoader requires that the cluster is on all the time.
- 5090 Views
- 5 replies
- 8 kudos
- 8 kudos
@Kaniz Fatma​ , If my cluster is not active, and I have uploaded 50 files in storage location, then where this Auto Loader will list out these 50 files. Will it use any checkpoint location, if yes, then how can I set the checkpoint location in Cloud ...
- 8 kudos
- 1346 Views
- 0 replies
- 1 kudos
I have created account but forgot to check community edition option while signing up. How this can be solved now.
I have created account but forgot to check community edition option while signing up. How this can be solved now.
- 1346 Views
- 0 replies
- 1 kudos
- 2752 Views
- 6 replies
- 4 kudos
Resolved! Databricks Certified Data Engineer Associate Certificate not received
Hello,I passed Databricks Certified Data Engineer Associate on 30 October 2022, but still didn't receive my certificate/badge. Could you please help me to obtain it?Regards,Ali
- 2752 Views
- 6 replies
- 4 kudos
- 4 kudos
Hi @ali.ganbarov ali.ganbarov​ We are really sorry for the delays.The certification has been issued but due to the lag in the system, it is taking time. Our team is working on it. Please visit the credible site once.Thanks and Regards
- 4 kudos
- 5837 Views
- 4 replies
- 7 kudos
Schedule job runs with different parameters?
Is it possible to schedule different runs for job with parameters? I have a notebook that generates data based on the supplied parameter but would like to schedule runs instead of manually starting them. I assume that this would be possible using the...
- 5837 Views
- 4 replies
- 7 kudos
- 7 kudos
You can pass parameters for your task. Each task type has different requirements for formatting and passing the parameters. https://docs.databricks.com/workflows/jobs/jobs.html#create-a-jobREST API can also pass parameters fro jobs. Tokens replace pa...
- 7 kudos
- 2437 Views
- 1 replies
- 2 kudos
Resolved! Cluster Configuration Best Practices
I have a cluster with the configuration of 400 GB RAM, 160 Cores.Which of the following would be the ideal configuration to use in case of one or more VM failures?Cluster A: Total RAM 400 GB Total Cores 160 Total VMs: 1 400 GB/Exec & 160 c...
- 2437 Views
- 1 replies
- 2 kudos
- 2 kudos
@Santhosh Raj​ can you please confirm cluster sizes you are taking are related to driver and worker node. how much you want to allocate to Driver and Worker? once we are sure about type of driver and worker we would like to pick, we need to enable au...
- 2 kudos
- 6309 Views
- 3 replies
- 3 kudos
Resolved! ipywidgets stopped displaying today
everything was working yesterday, but today it stopped working as below: The example from the DB website does not work either with the same error. The page source says This is affecting my work~~~a bit annoying, is DB people going to look into this ...
- 6309 Views
- 3 replies
- 3 kudos
- 3 kudos
Today its back to work! I got a pop up window sayingthis should be the reason it was broken
- 3 kudos
| User | Count |
|---|---|
| 1611 | |
| 768 | |
| 348 | |
| 286 | |
| 252 |
