- 1737 Views
- 1 replies
- 2 kudos
Jobs Data Pipeline Runtime Increase Significantly
Hi, I am facing an issue where one of my jobs taking so long since certain time, previously its only needs less than 1 hour to run a batch job that load json data and do a truncate and load to a delta table, but since june 2nd, it become so long that...
- 1737 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @krisna math​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 2 kudos
- 1756 Views
- 1 replies
- 1 kudos
Does JDBC and ODBC connection to Databricks Cluster or SQL Warehouse encrypted?
Hello, our consumers (Dell Boomi and other apps) needs data from Databricks. Databricks provides JDBC and ODBC drivers. Does JDBC and ODBC connection to Databricks Cluster or SQL Warehouse encrypted? I am talking about the data in-transit between Dat...
- 1756 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Rajesh Paul​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 1696 Views
- 1 replies
- 1 kudos
how to execute "Build your Chat Bot with Dolly Demo" in my own VM?
I am trying to execute Build your Chat Bot with Dolly Demo using my own VM. At the first steps they are executing this command %run ./_resources/00-init $catalog=hive_metastore $db=dbdemos_llm which is -as I understand- calling another python script...
- 1696 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @alaa migdady​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 3776 Views
- 1 replies
- 2 kudos
apply a function across multiple smaller dataframes created from one big dataframe in scala
The dataframe 'big_df' looks like the below| id| index| timestamp||:---- |:------:| -----:|| abc| 1| 11:00:00|| abc| 1| 11:00:10|| abc| 1| 11:00:20|| abc| 1| 11:00:30|| abc| 1| 11:00:40|| abc| 1| 11:00:50|| abc| 2| 11:01:00|| abc| 2| 11:01:10|| abc| ...
- 3776 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Sandesh Puligundla​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 2 kudos
- 1862 Views
- 1 replies
- 2 kudos
How to change databricks email alert notifications subject line?
Currently, our email notification subject shows for error in below format:<[AWS Account]> Error in run <RUN ID> of <Job Name>In our current databricks environment we have multiple environment jobs such as Dev/QA/UAT/Prod and it is very hard for us to...
- 1862 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Sanath Pabba​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 2 kudos
- 1687 Views
- 1 replies
- 1 kudos
Alternative to the inbuilt Scan method of the Databricks SDK for Golang
The implementation of Scan method in the Databricks SDK .If the default value of any column is NULL, in that case the Scan method return exception.What can we use as the alternative to this?While implementing the logic with Scan method as per the doc...
- 1687 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Shivansh Bhatnagar​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 1779 Views
- 1 replies
- 0 kudos
Alternative to Scan method in Databricks SDK for Golang
We want to fetch multiple rows/records with 100+ columns(that are dynamic as per the request) but in the doc the way shown is to scan each column(through Scan method) one by one rather scanning the complete object at once which is making it very tedi...
- 1779 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Shivansh Bhatnagar​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 19418 Views
- 1 replies
- 1 kudos
Platform and Approach Comparison
Do anyone have structure and crisp comparison between benefits of performing MLOps using below ways and what are the strong areas of each platform:a) Standalone Databricks where all pipelines and orchestration done on Databricks and external third pa...
- 19418 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Saurabh Singh​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 1209 Views
- 1 replies
- 0 kudos
Is it possible to create sql statements (after return) with cte's in databricks scalar functions?
The function returns an integer as a result of the query
- 1209 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Luis Lopez​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 2990 Views
- 4 replies
- 2 kudos
error: at least one column must be specified for the table.
error: at least one column must be specified for the table.
- 2990 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @anand R​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us so we ca...
- 2 kudos
- 9726 Views
- 5 replies
- 0 kudos
Resolved! Incremental Load without Last Modified Date and Primary Key field in Azure Data Factory to create bronze data in data bricks
I am trying to do incremental load in azure data factory. Most of the tables in the Oracle database doesn't have last modified date and Primary key column. Is there any way to do incremental loading without last modified date and primary key column?
- 9726 Views
- 5 replies
- 0 kudos
- 0 kudos
Hi @Selva Kumar Ponnusamy​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please te...
- 0 kudos
- 3549 Views
- 2 replies
- 2 kudos
I would like to create a schedule in Databricks that runs a job every two weeks on Monday night 0:00
This seems impossible with the cron that databricks is using but maybe I am wrong? However, if this is not possible it seems to me a missing feature, and thereby would like to suggest this feature to you
- 3549 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @Eelke van Foeken​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.T...
- 2 kudos
- 21588 Views
- 11 replies
- 7 kudos
Resolved! Insert into query showing error
Im trying to insert data from table with 7 columns to table with 37 columns
- 21588 Views
- 11 replies
- 7 kudos
- 5861 Views
- 6 replies
- 8 kudos
Resolved! My exam has suspended , Need help Urgently (10/06/2023)
Hello Team, I encountered Pathetic experience while attempting my 1st DataBricks certification. Abruptly, Proctor asked me to show my desk, after showing he/she asked multiple times.. wasted my time and then suspended my exam. I want to file a compla...
- 5861 Views
- 6 replies
- 8 kudos
- 8 kudos
Hi @Varad Manglekar​ Glad to hear!Help us build a vibrant and resourceful community by recognizing and highlighting insightful contributions. Mark the best answers and show your appreciation!
- 8 kudos
- 3832 Views
- 4 replies
- 2 kudos
Resolved! Voucher not received
Hi, I have attended the recent Lakehouse webinar (May) and completed Lakehouse fundamentals but still have not received the certification voucher. Kindly help.
- 3832 Views
- 4 replies
- 2 kudos
- 2 kudos
Thanks @Vidula Khanna​ for your help. I have raised a ticket as suggested for this.
- 2 kudos
| User | Count |
|---|---|
| 1619 | |
| 790 | |
| 485 | |
| 349 | |
| 287 |
