- 2690 Views
- 2 replies
- 3 kudos
Efficiently orchestrate data bricks jobs
Hi Team,How efficiently can orchestrate data bricks jobs which involve a lot of transformations, dependencies, and complexity?At source have a lot of SSIS packages that have complex dependencies and more transformation. We have the following opti...
- 2690 Views
- 2 replies
- 3 kudos
- 3 kudos
My question is, how do we reliably orchestrate multiple Databricks Jobs/Workflows that are running in a mixed latency and can write to the same silver and gold delta tables? Could you please suggest the best approach and practices for the same?
- 3 kudos
- 8730 Views
- 6 replies
- 4 kudos
Git Integration: selective check-in not working
We started noticing the problem about 3 weeks ago. Databricks' Git GUI fails to commit only the selected files. All the files that have been modified, added or removed since that last commit will get checked in even if it de-selected in the GUI.Has ...
- 8730 Views
- 6 replies
- 4 kudos
- 4 kudos
Hi @Lawrence Chen​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 4 kudos
- 9867 Views
- 4 replies
- 2 kudos
org.apache.spark.SparkException: Job aborted due to stage failure: Authorized committer failed while pushing dataframe to azure cosmos db.
I am writing data to the azure cosmos db using OLTP connector using below codecfg["spark.cosmos.write.strategy"]="ItemOverwrite" json_df.write.format("cosmos.oltp").options(**cfg).mode("APPEND").save()I am getting below error Please let me know i...
- 9867 Views
- 4 replies
- 2 kudos
- 2 kudos
Hi @Manasa Kalluri​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedba...
- 2 kudos
- 3082 Views
- 4 replies
- 2 kudos
Resolved! Adding tags to jobs from Tableau / Python (ODBC)
Hi all,We are using Azure Databricks.We would like to see if we can track usage of jobs initiated from Tableau /Python (via simba spark ODBC driver) One way we can think of is to add tag to the job. But we are not sure if we can add tags to job when ...
- 3082 Views
- 4 replies
- 2 kudos
- 2 kudos
@Vidula Khanna​ We can add custom values to the job clusters as well. When you create the job task, click on edit cluster details which will take you to the cluster configuration where you can add the custom tags. In case of Tableau queries, it would...
- 2 kudos
- 5681 Views
- 7 replies
- 4 kudos
Didn't receive Databricks badge
I just completed the Databricks Lakehouse Fundamentals exam, but while redirecting to credentials.databricks.com. I'm not able to find any badge provided to me for completion. Can you help me out? @Vidula Khanna​
- 5681 Views
- 7 replies
- 4 kudos
- 4 kudos
check your mail. yesterday I have completed they shared me through registered email
- 4 kudos
- 1908 Views
- 3 replies
- 0 kudos
Am using databricks trying to login​
Am using databricks trying to login​
- 1908 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Ige Makanjuola​ Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...
- 0 kudos
- 9262 Views
- 5 replies
- 5 kudos
Is it not possible to create multiple tables on same location in unity catalog?
I have tried 2 tables with different catalogs, schemas, column names and it throws LOCATION_OVERLAP. In spark we can create as many tables as needed on the same location.
- 9262 Views
- 5 replies
- 5 kudos
- 5 kudos
Hi @Ashish Singh​ I'm sorry you could not find a solution to your problem in the answers provided.Our community strives to provide helpful and accurate information, but sometimes an immediate solution may only be available for some issues.I suggest p...
- 5 kudos
- 2747 Views
- 3 replies
- 3 kudos
Level of databricks spark certificate exam
I am willing to take spark certificate exam , i have solved the practice test and it was very good , but i am afraid that the level of exam is harder than the practice exam . So does it really harder ?
- 2747 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Aly Ayman​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers you...
- 3 kudos
- 3929 Views
- 2 replies
- 1 kudos
Resolved! DLT Security in Transit and in Rest
do you have docs that explain more specifics? is it end-to-end encrypted in transit? for in rest, is it just the encryption we get from e.g. S3?
- 3929 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @Frank Munz​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 1 kudos
- 3147 Views
- 1 replies
- 5 kudos
When are you gonna fix this site?
Unacceptable user experience for this forum: What markdown support?My colleagues agree the email verification looks sketchy.The website is SUPER slow. (10-56 seconds per page)Uses color red for email verified?!No feedback on repeated logins.The fonts...
- 3147 Views
- 1 replies
- 5 kudos
- 5 kudos
Hi @Elon Musk​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedback wi...
- 5 kudos
- 1897 Views
- 2 replies
- 1 kudos
I would like to get rid of the error
the SPN we use for the mount points has access to the dataset in question, but for some reason I get this errorPlease find the attached screenshot for the error details.
- 1897 Views
- 2 replies
- 1 kudos
- 1 kudos
Hi @sai chandu palkapati​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from y...
- 1 kudos
- 6636 Views
- 3 replies
- 0 kudos
Resolved! Job aborted due to stage failure: ModuleNotFoundError
I'm getting this Failure Reason on a fairly simple streaming job. I'm running the job in a notebook. The notebook relies on a python module that I'm syncing to DBFS with `dbx`. Within the notebook generally, the module is available, i.e. `import mymo...
- 6636 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Benjamin Dean​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Than...
- 0 kudos
- 4808 Views
- 3 replies
- 4 kudos
Resolved! Load data into Azure SQL Database from Azure Databricks ( restricted table not a whole workspace tables)
Hi ,I want to share limited tables in my databricks workspace and users will connect to my databricks through Azure Data factory and will load data into Azure SQL. Is this possible using Delta Sharing? Or any other method or tool?
- 4808 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @Dayananthan Marimuthu​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your...
- 4 kudos
- 4373 Views
- 3 replies
- 3 kudos
Unable to find input by typing input in the Multiselect Widget
In the AWS databricks widgets.multiselect, I'm unable to find input by typing input in the mulitselect bar. It was working before. Although I can find the inputs by scrolling down the list, it's annoying if the list is long.​​Here's my script:measlis...
- 4373 Views
- 3 replies
- 3 kudos
- 3 kudos
Hi @Philip Teu​ Hope all is well! Just wanted to check in if you were able to resolve your issue and would you be happy to share the solution or mark an answer as best? Else please let us know if you need more help. We'd love to hear from you.Thanks!
- 3 kudos
- 7773 Views
- 3 replies
- 4 kudos
Resolved! Confusion in string comparison
Hello expertI am new to spark. I am using same price of code but getting different resultsWhen i am using below piece of code, i am getting errorpy4j.Py4JException: Method or([class java.lang.String]) does not existdf.filter(F.col("state").isNull() ...
- 7773 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @Saswata Dutta​ Thank you for your question! To assist you better, please take a moment to review the answer and let me know if it best fits your needs.Please help us select the best solution by clicking on "Select As Best" if it does.Your feedbac...
- 4 kudos
| User | Count |
|---|---|
| 1644 | |
| 1050 | |
| 1005 | |
| 791 | |
| 553 |
