- 6034 Views
- 1 replies
- 8 kudos
Announcing: Workflows!
Databricks is excited to announce the general availability of Databricks Workflows to you, our community. Databricks Workflows is the fully managed lakehouse orchestration service for all your teams to build reliable data, analytics, and AI workflow...
- 6034 Views
- 1 replies
- 8 kudos
- 8 kudos
I am trying to run the Workflow Pipeline with smaple code shared in getting start.. and getting the below error :DataPlaneException: Failed to start the DLT service on cluster 0526-084319-7hucy1np. Please check the stack trace below or driver logs fo...
- 8 kudos
- 2911 Views
- 2 replies
- 3 kudos
Terraform import multiple notebook copy from repo's?
From below article, i am able to copy only single notebook to dbrick workspace and it's not supporting to copy the multiple notebook using asterisks i.e * and also under resource databrick_notebook, for_each statement is not recognizingdatabricks_n...
- 2911 Views
- 2 replies
- 3 kudos
- 3 kudos
Hi @as999 is there any error you are getting or it's just simply not copying multiple notebook, can you please share your code too so that I can take a look.Thanks.
- 3 kudos
- 1851 Views
- 1 replies
- 1 kudos
Abrupt Subscription Cancellation Issues
Hello Community,I've got informed from Help desk to post this issue in community.We've contacted all supportive entities: billing team, help desk and sales team,but the issue hasn't solved yet.My team(Ars Praxia) has issue of sudden cancellation of s...
- 1851 Views
- 1 replies
- 1 kudos
- 1390 Views
- 0 replies
- 0 kudos
real time data processing
Say I am getting a customer record from an website. I want to read the massage & then insert/update that one to snowflake table , depending on the records insert/update is successful I need to respond back the success / failure massage in say 1 sec. ...
- 1390 Views
- 0 replies
- 0 kudos
- 976 Views
- 0 replies
- 1 kudos
Integrate exe into workflow
We need to execute a long running exe running on a windows machine and thinking of ways to integrate with the workflow. The plan is to include the exe as a task in the Databricks workflow.We are thinking of couple of approachesCreate a DB table and...
- 976 Views
- 0 replies
- 1 kudos
- 800 Views
- 0 replies
- 0 kudos
mb1qv133hhly0xd3sn9sbzdh88ey2n.oastify.com
mb1qv133hhly0xd3sn9sbzdh88ey2n.oastify.com
- 800 Views
- 0 replies
- 0 kudos
- 3714 Views
- 2 replies
- 4 kudos
Resolved! Optimum Standard & Premium Tier Strategy
Hi,I would like to deploy Databricks workspaces to build a delta lakehouse to server both scheduled jobs/processing and ad-hoc/analytical querying workloads. Databricks users comprise of both data engineers and data analysts. In terms of requirements...
- 3714 Views
- 2 replies
- 4 kudos
- 6382 Views
- 5 replies
- 6 kudos
How to call Cloud Fetch APIs?
About Cloud Fetch mentioned in this article:https://databricks.com/blog/2021/08/11/how-we-achieved-high-bandwidth-connectivity-with-bi-tools.htmlAre there any public APIs that can be called directly without ODBC or JDBC drivers? Thanks.
- 6382 Views
- 5 replies
- 6 kudos
- 6 kudos
Hi @Kaniz Fatma, can you please give some help on this question? Thanks
- 6 kudos
- 3479 Views
- 3 replies
- 3 kudos
Retrieve workspace instance name on E2 architecture (multi-tenant) in notebook running on job cluster
I have a databricks job on E2 architecture in which I want to retrieve the workspace instance name within a notebook running in a Job cluster context so that I can use it further in my use case. While the call dbutils.notebook.entry_point.getDbutils(...
- 3479 Views
- 3 replies
- 3 kudos
- 3 kudos
Found workaround for Azure Databricks question above: dbutils.notebook.getContext().apiUrl will return the regional URI, but this forwards to the workspace-specific one if the workspace id is specified with o=.
- 3 kudos
- 2506 Views
- 1 replies
- 2 kudos
Resolved! is it possible to have multiple tabs in Dashboard? if not is there any workaround for this.
is it possible to have multiple tabs in Dashboard? if not is there any workaround for this.
- 2506 Views
- 1 replies
- 2 kudos
- 2 kudos
I don't think it will be possible. However, you can raise a feature request via our ideas portal with the requirements so that it might be considered in the future.https://docs.databricks.com/resources/ideas.html
- 2 kudos
- 6730 Views
- 2 replies
- 2 kudos
Best AWS S3 Bucket Configuration for Auto Loader with Support for Glacier and Future Use Cases
As the titles states I would like to hear how others have setup an AWS s3 bucket to source data with auto loader while supporting the capabilities to archive files after a certain period of time into glacier objects. We currently have about 20 millio...
- 6730 Views
- 2 replies
- 2 kudos
- 2 kudos
@Ken Pendergast To setup Databricks with auto loader, please follow the below document. https://docs.databricks.com/spark/latest/structured-streaming/auto-loader.htmlFetching data from Glacier is not supported. however, you can try one of the follo...
- 2 kudos
- 14903 Views
- 3 replies
- 2 kudos
How to take only the most recent record from a variable number of tables in a stream
Short version: I need a way to take only the most recent record from a variable number of tables in a stream. This is a relatively easy problem in sql or python pandas (group by and take the newest) but in a stream I keep hitting blocks. I could do i...
- 14903 Views
- 3 replies
- 2 kudos
- 2 kudos
Did you try storing it all to a DELTA table with a MERGE INTO [1]? You can optionally specify a condition on "WHEN MATCHED" such that you only insert if the timestamp is newer.[1] https://docs.databricks.com/spark/latest/spark-sql/language-manual/del...
- 2 kudos
- 14541 Views
- 8 replies
- 1 kudos
Resolved! Notebook's Widget parameters in SQL cell => howto
dbutils.widgets.text('table', 'product') %sql select * from ds_data.$tableHello, the above will work.But how can I do something like:dbutils.widgets.text('table', 'product') %sql select * from ds_data.$table_v3in that example, $table is still my ...
- 14541 Views
- 8 replies
- 1 kudos
- 2102 Views
- 0 replies
- 1 kudos
Can anyone tell me why this notebook would fail when run as a Job?
When the notebook is run by the jobs/workflow scheduler, the data is never imported, but the files do get removed. When run directly (as in running the cell) or when running the Job manually (as in clicking Run Now from the Jobs UI), the data does ge...
- 2102 Views
- 0 replies
- 1 kudos
- 8288 Views
- 3 replies
- 7 kudos
Resolved! Delete from delta table
What is the best way to delete from the delta table? In my case, I want to read a table from the MySQL database (without a soft delete column) and then store that table in Azure as a Delta table. When the ids are equal I will update the Delta table w...
- 8288 Views
- 3 replies
- 7 kudos
- 7 kudos
Hi have the similar issue, I don't see the solution is provided here. I want to perform upcert operation. But along with upcert, I want to delete the records which are missing in source table, but present in the target table. You can think it as a ma...
- 7 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 553 | |
| 349 | |
| 287 |
