- 3639 Views
- 3 replies
- 1 kudos
Resolved! Autoload files in wide table format, but store it unpivot in Streaming Table
Hey, I get wide table format in csv file. Where each sensor have its own column. I want to store it in Delta Live Streaming Table. But since it is inefficient to process it and storage space, due to varying frequency and sensor amount. I want to tran...
- 3639 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @Simen Småriset​,Hope everything is going great.Just wanted to check in if you were able to resolve your issue. If yes, would you be happy to mark an answer as best so that other members can find the solution more quickly? If not, please tell us s...
- 1 kudos
- 8053 Views
- 2 replies
- 2 kudos
In the VSCode Databricks Extension how can one specify notebook parameters to a pass to a workflow job?
I have successfully used the VSCode extension for Databricks to run a notebook on a cluster from my IDE. However in order to test effectively without changing the source, I need a way to pass parameters to the workflow job.I have tried various ways ...
- 8053 Views
- 2 replies
- 2 kudos
- 2 kudos
@matthew kaplan​ I am not using widgets but what works is running it by pressing F5 in the python file you want to run.
- 2 kudos
- 3797 Views
- 3 replies
- 5 kudos
Issue with connection our Alation instance to Azure Databricks using the Unity Catalog OCF connector
When we try and do the above I am able to get the list of schemas. But when I select one to injest we are then getting issue due to it trying to access system.lineage.table_lineage. When I look in the System catalog I can only see a schema called inf...
- 3797 Views
- 3 replies
- 5 kudos
- 5 kudos
We are working with an agency that is a Databricks partner if that is what you mean.
- 5 kudos
- 3518 Views
- 3 replies
- 4 kudos
Resolved! Kept outputs
Hi everyone,I was wondering if perhaps someone of you could tell me which kinds of outputs are kept in a notebook after the cluster to which it is attached is terminated... Actually, I am asking it especially because I lost some visualization that I ...
- 3518 Views
- 3 replies
- 4 kudos
- 2509 Views
- 1 replies
- 2 kudos
Update query from Azure databricks to Oracle table
I need to update a single row on a on-prem Oracle table via jdbc connection.Please note, I don't want to append, just have to update a row, is it possible ?
- 2509 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Tony S​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 2 kudos
- 2805 Views
- 1 replies
- 0 kudos
- 2805 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Sourav Nandy​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 9726 Views
- 1 replies
- 2 kudos
getting error while runction show function
I was using String indexer, while fitting, transforming I didn't get any erro. but While runnign show function I am getting error, I mention the error beloworg.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 45.0 failed...
- 9726 Views
- 1 replies
- 2 kudos
- 2 kudos
Hi @Bojja Giri​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 2 kudos
- 31145 Views
- 2 replies
- 2 kudos
Resolved! Am I being charged for Starter Warehouse Pro?
When I go to add data, I see that the Starter Warehouse Pro cluster spun up after the first use and has been there for a long time. It does not show in my clusters and I can't find a way to shut it down. Am I being charged for this? If so, how do I s...
- 31145 Views
- 2 replies
- 2 kudos
- 17063 Views
- 4 replies
- 3 kudos
Databricks CI/CD Azure DevOps
Hi All. I have a scenario where there are few .sql scripts present in my repo. Is there any way we can execute those SQLs on Databricks via Azure DevOps CI/CD pipeline?Please help.
- 17063 Views
- 4 replies
- 3 kudos
- 3 kudos
Hi @Divyansh Jain​ Thank you for posting your question in our community! We are happy to assist you.To help us provide you with the most accurate information, could you please take a moment to review the responses and select the one that best answers...
- 3 kudos
- 4230 Views
- 1 replies
- 3 kudos
how to alter a column into an IDENTITY column
Here's me use case: I'm migrating out of an old DWH, into Databricks. When moving dimension tables into Databricks, I'd like old SKs (surrogate keys) to be maintained, while creating the SKs column as an IDENTITY column, so new dimension values get a...
- 4230 Views
- 1 replies
- 3 kudos
- 3 kudos
Hi @Ronen Levi​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 3 kudos
- 1920 Views
- 1 replies
- 0 kudos
How can I know if databricks auto-detected to use tuneFileSizesForRewrites?
We are having some issues with merge performance, so I went and read a bit in the documentation, I found this section:https://docs.databricks.com/delta/tune-file-size.html#autotune-file-size-based-on-workload"Databricks recommends setting the table p...
- 1920 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Ron Serruya​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 2114 Views
- 1 replies
- 1 kudos
ReadStream & writeStream at gold layer level
Hello, I have seen in many places readStream and writeStream in gold layer, Is it correct to use readStream and writeStream for gold layer ? knowing that a gold table is no not valid for streaming.is there some logic when to use readStream/ writeStr...
- 2114 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Ibrahim ISSOUANI​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 1 kudos
- 2530 Views
- 1 replies
- 0 kudos
Failed to import `Ray` on jupyter notebook.
While working on my school's Linux server, I encountered an issue while attempting to install and import Ray in my Jupyter Notebook. I successfully installed the package ray==2.4.0, but encountered an error when trying to import it, specifically stat...
- 2530 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Jaeseon Song​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 2885 Views
- 1 replies
- 0 kudos
Creating cluster from ADF linked service with Workspace init script
Similar issue: https://stackoverflow.com/questions/76220211/create-new-databricks-cluster-from-adf-linked-service-with-initscripts-from-abfsI am trying to create clusters using ADF linked service where the cluster is configured with a init script. As...
- 2885 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Oscar Dyremyhr​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 3627 Views
- 1 replies
- 4 kudos
How to use HTML tags in Alert templates?
According to the alert docs (here), HTML tags should work to format messages in a custom template. When I tried using them, it doesn't seem able to recognize them however and just returns the whole text.ie
- 3627 Views
- 1 replies
- 4 kudos
- 4 kudos
Hi @Adrian Quicoy​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 4 kudos
| User | Count |
|---|---|
| 1644 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
