Community Discussions
1002 Posts
Activity in Discussions
- 13 Views
- 1 replies
- 0 kudos
Databricks Asset Bundle Workflow Redeployment Issue
Hello All,In my Databricks workflows, I have three tasks configured, with the final task set to run only if the condition "ALL_DONE" is met. During the first deployment, I observed that the dependency "ALL_DONE" was correctly assigned to the last tas...
- 13 Views
- 1 replies
- 0 kudos
Latest Reply
- 0 kudos
Hi, I've attached two snapshots, one from the initial deployment and another from the second redeployment, where no changes were made to the code.
- 0 kudos
- 17687 Views
- 2 replies
- 3 kudos
What's the difference between a Global view and a Temp view?
The difference between Global and Temp is how the lifetime of the view is tied to the application:http://spark.apache.org/docs/latest/api/python/reference/api/pyspark.sql.DataFrame.createOrReplaceTempView.html?highlight=createorreplacetempview#pyspar...
- 17687 Views
- 2 replies
- 3 kudos
Latest Reply
- 3 kudos
Correct A Temp View is scoped to the SparkSession and dropped when that session closes. Each notebook runs in its own SparkSession. The Global Temp View is scoped to the cluster and dropped when the cluster re-starts or you drop it. ---------------...
- 3 kudos
- 46 Views
- 0 replies
- 0 kudos
Databricks OutOfMemory error on code that previously worked without issue
I have a notebook in Azure Databricks that does some transformations on a bronze tier table and inserts the transformed data into a silver tier table. This notebook is used to do an initial load of the data from our existing system into our new datal...
- 46 Views
- 0 replies
- 0 kudos
- 8312 Views
- 7 replies
- 4 kudos
Resolved! Pass date value as parameter in Databricks SQL notebook
I want to pass yesterday date (In the example 20230115*.csv) in the csv file. Don't know how to create parameter and use it here.CREATE OR REPLACE TEMPORARY VIEW abc_delivery_logUSING CSVOPTIONS ( header="true", delimiter=",", inferSchema="true", pat...
- 8312 Views
- 7 replies
- 4 kudos
Latest Reply
- 4 kudos
@Kaniz @sp1 @Chaitanya_Raju @daniel_sahal Hi Everyone,I need the same scenario on SQL code, because my DBR cluster not allowed me to run python codeError: Unsupported cell during execution. SQL warehouses only support executing SQL cells.I appreciate...
- 4 kudos
- 35 Views
- 0 replies
- 0 kudos
On the fly transformations on DLT tables
Hi, I am loading data from a kinesis data stream using DLT. CREATE STREAMING TABLE Consumers_kinesis_2 ( ..., unbase64(data) String, ... ) AS SELECT * FROM STREAM read_kinesis (...) Is it possible to directly cast, unbase64, and/or transform the resu...
- 35 Views
- 0 replies
- 0 kudos
- 66 Views
- 0 replies
- 0 kudos
connect timed out error - Connecting to SQL Server from Databricks
I am getting a connect timed out error when attempting to access a sql server. I can successfully ping the server from Databricks. I have used the jdbc connection and the sqlserver included driver and both result in the same error. I have also attemp...
- 66 Views
- 0 replies
- 0 kudos
- 32 Views
- 0 replies
- 0 kudos
Testosil Revealed - Real Customer Experience and Fact Check
Testosil is a total game-changer! Since starting it, I feel like a whole new person. More energy, sharper focus, and just feeling great overall. What I love most is its natural ingredients, no worries about anything artificial. Taking it is a breeze,...
- 32 Views
- 0 replies
- 0 kudos
- 68 Views
- 0 replies
- 0 kudos
New admin question: How do you enable R on a existing cluster?
Hello Community. I have a user trying to use R and receive the error message illustrated on the attachment. I can't seem to find correct documentation on enabling R on an existing cluster. Would anyone be able to point me in the right direction? Than...
- 68 Views
- 0 replies
- 0 kudos
- 41 Views
- 0 replies
- 0 kudos
Testosil Reviews (2024): Is It Worth Buying Or Not
Testosil rocks! It's been a total game-changer for me. Since I started using it, I've felt a real boost in my sprightliness and focus. What's awesome is that it's made with natural ingredients, so I feel good about what I'm putting into my body. Plus...
- 41 Views
- 0 replies
- 0 kudos
- 29 Views
- 0 replies
- 0 kudos
Impersonating a user
How do I impersonate a user? I can't find any documentation that explains how to do this or even hint that it's possible.Use case: I perform administrative tasks like assign grants and roles to catalogs, schemas, and tables for the benefit of busines...
- 29 Views
- 0 replies
- 0 kudos
- 57 Views
- 0 replies
- 0 kudos
Testosil Reviews (April 2024): Is It Worth or Not
Testosil delivers a powerhouse punch for vitality seekers. Within days of starting, I noticed a remarkable surge in energy levels, allowing me to power through the day with vigor. Not only did it improve my physical strength, but it also uplifted my ...
- 57 Views
- 0 replies
- 0 kudos
- 88 Views
- 0 replies
- 0 kudos
Notebook cell gets hung up but code completes
Have been running into an issue when running a pymc-marketing model in a Databricks notebook. The cell that fits the model gets hung up and the progress bar stops moving, however the code completes and dumps all needed output into a folder. After the...
- 88 Views
- 0 replies
- 0 kudos
- 11720 Views
- 8 replies
- 3 kudos
Resolved! Private PyPI repos on DBR 13+
We use a private PyPI repo (AWS CodeArtifact) to publish custom python libraries. We make the private repo available to DBR 12.2 clusters using an init-script as prescribed here in the Databricks KB. When we tried to upgrade to 13.2 this stopped wor...
- 11720 Views
- 8 replies
- 3 kudos
Latest Reply
- 3 kudos
I'm coming back to provide an updated solution that doesn't rely on the implementation detail of the user name (e.g., libraries) - which is not considered a contract and could potentially change and break in the future.The key is to use the --global ...
- 3 kudos
- 5466 Views
- 8 replies
- 8 kudos
Resolved! Installing libraries on job clusters
Simple question : what is the way to go to install libraries on job clusters ? There does not seem to be a "Libraries" tab on the UI as opposed to regular clusters. Does it mean that the only option is to use init scripts ?
- 5466 Views
- 8 replies
- 8 kudos
Latest Reply
- 8 kudos
@daniel_sahalflappy birdIs there another way to add libraries to the ADF Databricks task even faster?
- 8 kudos
- 106 Views
- 0 replies
- 0 kudos
Insufficient Permissions Issue on Databricks
I have encountered a technical issue on Databricks.While executing commands both in Spark and SQL within the Databricks environment, I’ve run into permission-related errors from selecting files from DBFS. "org.apache.spark.SparkSecurityException: [IN...
- 106 Views
- 0 replies
- 0 kudos
