- 5773 Views
- 4 replies
- 7 kudos
Databricks Academy content for Azure Databricks Customers
Hi! We've recently provisioned an Azure Databricks workspace and started building our pipelines. Do we qualify as Databricks 'customers' who have free access to all self-paced content on Databricks Academy? If so, how do we access it? We don't have a...
- 5773 Views
- 4 replies
- 7 kudos
- 7 kudos
They changed the registration process and added 'Additional Fields' section, where you can provide your company email address, that you use in Azure Databricks. This worked automatically for me and I can access the self-paced trainings for free now.
- 7 kudos
- 12902 Views
- 2 replies
- 5 kudos
How to restart the Spark session within the notebook without reattaching the notebook?
Hi All,I want to run an ETL pipeline in a sequential way in my DB notebook. If I run it without resetting the Spark session or restarting the cluster I am getting a data frame key error. I think this might be because of the Spark cache because If I r...
- 12902 Views
- 2 replies
- 5 kudos
- 5 kudos
Is there a solution to the above problem? I also would like to restart SparkSession to free my cluster's resources, but when callingspark.stop()the notebook automatically detach and the following error occurs:The spark context has stopped and the dri...
- 5 kudos
- 2098 Views
- 0 replies
- 0 kudos
Optimal Azure VM type for EventHub streaming
Hello,our spark jobs stream messages from Event Hub then transform it and finally the messages are peristed in storage. We plan to exercise cluster configurations for these jobs in order to find the optimal and procure Azure reservations. Furtemore, ...
- 2098 Views
- 0 replies
- 0 kudos
- 1580 Views
- 0 replies
- 1 kudos
Missed Associate Developer for Apache Spark 3.0 - Python Due to Power outage
Dear Databricks Certification Team,Unfortunately, I was unable to take the exam as scheduled due to an unforeseen power breakdown in my area. The power outage occurred just before the exam, rendering me unable to access the necessary resources to com...
- 1580 Views
- 0 replies
- 1 kudos
- 7039 Views
- 4 replies
- 0 kudos
Resolved! If 2 users run the same notebook on the same cluster, will they share the same Spark session?
Databricks docs here:https://docs.databricks.com/notebooks/notebook-isolation.htmlstate that "Every notebook attached to a cluster has a pre-defined variable named spark that represents a SparkSession." What if 2 users run the same notebook on the sa...
- 7039 Views
- 4 replies
- 0 kudos
- 0 kudos
The spark session is isolated at the notebook level and is not isolated at the user level. So, two users accessing the same notebook will be using the same spark session
- 0 kudos
- 2061 Views
- 0 replies
- 1 kudos
- 2061 Views
- 0 replies
- 1 kudos
- 13703 Views
- 2 replies
- 0 kudos
Databricks Logs some error messages while trying to read data using databricks-jdbc dependency
I have tried to read data from Databricks using the following java code.String TOKEN = "token..."; String url = "url..."; Properties properties = new Properties(); properties.setProperty("user", "token"); properties.setProperty("PWD", TOKEN); Con...
- 13703 Views
- 2 replies
- 0 kudos
- 0 kudos
@Binesh J​ - The issue could be due to the data type of the column is not compatible with getString() method in line#17. use getObject() method to retrieve the value as a generic value and then convert to string.
- 0 kudos
- 11962 Views
- 2 replies
- 1 kudos
Resolved! Snowflake query with time travel not working from Databricks while reading into Dataframe.
I am trying to read the changes data from snowflake query into the dataframe using Databricks.Same query is working in snowflake but not in Databricks. Both sides timezones and format are same for the timestamp. I am trying to implement changetrackin...
- 11962 Views
- 2 replies
- 1 kudos
- 1 kudos
you are format is wrong that's why you got an errortry thisSELECT * FROM TestTable CHANGES(INFORMATION => DEFAULT) AT(TIMESTAMP => TO_TIMESTAMP_TZ('2023-05-03 00:43:34.885','YYYY-MM-DD HH24:MI:SS.FF'))
- 1 kudos
- 6486 Views
- 3 replies
- 0 kudos
How to set the permissions to databricks jobs that created and run from Azure Data Factory(ADF)?
I would like to set the permissions to jobs such as granting "CAN_VIEW" or "CAN_MANAGE" to specific groups that run from ADF. It appears that we need to set permissions in pipe line where job runs from ADF, But I could not figure it out. ​​
- 6486 Views
- 3 replies
- 0 kudos
- 0 kudos
Thank you @Debayan Mukherjee​ and @Vidula Khanna​ for getting back to me. But, it didn't help my case. I am specifically looking for setting permissions to the job so that our team can see the job cluster including Spark UI with that privilege. ...
- 0 kudos
- 8809 Views
- 3 replies
- 0 kudos
I have a job with multiple tasks running asynchronously and I don't think its leveraging all the nodes on the cluster based on runtime.
I have a job with multiple tasks running asynchronously and I don't think its leveraging all the nodes on the cluster based on runtime. I open the Spark UI for the cluster and checkout the executors and don't see any tasks for my worker nodes. How ca...
- 8809 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi @Dave Hiltbrand​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 1836 Views
- 1 replies
- 0 kudos
How to get the table reference of cached query results?
Databricks caches query results for 24 hours. I would like to access the query results as if it is a table so that I can post process it. For example, do another query against it. The ask is similar to Snowflake RESULT_SCAN https://docs.snowflake.com...
- 1836 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @MAN LI​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 9666 Views
- 6 replies
- 3 kudos
Azure Storage Account inside Databricks cannot enable soft-delete.
Hello,When deploying any databricks workspace inside Azure, the storage account inside the databricks managed resource group is unable to apply any changes, including enabling soft-delete. Is there a way to enable it?Best regards,Jon
- 9666 Views
- 6 replies
- 3 kudos
- 2717 Views
- 1 replies
- 2 kudos
Take advantage of the Data + AI Summit Virtual Experience next week! Data + AI Summit is just a few days away! With data professionals from 155+ count...
Take advantage of the Data + AI Summit Virtual Experience next week!Data + AI Summit is just a few days away! With data professionals from 155+ countries already registered, this is truly the premier event for the global data, analytics and AI commun...
- 2717 Views
- 1 replies
- 2 kudos
- 2 kudos
Thank you for informing highlights of the upcoming sessions.
- 2 kudos
- 2182 Views
- 1 replies
- 0 kudos
- 2182 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @Muhammad Umer​ Great to meet you, and thanks for your question! Let's see if your peers in the community have an answer to your question. Thanks.
- 0 kudos
- 4332 Views
- 2 replies
- 4 kudos
docs.databricks.com
Databricks new Release : Full-page workspace browser includes ReposDatabricks plans to enable the full-page workspace browser experience that unifies Workspace and Repos by default. You’ll browse content in Databricks Repos alongside your workspace c...
- 4332 Views
- 2 replies
- 4 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 586 | |
| 349 | |
| 287 |
