cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

wellington
by New Contributor III
  • 281 Views
  • 2 replies
  • 0 kudos

Log notebook activities

Hi friends;I'm working on a project where we are 4 programmers. We are working in a single environment, using only the "Workspaces" folder. Each has its own user, which is managed by Azure AD.We had a peak in consumption on the 5th Feb. So I can see ...

  • 281 Views
  • 2 replies
  • 0 kudos
Latest Reply
wellington
New Contributor III
  • 0 kudos

Hi @Kaniz , thanks for your quick answer.There is no other way to monitor notebook runs. I ask this because adding tags to the cluster and workspace does not solve my problem, considering that everyone uses the same cluster and the same workspace.

  • 0 kudos
1 More Replies
Lucidity
by New Contributor II
  • 2084 Views
  • 3 replies
  • 1 kudos

Resolved! Deny assignment modification to allow attach/detach of disks in azure databricks

Our application does storage autoscaling on Azure. We would like to deploy our solution with Azure databricks. But even though the service principal associated with our application has the necessary roles and permissions to attach/detach a disk from ...

  • 2084 Views
  • 3 replies
  • 1 kudos
Latest Reply
Lucidity
New Contributor II
  • 1 kudos

Thank you for your replyIs there any way databricks provides to bypass the deny assignment for specific apps? I noticed in the deny assignment unity-catalog-access-connector has been provided exlusion under the excludePrincipals section. is there a w...

  • 1 kudos
2 More Replies
Data_Engineer3
by Contributor II
  • 1271 Views
  • 2 replies
  • 0 kudos

Resolved! spark context in databricks

Hi @all,In Azure Databricks,I am using structured streaming for each batch functionality, in one of the functions I am creating tempview with pyspark dataframe (*Not GlobalTempView) and trying to access the same temp view by using spark.sql functiona...

  • 1271 Views
  • 2 replies
  • 0 kudos
Latest Reply
Lakshay
Esteemed Contributor
  • 0 kudos

Do you face this issue without spark streaming as well? Also, could you share a minimal repo code preferably without streaming?

  • 0 kudos
1 More Replies
valjas
by New Contributor III
  • 1299 Views
  • 3 replies
  • 1 kudos

Clusters are really slow

We have two environments for our Azure Databricks. Dev and Prod. We had clusters created and tested in Dev environment, then they were exported to the prod environment through APIs. The clusters in Dev are performing as expected. Whereas, the cluster...

  • 1299 Views
  • 3 replies
  • 1 kudos
Latest Reply
Kaniz
Community Manager
  • 1 kudos

Hi @valjas, Workspace Creation and Cluster Performance: Actions taken during the creation of a workspace can indeed impact cluster performance. When setting up a workspace, consider the following factors: Configuration Settings: Ensure that the wor...

  • 1 kudos
2 More Replies
wellington
by New Contributor III
  • 3059 Views
  • 3 replies
  • 2 kudos

Resolved! Not able to access Account console page

Hello All,I'm not be able to access the account console page. I'm a portal admin, my workspace is premium, and yet the Databricks portal stays in a loop, always returning to the Workspaces Overview page and not going to the accounts console page so t...

Get Started Discussions
Account Console
Account Management
  • 3059 Views
  • 3 replies
  • 2 kudos
Latest Reply
wellington
New Contributor III
  • 2 kudos

Guys, the problem is that I was not signed in as a Global Administrator in Azure AD. After that, I reset all the Browser settings and I was able to do that. Here's the tip.

  • 2 kudos
2 More Replies
Farzana
by New Contributor II
  • 2391 Views
  • 3 replies
  • 0 kudos

Databricks cluster launch time

Hi Team,We have an @adf pipeline which will run some set of activities before #Azure databricks notebooks get called.As and when the notebooks are called our pipeline will launch a new cluster for every job with job compute as Standard F4 with a sing...

  • 2391 Views
  • 3 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Farzana , If the cluster pool has enough idle instances and the slow image download from the artefact storage is not the issue, then a few things can be done to reduce the cluster launch time. Preload the runtime on the instance pool to improve t...

  • 0 kudos
2 More Replies
Ajay-Pandey
by Esteemed Contributor III
  • 1626 Views
  • 3 replies
  • 0 kudos

Databricks Pub-Sub Data Recon

I am trying to setup a recon activity between GCP Pub-Sub and databricks, Is there any way to fetch the last 24hrs record count from Pub-Sub?I tried but not got any direct solution for it, It will be great if any one can suggest me the way t#pubsub, ...

  • 1626 Views
  • 3 replies
  • 0 kudos
Latest Reply
Anonymous
Not applicable
  • 0 kudos

Hi @Ajay-Pandey  Hope you are well. Just wanted to see if you were able to find an answer to your question and would you like to mark an answer as best? It would be really helpful for the other members too. Cheers!

  • 0 kudos
2 More Replies
Labels