- 1200 Views
- 1 replies
- 1 kudos
Spark event log frequently missing rollover logs
Hi,We are currently using the event log to gain some visibility to what spark is doing under the hood but we frequently find that rollover logs are missing and we are left with a partial set. Is there a reason for this or a bug being tracked? Is ther...
- 1200 Views
- 1 replies
- 1 kudos
- 1 kudos
I'm also having this issue , it seems to occur much more frequently on azure than aws, but even on aws jobs sometimes some of the logs will be missing
- 1 kudos
- 1197 Views
- 1 replies
- 0 kudos
Features tab under Machine learning shows error
Cannot resolve metastore id, go back. FeatureStoreError- Workspace Feature Store has been deprecated in the current workspace. Databricks recommends using Feature Engineering in Unity Catalog. Please reach out to system admin if you have any more que...
- 1197 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @rdvc,Could it be that this one workspace that has this problem is not configured to use Unity Catalog?
- 0 kudos
- 1223 Views
- 1 replies
- 0 kudos
How to access smtplib with ACL enables cluster
I am having a tough time to connect to smtp server from databricks with ACL enabled cluster , I am able to access with out ACL enabled cluster what can be done to access it via ACL enabled cluster import smtplibfrom email.mime.multipart import MIMEMu...
- 1223 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @unalluri9999 ,What error do you get? By ACL you mean IP access lists or something different? Your workspace is vnet injected?
- 0 kudos
- 4391 Views
- 0 replies
- 3 kudos
Databricks "Page Not Responding" Error Affecting All Users
For the past two days, our entire project team has been experiencing a recurring "Page Not Responding" error in Databricks. This issue occurs roughly every five minutes, making it impossible to run any code effectively.What's notable is that this pr...
- 4391 Views
- 0 replies
- 3 kudos
- 3622 Views
- 0 replies
- 0 kudos
Survey about cluster configuration and optimization
Hi everyone, I hope you are very well.In the research for my university we are conducting a survey to Databricks users, aimed to gather information about how cluster configuration and optimization is faced in the industry.If you like you can answer i...
- 3622 Views
- 0 replies
- 0 kudos
- 4411 Views
- 1 replies
- 1 kudos
Error running 80 task at same time in Job, how limit this?
Hi! I have a Job running to process multiple streaming tables. In the beginning, it was working fine, but now I have 80 tables running in this job, the problem is that all the runs are trying to run at the same time throwing an error. Is there a way ...
- 4411 Views
- 1 replies
- 1 kudos
- 1750 Views
- 1 replies
- 0 kudos
DABs vs API
DABs seem like a great way to build a project based workflow. However, I have a need to take a user's SQL and create a TABLE/VIEW based on it. We plan to use the API to submit the request but is this the best way going forward? Should we consider ...
- 1750 Views
- 1 replies
- 0 kudos
- 0 kudos
DABs use Terraform which in turn calls the API anyway. So, I guess it comes down to personal preference of deployment consistency, code maintenance etc. However, from your description of "taking a user's SQL" I'm assuming some sort of real-time appli...
- 0 kudos
- 1485 Views
- 1 replies
- 0 kudos
Assistance Required for Enabling Unity Catalog in Databricks Workspace
Hi,I hope this message finds you well.I am reaching out regarding a concern with Databricks Administrator privileges. I have an Azure subscription and I use Azure Databricks for my tutorials, but I currently do not have Global Administrator access, w...
- 1485 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello, each account should have at least one account administrator that is the one that has the permissions on the managed account and that can give you this access if applicable or assist you with the set up of the UC, you might need to ask internal...
- 0 kudos
- 2294 Views
- 1 replies
- 1 kudos
Disabling Default Personal Compute Policy in Terraform (AWS)
I'm looking to disable the Personal Compute Policy in Terraform, however I don't see an option to do this. According to this documentation here https://docs.databricks.com/en/admin/clusters/personal-compute.html#customize-the-personal-compute-policy ...
- 2294 Views
- 1 replies
- 1 kudos
- 1 kudos
I have the same issue in terraform-provider-google.I found how to do it from Admin UI (UI -> Settings -> Features Enablement -> disable Personal compute). At the same time I didn't find how to have the same behavior from Terraform.
- 1 kudos
- 5222 Views
- 3 replies
- 2 kudos
Resolved! Message queue to directly trigger job
Hi All,I'm very new to Databricks and trying to understand my use case a bit better.I have a databricks script / job that I want to be reactive to events outside of my databricks environment. A best case scenario would be if my script / job could aut...
- 5222 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @AmandaOhare ,You can use AWS Lambda to achieve that. You can setup queue trigger that will activate AWS lambda function. In that function you can call datbricks rest API that will launch workflow/job.
- 2 kudos
- 845 Views
- 1 replies
- 0 kudos
Unable to access databricls workspaces
Hi Team, Previously we are able to access databricks workspaces using username and password but now its asking for token, But we dont have access to mail account with username that we are using while login. Could you please assist on the issue how ca...
- 845 Views
- 1 replies
- 0 kudos
- 0 kudos
Anyone can help on this ?
- 0 kudos
- 2616 Views
- 1 replies
- 0 kudos
Resolved! Using Databricks Connect with PyCharm behind Proxy
I am using Databricks connect with PyCharm in order to locally develop my pipelines before deploying to Databricks. I set up authentication for databricks-connect using the following guide https://learn.microsoft.com/en-us/azure/databricks/dev-tools/...
- 2616 Views
- 1 replies
- 0 kudos
- 0 kudos
I managed to solve the problem by including my login and password in the HTTPS_PROXY environment variable.I set it to something like this HTTPS_PROXY = "http://username:password@proxy.company.com:port" and databricks-connect can create spark sessions...
- 0 kudos
- 1329 Views
- 3 replies
- 1 kudos
Can't login to new AWS databricks workspace
Just get our first AWS Databricks workspace created by following the instructions: https://docs.databricks.com/en/admin/workspace/create-uc-workspace.htmladopted the customer-managed VPC and private link for the setup. Upon reviewing the network conf...
- 1329 Views
- 3 replies
- 1 kudos
- 1 kudos
the problem resolved after networking team currently set the DNS to point to the endpoint. We can login through the workspace link now.
- 1 kudos
- 2736 Views
- 2 replies
- 0 kudos
Databricks REST API to fetch mount points
Is there a way to fetch workspace mount points ( mount infos) through REST API or SQL-query ? ( similar to the python API "display(dbutils.fs.mounts())" ) I couldn't find any REST API for the mounts in the official databricks API documentation ( ...
- 2736 Views
- 2 replies
- 0 kudos
- 10189 Views
- 4 replies
- 3 kudos
Resolved! Does REST API usage is chargeable ?
If we use REST API to perform some operation on the Databricks cluster- like running some SQL Statements, will that be chargeable, will it costs us.
- 10189 Views
- 4 replies
- 3 kudos
- 3 kudos
Yep, to sum up - anything that requires compute power and makes the cluster to spin up will be charged.
- 3 kudos
