- 3744 Views
- 3 replies
- 4 kudos
Job Owner Changes in Asset Bundle
When I deploy a fresh job, I'm set as owner, which makes sense because I'm creating it.Then at some point, the owner switches to a service principal. This is strange. I believe it happens when a scheduled job triggers.Can job owners change?
- 3744 Views
- 3 replies
- 4 kudos
- 4 kudos
Hi @T0M, how are you doing?You’re right about the statuses at the level above the job; however, my reference to IS_OWNER comes from the job-level statuses, as shown below:IS_OWNER exists and is valid for resource-level permissions (e.g., inside resou...
- 4 kudos
- 2100 Views
- 3 replies
- 1 kudos
Resolved! Error [UNAUTHORIZED_ACCESS]: Merging into a table with masked columns
Hi,I'm facing a weird error when trying to incrementally merge into a table with a masked column.Table is external, but catalog is managed. Using Unity Catalog.Error is:[UNAUTHORIZED_ACCESS] Unauthorized access: PERMISSION_DENIED: Path-based access t...
- 2100 Views
- 3 replies
- 1 kudos
- 1 kudos
Hello @thedatanerd , Thanks for the reply. Yes, you can access that by DeltaTable.forName if you know the table name or use MERGE into using SQL. If your question is answered,could you please accept this as a solution, as it would be useful to others...
- 1 kudos
- 2042 Views
- 5 replies
- 3 kudos
Resolved! unable to access databricks pages
Hello databricks team I am unable to access this URL, but I can see some discription available. I tried to search for this to solve a case but unable to open it. https://docs.databricks.com/_extras/notebooks/source/kb/clusters/event-log-replay.html h...
- 2042 Views
- 5 replies
- 3 kudos
- 3 kudos
Hello @Khaja_Zaffer! The notebook is no longer available at that URL. You can use this valid link: https://docs.databricks.com/aws/en/notebooks/source/kb/clusters/event-log-replay.html
- 3 kudos
- 2495 Views
- 1 replies
- 1 kudos
Resolved! mongodb spark
Hello Guys,Someone know a technique to perform write for a delta table to Mongo using connection mongodb org.mongodb.spark:mongo-spark-connector_2.12:10.5.0I have 1 bilion records to write Thanks
- 2495 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @seefoods ,Yes, it's well described at mongo db connector documentation page. To write data to MongoDB you need call the write function on your DataFrame object. This function returns a DataFrameWriter object, which you can use to specify the form...
- 1 kudos
- 809 Views
- 2 replies
- 2 kudos
Dropdown parameter in new SQL editor
New SQL editor UI doesn't allow adding a list of parameter values. Instead values has to be added one by one and that's just stupid. Also it doesn't allow using query results as values. In notebooks thats of course possible.
- 809 Views
- 2 replies
- 2 kudos
- 2 kudos
Hi @JuhaLeppanen ,If you'd like to see such features in SQL Editor please let know product team using Databricks Ideas portal.Feedback | Databricks
- 2 kudos
- 3491 Views
- 14 replies
- 11 kudos
Resolved! Spark UI Replay Notebook
I need to load the Spark UI events of a long-running job.I found this Databricks knowledge base article on the topic:https://kb.databricks.com/clusters/replay-cluster-spark-eventsHowever, the referenced notebook is no longer accessible.Does anyone st...
- 3491 Views
- 14 replies
- 11 kudos
- 11 kudos
Hi @der ,Here's the content of Event Log Replay notebook. You can use it to achieve your goals %scala dbutils.widgets.removeAll() dbutils.widgets.text("event_log_path", "", "event_log_path") val eventLogPath = dbutils.widgets.get("event_log_path"...
- 11 kudos
- 10622 Views
- 13 replies
- 1 kudos
Resolved! Limitations for Unity Catalog on single user access mode clusters
Hello! According to Databricks documentation on azure :"On Databricks Runtime 15.3 and below, fine-grained access control on single user compute is not supported. Specifically:You cannot access a table that has a row filter or column mask.You cannot ...
- 10622 Views
- 13 replies
- 1 kudos
- 1 kudos
@arzb @Nes_Hdr Command support for FGAC on dedicated compute was added in DBR 17.1: https://docs.databricks.com/aws/en/compute/single-user-fgac#support-for-ddl-show-describe-and-other-commands
- 1 kudos
- 21687 Views
- 3 replies
- 8 kudos
Resolved! How do I resolve problems when deploying a workspace with AWS Quickstart cloud formation template?
I am unable to deploy a workspace on AWS using Quickstart from my account console.Short description-You might receive one of the following common errors users face:Wrong credentialsElastic IP and VPC limit reachedRegion unavailableResolution-Wrong cr...
- 21687 Views
- 3 replies
- 8 kudos
- 8 kudos
CreateStorageConfiguraitonI encounter this issue, but the stack template is different from this example, and I dont have a password for my account.Are the solutions here still relevant?
- 8 kudos
- 1869 Views
- 2 replies
- 3 kudos
Resolved! Serverless Workspace Observability
I’m setting up observability for a Databricks serverless workspace on AWS and need some guidance.I know we can configure audit logs for S3 delivery, but I’m unsure if that alone is sufficient.For a complete observability setup especially when integra...
- 1869 Views
- 2 replies
- 3 kudos
- 915 Views
- 1 replies
- 3 kudos
Job Run Failed - "Cluster became unreachable during run" with Cause: "requirement failed: Execution
I'm encountering a failure while running a job in Databricks. The run fails with the following error message:Cluster became unreachable during run Cause: requirement failed: Execution is doneDetails:Runtime version: 15.4 LTS (includes Apache Spark 3....
- 915 Views
- 1 replies
- 3 kudos
- 3 kudos
Hello @apurvasawant I'm sorry you are seeing this behavior while using Jobs. Definitely, these messages don't help much. When this happens, I suggest taking a step back and reviewing the configuration of your Job and some troubleshooting: What is th...
- 3 kudos
- 1174 Views
- 3 replies
- 1 kudos
GIT automate DEV Databricks instance to PROD instance
Hello,Would like to use GIT to automate the process of syncing between a DEV Databricks instance and a PROD Databricks instance. Something like :On GIT console pull changes/sync with DEV Databricks.Have some kind of approval process in GIT (like some...
- 1174 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @adrianhernandez ,This can be easily achieved using Databricks Assets Bundles in combination with Azure Devops Pipeline (or github actions).So, typical CI/CD workflows looks something like this:Store: Store your Databricks code and notebooks in a ...
- 1 kudos
- 3622 Views
- 19 replies
- 2 kudos
No Longer Able to Create DeltaTables in ADLS Gen 2
Hi Community,Up until recently I was happily deleting deltaTables in ADLS Gen with their associated _delta_log table, and subsequently recreating the same table with a new _delta_log table.Now, after deleting a table with its associated _delta_log ta...
- 3622 Views
- 19 replies
- 2 kudos
- 2 kudos
Hi @szymon_dybczak , you must have a special version of Databricks Community Edition as I don't have those options after I select settings Shall I try from my premium paid for version of Databricks?
- 2 kudos
- 1355 Views
- 4 replies
- 4 kudos
Resolved! Workspace Catalog Connection to 'on-prem' MS SQL Server
Hi, first post/question here ...I'm trying to add a MS SQL Connection to my workspace (us-west-2) Catalog. Connection type is SQL Server, Host equal the public IP of that Server, port is the default 1433. (Server is a AWS EC2 instance)All attempts to...
- 1355 Views
- 4 replies
- 4 kudos
- 4 kudos
Thank You to all responses.I've added 44.234.192.32/28 and 52.27.216.188/32 as a (for now) All traffic inbound rule.But no success ...I'll keep looking and trying.We're currently evaluating with a Serverless Cloud workspace, but at the end might go t...
- 4 kudos
- 1525 Views
- 6 replies
- 2 kudos
Notebook Dashboard refreshes for all users when one user refreshes
Hi Team ,I have created a dashboard using the dashboard functionality of the notebook.But when one user refreshes the dashboard it refreshes for all the users with whom the dashboard is shared.What are the ways this issue can be resolved ?
- 1525 Views
- 6 replies
- 2 kudos
- 3021 Views
- 3 replies
- 0 kudos
Delta live tables - ignore updates on some columns
Hello Team,I have scenario where in apply_changes, i want to ignore updates on 1 column. Is there any way we can achieve this in Delta live tables?
- 3021 Views
- 3 replies
- 0 kudos
- 0 kudos
Hi there @Anish_2 , Yes you can do that Here is the doc link : https://docs.databricks.com/aws/en/dlt/cdc?language=PythonFor python you can simply add an attribute except_columns_list like thisdlt.apply_changes( target = "target", source = "users...
- 0 kudos
| User | Count |
|---|---|
| 1645 | |
| 793 | |
| 582 | |
| 349 | |
| 287 |
