- 8325 Views
- 1 replies
- 0 kudos
[Possible Bug] Repo Notebooks being modified without human interaction
Our production workspace has several Repos integrated with GitHub. These repos aways point to master and should never be modified manually by a human directly in the workspace as the pulls are triggered by a GitHub Actions workflow. This workflow cal...
- 8325 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jrosend , The issue you are facing is that there are minor changes occurring in the notebooks in your Databricks Repos, which are causing conflicts during the automatic update process triggered by the GitHub Actions workflow. These changes are no...
- 0 kudos
- 2827 Views
- 1 replies
- 0 kudos
Error: cannot create permissions: invalid character '<' looking for beginning of value
I'm trying to use terraform to assign a cluster policy to an account-level group (sync'd from AAD via SCIM)My provider is configured like thisprovider "databricks" {alias = "azure_account"host = "accounts.azuredatabricks.net"account_id = "%DATABRICKS...
- 2827 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @abhaigh , • The issue is related to applying a cluster policy to an account-level group using Terraform.• The error message indicates that the returned value from the API endpoint is not as expected. • To resolve the issue, follow these steps:...
- 0 kudos
- 2570 Views
- 1 replies
- 0 kudos
Struggling with UC Volume Paths
I am trying to setup my volumes and give them paths in the data lake but I keep getting this message:Input path url 'abfss://my-container@my-storage-account.dfs.core.windows.net/' overlaps with managed storage within 'CreateVolume' callThere WAS some...
- 2570 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @_YSF, The error message "Input path URL' abyss://my-container@my-storage-account.dfs.core.windows.net/' overlaps with managed storage within' CreateVolume' call" suggests that there is an issue with setting up volumes and giving them paths in th...
- 0 kudos
- 5947 Views
- 5 replies
- 2 kudos
Resolved! Unable to list service principal in Job details RUN AS
I added the service principal in Admin Settings > Service Principal and then enabled all the Configurations "allow cluster creation", "databricks SQL access" and "workspace access". In the Permission settings I have enabled "Service principal: Manage...
- 5947 Views
- 5 replies
- 2 kudos
- 2 kudos
For future readers - don't forget to add your email (e.g. me@foo.com) in the Service Principals permissions tab. This way, you will be able to see the newly-created service principal in the dropdown menu.
- 2 kudos
- 2145 Views
- 1 replies
- 0 kudos
Workspace creation via terraform provider fails on AWS
I'm trying to create a new workspace in a empty account. I have managed to create all the other resources without issues but when I try to create the workspace it fails with the following error:Error: cannot create mws workspaces: MALFORMED_REQUEST: ...
- 2145 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @DevOps, Your error is related to credentials_id your workspace configuration. The credentials_id is a reference to the Databricks credentials configuration ID, which is generated when you create a new set of credentials. This ID represents your ...
- 0 kudos
- 11441 Views
- 1 replies
- 0 kudos
Resolved! Connect Community Edition to Power BI Desktop
I have submitted this question several times to Databricks over the past few weeks, and I have gotten no response at all, not even an acknowledgement that my request was received.Please help.How can I connect a certain dataset in Databricks Community...
- 11441 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @OU_Professor, To connect a particular data set in Databricks Community Edition to the Power BI desktop, you can follow these steps: 1. Install Databricks JDBC driver: You can download the JDBC driver from the Databricks JDBC driver download page....
- 0 kudos
- 1382 Views
- 1 replies
- 0 kudos
Resolved! Clean up Databricks confidential computing resources
Hello All,I created a Databricks Premium Workspace for a Confidential Computing PoC. After creating a VM from Databricks UI, it came to notice that there is a new RG with managed identity, NAT Gateway, Public IP, security group, and a VNET (/16). I w...
- 1382 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @jaganadhg , To delete the resources once you complete the work involving confidential computing, you can use the DELETE API call provided in the given information. Here is an example of how to use it: bashcurl -X DELETE 'https://accounts.cloud.d...
- 0 kudos
- 1204 Views
- 2 replies
- 0 kudos
Extreme RocksDB memory usage
During migration to production workload, I switched some queries to use RocksDB. I am concerned with its memory usage though. Here is sample output from my streaming query: "stateOperators" : [ { "operatorName" : "dedupeWithinWatermark", "...
- 1204 Views
- 2 replies
- 0 kudos
- 0 kudos
Thank you for the input. Is there any particular reason why deduplication watermark makes it store everything and not just the key needed for deduplication? The 1st record has to be written to the table anyway, and its content is irrelevant as it jus...
- 0 kudos
- 2057 Views
- 1 replies
- 1 kudos
Resolved! Monitoring job metrics
Hi,We need to monitor Databricks jobs and we have made a setup where are able to get the prometheus metrics, however, we are lagging an overview of which metrics refer to what.Namely, we need to monitor the following:failed jobs : is a job failedtabl...
- 2057 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Bagger, You can monitor Databricks jobs and get the required metrics using a combination of Databricks features and Prometheus. Here's a general idea of how you could approach each metric you mentioned. 1. Failed jobs: Databricks provides a REST ...
- 1 kudos
- 1916 Views
- 1 replies
- 1 kudos
How can I install maven coordinates using init script?
Hi,I need to install the below maven coordinates on the clusters using databricks init scripts.1. coordinate: com.microsoft.azure:synapseml_2.12:0.11.2 with repo https://mmlspark.azureedge.net/maven2. coordinate: com.microsoft.azure:spark-mssql-conne...
- 1916 Views
- 1 replies
- 1 kudos
- 1 kudos
Hi @Ajay3, You can follow these steps to install the Maven coordinates via Databricks init scripts. 1. Create a shell script to install the Maven libraries on the cluster. Here is an example of how you can do it: bash#!/bin/bash# Install synapseml_2...
- 1 kudos
- 5130 Views
- 3 replies
- 1 kudos
Resolved! How to change Workspace Owner?
Our Databricks workspace was created by a personal account. Now the person has left the organization. We would like to change the owner to a Service account(preferably, else to an Admin account).Questions:Is it possible to change the owner of the wor...
- 5130 Views
- 3 replies
- 1 kudos
- 1 kudos
Hi @nihar_ghude , 1. Yes, changing the owner of the Databricks workspace is possible. As per the provided information, the account's owner can be a user or service account. 2. However, the process to change the owner is not directly mentioned in the...
- 1 kudos
- 5734 Views
- 5 replies
- 1 kudos
Resolved! Using a custom Hostname in Databricks CLI instead of per-workspace URL
Hi,At our organization, we have added front end privatelink connection to a Databricks workspace in Azure, and public access to the workspace is disabled. I am able to access the workspace UI with the private IP (in the browser), and able to call the...
- 5734 Views
- 5 replies
- 1 kudos
- 1 kudos
Hi @Kaniz_Fatma ,Thank you for the support.Really appreciate it.Thanks
- 1 kudos
- 1754 Views
- 3 replies
- 2 kudos
Resolved! Modularization of Databricks Workflows
Given the size of a Workflow may become too big to manage in a single Terraform project, what would be your recommendation as a best practice to manage and deploy the workflows via code to maintainer a predictable result between environments?Would it...
- 1754 Views
- 3 replies
- 2 kudos
- 2 kudos
Hi @jaznarro, Given the size of a Workflow may become too big to manage in a single Terraform project, breaking the Workflow into multiple ones with references between each other would be a recommended best practice. This approach can help maintain ...
- 2 kudos
- 823 Views
- 0 replies
- 0 kudos
terraform/databricks setting default_catalog_name
While configuring databricks, we've set the "default_catalog_name", which sets the default schema when users connect via an ODBC connection. While the naming isn't consistent, this does have one desired effect, that is, when users connect, it default...
- 823 Views
- 0 replies
- 0 kudos
- 708 Views
- 1 replies
- 0 kudos
Can I edit the ADLSg2 storage location for a schema?
I want to alter the schema and basically point it to a new path in the data lake #UnityCatalog
- 708 Views
- 1 replies
- 0 kudos
- 0 kudos
don't think so.You can alter the owner and dbproperties using the alter schema command, but not the location.https://learn.microsoft.com/en-us/azure/databricks/sql/language-manual/sql-ref-syntax-ddl-alter-schema
- 0 kudos
Connect with Databricks Users in Your Area
Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you won’t want to miss the chance to attend and share knowledge.
If there isn’t a group near you, start one and help create a community that brings people together.
Request a New Group-
Access control
1 -
ActiveDirectory
1 -
AmazonKMS
1 -
Apache spark
1 -
App
1 -
Availability
1 -
Availability Zone
1 -
AWS
5 -
Aws databricks
1 -
AZ
1 -
Azure
8 -
Azure Data Lake Storage
1 -
Azure databricks
6 -
Azure databricks workspace
1 -
Best practice
1 -
Best Practices
2 -
Billing
2 -
Bucket
1 -
Cache
1 -
Change
1 -
Checkpoint
1 -
Checkpoint Path
1 -
Cluster
1 -
Cluster Pools
1 -
Clusters
1 -
ClustersJob
1 -
Compliance
1 -
Compute Instances
1 -
Cost
1 -
Credential passthrough
1 -
Data
1 -
Data Ingestion & connectivity
6 -
Data Plane
1 -
Databricks Account
1 -
Databricks Control Plane
1 -
Databricks Error Message
2 -
Databricks Partner
1 -
Databricks Repos
1 -
Databricks Runtime
1 -
Databricks SQL
3 -
Databricks SQL Dashboard
1 -
Databricks workspace
1 -
DatabricksJobs
1 -
DatabricksLTS
1 -
DBFS
1 -
DBR
3 -
Dbt
1 -
Dbu
3 -
Deep learning
1 -
DeleteTags Permissions
1 -
Delta
4 -
Delta Sharing
1 -
Delta table
1 -
Dev
1 -
Different Instance Types
1 -
Disaster recovery
1 -
DisasterRecoveryPlan
1 -
DLT Pipeline
1 -
EBS
1 -
Email
2 -
External Data Sources
1 -
Feature
1 -
GA
1 -
Ganglia
3 -
Ganglia Metrics
2 -
GangliaMetrics
1 -
GCP
1 -
GCP Support
1 -
Gdpr
1 -
Gpu
2 -
Group Entitlements
1 -
HIPAA
1 -
Hyperopt
1 -
Init script
1 -
InstanceType
1 -
Integrations
1 -
IP Addresses
1 -
IPRange
1 -
Job
1 -
Job Cluster
1 -
Job clusters
1 -
Job Run
1 -
JOBS
1 -
Key
1 -
KMS
1 -
KMSKey
1 -
Lakehouse
1 -
Limit
1 -
Live Table
1 -
Log
2 -
LTS
3 -
Metrics
1 -
MFA
1 -
ML
1 -
Model Serving
1 -
Multiple workspaces
1 -
Notebook Results
1 -
Okta
1 -
On-premises
1 -
Partner
33 -
Pools
1 -
Premium Workspace
1 -
Public Preview
1 -
Redis
1 -
Repos
1 -
Rest API
1 -
Root Bucket
2 -
SCIM API
1 -
Security
1 -
Security Group
1 -
Security Patch
1 -
Service principal
1 -
Service Principals
1 -
Single User Access Permission
1 -
Sns
1 -
Spark
1 -
Spark-submit
1 -
Spot instances
1 -
SQL
1 -
Sql Warehouse
1 -
Sql Warehouse Endpoints
1 -
Ssh
1 -
Sso
2 -
Streaming Data
1 -
Subnet
1 -
Sync Users
1 -
Tags
1 -
Team Members
1 -
Thrift
1 -
TODAY
1 -
Track Costs
1 -
Unity Catalog
1 -
Use
1 -
User
1 -
Version
1 -
Vulnerability Issue
1 -
Welcome Email
1 -
Workspace
2 -
Workspace Access
1
- « Previous
- Next »
User | Count |
---|---|
22 | |
9 | |
8 | |
8 | |
6 |