Hi @nayan_wylde 

Yes, this is a common challenge when transitioning to Unity Catalog (UC) enabled clusters.
The installation of Maven packages from Artifactory repositories does work differently in UC environments,
but there are several approaches you can use to maintain your organization's approved patterns.

Current Limitations in UC Standard Mode:
In UC Standard mode clusters, the traditional methods of installing Maven packages directly through cluster libraries
or %pip install commands have restrictions due to the enhanced security model. This is why you're seeing success
with the manual JAR placement in volumes.

Recommended Solutions for Artifactory Integration:
1. Use Init Scripts with Artifactory Authentication
You can create an init script that downloads packages from your Artifactory during cluster startup:

#!/bin/bash
# Download from Artifactory with authentication
curl -u $ARTIFACTORY_USER:$ARTIFACTORY_TOKEN \
-o /databricks/jars/your-package.jar \
https://your-artifactory-url/path/to/package.jar

2. Databricks Asset Bundles (Recommended)
Configure your deployment pipeline to include Artifactory packages as part of your bundle deployment.
This maintains the approved pattern while working within UC constraints.

3. Custom Package Management: Create a standardized process where:
- Packages are pulled from Artifactory during your CI/CD pipeline
- JARs are placed in designated volumes/DBFS locations
- Cluster configurations reference these standard locations.

4. Unity Catalog Volumes with Automation: Set up an automated process that:
- Periodically syncs approved packages from Artifactory to UC volumes
- Uses service principals for authentication
- Maintains version control and dependency management

Is there a way we can automate the volumes to sync.

Yes, there are several ways to automate volume syncing in Databricks. Here are the main approaches:
1. Databricks Jobs with Scheduled Triggers
2. Using Delta Live Tables (DLT) for Data Syncing
3. Workflow Orchestration with Databricks Workflows
4. Real-time Sync with File Watchers
5. Using Unity Catalog APIs for Automation
6. Multi-Cloud Sync (if needed)

Best Practices for Volume Sync Automation
- Use Databricks Jobs for scheduled syncing
- Implement error handling and retry logic
- Add logging for monitoring sync operations
- Use incremental sync for large datasets
- Set up alerts for sync failures
- Consider bandwidth and cluster costs