- 6411 Views
- 5 replies
- 2 kudos
Cluster compute metrics
I want to fetch compute metrics(hardware, gpu and spark) and use them in certain dashboard on Databricks, however i'm not able to fetch them. i have tried GET API request and system tables. The system tables only have CPU utilization and memory utili...
- 6411 Views
- 5 replies
- 2 kudos
- 2 kudos
How can we store the CPU & Memory metrics for GCP databricks centrally and setup some alerts incase if the usage is high and monitor the performance.
- 2 kudos
- 2791 Views
- 3 replies
- 1 kudos
Resolved! Constantly Running Interactive Clusters Best Practices
Hello there, I’ve been creating an ETL/ELT Pipeline with Azure Databricks Workflows, Spark and Azure Data Lake. It should process in Near Real Time changes (A Change Data Capture process) from an Azure SQL Database. For that purpose, I will have sev...
- 2791 Views
- 3 replies
- 1 kudos
- 1288 Views
- 3 replies
- 0 kudos
Dashboard sharing in Databriks with Unity Catalog enabled
Hello,I am planning to deploy a workspace with Unity Catalog enabled. Deploying permissions in one place sounds like a good solution. It can even simplify dataset architecture by masking rows and columns.As an architect, I’m concerned about the user’...
- 1288 Views
- 3 replies
- 0 kudos
- 0 kudos
I will suggest for you to submit a feature request for this through https://docs.databricks.com/en/resources/ideas.html#ideas
- 0 kudos
- 1160 Views
- 1 replies
- 0 kudos
Clean Room Per Collaborator Pricing
For Clean Room Pricing it is mentioned50$/ collaborator per day.1) If I keep the clean room for may be 5 hrs and delete it later will I get charged for those 5hrs(10.41$)/collaborator or will I be charged 50$ regardless of active hrs of clean room fo...
- 1160 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @RohithChippa, Thanks for your questions! Please find below feedback: 1) If you keep the clean room for 5 hours and delete it later, you will be charged $50 per collaborator for that day, regardless of the number of active hours the clean room was...
- 0 kudos
- 1285 Views
- 1 replies
- 0 kudos
Databricks Community Edition Not working
Hello,Is the Databricks Community Edition working, as I am getting error every time and after enter my mail id for login and enter the pass-code, it again load the Sign-In page again and i have tried with multiple browsers as well.Can anyone help giv...
- 1285 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @AshishMehta, Have you tried cleaning cookies and cache? Also can you please try resetting your password?
- 0 kudos
- 1666 Views
- 0 replies
- 1 kudos
Turn Your Dataframes into an Interactive Tableau-Styled Drag-and-Drop UI for Visual Analysis
You can create Tableau-styled charts without leaving your notebook with just a few lines of code.Imagine this: You’re working within Databricks notebook, trying to explore your Spark/Pandas DataFrame, but visualizing the data or performing Explorator...
- 1666 Views
- 0 replies
- 1 kudos
- 3154 Views
- 6 replies
- 0 kudos
Migrating Service Principals from Non-Unity to Unity-Enabled Databricks Workspaces - Entitlements Mi
Hello Databricks Community,I am currently in the process of migrating Service Principals from a non-Unity workspace to a Unity-enabled workspace in Databricks. While the Service Principals themselves seem to be migrating correctly, I am facing an is...
- 3154 Views
- 6 replies
- 0 kudos
- 0 kudos
Do you have the option to open a support ticket? If not the case I would suggest to run an additional code that disable the entitlement for the new objects, as seems that the entitlement is not being properly passed in the original call
- 0 kudos
- 1860 Views
- 0 replies
- 3 kudos
How to make your Service Principal or Group Workspace admin?
Do you require for a Service Principal or a Group to have admin rights to allow automation or reduce the efforts in the process of adding the permission to each user. Solution For Service Principals: You need to be at least Workspace AdminYou can eit...
- 1860 Views
- 0 replies
- 3 kudos
- 1189 Views
- 3 replies
- 0 kudos
Unable to install UCX
Executing databricks labs install ucx via databricks cli and running into 'WorkspaceInstaller is not supposed to be executed in databricks runtime.' How do I resolve it?
- 1189 Views
- 3 replies
- 0 kudos
- 0 kudos
Databricks CLI is designed to work in your local terminal and connect to your workspace. The error is mentioning that the command is not allowed to be run in a Databricks cluster, as you are using the web terminal of a cluster in your workspace it w...
- 0 kudos
- 1905 Views
- 3 replies
- 2 kudos
Problems registering models via /api/2.0/mlflow/model-versions/create
Initially, I tried registering while logging a model from mlflow and got:"Got an invalid source 'dbfs:/Volumes/compute_integration_tests/default/compute-external-table-tests-models/test_artifacts/1078d04b4d4b4537bdf7a4b5e94e9e7f/artifacts/model'. Onl...
- 1905 Views
- 3 replies
- 2 kudos
- 2 kudos
I know, right! Look at the error message I posted:"Got an invalid source 'dbfs:/Volumes/compute_integration_tests/default/compute-external-table-tests-models/test_artifacts/1078d04b4d4b4537bdf7a4b5e94e9e7f/artifacts/model'. Only DBFS locations are cu...
- 2 kudos
- 2559 Views
- 2 replies
- 2 kudos
Azure Databricks to GCP Databricks Migration
Hi Team, Can you provide your thoughts on moving Databricks from Azure to GCP? What services are required for the migration, and are there any limitations on GCP compared to Azure? Also, are there any tools that can assist with the migration? Please ...
- 2559 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello Team, Adding to @sunnydata comments: Moving Databricks from Azure to GCP involves several steps and considerations. Here are the key points based on the provided context: Services Required for Migration:Cloud Storage Data: Use GCP’s Storage T...
- 2 kudos
- 1330 Views
- 1 replies
- 0 kudos
Structuring RAG Projects in Python Using
Understanding Retrieval-Augmented Generation (RAG)Retrieval-Augmented Generation (RAG) is a cutting-edge AI paradigm that enhances traditional generative models by integrating real-time data retrieval. By combining retrieval and generation, RAG ensur...
- 1330 Views
- 1 replies
- 0 kudos
- 0 kudos
Here's a demo using RAG LLM: https://www.databricks.com/resources/demos/tutorials/data-science-and-ai/lakehouse-ai-deploy-your-llm-chatbot
- 0 kudos
- 1641 Views
- 1 replies
- 0 kudos
Databricks Support
Hi everyoneI'm new to databricks and to this platform. My organization just got started with databricks and we're looking to procure/purchase enterprise support to help with things like training, set up, maintenance and development of our warehousing...
- 1641 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @victor_okrobodo, Please see: https://www.databricks.com/professional-services Let me know if you have any other questions.
- 0 kudos
- 3090 Views
- 4 replies
- 5 kudos
Data Modelling
What is the 'implicit' or 'by default' data model of databricks or unity catalog ? Is it Data Vault ?
- 3090 Views
- 4 replies
- 5 kudos
- 1323 Views
- 3 replies
- 0 kudos
Suggest ways to get unity catalog data to Aws s3 or sagemaker
Please suggest best ways to get databricks unity catalog data to Aws s3 or sagemaker. Data could be around 1gb in some tables and 20gb in others.currently sagemaker pipelines use data from s3 as batches in different parquet files. But now we would li...
- 1323 Views
- 3 replies
- 0 kudos
- 0 kudos
You could try by using Delta Sharing with your provider as mentioned in doc https://docs.databricks.com/en/delta-sharing/set-up.html
- 0 kudos
