- 3538 Views
- 3 replies
- 1 kudos
Resolved! Constantly Running Interactive Clusters Best Practices
Hello there, I’ve been creating an ETL/ELT Pipeline with Azure Databricks Workflows, Spark and Azure Data Lake. It should process in Near Real Time changes (A Change Data Capture process) from an Azure SQL Database. For that purpose, I will have sev...
- 3538 Views
- 3 replies
- 1 kudos
- 1649 Views
- 3 replies
- 0 kudos
Dashboard sharing in Databriks with Unity Catalog enabled
Hello,I am planning to deploy a workspace with Unity Catalog enabled. Deploying permissions in one place sounds like a good solution. It can even simplify dataset architecture by masking rows and columns.As an architect, I’m concerned about the user’...
- 1649 Views
- 3 replies
- 0 kudos
- 0 kudos
I will suggest for you to submit a feature request for this through https://docs.databricks.com/en/resources/ideas.html#ideas
- 0 kudos
- 1754 Views
- 1 replies
- 0 kudos
Clean Room Per Collaborator Pricing
For Clean Room Pricing it is mentioned50$/ collaborator per day.1) If I keep the clean room for may be 5 hrs and delete it later will I get charged for those 5hrs(10.41$)/collaborator or will I be charged 50$ regardless of active hrs of clean room fo...
- 1754 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @RohithChippa, Thanks for your questions! Please find below feedback: 1) If you keep the clean room for 5 hours and delete it later, you will be charged $50 per collaborator for that day, regardless of the number of active hours the clean room was...
- 0 kudos
- 1469 Views
- 1 replies
- 0 kudos
Databricks Community Edition Not working
Hello,Is the Databricks Community Edition working, as I am getting error every time and after enter my mail id for login and enter the pass-code, it again load the Sign-In page again and i have tried with multiple browsers as well.Can anyone help giv...
- 1469 Views
- 1 replies
- 0 kudos
- 0 kudos
Hello @AshishMehta, Have you tried cleaning cookies and cache? Also can you please try resetting your password?
- 0 kudos
- 2071 Views
- 0 replies
- 1 kudos
Turn Your Dataframes into an Interactive Tableau-Styled Drag-and-Drop UI for Visual Analysis
You can create Tableau-styled charts without leaving your notebook with just a few lines of code.Imagine this: You’re working within Databricks notebook, trying to explore your Spark/Pandas DataFrame, but visualizing the data or performing Explorator...
- 2071 Views
- 0 replies
- 1 kudos
- 2549 Views
- 0 replies
- 3 kudos
How to make your Service Principal or Group Workspace admin?
Do you require for a Service Principal or a Group to have admin rights to allow automation or reduce the efforts in the process of adding the permission to each user. Solution For Service Principals: You need to be at least Workspace AdminYou can eit...
- 2549 Views
- 0 replies
- 3 kudos
- 1427 Views
- 3 replies
- 0 kudos
Unable to install UCX
Executing databricks labs install ucx via databricks cli and running into 'WorkspaceInstaller is not supposed to be executed in databricks runtime.' How do I resolve it?
- 1427 Views
- 3 replies
- 0 kudos
- 0 kudos
Databricks CLI is designed to work in your local terminal and connect to your workspace. The error is mentioning that the command is not allowed to be run in a Databricks cluster, as you are using the web terminal of a cluster in your workspace it w...
- 0 kudos
- 2507 Views
- 3 replies
- 2 kudos
Problems registering models via /api/2.0/mlflow/model-versions/create
Initially, I tried registering while logging a model from mlflow and got:"Got an invalid source 'dbfs:/Volumes/compute_integration_tests/default/compute-external-table-tests-models/test_artifacts/1078d04b4d4b4537bdf7a4b5e94e9e7f/artifacts/model'. Onl...
- 2507 Views
- 3 replies
- 2 kudos
- 2 kudos
I know, right! Look at the error message I posted:"Got an invalid source 'dbfs:/Volumes/compute_integration_tests/default/compute-external-table-tests-models/test_artifacts/1078d04b4d4b4537bdf7a4b5e94e9e7f/artifacts/model'. Only DBFS locations are cu...
- 2 kudos
- 3530 Views
- 2 replies
- 2 kudos
Azure Databricks to GCP Databricks Migration
Hi Team, Can you provide your thoughts on moving Databricks from Azure to GCP? What services are required for the migration, and are there any limitations on GCP compared to Azure? Also, are there any tools that can assist with the migration? Please ...
- 3530 Views
- 2 replies
- 2 kudos
- 2 kudos
Hello Team, Adding to @sunnydata comments: Moving Databricks from Azure to GCP involves several steps and considerations. Here are the key points based on the provided context: Services Required for Migration:Cloud Storage Data: Use GCP’s Storage T...
- 2 kudos
- 1880 Views
- 1 replies
- 0 kudos
Structuring RAG Projects in Python Using
Understanding Retrieval-Augmented Generation (RAG)Retrieval-Augmented Generation (RAG) is a cutting-edge AI paradigm that enhances traditional generative models by integrating real-time data retrieval. By combining retrieval and generation, RAG ensur...
- 1880 Views
- 1 replies
- 0 kudos
- 0 kudos
Here's a demo using RAG LLM: https://www.databricks.com/resources/demos/tutorials/data-science-and-ai/lakehouse-ai-deploy-your-llm-chatbot
- 0 kudos
- 2264 Views
- 1 replies
- 0 kudos
Databricks Support
Hi everyoneI'm new to databricks and to this platform. My organization just got started with databricks and we're looking to procure/purchase enterprise support to help with things like training, set up, maintenance and development of our warehousing...
- 2264 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi @victor_okrobodo, Please see: https://www.databricks.com/professional-services Let me know if you have any other questions.
- 0 kudos
- 3435 Views
- 4 replies
- 5 kudos
Data Modelling
What is the 'implicit' or 'by default' data model of databricks or unity catalog ? Is it Data Vault ?
- 3435 Views
- 4 replies
- 5 kudos
- 1617 Views
- 3 replies
- 0 kudos
Suggest ways to get unity catalog data to Aws s3 or sagemaker
Please suggest best ways to get databricks unity catalog data to Aws s3 or sagemaker. Data could be around 1gb in some tables and 20gb in others.currently sagemaker pipelines use data from s3 as batches in different parquet files. But now we would li...
- 1617 Views
- 3 replies
- 0 kudos
- 0 kudos
You could try by using Delta Sharing with your provider as mentioned in doc https://docs.databricks.com/en/delta-sharing/set-up.html
- 0 kudos
- 1680 Views
- 2 replies
- 0 kudos
Unable to capture the Query result via JDBC client execution
As shown in below screenshots MERGE INTO command produces information about the result (num_affected_rows, num_updated_rows, num_deleted_rows, num_inserted_rows).Unable to get this information when the same query is being executed via JDBC client. Is...
- 1680 Views
- 2 replies
- 0 kudos
- 0 kudos
Delta API can help you get these details. Reference - https://docs.databricks.com/en/delta/history.html#history-schema
- 0 kudos
- 2215 Views
- 1 replies
- 0 kudos
DIAS 2023 -- recommend the training!
Did the SparkUI training yesterday with Mark Ott, and I highly recommend it. It was super helpful and provided a lot of clarity around some of the vaguer terms and metrics, and some surprise penalties.In-memory partition size is the the main thing to...
- 2215 Views
- 1 replies
- 0 kudos
- 0 kudos
Hi, I can't find this course... can you please share the full name of this course? Thanks in advance
- 0 kudos
