cancel
Showing results for 
Search instead for 
Did you mean: 
Get Started Discussions
Start your journey with Databricks by joining discussions on getting started guides, tutorials, and introductory topics. Connect with beginners and experts alike to kickstart your Databricks experience.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

menotron
by Valued Contributor
  • 1221 Views
  • 0 replies
  • 1 kudos

Turn Your Dataframes into an Interactive Tableau-Styled Drag-and-Drop UI for Visual Analysis

You can create Tableau-styled charts without leaving your notebook with just a few lines of code.Imagine this: You’re working within Databricks notebook, trying to explore your Spark/Pandas DataFrame, but visualizing the data or performing Explorator...

menotron_0-1735979649344.png menotron_0-1735977132616.png
  • 1221 Views
  • 0 replies
  • 1 kudos
Sudheer2
by New Contributor III
  • 2527 Views
  • 6 replies
  • 0 kudos

Migrating Service Principals from Non-Unity to Unity-Enabled Databricks Workspaces - Entitlements Mi

 Hello Databricks Community,I am currently in the process of migrating Service Principals from a non-Unity workspace to a Unity-enabled workspace in Databricks. While the Service Principals themselves seem to be migrating correctly, I am facing an is...

  • 2527 Views
  • 6 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Do you have the option to open a support ticket? If not the case I would suggest to run an additional code that disable the entitlement for the new objects, as seems that the entitlement is not being properly passed in the original call  

  • 0 kudos
5 More Replies
hiphopjay
by New Contributor II
  • 891 Views
  • 3 replies
  • 0 kudos

Unable to install UCX

Executing databricks labs install ucx via databricks cli and running into 'WorkspaceInstaller is not supposed to be executed in databricks runtime.' How do I resolve it?

  • 891 Views
  • 3 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

Databricks CLI is designed to work in your local terminal and connect to your workspace.  The error is mentioning that the command is not allowed to be run in a Databricks cluster, as you are using the web terminal of a cluster in your workspace it w...

  • 0 kudos
2 More Replies
unj1m
by New Contributor III
  • 1318 Views
  • 3 replies
  • 2 kudos

Problems registering models via /api/2.0/mlflow/model-versions/create

Initially, I tried registering while logging a model from mlflow and got:"Got an invalid source 'dbfs:/Volumes/compute_integration_tests/default/compute-external-table-tests-models/test_artifacts/1078d04b4d4b4537bdf7a4b5e94e9e7f/artifacts/model'. Onl...

  • 1318 Views
  • 3 replies
  • 2 kudos
Latest Reply
unj1m
New Contributor III
  • 2 kudos

I know, right! Look at the error message I posted:"Got an invalid source 'dbfs:/Volumes/compute_integration_tests/default/compute-external-table-tests-models/test_artifacts/1078d04b4d4b4537bdf7a4b5e94e9e7f/artifacts/model'. Only DBFS locations are cu...

  • 2 kudos
2 More Replies
Phani1
by Valued Contributor II
  • 1866 Views
  • 2 replies
  • 2 kudos

Azure Databricks to GCP Databricks Migration

Hi Team, Can you provide your thoughts on moving Databricks from Azure to GCP? What services are required for the migration, and are there any limitations on GCP compared to Azure? Also, are there any tools that can assist with the migration? Please ...

  • 1866 Views
  • 2 replies
  • 2 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 2 kudos

Hello Team, Adding to @sunnydata comments: Moving Databricks from Azure to GCP involves several steps and considerations. Here are the key points based on the provided context:   Services Required for Migration:Cloud Storage Data: Use GCP’s Storage T...

  • 2 kudos
1 More Replies
fmadeiro
by Contributor II
  • 946 Views
  • 1 replies
  • 0 kudos

Structuring RAG Projects in Python Using

Understanding Retrieval-Augmented Generation (RAG)Retrieval-Augmented Generation (RAG) is a cutting-edge AI paradigm that enhances traditional generative models by integrating real-time data retrieval. By combining retrieval and generation, RAG ensur...

  • 946 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Here's a demo using RAG LLM: https://www.databricks.com/resources/demos/tutorials/data-science-and-ai/lakehouse-ai-deploy-your-llm-chatbot

  • 0 kudos
victor_okrobodo
by New Contributor
  • 1030 Views
  • 1 replies
  • 0 kudos

Databricks Support

Hi everyoneI'm new to databricks and to this platform. My organization just got started with databricks and we're looking to procure/purchase enterprise support to help with things like training, set up, maintenance and development of our warehousing...

  • 1030 Views
  • 1 replies
  • 0 kudos
Latest Reply
Alberto_Umana
Databricks Employee
  • 0 kudos

Hi @victor_okrobodo, Please see: https://www.databricks.com/professional-services  Let me know if you have any other questions.

  • 0 kudos
sravs1
by New Contributor II
  • 2714 Views
  • 4 replies
  • 5 kudos

Data Modelling

What is the 'implicit' or 'by default' data model of databricks or unity catalog ? Is it Data Vault ?

  • 2714 Views
  • 4 replies
  • 5 kudos
Latest Reply
ZacharyHill
New Contributor II
  • 5 kudos

Thank you so much for the information. You made it easy for me.

  • 5 kudos
3 More Replies
Sa_1234
by New Contributor II
  • 921 Views
  • 3 replies
  • 0 kudos

Suggest ways to get unity catalog data to Aws s3 or sagemaker

Please suggest best ways to get databricks unity catalog data to Aws s3 or sagemaker. Data could be around 1gb in some tables and 20gb in others.currently sagemaker pipelines use data from s3 as batches in different parquet files. But now we would li...

  • 921 Views
  • 3 replies
  • 0 kudos
Latest Reply
Walter_C
Databricks Employee
  • 0 kudos

You could try by using Delta Sharing with your provider as mentioned in doc https://docs.databricks.com/en/delta-sharing/set-up.html   

  • 0 kudos
2 More Replies
inagar
by New Contributor
  • 1078 Views
  • 2 replies
  • 0 kudos

Unable to capture the Query result via JDBC client execution

As shown in below screenshots MERGE INTO command produces information about the result (num_affected_rows, num_updated_rows, num_deleted_rows, num_inserted_rows).Unable to get this information when the same query is being executed via JDBC client. Is...

Screenshot 2024-06-26 at 1.30.19 AM.png Screenshot 2024-06-26 at 1.36.10 AM (1).png Screenshot 2024-06-26 at 1.45.23 AM.png
  • 1078 Views
  • 2 replies
  • 0 kudos
Latest Reply
NandiniN
Databricks Employee
  • 0 kudos

Delta API can help you get these details.  Reference - https://docs.databricks.com/en/delta/history.html#history-schema

  • 0 kudos
1 More Replies
tigerhawkvok
by New Contributor
  • 1704 Views
  • 1 replies
  • 0 kudos

DIAS 2023 -- recommend the training!

Did the SparkUI training yesterday with Mark Ott, and I highly recommend it. It was super helpful and provided a lot of clarity around some of the vaguer terms and metrics, and some surprise penalties.In-memory partition size is the the main thing to...

  • 1704 Views
  • 1 replies
  • 0 kudos
Latest Reply
harshadsaglani
New Contributor II
  • 0 kudos

Hi, I can't find this course... can you please share the full name of this course? Thanks in advance 

  • 0 kudos
Demudu
by New Contributor II
  • 1437 Views
  • 3 replies
  • 4 kudos

Resolved! How to read Databricks UniForm format tables present in ADLS

We have Databricks UniForm format (iceberg) tables are present in azure data lake storage (ADLS) which has already integrated with Databricks unity catalog. How to read Uniform format tables using Databricks as a query engine?

  • 1437 Views
  • 3 replies
  • 4 kudos
Latest Reply
fmadeiro
Contributor II
  • 4 kudos

Query Using Unity Catalog:SQL:sqlCopiar códigoSELECT * FROM catalog_name.schema_name.table_name;PySpark:pythonCopiar códigodf = spark.sql("SELECT * FROM catalog_name.schema_name.table_name") df.display()Direct Access by Path: If not using Unity Catal...

  • 4 kudos
2 More Replies
egndz
by New Contributor II
  • 9055 Views
  • 2 replies
  • 0 kudos

Cluster Memory Issue (Termination)

Hi,I have a single-node personal cluster with 56GB memory(Node type: Standard_DS5_v2, runtime: 14.3 LTS ML). The same configuration is done for the job cluster as well and the following problem applies to both clusters:To start with: once I start my ...

egndz_2-1712845742934.png egndz_1-1712845616736.png
  • 9055 Views
  • 2 replies
  • 0 kudos
raktim_das
by New Contributor II
  • 1755 Views
  • 2 replies
  • 1 kudos

Creating table in Unity Catalog with file scheme dbfs is not supported

 code:# Define the path for the staging Delta tablestaging_table_path = "dbfs:/user/hive/warehouse/staging_order_tracking"spark.sql( f"CREATE TABLE IF NOT EXISTS staging_order_tracking USING DELTA LOCATION '{staging_table_path}'" )Creating table in U...

  • 1755 Views
  • 2 replies
  • 1 kudos
Latest Reply
SaiPrakash_653
New Contributor II
  • 1 kudos

I believe, using mount point only we can able to connect to our storage account - containers. If this is anti pattern by data bricks what is the way? Can you please explain what is external location of UC is it our local system folders or something n...

  • 1 kudos
1 More Replies

Join Us as a Local Community Builder!

Passionate about hosting events and connecting people? Help us grow a vibrant local community—sign up today to get started!

Sign Up Now
Labels