cancel
Showing results for 
Search instead for 
Did you mean: 
Page Title

Welcome to the Databricks Community

Discover the latest insights, collaborate with peers, get help from experts and make meaningful connections

107662members
57989posts
cancel
Showing results for 
Search instead for 
Did you mean: 
2024 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms

Databricks Named a Leader Again Databricks has been named a Leader in the 2024 Gartner® Magic Quadrant™ for Data Science and Machine Learning Platforms. This new report includes generative AI — a key addition in our opinion since the last report was ...

  • 66 Views
  • 1 replies
  • 1 kudos
yesterday
Join Us for an Exciting Community Social Event!

Hello, Community members! We are thrilled to invite you to our upcoming Community Social Event! This is a fantastic opportunity to engage with fellow community members, stay on top of all the latest Databricks announcements, and be part of an ama...

  • 237 Views
  • 0 replies
  • 0 kudos
Tuesday
Mosaic AI: Build and deploy production-quality Compound AI Systems

Over the last year, we have seen a surge of commercial and open-source foundation models showing strong reasoning abilities on general knowledge tasks. While general models are an important building block, production AI applications often employ Comp...

  • 116 Views
  • 0 replies
  • 1 kudos
Tuesday
Introducing AI/BI: Intelligent Analytics for Real-World Data

Today, we are excited to announce Databricks AI/BI, a new type of business intelligence product built from the ground up to deeply understand your data's semantics and enable anyone to analyze data for themselves. AI/BI is built on a compound AI syst...

  • 598 Views
  • 0 replies
  • 2 kudos
Tuesday

Community Activity

denanyree
by New Contributor
  • 2 Views
  • 0 replies
  • 0 kudos

Instagram Private Profile View

Private Insta Viewer has been a game-changer for me! Before discovering this tool, I often found myself wanting to see private profiles without the hassle of following them. With Private Insta Viewer, I can easily and discreetly view these profiles....

  • 2 Views
  • 0 replies
  • 0 kudos
ivanagarrett
by New Contributor
  • 5 Views
  • 0 replies
  • 0 kudos

View Instagram Private Profiles

Private Insta Viewer has been a fantastic tool for me! I often wanted to view private Instagram profiles without the need to follow them, and Private Insta Viewer provided the perfect solution. The process was incredibly easy and discreet, making it...

  • 5 Views
  • 0 replies
  • 0 kudos
timothybrinkley
by New Contributor
  • 23 Views
  • 0 replies
  • 0 kudos

Viewer Private Instagram 🌟

Private Insta Viewer has been an amazing discovery for me! I've often been curious about certain private Instagram profiles but didn't want to follow them. With Private Insta Viewer, I can now view these profiles effortlessly and discreetly. The pro...

  • 23 Views
  • 0 replies
  • 0 kudos
Ramakrishnan83
by New Contributor III
  • 4913 Views
  • 7 replies
  • 0 kudos

Renaming the database Name in Databricks

Team,Initially our team created the databases with the environment name appended. Ex: cust_dev, cust_qa, cust_prod.I am looking to standardize the database name as consistent name across environments. I want to rename to "cust". All of my tables are ...

  • 4913 Views
  • 7 replies
  • 0 kudos
Latest Reply
Avvar2022
Contributor
  • 0 kudos

You can also use “CASCADE” to drop schema and tables as well. It is recursive. 

  • 0 kudos
6 More Replies
dvmentalmadess
by Valued Contributor
  • 3570 Views
  • 10 replies
  • 2 kudos

Resolved! Data Explorer minimum permissions

What are the minimum permissions are required to search and view objects in Data Explorer? For example, does a user have to have `USE [SCHEMA|CATALOG]` to search or browse in the Data Explorer? Or can anyone with workspace access browse objects and, ...

  • 3570 Views
  • 10 replies
  • 2 kudos
Latest Reply
bearded_data
New Contributor III
  • 2 kudos

Circling back to this.  With one of the recent releases you can now GRANT BROWSE at the catalog level!  Hopefully they will be rolling this feature out at every object level (schemas and tables specifically).

  • 2 kudos
9 More Replies
runninsavvy
by Visitor
  • 20 Views
  • 0 replies
  • 0 kudos

Cannot pass arrays to spark.sql() using named parameter markers

Hello all,I am attempting to use named parameter markers as shown in this article: https://docs.databricks.com/en/sql/language-manual/sql-ref-parameter-marker.html#named-parameter-markersI can pass strings and numbers in perfectly fine, but the issue...

  • 20 Views
  • 0 replies
  • 0 kudos
FG
by New Contributor II
  • 3629 Views
  • 4 replies
  • 1 kudos

Running unit tests from a different notebook (using Python unittest package) doesn't produce output (can't discover the test files)

I have a test file (test_transforms.py) which has a series of tests running using Python's unittest package. I can successfully run the tests inside of the file with expected output. But when I try to run this test file from a different notebook (run...

image.png image
  • 3629 Views
  • 4 replies
  • 1 kudos
Latest Reply
sparklearner233
  • 1 kudos

Try this:import unittestimport os # Set the path to the directory containing the test filetest_dir = '/dbfs/mnt/repository/yunjchen/customer_ai_project/packages/visualization/test/model_performance' # Add the test directory to the Python pathos.sys.p...

  • 1 kudos
3 More Replies
eheinlein
by Visitor
  • 11 Views
  • 0 replies
  • 0 kudos

How to confirm a workspace ID via an api token?

Hello! We are integrating with Databricks and we get the API key, workspace ID, and host from our users in order to connect to Databricks. We need the to validate the workspace ID because we do need it outside of the context of the API key (with webh...

  • 11 Views
  • 0 replies
  • 0 kudos
Zacp
by New Contributor II
  • 52 Views
  • 2 replies
  • 0 kudos

Delta Sharing Azure Databricks Private Storage

I am looking for details on how Delta Sharing is configured on Azure. If your storage account is private, do you need to whitelist the IP for the recipient? Recipient is not using Databricks.

  • 52 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @Zacp, When setting up Delta Sharing on Azure, here are the steps you need to follow: As an Azure Databricks account admin: Log in to the Azure Databricks account console.In the sidebar, click on “Catalog.”Click the name of a metastore to open...

  • 0 kudos
1 More Replies
df_Jreco
by Visitor
  • 40 Views
  • 1 replies
  • 0 kudos

Custom python package iin Notebook task using bundle

Hi mates!I'n my company, we are moving our pipelines to Databricks bundles, our pipelines use a notebook that receives some parameters.This notebook uses a custom python package to apply the business logic based on the parameters that receive.The thi...

Community Discussions
databricks-bundles
  • 40 Views
  • 1 replies
  • 0 kudos
Latest Reply
df_Jreco
Visitor
  • 0 kudos

 Solved understanding the databricks.yml configuration!

  • 0 kudos
dollyb
by Contributor
  • 530 Views
  • 2 replies
  • 0 kudos

Resolved! Differences between Spark SQL and Databricks

Hello,I'm using a local Docker Spark 3.5 runtime to test my Databricks Connect code. However I've come across a couple of cases where my code would work in one environment, but not the other.Concrete example, I'm reading data from BigQuery via spark....

  • 530 Views
  • 2 replies
  • 0 kudos
Latest Reply
daniel_sahal
Esteemed Contributor
  • 0 kudos

@dollyb That's because when you've added another dependency on Databricks, it doesn't really know which one it should use. By default it's using built-in com.google.cloud.spark.bigquery.BigQueryRelationProvider.What you can do is provide whole packag...

  • 0 kudos
1 More Replies
aranjan99
by New Contributor III
  • 26 Views
  • 0 replies
  • 0 kudos

system.access.table_lineage table missing data

I am using the system.access.table_lineage table  to figure out the tables accessed by sql queries and the corresponding SQL queries. However I am noticing this table missing data or values very often.For eg for sql queries executed by our DBT jobs, ...

  • 26 Views
  • 0 replies
  • 0 kudos
trimethylpurine
by New Contributor
  • 64 Views
  • 2 replies
  • 0 kudos

Resolved! Gathering Data Off Of A PDF File

Hello everyone,I am developing an application that accepts pdf files and inserts the data into my database. The company in question that distributes this data to us only offers PDF files, which you can see attached below (I hid personal info for priv...

  • 64 Views
  • 2 replies
  • 0 kudos
Latest Reply
Kaniz
Community Manager
  • 0 kudos

Hi @trimethylpurine, Extracting data from PDF files and inserting it into a database is a common task. Here are a few options you can explore: Docparser: This tool allows you to convert PDF data into usable formats for databases like MySQL, Post...

  • 0 kudos
1 More Replies
NaeemS
by New Contributor II
  • 46 Views
  • 2 replies
  • 0 kudos

Feature Store Model Serving endpoint

Hi,I am trying to deploy my model which was logged by featureStoreEngineering client as a serving endpoint in Databricks. But I am facing following error:   The Databricks Lookup client from databricks-feature-lookup and Databricks Feature Store clie...

  • 46 Views
  • 2 replies
  • 0 kudos
Latest Reply
NaeemS
New Contributor II
  • 0 kudos

Hi @Kaniz ,Thanks for your response. But I have limited access to shared cluster and can not use it for this purpose. Can you tell me how I can prevent databricks-feature-lookup from being added in my requirements file when I am logging my model usin...

  • 0 kudos
1 More Replies
jcohen
by New Contributor
  • 28 Views
  • 0 replies
  • 0 kudos

Nexalyn Norge anmeldelser og opplevelser - fordeler og kjøp

Anmeldelser om Nexalyn Norge – Introduksjon:Anmeldeser om Nexalyn Norge - Introduksjon:Nexalyn er et produkt utviklet for å fremme generell helse og vitalitet blant menn. I en verden hvor velvære og fysisk form ofte er assosiert med suksess, er det v...

  • 28 Views
  • 0 replies
  • 0 kudos
Join 100K+ Data Experts: Register Now & Grow with Us!

Excited to expand your horizons with us? Click here to Register and begin your journey to success!

Already a member? Login and join your local regional user group! If there isn’t one near you, fill out this form and we’ll create one for you to join!

Top Kudoed Authors

Latest from our Blog

MLOps Gym - Evaluating Large Language Models with MLflow

This is the second part of a three-part guide on MLflow in the MLOps Gym series. In Part 1, “Beginners’ Guide to MLflow”, we covered Tracking and Model Registry components. In this article, we will f...

1876Views 0kudos

MLOps Gym - 13 Essential Tips for Writing Clean Code

As a data scientist developing ML models in Python on Databricks, you likely utilize notebooks for conducting training experiments.  The ML code you jot down in your notebooks might end up cluttered ...

8513Views 3kudos