cancel
Showing results for 
Search instead for 
Did you mean: 
Resources
cancel
Showing results for 
Search instead for 
Did you mean: 

Browse the Community

Knowledge Sharing Hub

Dive into a collaborative space where members like YOU can exchange knowledge, tips, and best practi...

37 Posts

DatabricksTV

Expert-produced videos to help you leverage Databricks in your analytics and machine learning projec...

7 Posts

Activity in Resources

Sujitha
by Community Manager
  • 44 Views
  • 0 replies
  • 0 kudos

Introducing the Databricks AI Fund

Databricks Ventures Launches New Fund to Extend our Ecosystem Leadership We launched Databricks Ventures in December 2021 as our strategic investment arm for funding innovative startups across the data, analytics and AI landscape — companies that sha...

Screenshot 2024-05-23 at 8.51.34 AM.png
  • 44 Views
  • 0 replies
  • 0 kudos
Sujitha
by Community Manager
  • 55 Views
  • 0 replies
  • 0 kudos

Announcing General Availability of Liquid Clustering

Out-of-the-box, self-tuning data layout that scales with your data We’re excited to announce the General Availability of Delta Lake Liquid Clustering in the Databricks Data Intelligence Platform. Liquid Clustering is an innovative data management tec...

Screenshot 2024-05-23 at 8.45.12 AM.png
  • 55 Views
  • 0 replies
  • 0 kudos
MichTalebzadeh
by Contributor
  • 96 Views
  • 1 replies
  • 0 kudos

A handy tool called spark-column-analyser

I just wanted to share a tool I built called spark-column-analyzer. It's a Python package that helps you dig into your Spark DataFrames with ease.Ever spend ages figuring out what's going on in your columns? Like, how many null values are there, or h...

Knowledge Sharing Hub
Generative AI
python
spark
  • 96 Views
  • 1 replies
  • 0 kudos
Latest Reply
MichTalebzadeh
Contributor
  • 0 kudos

An example added to README in GitHubDoing analysis for column PostcodeJson formatted output{"Postcode": {"exists": true,"num_rows": 93348,"data_type": "string","null_count": 21921,"null_percentage": 23.48,"distinct_count": 38726,"distinct_percentage"...

  • 0 kudos
DavidOBrien
by New Contributor
  • 509 Views
  • 1 replies
  • 0 kudos

Editing value of widget parameter within notebook code

I have a notebook with a text widget where I want to be able to edit the value of the widget within the notebook and then reference it in SQL code. For example, assuming there is a text widget named Var1 that has input value "Hello", I would want to ...

  • 509 Views
  • 1 replies
  • 0 kudos
Latest Reply
anardinelli
New Contributor
  • 0 kudos

Hi @DavidOBrien, how are you? You can try the following approach: # Get the current value of the widget current_value = dbutils.widgets.get("widget_name") # Append the new value to the current value new_value = current_value + "appended_value" # Se...

  • 0 kudos
youssefmrini
by Honored Contributor III
  • 65 Views
  • 0 replies
  • 1 kudos

Schema evolution clause added to SQL merge syntax

You can now add the WITH SCHEMA EVOLUTION clause to a SQL merge statement to enable schema evolution for the operation. For more information: https://docs.databricks.com/en/delta/update-schema.html#sql-evo   #Databricks

  • 65 Views
  • 0 replies
  • 1 kudos
Hubert-Dudek
by Esteemed Contributor III
  • 80 Views
  • 0 replies
  • 0 kudos

VariantType + Parse_json()

In Spark 4.0, there are no more data type mismatches when converting dynamic JSONs, as the new data type VariantType comes with a new function to parse JSONs. Stay tuned for 4.0 release.

variant.png
  • 80 Views
  • 0 replies
  • 0 kudos
youssefmrini
by Honored Contributor III
  • 97 Views
  • 0 replies
  • 1 kudos

Type widening is in Public Preview

You can now enable type widening on tables backed by Delta Lake. Tables with type widening enabled allow changing the type of columns to a wider data type without rewriting underlying data files.   For more information:https://docs.databricks.co...

  • 97 Views
  • 0 replies
  • 1 kudos
Sujitha
by Community Manager
  • 172 Views
  • 1 replies
  • 2 kudos

Supercharge Your Code Generation

We are excited to introduce Databricks Assistant Autocomplete now in Public Preview. This feature brings the AI-powered assistant to you in real-time, providing personalized code suggestions as you type. Directly integrated into the notebook and SQL ...

Screenshot 2024-05-21 at 10.16.56 AM.png
  • 172 Views
  • 1 replies
  • 2 kudos
Latest Reply
Ajay-Pandey
Esteemed Contributor III
  • 2 kudos

Great features, help to write code much faster 

  • 2 kudos
Sujitha
by Community Manager
  • 399 Views
  • 0 replies
  • 2 kudos

Accelerating the Scientific AI Revolution

TetraScience and Databricks Join Forces To Transform Scientific Research, Development, Manufacturing, and Quality Control in Life Sciences BOSTON & SAN FRANCISCO, May 20th, 2024 - TetraScience and Databricks today announced a strategic partnership de...

Screenshot 2024-05-21 at 10.32.56 AM.png
  • 399 Views
  • 0 replies
  • 2 kudos
Sujitha
by Community Manager
  • 226 Views
  • 0 replies
  • 1 kudos

Deploying Third-party models securely with the Databricks Data Intelligence Platform and HiddenLayer

The ability for organizations to adopt machine learning, AI, and large language models (LLMs) has accelerated in recent years thanks to the popularization of model zoos – public repositories like Hugging Face and TensorFlow Hub that are populated wit...

Screenshot 2024-05-21 at 10.25.36 AM.png
  • 226 Views
  • 0 replies
  • 1 kudos
DatabricksGuide
by Community Manager
  • 234 Views
  • 0 replies
  • 0 kudos

Getting Started with Databricks - Building a Forecasting Model on Databricks

Building a Forecasting Model on Databricks: A Step-by-Step Guide This guide offers a detailed, step-by-step approach for building a forecasting model on Databricks. By leveraging the power of Databricks, you will unlock new potentials in your data w...

DatabricksGuide_0-1715968895219.gif DatabricksGuide_0-1715967500817.gif DatabricksGuide_1-1715967500776.gif DatabricksGuide_2-1715968440657.png
  • 234 Views
  • 0 replies
  • 0 kudos
Sujitha
by Community Manager
  • 28493 Views
  • 52 replies
  • 111 kudos

🔔 Attention Databricks Academy Users: SSO Implementation Incoming! Secure Your Account Today!

On May 18th a new SSO user experience will launch in Customer and Partner Academy. In order to secure your account, please see the instructions below on how to update and verify that your data is correct and complete. Please ensure that you have two ...

Sujitha_0-1715596765217.png Sujitha_1-1715596782895.png Screenshot 2024-05-06 at 11.56.58 AM.png
  • 28493 Views
  • 52 replies
  • 111 kudos
Latest Reply
DavidStruzik
New Contributor
  • 111 kudos

I'm a consultant. I work with multiple companies, but I don't have a main one. I'm not comfortable setting any of my emails in clients' domains on my profile. Is it OK for me to keep using my personal one?

  • 111 kudos
51 More Replies
Sujitha
by Community Manager
  • 528 Views
  • 0 replies
  • 0 kudos

Building DBRX-class Custom LLMs with Mosaic AI Training

We recently introduced DBRX: an open, state-of-the-art, general-purpose LLM. DBRX was trained, fine-tuned, and evaluated using Mosaic AI Training, scaling training to 3072 NVIDIA H100s and processing more than 12 trillion tokens in the process. Train...

Screenshot 2024-05-17 at 6.33.11 PM.png
  • 528 Views
  • 0 replies
  • 0 kudos
Sujitha
by Community Manager
  • 866 Views
  • 0 replies
  • 1 kudos

Accurate, Safe and Governed: How to Move GenAI from POC to Production

In the realm of AI, achieving accuracy is paramount. The publication delves into techniques for refining models to ensure they reliably deliver precise outcomes in real-world scenarios. It covers methodologies such as continuous monitoring, data augm...

Screenshot 2024-05-17 at 6.25.11 PM.png
  • 866 Views
  • 0 replies
  • 1 kudos
DatabricksGuide
by Community Manager
  • 618 Views
  • 0 replies
  • 0 kudos

Getting Started with Databricks - Ingest to Analytics & BI

Getting Started with Databricks - From Ingest to Analytics & BIIntroduction to DatabricksAnalytics & BI on DatabricksIngest Setup Steps [20 minutes]Step 0: Check your required prerequisitesStep 1: Access and start your warehouseStep 2: Connect your w...

DatabricksGuide_0-1714140949313.png DatabricksGuide_1-1714141186202.gif DatabricksGuide_4-1714141571707.gif DatabricksGuide_5-1714141715140.png
  • 618 Views
  • 0 replies
  • 0 kudos
Top Kudoed Authors