Using generative AI to generate descriptions

monke64
New Contributor

‎09-20-2024 12:46 AM

Hi all, I would like to check if there is a function in databricks notebook that allows users to get AI generated descriptions of dataframes within the session similar to the AI generated descriptions for tables? This is because I am wondering it it would be possible to iteratively go through multiple dataframes to generate and store their descriptions. Any help would be much appreciated.

0 Kudos

vikasgautam
Databricks Employee
Databricks Employee

‎09-20-2024 11:00 PM

You can use ai_query function to achieve this in SQL. It takes an LLM endpoint and a prompt as input.

Here's an example with DBRX and a specific Indian manufacturing industry-related prompt.

create
or replace function ai_create_table_metadata (table_name STRING) returns string return ai_query(
'databricks-dbrx-instruct',
CONCAT(
'Generate a short business description for the below table name in the context of the Indian Manufacturing Industry:: ',
table_name ));

select
table_name,
ai_create_table_metadata(table_name) as table_desc
from
system.information_schema.tables
where
table_name = 'your_table_name';
0 Kudos

Nidhig631
Databricks MVP

‎03-11-2026 04:12 AM

Thanks @vikasgautam  for sharing this solution.

0 Kudos

michael365
New Contributor III

‎03-13-2026 04:29 AM

You can also use Genie Code (which was named Assistent before) and put in some instructions as below. This would then be used and applied once the developer uses the Genie Code.

Or alternatively you can create an Agent Skill and put those instructions with the Agent skill file. In addition you can define some key words for the Skill like "generate descriptions" which enables Genie Code to perform the instructions once you enter those two words in a prompt.

In general there is high potential with Genie Code and Agent Skills to automatically:

  • check Code in terms of coding guidelines
  • check for logic and usage of typical coding instructions
  • add documentation
  • etc.

 

## Header documentation

Add a "--" and 50 "=" characters as first line to code
Add a "--" and name of user as author to header as comment 
Add a "--" and timestamp including Day, Date, Time of last updated to header as comment
Adda "-" and  50 "=" characters after author and timestamp
## General
Use consistent and descriptive identifiers and names.
Make judicious use of white space and indentation to make code easier to read.
Store ISO 8601 compliant time and date information (YYYY-MM-DDTHH:MM:SS.SSSSS)
Include comments in SQL code where necessary.
Names must begin with a letter and may not end with an underscore.
Only use letters, numbers and underscores in names.

 

0 Kudos