cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

Using UDF in an insert command

Dp15
Contributor

Hi,
I am trying to use a UDF to get the last day of the month and use the boolean result of the function in an insert command. Please find herewith the function and the my query.

function:

import calendar
from datetime import datetime, date, timedelta


def getMonthEndDate(curr_dt: int๐Ÿ˜ž
    curr_dt = datetime.strptime(str(curr_dt),'%Y%m%d').date()
    print("Provided date is: ",curr_dt)
    me_date=getLastBusinessDateInMonth(curr_dt.year,curr_dt.month)
    print("Last Business Day of the Month is: ",me_date)
    if me_date == curr_dt:
        return True
    else:
        return False

def getLastBusinessDateInMonth(year: int, month: int) -> int:
    return date(year, month, max(calendar.monthcalendar(year, month)[-1][:5]))

spark.udf.register("isMonthEnd1", getMonthEndDate)
 %sql
INSERT INTO test.metadata(id, reporting_date, run_id, run_type, batch_type, is_latest, create_date, create_time,is_month_end)
VALUES(11, 20240409, 00, 0, 'Test', 1, NULL, NULL,isMonthEnd1(20240409));

when I execute this notebook I am getting this following error
Could someone please help?

AnalysisException: [INVALID_INLINE_TABLE.CANNOT_EVALUATE_EXPRESSION_IN_INLINE_TABLE] Invalid inline table. Cannot evaluate the expression "isMonthEnd1(20240409)" in inline table definition.

 

 

1 REPLY 1

Dp15
Contributor

Thank you @Retired_mod for your detailed explanation

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group