Databricks Community

yatharth

SummaryWhen configuring a Jobs schedule → Interval periodic trigger in Databricks, the “Next run” timestamp changes inconsistently when the interval value is modified, even though the base schedule (start time) remains the same. The next run appears ...

yatharth · 04-02-2024

Hi Community i am try to create lzo-codec in my dbfs using:https://docs.databricks.com/en/_extras/notebooks/source/init-lzo-compressed-files.htmlbut i am facing the errorCloning into 'hadoop-lzo'... The JAVA_HOME environment variable is not defined c...

yatharth · 03-12-2024

Hi databricks:I have a job where I am saving my data in json format lzo compressed which requires the library lzo-codecon shifting to graviton instances I noticed that the same job started throwing exceptionCaused by: java.lang.RuntimeException: nati...

yatharth · 03-05-2024

Hi Databricks Community,I am looking for a formula/way to calculate the estimated cost for a job run, for which I have a few questions:1. Is there any formula to calculate the cost of any job like -> [(EC2 per hr cost) * (total time job ran)]and when...

yatharth · 02-19-2024

I was trying the new lakehouse monitoring feature for one of my unity tables, and when I create a monitoring dashboard for my table, the 2 metric tables({output_schema}.{table_name}_profile_metrics and {output_schema}.{table_name}_drift_metrics) are ...

yatharth

Thanks @Ashwin_DSA for that detailed answer, that surely solves my query but I see this as a classic UX bug disguised as feature

yatharth · 04-03-2024

I am facing the same type of issue and now changing back to old UI option is not available, cause of this issue i am not able to view cluster metrics

yatharth · 03-13-2024

For more context, Please use the following code to replicate the error:# Create a Python list containing JSON objectsjson_data = [ { "id": 1, "name": "John", "age": 25 }, { "id": 2, "name": "Jane", "...

yatharth · 03-07-2024

I am exploring the exact thing which you need, and I found out aws glue provides the same type of thing where it writes its own data quality rules, I tried creating the same with Databricks Assistant but not even close

yatharth · 03-06-2024

This looks a little bit confusing to me, I'm looking for a more straight forward answer, more like a simple formulaThanks though for your reply

Databricks Community

User Stats

User Activity

Bug Report: Incorrect “Next Run Time” Calculation for Interval Periodic Schedules

Unable to build LZO-codec

LZO codec not working for graviton instances

Databricks Job cost(AWS)

Metric Tables not created automatically

Re: Bug Report: Incorrect “Next Run Time” Calculation for Interval Periodic Schedules

Re: cluster creation - access mode option

Re: LZO codec not working for graviton instances

Re: What kind of data quality rules that can be run using unity catalog

Re: Databricks Job cost(AWS)