Databricks

yatharth · 04-02-2024

Hi Community i am try to create lzo-codec in my dbfs using:https://docs.databricks.com/en/_extras/notebooks/source/init-lzo-compressed-files.htmlbut i am facing the errorCloning into 'hadoop-lzo'... The JAVA_HOME environment variable is not defined c...

yatharth · 03-12-2024

Hi databricks:I have a job where I am saving my data in json format lzo compressed which requires the library lzo-codecon shifting to graviton instances I noticed that the same job started throwing exceptionCaused by: java.lang.RuntimeException: nati...

yatharth · 03-05-2024

Hi Databricks Community,I am looking for a formula/way to calculate the estimated cost for a job run, for which I have a few questions:1. Is there any formula to calculate the cost of any job like -> [(EC2 per hr cost) * (total time job ran)]and when...

yatharth · 02-19-2024

I was trying the new lakehouse monitoring feature for one of my unity tables, and when I create a monitoring dashboard for my table, the 2 metric tables({output_schema}.{table_name}_profile_metrics and {output_schema}.{table_name}_drift_metrics) are ...

yatharth · 02-13-2024

I wish to run aws CLI command in databricks, is there a way i can achieve the same, to be more specific i would like to run:aws cloudwatch get-metric-statistics --metric-name BucketSizeBytes --namespace AWS/S3 --start-time 2017-03-06T00:00:00Z --end-...

yatharth

I am facing the same type of issue and now changing back to old UI option is not available, cause of this issue i am not able to view cluster metrics

yatharth · 03-13-2024

For more context, Please use the following code to replicate the error:# Create a Python list containing JSON objectsjson_data = [ { "id": 1, "name": "John", "age": 25 }, { "id": 2, "name": "Jane", "...

yatharth · 03-07-2024

I am exploring the exact thing which you need, and I found out aws glue provides the same type of thing where it writes its own data quality rules, I tried creating the same with Databricks Assistant but not even close

yatharth · 03-06-2024

This looks a little bit confusing to me, I'm looking for a more straight forward answer, more like a simple formulaThanks though for your reply

yatharth · 02-22-2024

For me getting the admin access for the unity catalog solved it, as only admins can create tables, hence if tables are not getting created automatically, I'll suggest to check if you have permission to create any sort of table manually using the UI o...

Databricks

User Stats

User Activity

Unable to build LZO-codec

LZO codec not working for graviton instances

Databricks Job cost(AWS)

Metric Tables not created automatically

AWS CLI Commands

Re: cluster creation - access mode option

Re: LZO codec not working for graviton instances

Re: What kind of data quality rules that can be run using unity catalog

Re: Databricks Job cost(AWS)

Re: Metric Tables not created automatically