Databricks

Maverick1 · 06-16-2022

As of now, if I try to list the jobs via "list job" API then there is a limit of 25 jobs only.Is there a way to list all the available/visible jobs to a user?

Maverick1 · 05-31-2022

1. How to pass dynamic variable values like "sysdate" to a job parameters, so that it will automatically take the updated values on the fly.2. How to run multiple instance of set of tasksin a job (for different parameters). For e.g the same pipeline ...

Maverick1 · 05-20-2022

Is there any way to overwrite a partition in delta table without specifying each and every partition in replace where. For non dated partitions, this is really a mess with delta tables.Most of my DE teams don't want to adopt delta because of these gl...

Maverick1 · 03-21-2022

I was trying to see whether it would be possible to refer a library from a Repos path (as “files in repos” feature is already available).I can see the “.whl” file in the repo itself but it is not visible under the “Workspace” option of “python wheel”...

Maverick1 · 03-04-2022

Can an mlflow registered model automatically infer the online feature store table, if that model is trained and logged via a databricks feature store table and the table is pushed to an online feature store (like AWS RDS)?

Maverick1 · 06-23-2022

@Kaniz Fatma @Arvind Ravish :Yes the user can generate the API token but the limit is on the "get jobs" API level in version 2.1.One workaround that I found is to create a while loop function and pass the offset param variable to "list job" API wi...

Maverick1 · 06-06-2022

@Chetan Kardekar :Hubert's answer is not relevant in this case. His answer might be correct if someone is performing data warehousing scenario's having primary keys etc. and completely using SQL. Programatic API code doesn't usually use the MERGE st...

Maverick1 · 05-20-2022

@Kaniz Fatma : Hi Fatima, mlflow-export-import is still not a recommended solution and has no support from Databricks. This is clarified by Databricks SA's themselves.I have developed an another approach where I am capturing the statistics of a part...

Maverick1 · 04-25-2022

@Kaniz Fatma : Unfortunately No. The above implementation always assumes that image is hosted in ECR repository.

Maverick1 · 04-02-2022

As of now, The only resolutions for this query are:Either push/deploy the library to an DBFS or S3 path.Either deploy the library on a repos path and then refer it inside a notebook via passing repos path.Accessing Workspace path or root "file:/" pat...

Databricks

User Stats

User Activity

How to get the list of all jobs available for a particular user?

Dynamic variable and multi-instance tasks.

Is there any way to overwrite a partition in delta table without specifying each and every partition in replace where? For non dated partitions, this is really a mess with delta tables.

How to refer the ".whl" files from repos into a Databricks "python wheel" task in Databricks jobs?

How to infer the online feature store table via an mlflow registered model, which is deployed to a sagemaker endpoint?

Re: How to get the list of all jobs available for a particular user?

Re: Is there any way to overwrite a partition in delta table without specifying each and every partition in replace where? For non dated partitions, this is really a mess with delta tables.

Re: Migration from self-managed MLflow to Databricks managed MLflow.

Re: How to deploy mlflow models to sagemaker endpoints where sagemaker refers the private docker registry?

Re: How to refer the ".whl" files from repos into a Databricks "python wheel" task in Databricks jobs?