topic Re: How can we fetch application id in serverless compute in databricks? in Data Engineering

How can we fetch application id in serverless compute in databricks?

momita_s — Wed, 22 Jan 2025 13:50:33 GMT

Hi All,

How can we fetch application id in serverless compute in databricks? We are working to use serverless compute for some jobs. The issue is we are not able to fetch application id in notebook. Earlier we were using spark.sparkContext.applicationId, but sparkContext is not supported in serverless compute. We also tried to use spark.conf.get("spark.app.id"), but this is also not working. Please let me know if anyone has any inputs on this.

Thanks

Re: How can we fetch application id in serverless compute in databricks?

Alberto_Umana — Wed, 22 Jan 2025 14:30:24 GMT

Hi @momita_s,

Thanks for your question. As you mentioned above parameters/commands are not available in serverless, I did not find a way to retrieve the applicationID on serverless, I would check internally but this likely requires a feature request.

Re: How can we fetch application id in serverless compute in databricks?

momita_s — Wed, 22 Jan 2025 16:08:31 GMT

Hi @Alberto ,
Thanks for the reply, please let me know if you find anything on this after discussing with internal team.