topic Re: How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook? in Data Engineering

How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?

VeeraNerella — Tue, 30 Aug 2022 19:54:34 GMT

%sh

export a

a=100

echo $a

OUTPUT: 100

in the below cell trying to use the variable a, but it's not printing the value

%sh

echo $a

OUTPUT:

Re: How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?

Hubert-Dudek — Tue, 30 Aug 2022 19:59:26 GMT

Hi, bash is not supported as notebook language. Instead, it is a magic cell executing shell script on the driver node.

Re: How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?

VeeraNerella — Tue, 30 Aug 2022 20:22:58 GMT

Okay...

In data bricks notebook, is there a way to declare a bash variable as a global variable and use that variable's value in the subsequent cells in the notebook?

like mentioned in the attachment

Re: How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?

Hubert-Dudek — Tue, 30 Aug 2022 21:01:04 GMT

You can configure custom environment variables

On the cluster configuration page, click the Advanced Options toggle.
Click the Spark tab.
Set the environment variables in the Environment Variables field.

Re: How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?

VeeraNerella — Tue, 13 Sep 2022 00:12:49 GMT

Hi Kaniz, seems this is limitation in databricks notebook(S)

"How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?"

So alternatively, i used all commands in the same cell and moved forward, i didn't find any alternative as of now