cancel
Showing results forย 
Search instead forย 
Did you mean:ย 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results forย 
Search instead forย 
Did you mean:ย 

How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?

VeeraNerella
New Contributor III

%sh

export a

a=100

echo $a

OUTPUT: 100

in the below cell trying to use the variable a, but it's not printing the value

%sh

echo $a

OUTPUT:

4 REPLIES 4

Hubert-Dudek
Esteemed Contributor III

Hi, bash is not supported as notebook language. Instead, it is a magic cell executing shell script on the driver node.

Okay...

In data bricks notebook, is there a way to declare a bash variable as a global variable and use that variable's value in the subsequent cells in the notebook?

like mentioned in the attachment

Hubert-Dudek
Esteemed Contributor III

You can configure custom environment variables

  1. On the cluster configuration page, click the Advanced Options toggle.
  2. Click the Spark tab.
  3. Set the environment variables in the Environment Variables field.

Hi Kaniz, seems this is limitation in databricks notebook(S)

"How to use a bash variable, which holds a value to the below subsequent cells in Data bricks notebook?"

So alternatively, i used all commands in the same cell and moved forward, i didn't find any alternative as of now

Connect with Databricks Users in Your Area

Join a Regional User Group to connect with local Databricks users. Events will be happening in your city, and you wonโ€™t want to miss the chance to attend and share knowledge.

If there isnโ€™t a group near you, start one and help create a community that brings people together.

Request a New Group