topic Re: Why am I getting NameError name ' ' is not defined in another cell? in Data Engineering

Why am I getting NameError name ' ' is not defined in another cell?

Ank — Mon, 22 Aug 2022 23:33:12 GMT

I defined a dictionary variable Dict, populated it, and print(dict) in the first cell of my notebook. In the next cell, I executed the command print(dict) again. However, this time it gave me an error

NameError: name 'Dict is not defined

How can that be? Why am I able to print this variable in one cell, but unable to do so in the next cell?

Re: Why am I getting NameError name ' ' is not defined in another cell?

Morg — Mon, 31 Jul 2023 12:20:54 GMT

I am facing the same issue while executing a job currently. Was a cause/solution found for this?

Re: Why am I getting NameError name ' ' is not defined in another cell?

Tharun-Kumar — Tue, 01 Aug 2023 04:48:30 GMT

@Ank @Morg

One possible scenario, when this could happen is the variable (dict) was defined in a python environment and it was called in a scala environment or the vice versa.

Re: Why am I getting NameError name ' ' is not defined in another cell?

Tharun-Kumar — Tue, 01 Aug 2023 04:49:20 GMT

A variable defined in a particular language environment will be available only in that environment. It will not be available in other environments. Can you confirm?

@Ank @Morg

Re: Why am I getting NameError name ' ' is not defined in another cell?

Morg — Wed, 09 Aug 2023 13:19:03 GMT

I can confirm that your answer at least pointed me in the right direction.

It seems my issue wasn't exactly the same as I was instead using the magic pip command to install a notebook scoped package:
%pip install <some-package>

Anything defined within that command cell seems to be lost so I had to separate this into it's own command before everything else.

Thank you for the help!

Re: Why am I getting NameError name ' ' is not defined in another cell?

erigaud — Wed, 09 Aug 2023 14:20:58 GMT

Running pip install restarts the interpreter, meaning that any variable defined prior to the pip install is lost, so indeed the solution is so run the pip install first, or better is to add the library you want to installl directly to the cluster configuration