topic How can I fix strange magic command behaviour in Databricks notebooks? in Data Engineering

How can I fix strange magic command behaviour in Databricks notebooks?

jowziph — Wed, 28 Sep 2022 12:16:25 GMT

Hi,

I have encountered a strange issue when using magic commands on my Azure Databricks workspace.

When I create a cell and type the magic command for a language which is not default in the notebook, the code is instead interpreted as the default notebook language.

Here you can see a notebook with python as the default language, with a sql magic command:

And here, the same notebook. But with python code written in the cell.

However, if I create a cell at the end of the notebook (not in between existing cells). And then set the language to SQL using this button, the code will be correctly interpreted as SQL.

This issue first arose this week on Monday the 26th of September. It has broken a lot of my existing notebooks which use "%sql".

I have tried restarting my cluster, creating a new cluster, creating a new notebook, and changing my notebook default language. But none of these have fixed the issue.

Thanks,

Joe

Re: How can I fix strange magic command behaviour in Databricks notebooks?

jowziph — Wed, 28 Sep 2022 14:40:11 GMT

Hi,

Yeah this is happening when magic command is used in between two cells of the default language. But this is not the only case.

This is happening when magic command is manually typed out (e.g. %sql or %scala in a python notebook). OR when magic command is selected from the language button and the cell is not a brand new cell at the bottom of the notebook.

The issue is making it difficult for me to develop multi language notebooks.

Re: How can I fix strange magic command behaviour in Databricks notebooks?

Debayan — Wed, 28 Sep 2022 13:30:00 GMT

Hi, Is this happening when magic command is used in between two cells of the default language?

You can override the default language in a cell by clicking the language button and selecting a language from the dropdown menu. Alternately, you can use the language magic command %<language> at the beginning of a cell. The supported magic commands are: %python , %r , %scala , and %sql .