cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 
Help
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Turn on suggestions
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

Start a conversation
shamly
by • New Contributor III
  • 4703 Views
  • 4 replies
  • 4 kudos

Urgent - Use Python Variable in shell command in databricks notebook

I am trying to read a csv and do an activity from azure storage account using databricks shell script. I wanted to add this shell script into my big python code for other sources as well. I have created widgets for file path in python. I have created...

Data Engineering
Reply
  • 4703 Views
  • 4 replies
  • 4 kudos
View Replies
Latest Reply
SS2
Valued Contributor
  • 4 kudos

You can mount the storage account and then can set env level variable and can do the operation that you want.

  • 4 kudos
3 More Replies
Data_Engineer3
by • Contributor III
  • 2306 Views
  • 2 replies
  • 1 kudos

Unable to access Scala and python variables in-between shells in same notebook.

I am facing issue in while accessing python data frame in Scala shell and vice versa. I am getting error variable not defined.

Data Engineering
Reply
  • 2306 Views
  • 2 replies
  • 1 kudos
View Replies
Latest Reply
tomasz
Databricks Employee
  • 1 kudos

The context is not shared between Scala and Python so you won't be able to access the same variables directly. However you can use createOrReplaceTempView to create a temporary view of your dataframe and read it in the other language with read_df = s...

  • 1 kudos
1 More Replies
Labels
Top