cancel
Showing results for 
Search instead for 
Did you mean: 
Data Engineering
Join discussions on data engineering best practices, architectures, and optimization strategies within the Databricks Community. Exchange insights and solutions with fellow data engineers.
cancel
Showing results for 
Search instead for 
Did you mean: 

Forum Posts

HariharaSam
by Contributor
  • 16493 Views
  • 3 replies
  • 4 kudos

Using variables in Spark SQL

Is there a way to declare variables in Spark SQL like we do it in T-SQL?

  • 16493 Views
  • 3 replies
  • 4 kudos
Latest Reply
Debayan
Databricks Employee
  • 4 kudos

Could you please follow the below link and let us know if this helps? https://community.databricks.com/s/question/0D53f00001HKHa3CAH/how-do-i-pass-parameters-to-my-sql-statements

  • 4 kudos
2 More Replies
lizou
by Contributor II
  • 5686 Views
  • 6 replies
  • 3 kudos

Resolved! Random error related to dynamic variables in SQL - at least one column must be specified (shallowclone table)

Environment: AzureA data transformation template that take s table name as variable. The variable is set in separate notebook %run ../_configShallow clone is used to generate a staging table with exact same columns of target table.A random error occ...

image
  • 5686 Views
  • 6 replies
  • 3 kudos
Latest Reply
Anonymous
Not applicable
  • 3 kudos

@lizou​ - That's great! I'm so glad. Would you be happy to share your solution with us and then mark the answer as best? That will help others find the solution more quickly.

  • 3 kudos
5 More Replies
Siddhesh2525
by New Contributor III
  • 5753 Views
  • 2 replies
  • 6 kudos

How to pass dynamic value in databricks

I have separate column value defined in 13 diffrent notebook and i want merge into 1 databrick notebook and want to pass dynamic parameter using databrick so it will help me to run in single databricks notebook .

  • 5753 Views
  • 2 replies
  • 6 kudos
Latest Reply
Prabakar
Databricks Employee
  • 6 kudos

Hi @siddhesh Bhavar​ you can use widgets with the %run command to achieve this. https://docs.databricks.com/notebooks/widgets.html#use-widgets-with-run%run /path/to/notebook $X="10" $Y="1"

  • 6 kudos
1 More Replies
Labels