Databricks Community

Regarding --files option in spark submit task of Databricks jobs, would like to understand how it works and what is the syntax to pass multiple files to --files? I tried using --files and --py-files and my understanding is, it should make available t...

I have a spark submit job which is running one python file called main.py.The other file is alert.py which is being imported in main.py.Also main.py is using multiple config files.Alert.py is passed in --py-files and other config files are passed as ...

I have code files (.hql) in s3 which was running on Hive. But now, I need to run them on the Databricks cluster. I can rename the files to .sql and add comment "-- Databricks notebook source" on top of the files to treat them as single cell notebook....