Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

[KVNARK]

Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective. can someone throw some light on this. Currently learning Pyspark (basics of Pythion in handling the data) and also learning Databricks SQL by free learning provided in data bricks portal itself. Any guidance would be much appreciated.

[Ajay-Pandey]

You can use spark sql to build your ETS

[KVNARK]

Databricks SQL doesn't require any prior knowledge on Pyspark right. But to build ETL pipelines, we may need to learn Pyspark, I mean for creating the ETL pipelines. Is that correct?

[Ajay-Pandey · Accepted solution]

If you want u can use SQL alone to build a pipeline but if you have knowledge about pyspark it will add more advantages.

[KVNARK]

Thank you @Ajay Pandey​ 

[AmarK]

@KVNARK .​ Could you please select @Ajay Pandey​ answer as Best answer if it helped you?

[Anonymous]

You can also learn the Scala API if you don't want to learn Python. It might be a valuable skill to learn Python. Either way, you can use SQL or Python for most applications. Some things such as AutoML don't require either.

[NhatHoang]

You can use Scala or R (with SparkR) if you do not want to use Python. :)​

[KVNARK]

Thanks All for your valuable suggestions!