Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

KVNARK
Honored Contributor II

‎11-22-2022 04:36 AM

Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective. can someone throw some light on this. Currently learning Pyspark (basics of Pythion in handling the data) and also learning Databricks SQL by free learning provided in data bricks portal itself. Any guidance would be much appreciated.

Labels:

Ajay-Pandey
Databricks MVP

‎11-22-2022 04:40 AM

You can use spark sql to build your ETS

Ajay Kumar Pandey

KVNARK
Honored Contributor II
In response to Ajay-Pandey

‎11-22-2022 04:42 AM

Databricks SQL doesn't require any prior knowledge on Pyspark right. But to build ETL pipelines, we may need to learn Pyspark, I mean for creating the ETL pipelines. Is that correct?

Ajay-Pandey
Databricks MVP
In response to KVNARK

‎11-22-2022 04:47 AM

If you want u can use SQL alone to build a pipeline but if you have knowledge about pyspark it will add more advantages.

Ajay Kumar Pandey

View solution in original post

KVNARK
Honored Contributor II

‎11-22-2022 04:51 AM

Thank you @Ajay Pandey​ 

AmarK
Databricks Employee
Databricks Employee

‎11-22-2022 05:30 AM

@KVNARK .​ Could you please select @Ajay Pandey​ answer as Best answer if it helped you?

Anonymous
Not applicable

‎11-22-2022 06:11 AM

You can also learn the Scala API if you don't want to learn Python. It might be a valuable skill to learn Python. Either way, you can use SQL or Python for most applications. Some things such as AutoML don't require either.

NhatHoang
Valued Contributor II

‎11-23-2022 12:28 AM

You can use Scala or R (with SparkR) if you do not want to use Python. :)​

KVNARK
Honored Contributor II

‎11-23-2022 01:01 AM

Thanks All for your valuable suggestions!