11-22-2022 04:36 AM
Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective. can someone throw some light on this. Currently learning Pyspark (basics of Pythion in handling the data) and also learning Databricks SQL by free learning provided in data bricks portal itself. Any guidance would be much appreciated.
11-22-2022 04:47 AM
If you want u can use SQL alone to build a pipeline but if you have knowledge about pyspark it will add more advantages.
11-22-2022 04:40 AM
You can use spark sql to build your ETS
11-22-2022 04:42 AM
Databricks SQL doesn't require any prior knowledge on Pyspark right. But to build ETL pipelines, we may need to learn Pyspark, I mean for creating the ETL pipelines. Is that correct?
11-22-2022 04:47 AM
If you want u can use SQL alone to build a pipeline but if you have knowledge about pyspark it will add more advantages.
11-22-2022 04:51 AM
Thank you @Ajay Pandey
11-22-2022 05:30 AM
@KVNARK . Could you please select @Ajay Pandey answer as Best answer if it helped you?
11-22-2022 06:11 AM
You can also learn the Scala API if you don't want to learn Python. It might be a valuable skill to learn Python. Either way, you can use SQL or Python for most applications. Some things such as AutoML don't require either.
11-23-2022 12:28 AM
You can use Scala or R (with SparkR) if you do not want to use Python. :)
11-23-2022 01:01 AM
Thanks All for your valuable suggestions!
Join our fast-growing data practitioner and expert community of 80K+ members, ready to discover, help and collaborate together while making meaningful connections.
Click here to register and join today!
Engage in exciting technical discussions, join a group with your peers and meet our Featured Members.