topic Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective. in Data Engineering

Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

KVNARK — Tue, 22 Nov 2022 12:36:38 GMT

Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective. can someone throw some light on this. Currently learning Pyspark (basics of Pythion in handling the data) and also learning Databricks SQL by free learning provided in data bricks portal itself. Any guidance would be much appreciated.

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

Ajay-Pandey — Tue, 22 Nov 2022 12:40:26 GMT

You can use spark sql to build your ETS

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

KVNARK — Tue, 22 Nov 2022 12:42:54 GMT

Databricks SQL doesn't require any prior knowledge on Pyspark right. But to build ETL pipelines, we may need to learn Pyspark, I mean for creating the ETL pipelines. Is that correct?

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

Ajay-Pandey — Tue, 22 Nov 2022 12:47:17 GMT

If you want u can use SQL alone to build a pipeline but if you have knowledge about pyspark it will add more advantages.

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

KVNARK — Tue, 22 Nov 2022 12:51:12 GMT

Thank you @Ajay Pandey

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

AmarK — Tue, 22 Nov 2022 13:30:02 GMT

@KVNARK . Could you please select @Ajay Pandey answer as Best answer if it helped you?

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

Anonymous — Tue, 22 Nov 2022 14:11:12 GMT

You can also learn the Scala API if you don't want to learn Python. It might be a valuable skill to learn Python. Either way, you can use SQL or Python for most applications. Some things such as AutoML don't require either.

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

NhatHoang — Wed, 23 Nov 2022 08:28:56 GMT

You can use Scala or R (with SparkR) if you do not want to use Python. :)

Re: Can we use Databricks or code in data bricks without learning Pyspark in depth which is used for ETL purpose and data engineering perspective.

KVNARK — Wed, 23 Nov 2022 09:01:08 GMT

Thanks All for your valuable suggestions!