https://community.databricks.com/t5/data-engineering/can-i-run-data-engineering-data-transformation-jobs-using/m-p/22522#M15434 <P>Generally, interactive clusters and jobs are better suited for data engineering and transformations as they support more than just SQL. However, if you are using pure SQL, then endpoints can be used for data transformations. All of the Spark SQL functionality is available in DSQL so you can create tables, insert/update data, and run multiple SQL statements within a single DSQL query. </P><P></P><P>Note that DSQL queries are mostly purposed for visualizations and data acquisition. </P> Fri, 18 Jun 2021 20:52:21 GMT Ryan_Chynoweth 2021-06-18T20:52:21Z https://community.databricks.com/t5/data-engineering/can-i-run-data-engineering-data-transformation-jobs-using/m-p/22521#M15433 Fri, 18 Jun 2021 19:27:07 GMT https://community.databricks.com/t5/data-engineering/can-i-run-data-engineering-data-transformation-jobs-using/m-p/22521#M15433 aladda 2021-06-18T19:27:07Z https://community.databricks.com/t5/data-engineering/can-i-run-data-engineering-data-transformation-jobs-using/m-p/22522#M15434 <P>Generally, interactive clusters and jobs are better suited for data engineering and transformations as they support more than just SQL. However, if you are using pure SQL, then endpoints can be used for data transformations. All of the Spark SQL functionality is available in DSQL so you can create tables, insert/update data, and run multiple SQL statements within a single DSQL query. </P><P></P><P>Note that DSQL queries are mostly purposed for visualizations and data acquisition. </P> Fri, 18 Jun 2021 20:52:21 GMT https://community.databricks.com/t5/data-engineering/can-i-run-data-engineering-data-transformation-jobs-using/m-p/22522#M15434 Ryan_Chynoweth 2021-06-18T20:52:21Z