topic Re: connect databricks to teradata in Data Engineering

connect databricks to teradata

Rishabh-Pandey — Thu, 22 Dec 2022 10:24:40 GMT

hey i want to know can we connect databricks to the teradata database and if yes what will be the procedure ??? help would be appreciated

Re: connect databricks to teradata

User16255483290 — Thu, 22 Dec 2022 19:29:12 GMT

@Rishabh Pandey You can follow the below docs on how to connect to teradata from databricks.

https://www.cdata.com/kb/tech/teradata-jdbc-azure-databricks.rst

https://medium.com/@Tulga.Unlusoy/accessing-teradata-from-databricks-for-rapid-experimentation-in-data-science-and-analytics-projects-4c5e0b558b1

Re: connect databricks to teradata

Harshjot — Fri, 23 Dec 2022 12:40:13 GMT

Hi @Rishabh Pandey just add Teradata JDBC jar to your data bricks cluster.

https://spark.apache.org/docs/latest/sql-data-sources-jdbc.html

Re: connect databricks to teradata

jose_gonzalez — Tue, 27 Dec 2022 23:16:33 GMT

use the JDBC driver from here https://docs.databricks.com/integrations/jdbc-odbc-bi.html

Re: connect databricks to teradata

Rishabh-Pandey — Wed, 28 Dec 2022 07:47:59 GMT

thanks

Re: connect databricks to teradata

Rishabh-Pandey — Wed, 28 Dec 2022 07:48:08 GMT

thanks

Re: connect databricks to teradata

Rishabh-Pandey — Wed, 28 Dec 2022 07:48:16 GMT

thanks

Re: connect databricks to teradata

JasonMamoa — Wed, 08 Feb 2023 18:14:04 GMT

Would this connection be encrypted end to end?

Re: connect databricks to teradata

BroData — Mon, 12 May 2025 12:07:04 GMT

There are two main ways to connect to Teradata from Databricks using Python.

Way 1: Using Python Libraries (e.g., sqlalchemy, pyjdbc, pyodbc, jaydebeapi, and so on)

Pros: Provides a comprehensive solution, allowing us to: Query data, Trigger stored procedures, Perform other advanced database operations.

Cons: Only utilizes the driver node of the Databricks cluster. So, this way does not leverage the full distributed power of the Databricks cluster, which can lead to performance limitations for large datasets.

Way 2: Using PySpark and Spark JDBC API

Step 1: Install the Maven library terajdbc on the Databricks cluster.

Step 2: Read & Write

df = spark.read.format('jdbc') \
.option('driver', 'com.teradata.jdbc.TeraDriver') \
.option('url', 'jdbc:teradata://<host_name>/DBS_PORT=<port_number>,TMODE=ANSI,logmech=ldap') \
.option('user', '<user>') \
.option('password', '<password>') \
.option('query', '<query>') \
.load()

df.display()

df.write.format('jdbc') \
.option('driver', 'com.teradata.jdbc.TeraDriver') \
.option('url', 'jdbc:teradata://<host_name>/DBS_PORT=<port_number>,TMODE=ANSI,logmech=ldap') \
.option('user', '<user>') \
.option('password', '<password>') \
.option('dbtable', '<target_db>.<target_table>') \
.mode('<write_mode>') \
.save()

Note - a. Ensure the URL is correctly configured. b. Provide valid user credentials with appropriate access. c. Ensure <query> or <db_name>.<table_name> is accessible by the <user>.

Pros: Fully utilizes the distributed computing power of the Databricks cluster. So, this way offers excellent performance for reading and writing large datasets.

Cons: Spark JDBC API is primarily for DataFrame-based data I/O, not procedural/transactional logic. So, this way supports limited operations (like, we can't execute stored procedures and some other advanced database operations).

Thanks & Regards,

BroData