- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
10-08-2025 09:02 AM
If you are on a Spark version that supports .option("query", ...), you can do:
df = spark.read \
.format("jdbc") \
.option("url", jdbc_url) \
.option("query", "SELECT TOP 10 * FROM Customer") \
.option("user", "xxxx") \
.option("password", "xxx") \
.option("driver", "com.netsuite.jdbc.openaccess.OpenAccessDriver") \
.load()
display(df)
But note that some JDBC drivers, including NetSuite, may have restrictions, so review your Spark and JDBC connector documentation and test with preliminary table queries first.
Troubleshooting Common Issues
-
Jar File Location: Confirm the .jar is on all Spark worker nodes and that the cluster recognizes the
com.netsuite.jdbc.openaccess.OpenAccessDriverclass. -
NetSuite Permissions: Double-check that the user/role (AccountID, RoleID) has correct access rights to view the "Customer" table.
-
Port and Endpoint: Ensure firewall/network settings allow traffic to the required NetSuite JDBC endpoint and port.
-
Driver Class: Confirm the driver class is spelled correctly and matches your .jar file.
-
Error Messages: Carefully check any exception traceback for specific hints; common errors include authentication failures, missing drivers, or SQL syntax not supported by NetSuite JDBC.
Recommendations
-
Start with
.option("dbtable", "Customer")for a basic read. -
Gradually introduce more complex queries if
.option("query", ...)is supported. -
Review NetSuite JDBC and Spark documentation for compatibility notes.