topic Re: Databricks Logs some error messages while trying to read data using databricks-jdbc dependency in Data Engineering

Databricks Logs some error messages while trying to read data using databricks-jdbc dependency

Binesh — Tue, 16 May 2023 17:08:05 GMT

I have tried to read data from Databricks using the following java code.

String TOKEN = "token...";
String url = "url...";
 
Properties properties = new Properties();
properties.setProperty("user", "token");
properties.setProperty("PWD", TOKEN);
 
Connection connection = DriverManager.getConnection(url, properties);
String query = "select * from schema.table";
 
Statement statement = connection.createStatement();
ResultSet rs = statement.executeQuery(query);
 
while(rs.next()) {
    int colCount = rs.getMetaData().getColumnCount();
    for(int col=1; col<=colCount; col++){
        System.out.print(rs.getString(col) + " ");
    }
 
    System.out.println();
}

But every time when I run this java program, some errors are being logged by StatusLogger and I don't know how to resolve those errors.

Note:

Apart from databricks-jdbc dependency, I also have log4j as a dependency.

Things I have noticed:

If I remove log4j from my dependency, no errors are being logged in the console.

Can someone please help me resolving this issue?

Can someone also please explain me why databricks-jdbc has an internal log4j instead of using log4j as its dependency?

Re: Databricks Logs some error messages while trying to read data using databricks-jdbc dependency

shan_chandra — Thu, 15 Jun 2023 17:56:36 GMT

@Binesh J - The issue could be due to the data type of the column is not compatible with getString() method in line#17. use getObject() method to retrieve the value as a generic value and then convert to string.

Re: Databricks Logs some error messages while trying to read data using databricks-jdbc dependency

Binesh — Fri, 23 Jun 2023 16:39:40 GMT

@Shanmugavel Chandrakasu - Thanks for the reply.

I have tried using the getObject() method in line#17 but I can still see those error logs in my console. I have tried debugging my program and found out that these exceptions were logged when line#14 is getting executed.

Can you also please explain to me why databricks-jdbc has an internal log4j instead of using log4j as its dependency?