topic Re: Internal Errors in Databricks Queries via Go Library. in Data Engineering

Internal Errors in Databricks Queries via Go Library.

Vetrivel — Wed, 16 Oct 2024 06:01:29 GMT

While executing a query directly from the Databricks editor for the entire table, no errors are observed. However, when querying through the Go library, we encounter an internal error.

Your guidance on this issue would be greatly appreciated.

Re: Internal Errors in Databricks Queries via Go Library.

gchandra — Thu, 17 Oct 2024 02:27:17 GMT

Can you share the sample code, error message screenshot, and the number of rows you are trying to load via Go CLI that caused the error?

Re: Internal Errors in Databricks Queries via Go Library.

michael569gardn — Thu, 17 Oct 2024 04:47:20 GMT

@Vetrivel wrote:
While executing a query directly from the Databricks editor for the entire table, no errors are observed. However, when querying through the Go library, we encounter an internal error.
Your guidance on this issue would be greatly appreciated.

Hello @Vetrivel floridablue ,
It sounds like a frustrating issue! Since the query works in the Databricks editor but not through the Go library, it might be a configuration or environment issue. Have you tried updating the Go SDK to the latest version and ensuring all dependencies are correctly set up? If the problem persists, consider checking the Databricks community forums or GitHub issues for similar cases.

Best Regards,
Michael Gardner

Re: Internal Errors in Databricks Queries via Go Library.

Vetrivel — Thu, 17 Oct 2024 04:54:42 GMT

Attached the error screenshot for the same.

Re: Internal Errors in Databricks Queries via Go Library.

gchandra — Thu, 17 Oct 2024 11:24:53 GMT

Can you share the script and details like the number of rows, etc.?

Re: Internal Errors in Databricks Queries via Go Library.

Vetrivel — Thu, 17 Oct 2024 13:10:57 GMT

I have attached a sample script (not the exact version but similar logic), and we are attempting to retrieve 150 million rows. Same SQL is working for fetching 1 million rows but fails when we are trying to fetch all rows in table.
Could you please advise on the maximum number of rows that can be retrieved?