topic Issues around float datatypes in Get Started Discussions

Issues around float datatypes

inagar — Fri, 12 Jul 2024 14:00:36 GMT

Float data from file, is never getting copied to the temp_table created though the schema matches.
As a workaround, using CREATE TABLE [USING] which is able to insert the file data to a temp_table. Is this a known issue?

COPY INTO issue is explained with screenshots below:

Structure of the table:

Another issue that we are observing in loading data to the Float column of a table, we see random digits are being appended. Is there any logic behind appending/truncating float values ?

Re: Issues around float datatypes

Witold — Fri, 12 Jul 2024 14:51:41 GMT

You're inferring the schema ("spark is guessing it") from the CSV. In your case either double or decimal is used. Instead, provide the schema while reading the files.

These "rounding errors" usually occur during floating-point calculations. Please make sure to use the same data type anywhere. Databricks supports at least three: FLOAT, DOUBLE and all variations of DECIMAL.

Re: Issues around float datatypes

RishabhTiwari07 — Fri, 19 Jul 2024 12:36:58 GMT

Hi @inagar ,

Thank you for reaching out to our community! We're here to help you.

To ensure we provide you with the best support, could you please take a moment to review the response and choose the one that best answers your question? Your feedback not only helps us assist you better but also benefits other community members who may have similar questions in the future.

If you found the answer helpful, consider giving it a kudo. If the response fully addresses your question, please mark it as the accepted solution. This will help us close the thread and ensure your question is resolved.

We appreciate your participation and are here to assist you further if you need it!

Thanks,

Rishabh