Data Types - varchar(2147483647)

yumnus — Wed, 27 Nov 2024 10:26:37 GMT

Hi,

when I read a PostgreSQL table containing a custom datatype, it gets translated to VARCHAR(2147483647).

I would like to understand how Databricks and Delta handle this scenario. Specifically, does Delta store all the bytes for the maximum length of 2147483647 characters, or does it only store the bytes for the actual characters used?

For example, if I use only 1000 characters instead of the full capacity, does Delta still allocate storage for the entire 2147483647 characters?

Thank you and best regards

Re: Data Types - varchar(2147483647)

Alberto_Umana — Wed, 27 Nov 2024 13:50:59 GMT

Hello @yumnus,

Delta does not allocate storage for the entire length (2147483647) characters if you only use a portion of it. Instead, Delta stores only the bytes for the actual characters used.

topic Re: Data Types - varchar(2147483647) in Data Engineering

Data Types - varchar(2147483647)

Re: Data Types - varchar(2147483647)