Great question, and you're right that it's not a storage limitation — the variant encoding does have a null type (ID 0), as your Parquet link shows. The answer lives in Spark's type system, and there's actual evidence in the source and JIRAs. Fair warning: no design doc states this in one sentence, so what follows is reconstructed from the code and the tickets — but the pieces fit tightly.

Part 1: the top-level cast doesn't really "work" — it's vacuous. Spark's generic rule allows NullType → any type at analysis, and at runtime the nullSafeEval guard short-circuits: NULL in, NULL out, the cast function never executes. There's literally a comment in Cast.scala saying primitive-level casts never reach the NullType branch because of that guard — only *nested* null-type fields inside a struct reach it, and that branch throws cannotCastFromNullTypeError. So CAST(void_col AS VARIANT) never encodes a variant null; it hands you a SQL NULL that happens to be typed VARIANT. Your screenshot shows exactly that: plain nulls, no variant value ever built.

Part 2: the nested case is rejected because there, Spark would have to actually encode something — and that forces a choice it deliberately refuses to make: conflating SQL NULL with variant null. Those are different things in the variant model:

SELECT parse_json('null') IS NULL; -- false: a non-null VARIANT holding a variant-null
SELECT CAST(NULL AS VARIANT) IS NULL; -- true: a SQL NULL, nothing was encoded
SELECT to_json(parse_json('null')); -- 'null'
SELECT to_json(CAST(NULL AS VARIANT)); -- NULL

SPARK-51576 shows the project actively guarding this boundary: variant is the only type where a non-null value can cast into a null value (parse_json('null')::string → NULL), and the cast rules were tightened precisely because that conflation leaks. Now look at your struct: every value in a VOID field is a SQL NULL — typed absence. Encoding it into a variant object would require writing variant null, i.e. manufacturing "a value that says null" out of "no value." Round-trip that back out and you can no longer tell which one you started with. Rather than pick a lossy convention, the cast is rejected at analysis — hence CAST_WITHOUT_SUGGESTION, an error class that means "there is no correct suggestion to offer."

There's also a longer historical thread: NullType/VOID in Spark is a type-inference placeholder (so that literal NULLs type-check), not a materializable type. It's been progressively banned from contexts that persist or construct values — you can't create tables with VOID columns, and nested null-type casts used to crash outright with a MatchError (SPARK-27671) before becoming a proper error. Excluding it from variant *construction* while letting the vacuous top-level cast pass through is consistent with that stance: the asymmetry you found isn't VOID being partially supported in variant — it's that the top-level case never touches variant at all.

So the design decision, best reconstructed: (1) SQL NULL ≠ variant null, and casts must not silently convert one into the other; (2) VOID is a placeholder type that never materializes into constructed values. Top-level slips through because nothing is constructed; nested fails because something would have to be.