I have a table pulled from Snowflake via a foreign catalog that's only returning 100 rows in Databricks even though there's billions of records in the source Snowflake table. What steps should I take to diagnose the limitation of rows?
Sorry, this ended up being user error. I was querying a different table in Snowflake than Databricks. The query federation has been working as it should.
The Snowflake tables aren't backed by Iceberg, so I can't setup catalog federation. The EXPLAIN for a count query returns 'SnowflakePlan [count(1)#13394L], MapPartitionsRDD[698] at mapPartitions at SnowflakeRelation.scala:149' both with and without a...
`SELECT COUNT(*)` with & without a `LIMIT 1000` both return a count of 100. I did run `REFRESH FOREIGN TABLE` before the queries. Currently query federation is being used. I'm going to try to setup catalog federation, since that would be preferable f...
