- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Report Inappropriate Content
2 weeks ago
Hi dax60,
What you are seeing comes down to the difference between processing limits and UI rendering limits in Visual Data Prep. When you set Rows scanned to Max you are instructing the engine to process all records from the source table for your interactive session. However, the preview pane itself has a separate cap on how much data it can render in your browser. The Truncated data warning is the key here as it indicates the UI is only displaying a subset of the data it actually fetched.
Preview rendering limit is not strictly based on a hardcoded row count and hence acts differently for different tables. It's dictated by the overall memory footprint of the data payload. Tables with wider columns, higher column counts, or heavier string sizes will hit that rendering ceiling much faster, causing one table to truncate at 2,692 rows while a narrower table might display entirely.
The 4,168 count in the direct SQL query is the source of truth. The truncation is purely a limitation of the interactive preview canvas and does not apply to the actual compute execution. Scheduled jobs and pipeline runs bypass these UI preview limits and will always process the full dataset.
To verify this in your workspace, you can write the output of this Source operator to a temporary target table. Trigger a full pipeline run and when you query that new target table via SQL, you will likely see all 4,168 rows successfully materialized.