Monitor Materialization Refresh Status
Monitor Materialization Refresh Status
For transformation datasets that have been enabled with materialization (set up with NinjaCat support), the Dataset Details page shows an additional Materialization panel alongside the publish status. This panel surfaces the current state of the materialized dataset's refresh pipeline directly in the UI — no need to contact support or inspect backend logs.
NoteThe Materialization panel only appears for datasets that have been flagged as materialized. It will not be visible on standard transformation datasets. This feature can only be enabled by the Ninjacat team, on an as-needed basis.
The Materialization panel
The panel contains the following fields, displayed using their exact UI labels:
| Field | Description |
|---|---|
| Refresh Mode: | How the materialized dataset refreshes — for example, INCREMENTAL or AUTO. |
| Reason: | Shown when there is a refresh or downgrade reason to surface. |
| Last Refresh: | The result of the most recent refresh attempt — for example, SUCCEEDED or FAILED. |
| Error: | When Last Refresh is FAILED, an Error: line appears beneath it with the specific error detail. |
| Freshness: | How current the materialized data is. |
| Lag vs Target: | How far behind the target lag the data currently is. An Over target indicator appears when the lag exceeds the configured target. |
| Scheduling: | Whether refreshes are currently Scheduled or Suspended. |
| Estimated Rows: | The estimated row count of the materialized dataset. |
| Size: | The storage size of the materialized dataset. |
Refresh Mode cost note
INCREMENTAL refreshes are preferred as only portions of the data that have been modified need to be computed for change. This means that updates to data are available more quickly and with a lower compute cost.
However, several factors can prevent INCREMENTAL refreshes. When Refresh Mode is not INCREMENTAL (AUTO or another full-refresh mode), the frequency of data refreshes is reduced, and changes to source data require that the full target table be rebuilt. When running in an AUTO or FULL refresh mode the panel may display the following notice:
"Full/Auto refresh mode may incur additional costs due to compute requirements."
Review this with your NinjaCat account team if you have questions about the performance or compute cost implications for your materialized datasets.
Suspended refreshes
When Scheduling is Suspended, the panel displays a Refreshes Suspended callout. Suspended refreshes mean the materialized dataset is no longer being updated on its schedule. This is typically due to errors in the SQL. Try republishing the current version of the transformation and/or editing the transformation to correct any errors. Contact NinjaCat support if this does not resolve the issues with a materialized transformation.
Served as a view
If a materialized dataset is currently being served as a view rather than a materialized table, this means the Transformation is unable to be processed as a Materialized view. The panel will display the following notice instead of the standard refresh fields:
"This materialized transformation is currently served as a view, so there is no dynamic-table refresh status."
This typically occurs during initial setup or following a refresh failure that caused a downgrade from materialized table to view. Contact support for assistance with this issue.
Troubleshooting with the Materialization panel
| Scenario | What to check |
|---|---|
| Data appears stale | Check Freshness and Lag vs Target. An Over target indicator means data is behind schedule. |
| Last refresh failed | Review the Error: detail line beneath Last Refresh for the specific failure reason. |
| Refreshes stopped running | Check Scheduling — a Refreshes Suspended callout means the refresh schedule is paused. Try editing the transformation to correct any errors and re-save. Contact NinjaCat support for additional assistance. |
| Panel shows "served as a view" | The dataset has been downgraded from a materialized table to a view. Check the Error: line (if visible) and contact NinjaCat support. |
| Unexpected compute costs | Review Refresh Mode — AUTO or full-refresh modes consume more compute. Check the message for why Snowflake is unable to enable incremental updates and attempt to modify the query to address the underlying issues. |
Common issues that force FULL or AUTO refresh
To reduce data latency and compute, it's best to structure any SQL transformation so that INCREMENTAL refreshes can be applied.
- Avoid the use of DOUBLE or FLOAT data types. When building queries make sure to avoid the use of floating point numbers. Use a static precision/scale NUMBER(scale, precision) data type (ie:
NUMBER(20, 4)) - Use Common Table Expressions (CTEs) instead of nested SQL statements. Meaning, avoid `SELECT ... FROM ( SELECT ... ) WHERE ...
- Don't use functions or keywords like
CURRENT_TIMESTAMP,ANY_VALUE(...), etc that change on every execution
Related articles
- Track Changes to Transformation Datasets with Version History
- Transform Data with SQL Transform Builder
- Managing Your Datasets in Data Cloud
Updated 27 days ago