Guides
Shape Help Center

How Historical Data Syncing Works in NinjaCat

NinjaCat supports historical data syncing when data is actively required by the platform. Simply connecting a data source to an account makes data available, but it does not trigger a sync on its own.

Historical syncing begins only when a connected data source is used by an active system such as a dataset, report, dashboard, or monitoring feature.

This article explains:

  • What actions actually trigger historical data syncing
  • How different NinjaCat features request historical data
  • How historical depth is determined
  • Known limitations and when Support assistance may be required

What Triggers Historical Data Syncing in NinjaCat?

Historical data syncing starts only when data is requested by an active consumer within NinjaCat.

Examples of features that trigger syncing include:

  • Data Cloud datasets
  • Reports
  • Dashboards
  • Other connected systems that actively query provider data

Important
Adding a data source connection to an account does not trigger historical syncing by itself. Syncing occurs only when that connection is used by a feature that requires data.

Common Scenarios That Trigger Historical Syncing

Scenario 1: Using a Data Source in a Dataset

When you create a Data Cloud dataset using accounts with existing data source connections, NinjaCat:

  • Requests historical data from provider APIs
  • Pulls data based on the dataset's configuration
  • Applies the dataset's backfill date and sync rules

Example:
You create a Google Ads dataset and select five client accounts with active Google Ads connections. NinjaCat pulls historical Google Ads data based on the dataset backfill date and provider limitations.

Scenario 2: Running a Report or Dashboard

When a report or dashboard is generated using provider data, NinjaCat:

  • Requests historical data needed to satisfy the selected date range
  • Pulls data from connected providers as required
  • Stores and caches data according to internal system rules

Example:
You generate a report with a "Last 12 Months" date range using Facebook Ads data. NinjaCat pulls the historical Facebook Ads data required to generate that report.

How Historical Data Depth Is Determined

There is no single global historical limit that applies to all NinjaCat data. Historical depth is determined by several factors working together.

About the ~2-Year Historical Data Default

NinjaCat applies a configurable per-provider default to how far back historical data can be retrieved for a new sync. This default is roughly 2 years (~730 days), but it is not a single universal hard-coded limit — the actual historical window may be shorter or longer depending on the data source. Each provider can carry its own value, so the effective limit varies by source.

A few important clarifications:

  • The historical floor is dynamic, not a fixed date. The oldest retrievable date is calculated relative to today (roughly "today minus the provider's window"), so it shifts forward by one day every day. There is no fixed calendar cutoff.
  • The window is provider-specific. Because the default is configurable per provider, some sources allow more history and some allow less. Treat ~2 years as a typical starting point, not a guarantee.
  • Data already stored stays available. The default governs how far back a new retrieval can reach. Long-tenured accounts that have been pulling data for years may already hold five or more years of stored historical data, and that data remains available.
  • Provider limits can make the effective window shorter. Some providers, metrics, or dimensions support far less history (see Provider and API Limitations below).

Extending an individual manual sync

When running a manual sync, there is an option in the manual sync screen to extend that individual sync's start date beyond the standard window. This applies only to the single manual sync you are running — it does not change the per-provider default or affect automated/nightly syncs.

If you are unsure whether to use this option, or you need historical data beyond the default window and aren't sure how to retrieve it, contact NinjaCat Support and we'll help you determine the best approach for your situation.

Note on date validation and who can relax it
The option that relaxes the historical-floor check (ignore_min_date_validation) is exposed in the manual sync UI only to super-admins — a normal authenticated user will not see the toggle. (Note that the server-side endpoint itself does not enforce a permission check; the gating is applied in the UI.) When the historical-floor check is relaxed, it only relaxes the historical (past-date) floor. Future-date validation is always enforced — a start or end date later than the current date is rejected (HTTP 400) regardless of any setting. You cannot request data with a future date.

1. Account Tenure and System History

  • Newer accounts generally begin with a limited historical window
  • Long-tenured accounts may already contain five or more years of historical data
  • Data that was pulled and stored previously remains available unless explicitly removed

2. Feature Configuration

How far back NinjaCat retrieves data depends on how the data is requested:

  • Datasets:
    Controlled by the dataset backfill date and sync configuration
  • Reports and Dashboards:
    Controlled by the selected date range and earliest reporting date

If a requested date range exceeds what the system or provider supports, NinjaCat defaults to the maximum available data.

3. Provider and API Limitations

Historical availability is also constrained by provider APIs, which are outside of NinjaCat's control.

Important considerations:

  • Each provider defines its own historical lookback limits
  • Limits can vary within the same provider
  • Some metrics or dimensions may only support 30, 90, or 180 days of history
  • Other fields may support multiple years of data

These limits are defined by provider APIs and may change over time.

Understanding "All Available Historical Data"

When NinjaCat retrieves historical data, "all available" means:

  • Data accessible based on:
    • Provider API limits
    • Specific metrics and dimensions requested
    • Feature configuration (dataset, report, dashboard, etc.)
    • Account tenure and system history

It does not guarantee that:

  • Every metric has the same historical depth
  • All providers support multi-year lookbacks
  • Data extends back to account creation

Known Limitation: Trade Desk & Simpli.fi Retroactive Accounts

There is a known limitation when working with Trade Desk and Simpli.fi.

The Scenario

  • Trade Desk or Simpli.fi data is already syncing in NinjaCat
  • A new advertiser account is added to the MCC after the initial NinjaCat connection
  • That advertiser contains historical data that was not previously accessible

In this case, NinjaCat cannot automatically backfill historical data for just the newly added account.

Why This Happens

For Trade Desk and Simpli.fi, retrieving historical data for a single newly added advertiser requires reprocessing data across the entire agency. This creates significant system load and must be handled manually.

What To Do

If this applies to you:

  1. Contact NinjaCat Support or your Customer Success Manager
  2. Provide:
    • Provider name (Trade Desk or Simpli.fi)
    • Account(s) requiring historical data
    • Requested date range
    • Business justification
  3. Support will evaluate feasibility based on system capacity

Processing times typically range from 24–72 hours, depending on data volume.

When to Contact Support for Historical Data

You may need manual assistance if:

  • You need data beyond the system's default historical behavior
  • Provider APIs prevented data retrieval
  • You notice gaps within an expected date range
  • A provider connection was reauthenticated and historical data is missing
  • You are affected by the Trade Desk or Simpli.fi limitation

Monitoring Sync Progress

You can monitor syncing from the relevant feature:

  • Datasets:
    Data Cloud → Datasets → Sync status
  • Reports and Dashboards:
    During generation or load time

Sync duration varies based on:

  • Number of accounts
  • Date range requested
  • Provider performance
  • Data volume

Key Takeaways

  • Connecting a data source does not trigger syncing by itself
  • Syncs occur only when data is actively requested
  • The ~2-year (~730-day) historical window is a configurable per-provider default — not a universal hard limit — and may be shorter or longer by data source; the floor is dynamic and shifts daily, and previously stored data remains available
  • Historical depth depends on configuration, tenure, and provider APIs
  • Some edge cases require manual Support involvement

Related Articles


Updated about 1 month ago

Did this page help you?