Guides
Shape Help Center
Start typing to search…

Data, NinjaCat, and You

All of our efforts go into accessing, aggregating, and delivering your data in the most helpful way we can. For this effort, we maintain an always growing number of direct connections to data source providers. Inside these direct connections, data is requested and delivered through APIs (Application Programming Interfaces) These APIs are created and managed by the data source developer and leveraged by us to gather your data from them.

Not all APIs are created equal. While using NinjaCat you may be interacting with multiple data sources in a single template. Maybe even in a single widget! Pulling data from multiple APIs leads to inconsistent experiences in data display. In order to increase consistency across data source behavior and in data display, NinjaCat employs a data warehouse strategy across all of our data sources.

Expected Behavior

In brief, to supply data to the Template Builder, Reports, and Dashboards, NinjaCat:

A) At a default time, every morning receives data from

  • APIs for direct data source connections
  • Server connections like SQL, FTP, and Amazon S3 for custom data
  • Data sources that we don't have API connections for via URL, automated Email, and Google Sheets

B) Warehouses that data in our own data warehouse

C) Delivers the warehoused data when requested in:

  • The NinjaCat Template Builder
  • Reports
  • Dashboards
📘

Data from the current day may show slight discrepancies compared to the data source platform due to the timing of the daily refresh. These discrepancies are automatically corrected by the next day's refresh.

Change Default Refresh Hour

1. Go to the Setup Settings

  1. Click on the desired Data Source Network
  2. Click on the field under the "Refresh Hour" label
  3. Select the desired hour from the resulting drop-down menu
  4. Click the "Save Network" button near the bottom left

Manually Refresh the Data Warehouse

In addition to the scheduled daily refresh, you can manually trigger a data warehouse refresh at any time. This is useful when:

  • You've updated data in a custom data source (e.g., Google Sheets, SQL, Amazon S3) and need the changes reflected immediately
  • You need to verify that recent data source changes are pulling correctly
  • A report or dashboard needs up-to-date data before the next scheduled refresh

How to Refresh the Data Warehouse

1. Go to the Setup Settings

  1. Click the Refresh Data Warehouse button
  2. In the dialog that appears:
  • Select the Network Type from the dropdown to choose which data source to refresh
  • Choose the specific Network (account/connection) you want to refresh, or select all
  1. Click Refresh to start the process
📘

The refresh is queued and processed in the background. Depending on system load and the size of the data source, it may take several minutes to complete. You can continue working in NinjaCat while the refresh processes.

🚧

Important: The Refresh Data Warehouse button refreshes the cached data in NinjaCat's warehouse. It does not pull new data from the external source — it re-processes and re-caches data that has already been ingested. If you need to pull new data from an external source ahead of the next scheduled sync, you may also need to trigger a backfill or wait for the next scheduled data pull.

Permissions

The ability to refresh the data warehouse is controlled by user role permissions. If you don't see the Refresh Data Warehouse button, contact your NinjaCat administrator to verify your role includes this permission.

For custom data sources, the Allow Cache Clear On Custom Data Sources setting must also be enabled at the agency level. See Creating and Customizing User Roles for more information on managing permissions.

Updated 16 days ago