Incident: June 22 – July 3

📘

What Happened?

Between late June and early July, a change on Google's side to how their advertising API limits data-heavy requests caused Shape's Google Ads spend data to update less frequently than normal. During this period, some Google Ads campaigns refreshed their spend more slowly than usual, which in some cases delayed the automated budget pacing Shape is designed to perform — and could allow a campaign to run past its budget before pacing caught up.

Impact Summary

• Google Ads spend figures in Shape updated on a delay for a period between roughly June 22 and July 3.
• Automated budget pacing (Cruise Control) and automated pausing (AutoPilot) could act on spend data that was behind real time.
• The greatest exposure was on budgets at or near their cap with automation enabled, where the delay could contribute to overspend.
• Other advertising platforms were unaffected.

Timeline

Sunday, June 22 (Onset)

Google Ads began intermittently rejecting the requests Shape uses to retrieve campaign spend. Errors were low at first and grew over the following days.

Friday, June 26 – Monday, June 29 (Escalation)

The rejections increased and became more frequent as end-of-month activity peaked. We opened a support case with Google and began deploying mitigations.

Tuesday, June 30 (Peak and turning point)

Errors reached their highest point. That evening we deployed a change that streamlined how Shape requests Google Ads data, and spend syncing recovered to normal within the hour.

Wednesday, July 1 – Thursday, July 3 (Resolution)

Error levels fell back to normal and Google Ads spend data was updating on its regular schedule again.

Root Cause

Google introduced a stricter limit on the computational "cost" of certain data-heavy queries. Shape's routine per-campaign spend request pulled a broad set of metrics that pushed it over Google's new limit, so Google began throttling that request. When those requests were throttled, the affected campaigns' spend figures did not refresh on schedule, and Shape's automated pacing acted on older data until the next successful update.

Our Response

Immediate Actions (June 29 – July 3)

• Opened a priority support case with Google.
• Streamlined the data Shape requests on each budget check so it stays within Google's limits — impression-share data now refreshes once daily through a separate process rather than on every check.
• Adjusted how these requests are scheduled and retried to avoid bursts.

Monitoring Improvements (Completed early July)

• Added detailed capture of Google's throttling responses so this signal is visible immediately.
• Added alerting on this specific error so we detect any recurrence within minutes.

What We're Doing to Prevent This

• ✅ Streamlined the core spend query so it stays within Google's limits.
• ✅ Added real-time monitoring and alerting for Google API throttling.
• ✅ Added an automatic rate limiter and retry-with-backoff for Google requests.
• 🔄 Working with Google to pursue ad-account reimbursement for affected spend.
• 🔄 Continuing to harden the remaining Google data paths against similar limits.

Our Commitment to You

Shape exists to keep your budgets pacing accurately, and we hold ourselves to that. We've resolved the underlying issue, strengthened our monitoring so we catch this class of change from Google far sooner, and we're actively working affected accounts.

Questions or Concerns?

If you have questions about a specific budget or account, please reach out to our support team and we'll be glad to help.