Table of contents
Research context
This protocol measures how often a third-party API outage reaches an offshore support queue without a prepared fallback. The unit is one vendor dependency on one customer path at the observation date. A fallback means a written, runnable response that names the owner, the degraded mode, and the customer message. The study describes outage preparedness, not the reliability of the vendor.
Define the vendor list before collection. Include payment, messaging, identity, mapping, shipping, and any API that sits inside checkout, login, or support. Record the contract owner and the support channel for each. Publish the list with a version so a later reviewer can reproduce the scope.
Define a fallback by the customer symptom it addresses, the degraded mode, the decision owner, and the evidence of a test. Require the degraded mode to be usable by the people on shift without extra approval. Record the customer message and the update cadence. A contact number alone is not a fallback.
Set the population as every vendor on the agreed customer path on the observation date. Include vendors with a recent incident and vendors with no incident in the window. Record the vendors that could not be assessed and the reason. Report them separately from vendors that failed the check.
Classify each vendor by fallback evidence. Use categories such as no written note, a note never tested, a note tested within the review window, and a test that failed. Do not merge the categories after seeing the result. The categories point to different fixes.
Record whether the vendor publishes a status page and whether the team monitors it. Note the contract response terms and the escalation path. Record the named decision owner and the backup. A vendor with a status page and no internal owner is still unprepared.
Report the number of vendors, the number with a tested fallback, and the number in each category. Show counts by customer path and by contract size. Present the vendors with no note and the vendors with a failed test as separate rows. Use the same denominator for every rate.
Separate a missing fallback from a fallback that failed in a drill. A missing fallback has no written response, while a failed fallback has evidence and a defect. Report the two categories separately because the fixes differ. Record the drill date and the failed step beside each finding.
Inspect the vendors that share a customer path or a contract owner. A cluster of unprepared vendors may mean the team never mapped the path end to end. Compare the pattern with the incident history. A single gap can be an oversight, while a cluster is a planning gap.
Practical application
Do not treat a vendor's uptime claim as proof that the customer path is safe. The claim may exclude the endpoint the application uses or the region the customers reach. Record the specific service and region in scope. Report the limits beside the result.
The record cannot show whether a real outage would be handled well under pressure. It can show only whether a written fallback existed and whether a drill exercised it. A missing fallback is not proof of a past failure, and a tested one is not proof of a fast recovery. The protocol also cannot see a vendor added outside the contract process.
A useful additional view is the time from the first customer report to the first public message. Record the detection time, the declaration time, and the message time for each drill. That field turns a preparedness count into a review of customer communication speed. Record both times on the same clock and with the same incident reference so a later reviewer can compare them.
Record the data the degraded mode cannot accept and how the team recovers it. A fallback that queues payments must have a reconciliation step and an owner. Test the recovery path, not only the blocked state. Report any mode that loses an order or sends a duplicate.
Check the tabletop record for the roles that attended. A drill without support or the client owner may miss the customer message and the approval path. Record the attendees and the gaps they found. Report a drill that only engineering attended as partial evidence.
Give the result to the client technical owner and the offshore delivery lead with one observable change. The change might be a vendor contact card, a degraded mode, a decision owner, or a quarterly tabletop. Record the owner, the effective date, and the measure that will show whether the change worked. Repeat the protocol with the same definitions after the change.
Close the review with one change and a date to re-measure. The change should address the largest missing or failed group, not the easiest vendor to document. Record the expected effect and the evidence that will prove it. A fallback note that is never rehearsed is a document, not a capability.
Build the vendor sample from the customer path rather than from the procurement list. A contract may cover a service the application never calls, while a free API may sit inside checkout. Trace one real order or login and record every external call. Use that trace to correct the list.
Use two evidence sources for each vendor, such as the fallback note and a drill record. The note proves a response was written, while the drill proves someone tried it. Record the date and the roles that attended. A note with no drill is a plan, not a capability.
Operating controls
Vendors with recent incidents are overrepresented because they are easy to remember. Include the vendor that has never failed and the one added last quarter. Record when the vendor entered the customer path. A new vendor often has no fallback because nobody updated the note.
Record the detection source and the alert route for each vendor. A status page that nobody watches does not shorten detection. Compare the first report time with the first internal alert. Report the gap and the owner of the monitoring rule.
Report the result by customer path as well as by count. A payment vendor and a map widget do not carry the same consequence. Record the path and the blocked action beside each finding. The reader should see which gap would stop revenue.
Repeat the measure after a tabletop and after any vendor change. A drill can turn a missing note into a tested one, but only if the team records the fix. Compare the categories with the earlier window. The trend shows whether the drills are changing the response.
Check the data the degraded mode cannot accept and the reconciliation owner. A fallback that queues payments without a replay step creates a second backlog. Record the replay test and the duplicate check. Report the mode as partial when recovery is unproven.
Treat the protocol as a review of customer communication and dependency planning. A missing fallback often means nobody mapped the path end to end. Ask who would have written the note and when. The answer usually reveals an ownership gap rather than a documentation gap.
Record the vendor's own incident history for the service and the region in scope. A vendor with a strong overall record can still fail repeatedly on the endpoint the application uses. Note the last incident that affected the customer path. Report the history beside the fallback status.
Set a review trigger for every new vendor or new integration. A vendor added after the last drill will not appear in the old fallback note. Require a contact, a symptom, and a degraded mode before the integration reaches production. Record the approval date. The trigger keeps the list from aging silently.
Record the last time the degraded mode was used in a real event as well as in a drill. A mode used under pressure may have been changed by the people on shift. Compare the note with what they actually did. Report the difference and update the note.
Sources
Frequently asked questions
What should happen first?
Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.
Who approves production changes?
The client's technical owner keeps final merge and release authority.
Related research
How long do production logs retain customer data in an outsourced debugging lane?
