Outsourced ProgrammersPowered by StealthAgents

What share of outsourced postmortems close their action items?

A protocol for measuring postmortem action completion, the age of open actions, and the difference between a merged change and a verified fix.

What share of outsourced postmortems close their action items?
WORK_LANE / RESEARCH BRIEF / DELIVERYREVIEW QUEUEACCEPTANCEPASSowner review requiredEVIDENCE TRAILTICKET → BRANCH → REVIEW → CLIENT RELEASE
Table of contents
  1. Research context
  2. Practical application
  3. Operating controls
  4. Sources
  5. Frequently asked questions

Research context

This protocol measures the share of postmortem action items that are completed and verified within a stated period. The unit is one action item from one postmortem, identified by the postmortem and the action identifier. Completion means the change is live and the evidence is recorded. The study describes follow-through, not the quality of the incident review.

Define the postmortem population as every review completed in the window, including reviews with no actions. The unit is the action, not the postmortem, because one review can produce several actions with different owners. A postmortem with no actions is a separate finding and should be reported. Test the unit on two reviews before collection begins.

Define an action as a concrete change with an owner, a due date, and a stated evidence standard. A general recommendation without an owner is not an action for this measure. Record the wording and the owner. The definition should distinguish an action from a discussion note.

Define completion as the change being live and verified by the evidence named at creation. A merged pull request, a ticket marked done, or a meeting note is not completion by itself. Record the evidence and the verifier. The definition should be applied the same way to every action.

Set the scope to one delivery team, one incident process, and a fixed window. Record the postmortem template, the review cadence, and the person who accepts actions. If the template changes during the window, split the sample at the change date. A new template can change the number and quality of actions.

Record the postmortem identifier, the action text, the owner, the due date, the status, the evidence, the verifier, and the close date. Add the incident severity and the customer impact. Keep the action linked to the postmortem and the change record. A second reviewer should be able to confirm the status without asking the owner.

Practical application

Separate an action that is complete from one that is partially complete. A partial action may have a merged change that is not deployed or a deployed change with no test. Record the missing step. The study should not count a partial action as complete.

Check the actions that were reassigned or had their due date moved. Record the original owner, the new owner, and the number of date changes. A reassignment is not a completion, and a moving due date can hide a stalled action. Report the count separately.

Report the number of postmortems, the number of actions, the share completed on time, the share completed late, and the share still open. Show the counts by severity and by owner group. Present the median age of open actions and the oldest open action. Use the same denominator for every rate and show actions with missing evidence.

Separate an action that is open because the work is hard from one that is open because nobody owns it. A hard action may need a design decision or a budget, while an unowned action needs a name. Report the two categories separately so the client can choose the right response. Record the blocking dependency beside each open action.

Inspect the actions that repeat across postmortems. A repeated action may mean the earlier change was never verified or did not address the cause. Treat the repeat as a finding about follow-through. Compare the wording and the owner with the earlier action.

Operating controls

Do not treat a high completion rate as proof that the incidents will not recur. An action can be completed and still fail to change the outcome. Report the actions that were completed but followed by a similar incident. The study measures follow-through, not effectiveness.

The record cannot show whether an action was the right response or whether the postmortem identified the true cause. It can show only whether the team completed and verified the change it committed to. A completed action is not proof of safety, and an open action is not proof of negligence. The protocol also cannot capture work that was done without a ticket.

A useful additional view is the share of actions that were closed by the same person who wrote them without a second verifier. That share describes the independence of the close, not the quality of the change. Record the verifier and the relationship to the owner. The field helps a reviewer interpret the completion rate.

Give the result to the client technical owner and the offshore delivery lead with one observable change. The change might be a weekly action review, a named verifier for high-severity actions, a shorter action template, or a rule that closes only with evidence. Record the owner, the effective date, and the measure that will show whether the change worked. Repeat the protocol with the same definitions after the change.

Close the review with one change and a date to re-measure. The change should address the most common reason an action stays open, not the one that is easiest to report. Record the expected effect and the evidence that will prove it. A postmortem process that never checks its own actions is a writing exercise.

Sources

  1. Google SRE Book, postmortem culture
  2. PagerDuty incident response
  3. Atlassian incident management
  4. NIST SP 800-53 security and privacy controls

Frequently asked questions

What should happen first?

Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.

Who approves production changes?

The client's technical owner keeps final merge and release authority.

Related research

How long does a vendor escalation take to reach a named owner?

Build a programming lane your technical owner can review.

Bring the stack, backlog, acceptance checks, and release boundaries. We will help scope the role around the work.

Discuss your programming needs