Table of contents
Research context
This protocol asks how often an offshore estimate changes after refinement and before commitment. The unit is one work item that received an initial estimate and a later refined estimate during the window. The study measures the direction and size of the change, the reason recorded, and the schedule effect. It does not judge whether the first estimate was honest or whether the team is fast.
An estimate change is not automatically an error. Refinement exists to replace uncertainty with information, and a changed number can be the sign that the process worked. The protocol should distinguish a change caused by new evidence from a change caused by pressure or a forgotten requirement. The reason field is what makes that distinction possible.
Set the scope to one offshore team, one backlog, and one refinement process. Record the estimation scale, the participants, and the definition of commitment at the start. If the scale or the ceremony changes during the window, keep the earlier definition for the full sample or analyze the change as a separate period. A new planning cadence can look like better estimation when it is only a different rhythm.
Record who requested the change and when. A change requested by the client after a scope discussion has a different meaning from one requested by the delivery lead the day before commitment. The requester and the timing are part of the evidence. Without them, the team can only guess why the number moved.
Define the initial estimate as the value recorded at the first sizing session, and the refined estimate as the value accepted at commitment. A work item that is split creates new units linked to the original. A work item removed from the backlog remains in the sample with a removal reason. Each definition needs a positive and a boundary example so two reviewers classify the same item the same way.
Measure the size of the change in steps rather than only in hours. A move from three to five points is easier to compare across items than a move from twelve to nineteen hours. If the team uses hours, define the band boundaries in advance. Consistent bands make the report readable without a statistics lesson.
Record the item type, the initial size, the refined size, the number of refinement sessions, the participants, and the stated reason for the change. Add the sprint in which it was committed and whether it shipped. Keep the original estimate visible rather than overwriting it. A second reviewer should be able to see the movement without reconstructing it from memory.
Separate the first estimate from the commitment estimate in the backlog history. Many tools overwrite the original value, which makes the movement invisible. Keep the first value in a custom field or a linked note. A team that cannot see the original number cannot learn from the change.
Practical application
Report the number of items, the share that changed, the median absolute change, and the share that changed by more than one size step. Show the counts by item type and by initial size. Present the share of changes attributed to missing information, hidden dependency, scope change, and technical discovery. Use the same denominator for every rate and show items with missing reasons.
Check whether the refined estimate included a contingency that the first estimate omitted. A deliberate buffer is not the same as a discovery, and the two should not be pooled. Record the reason as contingency when that is what happened. The distinction matters when the client asks why the total grew.
Inspect the largest movements and describe the pattern without naming the engineer. A cluster in one item type may point to a recurring unknown, such as data migration or third-party access. A cluster at the end of a sprint may point to pressure to commit before refinement is finished. Report items that were split separately because a split is a legitimate response to new information.
Look for items that changed size more than once. A single revision may be healthy, while repeated changes suggest the item is not ready for commitment. Record the number of revisions per item and report the distribution. An item with three revisions may need a spike or a smaller scope before it enters a sprint.
The record cannot show whether the refined estimate was more accurate or whether the team simply added contingency. It can show only that the number moved and what reason was recorded. Estimate movement is not proof of poor planning, because new information should change a plan. The protocol also cannot capture pressure that was applied in a conversation and never written down.
Do not compare estimates across teams with different scales or definitions of done. A five in one team may be a three in another. Keep the analysis inside one team and one scale, or normalize with a documented rule. Cross-team comparisons without that work are misleading.
Give the result to the delivery lead and the client owner and ask them to choose one observable change. The change might be a required spike before sizing, a definition of ready, a refinement checklist, or a rule that committed items cannot change size without a scope note. Record the owner, the effective date, and the measure that will show whether the change worked. Repeat the protocol with the same definitions.
Ask the engineers who did the work whether the refined estimate matched the effort they experienced. The answer is subjective, but it can reveal a systematic gap that the numbers hide. Record the response as a short note rather than a score. Use it to choose the next refinement improvement.
Operating controls
A useful additional view is the share of changed items that still shipped in the committed sprint. An estimate that moved but did not affect delivery may be a sign of healthy re-planning. An estimate that moved and pushed the item out is a schedule event. Report both so the reader can see the consequence.
Close the review with one change to the refinement process and a date for the next sample. The change might be a definition of ready, a required spike, or a rule that scope changes need a written note. Record the owner and the measure. Repeat the protocol with the same definitions and compare the same fields.
Record whether the item had a written acceptance criterion at the first estimate. An estimate made before the criterion exists is a guess about scope, and a later change is expected. If most large movements come from items without criteria, the fix belongs in the definition of ready rather than the estimation session. Show the split in the report.
Compare the refined estimate with the actual effort for items that shipped during the window. The comparison is not a performance score, because actual effort includes interruptions and waiting. It is a check on whether the team's scale is stable enough to learn from. Report the median absolute difference and the number of items with no actuals.
Check whether the same few people dominate the refinement discussion. A session where one architect speaks for most of the time may produce estimates that the implementers do not share. Record who estimated and who will do the work. If those groups differ, ask the implementers to confirm the number before commitment.
Record the number of participants in each refinement session and whether the implementer was present. A session that estimates work without the person who will do it can produce a number that nobody owns. If the implementer was absent, mark the estimate as provisional. The report can then show whether provisional estimates move more often than shared ones.
Check whether the team uses a reference story or a fixed scale anchor. Without an anchor, each session can drift and a five can mean different things in different months. Record the anchor used and whether it was visible during the session. A stable anchor is a cheap improvement that does not require a new tool.
Sources
Frequently asked questions
What should happen first?
Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.
Who approves production changes?
The client's technical owner keeps final merge and release authority.
Related research
How long does a new offshore engineer take to make a first production change?
