Table of contents
Research context
This protocol asks what share of offshore pull requests need a second review round and why. The unit is one pull request that received a substantive review decision during the window. The study measures whether the change returned for another round, the reason for the return, the time added, and whether the second round found a defect or only a preference. It does not evaluate the engineer's overall performance.
Set the scope to one repository, one review policy, and one engineering lane. Record the branch protection rules, the required reviewers, and the definition of a substantive review at the start. If the policy changes during the window, keep the earlier definition for the full sample or analyze the change as a separate period. A new required check can look like more rework when it is only a different gate.
Define the population as every pull request that reached a review decision during the window, including changes that were closed without merge. The unit is the pull request, not the commit, because a force-push can create many commits inside one review. A pull request that is reopened after being closed is a new unit with a link to the original. A stacked series counts as separate units.
Record the change type, risk tier, size band, author, reviewer, and whether the change touched a test. These strata matter because a dependency bump and an authentication change carry different review needs. Include changes from every author and reviewer so the result is not dominated by one pairing. Test the unit definition on two ordinary pull requests before collection begins. Use one size band scheme for the whole sample.
Define a review round as a reviewer decision that requests changes or approves with comments that require a new push. Define a second round as the next substantive review after that push. A comment that only asks a question and does not block merge is not a round. A rebase or a merge from main is not a round unless it changes the reviewed behavior. Each definition needs a positive and a boundary example.
Do not count an automated check failure as a review round unless a human reviewer also requested changes. Do not count a title edit or a label change as a new round. A reviewer who approves and then comments on a later line has still completed one round. Record the rule in the protocol so two reviewers classify the same history consistently. The unit is the human review decision, not every event in the timeline.
For each pull request, capture the author, reviewer, size, files changed, risk tier, first review time, decision, second review time, and final merge time. Add the reason for the second round, the number of comments, and whether a test changed between rounds. Record whether the change was eventually closed without merge. Keep the review history with the sample so a second reviewer can repeat the count.
Practical application
Also capture whether the first review named the exact change required. A comment that says this needs work without naming the gap often produces another round that a clearer request would have avoided. Record the category of the first blocking comment. A second reviewer should be able to classify the cause from the comment text and the diff.
Draw the sample from the repository rather than from a list of known rework. Record the export date, the filters, the row count, and the selection rule. Include pull requests from every author and reviewer so the result is not dominated by one difficult area. If the history is ordered by creation date, include every pull request in the window. Exclude drafts that never reached review and state that exclusion.
Collect the pull request as it existed at the final decision, not as it looks after later edits. If a review comment was deleted, record the deletion and treat the round as incomplete. Do not infer a second round from a long comment thread without a decision. Note the collection date separately from the review date so the reader can see how much of the record has aged. Keep the raw export with the analysis.
A useful additional measure is the time from the first review request to the first substantive comment. A quick first response can still lead to rework if the reviewer only looked at the title. Show the response time beside the rework rate. That view separates a slow queue from a review that starts late and then asks for a rewrite. Report both measures together.
Report the number of pull requests, the share with a second round, and the median time added by the second round. Show the counts by risk tier, size band, author, and reviewer. Present the share of second rounds caused by a missing test, an unclear requirement, a defect, and a preference. Use the same denominator for every rate and show the number of requests with incomplete history. Show the raw counts beside each rate.
Inspect the second rounds and describe the pattern without naming the engineer. A cluster in one area may point to an unstable test environment, while a cluster in one reviewer may point to an unwritten preference. Compare the result with the risk tier and the presence of a test change. Report closed-without-merge pull requests separately because they carry a different cost.
A useful additional view is whether the second round changed the behavior or only the style. A style-only round suggests the first review did not state the standard early. A behavior-changing round may be a healthy catch. Record the category and the line of code that changed. That view helps the owner decide whether to invest in a review checklist or in earlier design review.
Operating controls
The review should separate rework from iteration. A second round that catches a real defect is a success, while one that repeats an earlier comment is a process failure. Record the reason and the reviewer who requested it. That split prevents the team from optimizing for a low round count while missing defects. Keep the defect category in the record.
The record cannot show whether the reviewer had enough context or whether the author understood the requirement. It can show only the comments, pushes, and decisions. A second round is not proof of poor work, and a single round is not proof of quality. The protocol also cannot capture a review that happened in a call and left no written trace.
Repository history can omit comments from deleted accounts, force-pushes that erase a round, and reviews conducted in another tool. A pull request that was rebased may look shorter than it was. Record those gaps and their treatment instead of dropping them silently. If the author and reviewer are the same person, note the independence limitation and analyze that group separately. Keep the original review version with the sample.
A useful check is whether the review packet named the acceptance evidence before the first review. Ask the reviewer what they had to infer. That answer tests the packet, not the person. Add the missing evidence field to the template. A packet that lets the reviewer start without a call is the goal.
Give the result to the engineering lead and the review group and ask them to choose one observable change. The change might be a required test note, a smaller size band, a named reviewer backup, or a clearer comment template. Record the owner, the effective date, and the measure that will show whether the change worked. Avoid changing the size rule and the review template at the same time.
Repeat the protocol with the same definitions after the change and compare the same fields. If second rounds fell but escaped defects rose, the first review may have become too shallow. If review time rose but rework fell, the team may be trading speed for a better first pass. Close the review when the owner accepts the finding and the next sample is scheduled.
The final view is the author's time to a merged change, not only the reviewer's queue. A pull request that waits two days and then needs one round may be slower than one that gets a fast first review and two rounds. Record both the wait and the rounds. That view keeps the team focused on delivery rather than on a single metric.
Sources
Frequently asked questions
What should happen first?
Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.
Who approves production changes?
The client's technical owner keeps final merge and release authority.
Related research
How long does access provisioning take for a new offshore developer?
What share of outsourced QA defects are reopened after a fix?