Table of contents
Research context
This protocol measures the share of offshore load tests that run against production data. The unit is one load test window with a stated target and a stated data source. A production-data run means the test read or wrote live customer records, even if the target was a staging system. The study describes test isolation, not the accuracy of the performance result.
Define the load tests in scope before collection. Include capacity tests, soak tests, and failure tests that the team ran for the client in the window. Record the ticket, the date, the target environment, and the data source for each. Publish the list with a version so a later reviewer can reproduce it.
Define the target environment from the deployment inventory rather than the test script name. A test that points at a production host is a production test even when the file is called staging. Record the host, the account, and the database used. Test the classification on the first three tests.
Define the data source from the accounts and the connection strings used during the run. A copy of production data restored into staging is still production data. So is a read replica used for the test. Record the source and the date of the copy.
Check whether the test used synthetic identities or real customer records. Look for real email addresses, phone numbers, and payment values in the test dataset. Record the check method and the person who ran it. Report the tests that could not be checked.
Practical application
Check whether the integrations were sandboxed. A test that can send real email, charge a card, or ship a parcel is a production-data run even with synthetic records. Record the integration state for each test. Report the tests with an unknown integration state separately.
Report the number of tests, the number on production data, and the number on a production host. Show the counts by test type and by team. Present the tests with no recorded data source and the tests with no stop owner separately. Use the same denominator for every rate.
Separate a production-data run from a production-target run. A staging test with a restored production copy is one category, while a test against a production endpoint is another. Report the two categories separately because the controls differ. Record the evidence beside each finding.
Inspect the tests around a release deadline or a performance incident. A cluster may mean the team skipped the data preparation to save time. Compare the pattern with the release calendar. A single run can be a mistake, while a cluster is a scheduling gap.
Do not treat a clean test result as proof that the data was safe. A test can complete without an incident while still exposing live records to the load generator and its logs. Sample the test artifacts for real values. Report what the sample showed.
Operating controls
The record cannot show whether a customer was affected or whether the data was copied outside the environment. It can show only whether the test used production records and whether the integrations were sandboxed. A production-data run is not proof of harm, and a synthetic run is not proof of isolation. The protocol also cannot see a test run by a vendor outside the ticket process.
A useful additional view is the share of tests with a named stop owner and a written stop signal. That share describes the ability to stop a run before it affects customers. Record the owner and the signal for each test. That field turns a data count into a review of operational control.
Give the result to the client technical owner and the offshore delivery lead with one observable change. The change might be a synthetic data generator, a sandbox integration, a stop-owner rule, or a test ticket template. Record the owner, the effective date, and the measure that will show whether the change worked. Repeat the protocol with the same definitions after the change.
Close the review with one change and a date to re-measure. The change should address the largest production-data category, not the easiest test to classify. Record the expected effect and the evidence that will prove it. A load test that is never checked for data source is a performance exercise with an unmeasured risk.
Sources
Frequently asked questions
What should happen first?
Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.
Who approves production changes?
The client's technical owner keeps final merge and release authority.
