Table of contents
Research context
This protocol asks how often offshore test environments contain production personal data. The unit is one environment snapshot, identified by the environment, the dataset, and the observation date. The study measures whether personal data is present, whether it is masked, whether an owner approved the dataset, and whether the data is past its retention date. It does not evaluate the security of the application that produced the data.
Set the scope to one organization, one test environment, and one data classification scheme. Record the systems that copy data into the environment, the masking tools, and the retention policy at the start. If the policy changes during the window, keep the earlier definition for the full sample or analyze the change as a separate period. A new masking tool can look like an improvement when it is only a different process.
Define the population as every dataset that is present in the test environment during the observation window, including datasets that are not currently used by a test. The unit is the dataset, not the table, because one load can create several related tables. A dataset refreshed from the same source is a new unit with a new creation time. A backup of the same dataset is a separate unit.
Record the data source, the owner, the creation date, the retention date, and the masking method. These strata matter because a synthetic dataset and a raw production extract carry different risk. Include datasets with no owner, because the absence is part of the finding. Test the unit definition on two ordinary environments before collection begins. Use one classification label per dataset.
Define personal data as information that can identify a living person directly or in combination with other records. Define production personal data as personal data copied from a production system or a customer export. Define masked as a state in which direct identifiers are replaced and the re-identification risk has been reviewed. Define approved as a dataset with an owner, a purpose, and a retention date recorded before the load.
Do not count a synthetic record that resembles a real person as production personal data without evidence that it came from production. Do not count an email address in a test fixture that belongs to the test team. A dataset with a masking label but no rule or review is not automatically masked. Each definition should include one positive example and one boundary example.
For each dataset, capture the environment, source, owner, creation date, retention date, masking method, and row count. Add the field categories that contain personal data and the evidence that masking was applied. Record whether the dataset is used by an active test. Keep the evidence in a restricted store rather than in the general test documentation. Store the packet in the restricted evidence store.
Practical application
Also capture the access list and the last access date. These fields help explain a dataset that is old but still reachable by a broad group. If the access list comes from a group rather than a named user, record the group owner. A second reviewer should be able to classify the dataset without opening the raw personal records. Keep named users separate from group access.
Draw the sample from the environment inventory and the data-loading system rather than from a list of known incidents. Record the export date, the filters, the row count, and the selection rule. Include datasets from every source and every owner so the result is not dominated by one team. If the inventory is ordered by creation date, include every dataset in the window.
Collect the dataset metadata as it existed at the observation date, not as it looks after a later cleanup. If a dataset was deleted during the study, record the deletion date and the evidence. Do not inspect raw personal records to prove a point when metadata can answer the question. Note the collection date separately from the load date so the reader can see the age distribution.
A useful additional measure is the share of datasets whose masking rule was applied by an automated job and verified by a second person. A manual masking step can be correct, but it is harder to reproduce. Show the method beside the approval status. That view helps the owner decide whether the gap is in tooling, review, or the data request process.
Report the number of datasets, the share with production personal data, the share that is masked, and the share with a current approval. Show the counts by source, owner, and environment. Present the median age and the number of datasets past their retention date. Use the same denominator for every rate and show the number of datasets with incomplete metadata.
Inspect the findings and describe the pattern without naming individuals or exposing records. A cluster in one source may point to a copy job that bypasses the masking tool, while a cluster in one owner may point to a missing request process. Compare the result with the retention date and the access list. Report synthetic datasets separately so they do not distort the rate.
A useful additional view is the number of people who can restore a backup of the test environment. A dataset can be deleted from the live database and still be reachable through a snapshot. Show the restore access beside the live access. That view helps the owner decide whether the retention rule covers the backup path as well as the primary store.
Operating controls
The review should separate a policy gap from a tooling gap. A team that knows the rule but cannot mask a nested document needs a better tool, while a team that copies data because the request takes too long needs a faster approved path. Record the reason for each finding. That split prevents the same problem from returning after a training session.
The record cannot prove that a dataset contains no personal data just because no identifier field was found. Free-text notes, file attachments, and combined fields can hide personal information. It also cannot prove that a masked dataset is safe without a re-identification review. The protocol measures presence and evidence, not the probability of harm. It measures evidence, not the chance of harm.
Environment inventories can omit datasets created by an engineer outside the approved pipeline. A local copy, a downloaded export, or a vendor sandbox may never appear in the inventory. Record those gaps and their treatment instead of dropping them silently. If the person who loaded the data also approved it, note the independence limitation. Ask the platform owner to confirm the list.
A useful check is whether the environment can be rebuilt from synthetic data without losing the test coverage that made the production copy attractive. Ask the test owner which cases require realistic volume or edge distributions. That answer shows whether the copy is necessary or merely convenient. The finding should inform the data request process, not just the cleanup list.
Give the result to the data owner and the engineering lead and ask them to choose one observable change. The change might be a pre-load approval, an automated masking job, a shorter retention default, a restore-access review, or a synthetic-data path for common tests. Record the owner, the effective date, and the measure that will show whether the change worked. Avoid changing the tool and the policy at the same time.
Repeat the protocol with the same definitions after the change and compare the same fields. If production data fell but unapproved synthetic data rose, the team may be avoiding the request process rather than improving privacy. If approvals improved but retention did not, the deletion job may be the real gap. Close the review when the owner accepts the finding and the next sample is scheduled.
The final view is the customer promise attached to the data. A privacy notice or contract may limit where personal data can be processed and who can access it. Record the relevant promise and the environment location. That context helps the owner decide whether a finding is a policy breach, a control gap, or both. Record the next review date with the sample.
Sources
Frequently asked questions
What should happen first?
Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.
Who approves production changes?
The client's technical owner keeps final merge and release authority.
Related research
What share of outsourced QA defects are reopened after a fix?