Table of contents
Research context
This protocol measures how long production logs retain customer data in an outsourced debugging lane. The unit is one log stream on one environment at the observation date. Customer data means any field that identifies a person or reveals a private transaction, including email, phone, address, and payment details. The study describes log hygiene, not the quality of the application.
Define the log streams and the systems in scope before collection. Include application logs, proxy logs, queue logs, scheduled job output, and any vendor log the team can query. Record the retention setting and the storage owner for each. Publish the list with a version so a later reviewer can reproduce the scope.
Define a stream by its source, environment, retention period, access role, and export path. Record whether redaction happens in the application, the collector, or only the viewer. Note the encryption key owner and the deletion mechanism. Test the definition on the first three streams.
Set the population as every stream in the agreed list on the observation date. Include streams that are rarely queried because those often keep the oldest data. Record the streams that could not be inspected and the reason. Report them separately from streams that failed the check.
Classify each stream by retention period. Use bands such as seven days or less, eight to thirty days, thirty-one to ninety days, and more than ninety days. Do not merge the bands after seeing the result. The bands make comparison across months meaningful.
Record whether sensitive fields are redacted before storage and who approved the field list. Note the last test of the redaction rule and the result. Record the named access owner and the export approver. A stream with a short retention and no redaction is still a risk.
Report the number of streams, the number within the written retention policy, and the number in each band. Show counts by environment and by data category. Present the streams with no redaction test and the streams with a failed test as separate rows. Use the same denominator for every rate.
Separate a retention overrun from a redaction failure. A retention overrun keeps data longer than the policy allows, while a redaction failure stores a sensitive field it should have removed. Report the two categories separately because the fixes differ. Record the policy date and the evidence beside each finding.
Inspect the streams that share a framework or a logging library. A cluster of failures may mean a default writes the full request on an exception. Compare the pattern with the library versions and the deployment history. A single leak can be a mistake, while a cluster is a design gap.
Practical application
Do not treat a short retention as proof that the stream is safe. A seven-day stream can still expose a token to everyone with query access. Check the access role and the export path as well as the retention. Report the access finding beside the retention result.
The record cannot show whether a customer was harmed or whether an exported file still exists on a laptop. It can show only the configured retention and the redaction evidence in the inspected systems. A long retention is not proof of a breach, and a short one is not proof of good hygiene. The protocol also cannot see a log written by a vendor outside the agreed list.
A useful additional view is the number of open exports and their destinations. An export that lands in a shared drive or a personal download folder escapes the retention rule. Record the export date, the destination, and the ticket. That field turns a configuration count into a review of data movement.
Record how the team handles an incident hold. A hold can extend retention for evidence, but it needs an owner, a scope, and an end date. Check that the hold does not keep unrelated customer data. Report holds that have no review date.
Sample one stream and search for patterns the rule should have removed. Use a test account and a synthetic record rather than a real customer. Record the query, the result, and the person who ran it. Report a sample that could not be run because access was missing.
Give the result to the client data owner and the offshore delivery lead with one observable change. The change might be a field rule, a structured logger, a retention setting, an export control, or a monthly sample. Record the owner, the effective date, and the measure that will show whether the change worked. Repeat the protocol with the same definitions after the change.
Close the review with one change and a date to re-measure. The change should address the largest retention overrun or redaction failure, not the easiest stream to shorten. Record the expected effect and the evidence that will prove it. A retention rule that is never sampled is a setting, not a control.
Build the sample from the logging configuration and the storage console, not from the dashboards the team uses. A stream can exist without a dashboard and still hold sensitive fields. Include the error stream and the scheduled job output. Record how each stream entered the sample.
Use two evidence sources for each stream, such as the stored record and the logger configuration. The record shows what was written, while the configuration shows what the rule intended. Compare them on the same timestamp. A mismatch is a finding even when the rule looks correct.
Operating controls
Streams with recent incidents are overrepresented because they were queried. Include a stream that nobody has opened in months and a stream created by a new service. Record the last query date. An unread stream can still retain customer data.
Record the retention setting, the deletion job, and the access role for each stream. A policy on paper does not prove that old data was removed. Sample the oldest record and check its date. Report the result with the policy version.
Report the result by data category as well as by count. A stream with authentication tokens deserves a stricter finding than one with request timings. Record the field list beside the stream. The reader should see which retention gap carries the most exposure.
Repeat the measure after a logger or framework upgrade, because defaults change. Keep the same bands and the same sample method. Record the streams that moved categories. A stable policy with a changed library can still produce a new leak.
Check every export destination and every open export ticket. An export can outlive the stream and sit on a laptop or a shared drive. Record the export date, the destination, and the retention rule applied. Report open exports as a separate finding.
Treat the protocol as a review of data handling, not a judgment of the support team. A long retention often comes from a default nobody reviewed. Ask who set the value and when it was last checked. The answer points to the configuration change.
Record the log level used in production for each stream. A debug level enabled for one investigation can multiply the data and the exposure. Note who can change the level and whether the change expires. Report any stream left at a verbose level after the ticket closed.
Check the vendor and platform logs that the team can query but does not own. A support tool or a proxy may retain request bodies under a separate policy. Record the vendor, the retention, and the access path. Report the streams that have no review owner on the client side.
Record who can change the retention policy and how a change is approved. A policy edited by one engineer during an incident can leave the stream outside the agreed rule. Check the audit log for recent changes. Report the change and the approval beside the policy.
Sources
Frequently asked questions
What should happen first?
Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.
Who approves production changes?
The client's technical owner keeps final merge and release authority.
Related research
How often do third-party API outages reach an offshore support queue without a fallback?
