Outsourced ProgrammersPowered by StealthAgents

What share of outsourced QA test accounts are disabled after a sprint?

A protocol for counting test identities, checking disable evidence, and separating an account left active from one that was never owned.

What share of outsourced QA test accounts are disabled after a sprint?
WORK_LANE / RESEARCH BRIEF / QUALITYREVIEW QUEUEACCEPTANCEPASSowner review requiredEVIDENCE TRAILTICKET → BRANCH → REVIEW → CLIENT RELEASE
Table of contents
  1. Research context
  2. Practical application
  3. Operating controls
  4. Sources
  5. Frequently asked questions

Research context

This protocol measures the share of QA test accounts that are disabled after a sprint in an outsourced delivery lane. The unit is one test account on one environment at the observation date. Disabled means the login fails and the account cannot act, not merely that a ticket says it should. The study describes access lifecycle evidence, not the quality of the QA work.

Define the environments and account types in scope before collection. Include personal logins, shared fixtures, service accounts, and credentials embedded in automated tests. Record the identity provider and the secret store for each. Publish the list with a version so a later reviewer can reproduce the scope.

Define an account by its owner, purpose, environment, role, creation date, and expiry date. Record the sprint or release that justified it. Note whether it can approve a payment, send a message, or change a setting a customer would notice. Test the definition on the first five accounts.

Set the population as every test account in the agreed environments on the observation date. Include accounts created for a closed sprint and accounts that no test currently uses. Record the accounts that could not be inspected and the reason. Report them separately from accounts that failed the disable check.

Classify each account by lifecycle evidence. Use categories such as disabled with a verified failed login, disabled on paper only, active with a current purpose, and active with no owner. Do not merge the categories after seeing the result. The categories point to different fixes.

Record the named owner, the backup, and the date the purpose was last confirmed. Note the secret storage location and the last rotation. Record whether multi-factor authentication is enabled where the platform supports it. An account with an owner and no expiry is still a lifecycle gap.

Report the number of accounts, the number disabled with evidence, and the number in each category. Show counts by environment and by role. Present the active accounts with no owner and the accounts with a failed disable check as separate rows. Use the same denominator for every rate.

Separate an account left active from an account that was never owned. An active account may have a current purpose, while an unowned account has no one who can justify it. Report the two categories separately because the fixes differ. Record the evidence beside each finding.

Inspect the accounts created around a closed sprint or a departing contractor. A cluster of active accounts may mean the close-out step was never run. Compare the pattern with the sprint calendar and the personnel record. A single leftover can be an oversight, while a cluster is a process gap.

Practical application

Do not treat a disabled flag in a spreadsheet as proof that the login fails. Test the login or query the identity provider for the account state. Check the sessions and tokens that were issued before the disable. Report the accounts whose disable step did not revoke existing access.

The record cannot show whether an active account was used maliciously or whether a customer data set was reached. It can show only whether the account could act in the inspected environment. An active account is not proof of a breach, and a disabled one is not proof of good hygiene. The protocol also cannot see an account created outside the agreed identity provider.

A useful additional view is the number of active accounts per closed sprint. A sprint with twelve surviving accounts is a different finding from one with a single fixture. Record the sprint and the count. That field turns a total into a review of which close-out failed.

Record the role and the data each active account can reach. An admin fixture left active is a different risk from a read-only customer login. Note the approval for the role. Report the accounts whose role is broader than the test requires.

Check the secret store for credentials that belong to disabled accounts. A disabled login with a live API token can still act. Record the token name, the last rotation, and the revocation evidence. Report the accounts whose tokens were not revoked with the login.

Give the result to the client technical owner and the offshore delivery lead with one observable change. The change might be a default expiry, a close-out checklist, a fixture owner, or a token revocation step. Record the owner, the effective date, and the measure that will show whether the change worked. Repeat the protocol with the same definitions after the change.

Close the review with one change and a date to re-measure. The change should address the largest active or unowned group, not the easiest account to delete. Record the expected effect and the evidence that will prove it. An expiry date that nobody checks is a suggestion, not a control.

Build the sample from the identity provider export and the test repository, not from the test plan. The export shows what exists, while the repository shows what the automation still calls. Compare the two and record the accounts that appear in only one place. The mismatches are often the oldest fixtures.

Use two evidence sources for each account, such as the identity provider state and a login attempt. The provider shows the configured state, while the attempt shows the effective result. Record both outcomes. A provider that says disabled and a login that succeeds is a finding.

Operating controls

Accounts used in recent testing are overrepresented because the team remembers them. Include an account from a closed sprint and one created by a new service. Record the last login date. A dormant account with a live token is still a risk.

Record the account creation path and the approval. An account created by asking a colleague in chat leaves no record and no expiry. Note the ticket and the approver. Report the accounts whose creation cannot be explained.

Report the result by role and data access as well as by count. An admin account and a read-only fixture do not carry the same risk. Record the reachable data beside the status. The reader should see which active account carries the most exposure.

Repeat the measure after a sprint close, a handover, or an identity provider change. Keep the same categories and the same environment list. Record the accounts that moved between categories. A stable count with a changed team can still hide a new ownerless login.

Check the service accounts used by automated tests for a named owner and a rotation date. An automation identity with no owner is an orphan even when the tests are healthy. Record the rotation and the person who can perform it. Report the fixtures that share a secret with a human account.

Treat the protocol as a review of access lifecycle, not a judgment of the QA engineers. An active account often means the sprint close-out never included identity cleanup. Ask who was supposed to disable it and when. The answer points to the missing step rather than a careless person.

Record the customer data classes reachable from each active account and the masking rule applied. A test account that sees unmasked customer records is a privacy finding, not only an access finding. Note the rule and the owner. Report the accounts that reach data the test does not need.

Set a rule that no test account outlives its sprint without a written extension and a named owner. Write the rule into the test guide and the sprint close-out checklist. Record the extension approver and the review date. The rule turns a good intention into a check that can be verified.

Sources

  1. OWASP Application Security Verification Standard
  2. NIST SP 800-53 security and privacy controls
  3. OWASP Web Security Testing Guide
  4. CIS Controls

Frequently asked questions

What should happen first?

Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.

Who approves production changes?

The client's technical owner keeps final merge and release authority.

Related research

How long do revoked offshore credentials stay usable after an exit?

Build a programming lane your technical owner can review.

Bring the stack, backlog, acceptance checks, and release boundaries. We will help scope the role around the work.

Discuss your programming needs