On this page
Research context
This protocol asks how much of a third-party SDK's risk is documented before the team adopts it. The unit is one SDK evaluated for a specific integration during the observation window. The study scores the evidence available for license, data access, maintenance, failure behavior, and exit planning. It does not judge whether the SDK is good, only whether the team can see the risk before shipping.
Set the scope to one product, one engineering team, and one review process. Record the decision record template, the license policy, and the approval path in force at the start. If the template changes during the window, keep the earlier definition for the full sample or analyze the change separately. A new required field can look like better diligence when it is only a new form.
Define the population as all SDKs that reached an adoption decision during the window, including SDKs that were rejected. The unit is the SDK and the specific integration, identified by the package and the feature. A later upgrade of the same SDK is a new unit if the major version changes. A different feature using the same SDK is a separate unit.
The review should distinguish a risk that is unknown from a risk that is known and accepted. An unknown data practice is not the same as a documented telemetry call that the product owner approved. Record the status of each category as verified, claimed, or unknown. The decision should state which unknown risks the owner is willing to carry and why. A short table with one row per category makes that decision visible to the next reviewer.
Record the package, the version, the feature, the team, and whether the SDK was already in use elsewhere. These strata matter because a package already approved for another feature has a different review burden. Include rejected SDKs, because the reason for rejection is part of the evidence. Test the unit definition on two ordinary evaluations before collection begins.
Define license evidence as a recorded license name and version, with the obligations that apply to the distribution model. Define data evidence as a list of the data the SDK can access and the network calls observed in a test session. Define maintenance evidence as the release history, the issue response, and the security advisory record. Define failure evidence as the observed behavior under timeout, permission denial, and offline conditions.
Do not count a marketing page as license evidence. Do not count a privacy policy as data evidence unless it is matched to the observed network calls. A package with a recent release but no response to security issues is not necessarily maintained. Each definition should include one positive example and one boundary example so a second coder can apply it consistently.
Practical application
A vendor's security page is not evidence of the SDK's behavior in the product. The team should test the package in the same build and runtime it will ship, because a browser bundle and a server bundle can expose different code. Record the environment and the test method. A test that runs only in a development sandbox may miss a production-only path.
For each evaluation, capture the package, version, feature, license, distribution model, and the reviewer. Add the permissions requested, the network calls observed, the release history, the open advisories, and the failure tests. Record the bundle and startup measurements, the exit plan, and the decision. Code each field as present, missing, or not applicable.
Also record whether the review happened before the integration code was written and whether the decision record was approved by the technical owner. These process fields help explain a risk that was visible but ignored. If a field was filled with a vendor claim rather than a test, mark it as claimed rather than verified. A second coder should be able to reproduce the score from the packet.
Draw the sample from the dependency inventory and the decision records rather than from a list of recent incidents. Record the export date, the filters, the row count, and the selection rule. Include rejected and deferred SDKs so the result shows the full decision set. If the population is ordered by decision date, a fixed interval spreads the sample across the window.
The exit plan should include the data the SDK stores, not only the code that calls it. A replacement may need to migrate local caches, queued events, or identifiers that the old SDK created. Record the storage locations and the migration step. If the data cannot be exported, that limitation belongs in the adoption decision.
Collect the evidence as it existed at the adoption decision, not as it looks after a later incident. If the decision record was updated after the fact, record the update time and analyze the original separately. Do not reconstruct a missing test from the vendor's documentation. Note the collection date separately from the decision date so the reader can see how much of the record has aged.
Report the share of evaluations with verified license, data, maintenance, failure, and exit evidence. Show the counts by feature area and by whether the SDK was new to the team. Present the median number of missing fields and the share of decisions made before the integration code was written. Use the same denominator for every rate.
Operating controls
Inspect the evaluations with the weakest evidence and describe the missing category. A cluster of missing data evidence may point to the test process, while a cluster of missing maintenance evidence may point to the vendor research. Compare the result with the risk of the feature, because a low-risk integration may not need the same depth. Report the rejected decisions separately.
The review should be repeated when the SDK's major version changes or when the vendor is acquired. A package that was acceptable last year may have different terms or a different maintenance team. Record the trigger for the next review rather than relying on a calendar reminder. The trigger makes the review part of the change process.
A useful additional view is the time between the review and the first production release. A review that happens after the code is written is a ratification, not a decision. Plot the timing and describe the cases where the review came late. That pattern is often more actionable than the total score.
The record cannot show whether the SDK will behave well in production or whether the vendor will remain reliable. It can show only what the team knew before adoption. A complete record is not proof of a safe choice, and an incomplete one is not proof of a bad choice. The protocol measures the visibility of risk, not the risk itself.
Decision records can omit conversations, vendor calls, and tests that were run but not documented. An SDK that was approved through an architecture meeting may look undocumented in the file. Record those cases and their treatment instead of dropping them silently. If the same person wrote the integration and approved the record, note the independence limitation.
Give the result to the engineering lead and the security reviewer and ask them to choose one observable change. The change might be a required network capture, a license field, or a rule that the review happens before the first commit. Record the owner, the effective date, and the measure that will show whether the change worked. Avoid changing the template and the approval path at the same time.
Repeat the protocol with the same definitions after the change and compare the same fields. If evidence improved but late reviews did not, the process may be producing better paperwork after the decision. If data evidence improved but failure evidence did not, the team may be testing the happy path only. Close the review when the owner accepts the finding and the next evaluation is scheduled.
Sources
Frequently asked questions
What should happen first?
Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.
Who approves production changes?
The client's technical owner keeps final merge and release authority.
Related research
What survives a code handover when an outsourced engineer rotates?
Do accessibility fixes close the defect or only the reported symptom?