Outsourced ProgrammersPowered by StealthAgents

How much time do outsourced CI pipelines lose to cold caches?

A protocol for counting cache-miss builds, timing the rebuild, and separating a cold cache from a pipeline that never used one correctly.

How much time do outsourced CI pipelines lose to cold caches?
WORK_LANE / RESEARCH BRIEF / RELEASEREVIEW QUEUEACCEPTANCEPASSowner review requiredEVIDENCE TRAILTICKET → BRANCH → REVIEW → CLIENT RELEASE
Table of contents
  1. Research context
  2. Practical application
  3. Operating controls
  4. Sources
  5. Frequently asked questions

Research context

This protocol measures how much build time an outsourced delivery lane loses when its CI caches are cold. The unit is one pipeline run on one job at the observation date. A cold cache means the run restored no usable cache and rebuilt the dependencies or artifacts from source. The study describes pipeline timing, not the productivity of the engineering team.

Define the pipelines and jobs in scope before collection. Include package installs, compilation, test setup, and container builds. Record the runner class and the cache backend for each. Publish the scope with a version so a later reviewer can reproduce it.

Define a cold run by the cache key, the restore result, and the steps that executed from source. Record the job, the commit, and the trigger. Note whether the run was a pull request, a merge, or a scheduled build. Test the definition on the first ten runs.

Set the population as every pipeline run in the agreed window. Include runs that hit the cache, runs that missed, and runs that were cancelled after the build step. Record the runs that could not be inspected and the reason. Report them separately from runs that failed the cold-build test.

Classify each run by cache evidence. Use categories such as full hit, partial restore, clean miss, and cache disabled. Do not merge the categories after seeing the result. The categories point to different fixes.

Record the job duration, the time spent restoring the cache, and the time spent rebuilding. Note the download size and the runner's network class. Record the named pipeline owner and the cache size budget. A restore that takes longer than the rebuild is a net loss.

Report the number of runs, the number that were cold, and the median and slowest cold duration. Show counts by job and by trigger type. Present the clean misses and the disabled-cache runs as separate rows. Use the same denominator for every rate.

Separate a cold cache caused by a key change from one caused by an expiry or a failed store. A key change may be correct, while an expiry is a policy problem. Report the two categories separately because the fixes differ. Record the trigger and the date beside each finding.

Inspect the runs around a toolchain upgrade or a lockfile change. A cluster of cold builds may be expected, while a cluster with no change is a cache defect. Compare the pattern with the dependency history and the runner image updates. A single miss can be normal, while a cluster is a gap.

Practical application

Do not treat a long build as proof that the cache is cold. A cold run can be fast on a small project, and a warm run can be slow on a large one. Compare the same job and commit range across runs. Report the difference between a slow pipeline and a missing cache.

The record cannot show whether the delay blocked a release or whether an engineer waited idle. It can show only the time spent rebuilding when a cache was unavailable. A cold build is not proof of waste, and a warm build is not proof of correctness. The protocol also cannot see a local build that bypassed the pipeline.

A useful additional view is the share of cold time spent on each dependency group. Record the package manager, the compiler, and the image layer. That field turns a total into a review of which cache is worth fixing first. Report the largest group beside the total.

Record whether the pipeline still passes when the cache is disabled. A fast pipeline that depends on a stale cache can hide a missing dependency. Note the last cold-build result and its date. Report the jobs that have never run without a cache.

Check the cache key for the lockfile, toolchain, flags, and runner image. A key that omits one input restores the wrong output after a change. Record the key inputs and the owner. Report the jobs whose key cannot be traced to those inputs.

Give the result to the client technical owner and the offshore delivery lead with one observable change. The change might be a key input, a size budget, a scheduled cold build, or a restore timeout. Record the owner, the effective date, and the measure that will show whether the change worked. Repeat the protocol with the same definitions after the change.

Close the review with one change and a date to re-measure. The change should address the largest cold-time group, not the easiest job to adjust. Record the expected effect and the evidence that will prove it. A cache policy that is never measured is a guess about last quarter's pipeline.

Build the sample from pipeline logs and cache service records, not from the build script. The script shows the intended steps, while the logs show the time each step consumed. Compare the two and record the jobs that appear in only one source. The mismatches are often the restore steps that fail silently.

Use two evidence sources for each run, such as the pipeline log and the cache hit record. The log proves the step ran, while the cache record proves whether a restore happened. Compare them on the same clock. A log without a cache record is hard to classify.

Operating controls

Runs around a release are overrepresented because they are watched. Include a routine pull-request run and a nightly job. Record the trigger and the branch. A nightly job often carries the worst cold-cache cost because nobody waits for it.

Record the runner image version and the toolchain version for each job. A new image can invalidate every cache without a code change. Note the date of the image update and the owner. Report the jobs whose cold runs started on an image update day.

Report the result by wall-clock time and by engineer wait time as well as by count. A nightly cold build costs machine time, while a pull-request cold build costs a person waiting. Record the difference beside the finding. The reader should see which delay matters most.

Repeat the measure after the first key fix and keep the categories unchanged. A drop in clean misses is progress, but it is not yet a stable pipeline. Compare the next window with the same definitions. The trend matters more than a single release.

Check whether the cache stores secrets or production data. A cache that leaks credentials is a security finding even when the timing is good. Record the scan result and the owner. Report the jobs whose cache has never been inspected.

Treat the protocol as a review of pipeline design, not a judgment of the engineers who wait on it. A cold cache often means the key was copied from a template without the toolchain input. Ask who owns the pipeline file. The answer points to the missing review rather than a slow team.

Keep the cache evidence with the pipeline configuration so a reviewer can connect a delay to a real key change. Record the key, the restore result, and the job duration. A claim that cannot be tied to a run is only an impression. The record should survive a change of pipeline owner.

Set a rule that every cache key names its inputs and every release cycle includes one cache-disabled build. Write the rule into the pipeline documentation and the release checklist. Record the exception process and the approver. The rule turns a good intention into a check that can be verified.

Sources

  1. GitHub Actions dependency caching
  2. Docker build cache
  3. npm ci
  4. Turborepo caching

Frequently asked questions

What should happen first?

Begin with a bounded ticket, approved access, explicit acceptance evidence, and a named reviewer.

Who approves production changes?

The client's technical owner keeps final merge and release authority.

Related research

How often does an offshore release wait on client approval?

Build a programming lane your technical owner can review.

Bring the stack, backlog, acceptance checks, and release boundaries. We will help scope the role around the work.

Discuss your programming needs