Review all case details
Every stage, evidence variant, option, consequence and draft perspective for the seven cases, without playing each path. All organisations and figures are fictional. Earlier choices are shown as letters, for example "AB".
01 Onboarding and acceptance · version 1Digital onboarding and acceptance expansionActivation is stalling after verification was tightened, while the business wants POS, gateway and online acceptance launched for a new merchant segment. Recover conversion, keep verification intact and set a launch scope that can be supported.
Fictional acquirer and merchants. Figures are illustrative. Verification requirements are described in terms of the acquirer's own policy, not any specific jurisdiction's law.
Stage 1 · Activation is stalling Week 6
Completed applications fell from 61% to 44% after a stricter document step. Sales wants the old journey back. Risk wants to keep the new one. Nobody has shown where applicants actually leave.
Situation and evidence (1 variant)
The new step asks for a second identity document and a live photo. Drop-off is concentrated on mobile and on sole traders. Manual review queues have grown to four working days.
Funnel and review signals
- KYC completion before activation. Gap or risk.
Required on every path under the acquirer's own policy. No option in this case activates a merchant before verification is complete.
- Application completion. Gap or risk.
44% (was 61%)
- Drop-off at document step, mobile. Caution.
38% of starters
- Manual review backlog. Caution.
4 working days, 310 cases
- Reason codes on review decisions. Gap or risk.
Recorded on 52% of decisions
What do you change first?
a · Keep the checks, fix capture and review
Same verification standard. Redesign mobile capture, add save and resume, and make reason codes mandatory on every review decision.
Protects: The verification standard and an explainable review trail.
Costs: Several weeks of lower conversion while fixes ship.
Conditions: When drop-off is mainly friction rather than applicants who cannot pass.
Earlier choices: Any earlier path
Mobile completion recovers to 53% within three weeks. Every review decision now carries a reason a reviewer or auditor can read.b · Tier verification by assessed risk
Lower-risk segments get a document set that still meets policy minimums; higher risk keeps the full step. Compliance signs off the tiers.
Protects: Conversion where extra friction adds little risk insight.
Costs: A tiering model that needs a named owner, documentation and periodic review.
Conditions: When compliance confirms every tier meets the policy minimum and the tiering logic can be explained to a reviewer.
Earlier choices: Any earlier path
Compliance approves two tiers. Completion reaches 57%, and the tiering rules now need an owner and quarterly review.Risk flags: model-owner
c · Activate first and verify within 30 daysUnsafe
Let merchants start accepting cards immediately and complete verification afterwards.
This bypasses KYC. It is shown because it is commonly proposed, not because it is a valid option.
Protects: Short-term activation numbers.
Costs: Merchants process payments before verification is complete, which the acquirer's policy does not allow.
Conditions: Not defensible here. Policy requires verification before activation. Faster review is the legitimate lever.
Earlier choices: Any earlier path
Compliance stops the change before release. Two weeks are lost and the review backlog is still four days.Risk flags: kyc-bypass
Stage 2 · Three channels, one merchant Week 12
The new segment wants POS terminals, a payment gateway and online checkout. Each channel has its own onboarding system today, so the same merchant would be verified and set up three times.
Situation and evidence (3 variants)
The review trail is now consistent, so one verified merchant record could be reused across channels.
Integration estate
- KYC completion before activation. Gap or risk.
Required on every path under the acquirer's own policy. No option in this case activates a merchant before verification is complete.
- Merchant record. Gap or risk.
Three separate records today: POS, gateway, online
- Terminal estate. Caution.
Two terminal models; one needs a software update for contactless limits
- Gateway integration. Caution.
Hosted payment page ready; direct API needs a security review
- Review decisions. In place.
Reason codes on 100% of new decisions
Tiered verification is live, so the merchant record must carry the tier and its evidence to every channel.
Integration estate
- KYC completion before activation. Gap or risk.
Required on every path under the acquirer's own policy. No option in this case activates a merchant before verification is complete.
- Merchant record. Gap or risk.
Three separate records today: POS, gateway, online
- Terminal estate. Caution.
Two terminal models; one needs a software update for contactless limits
- Gateway integration. Caution.
Hosted payment page ready; direct API needs a security review
- Verification tier. Caution.
Must travel with the merchant record to every channel
The activation argument has cost time and trust with compliance. Integration planning starts late, and any shortcut will now be examined closely.
Integration estate
- KYC completion before activation. Gap or risk.
Required on every path under the acquirer's own policy. No option in this case activates a merchant before verification is complete.
- Merchant record. Gap or risk.
Three separate records today: POS, gateway, online
- Terminal estate. Caution.
Two terminal models; one needs a software update for contactless limits
- Gateway integration. Caution.
Hosted payment page ready; direct API needs a security review
- Compliance relationship. Gap or risk.
Strained after the activate-first proposal
How do you structure the rollout?
a · One merchant record, channels in waves
Build a single verified merchant record. Launch POS first, then gateway and online on the same record.
Protects: Consistent verification and one source of truth across channels.
Costs: Online and gateway merchants wait for wave two.
Conditions: When most of the segment's value is in-store, or when online fraud controls are not yet proven.
Earlier choices: Any earlier path
POS launches on the shared record. Gateway and online follow six weeks later with no duplicate verification.b · All three channels together
Launch POS, gateway and online at once on a shared record, with a larger integration team.
Protects: The omnichannel proposition the segment asked for.
Costs: Three integration risks land in the same week, with shared people on support.
Conditions: When the gateway security review is complete and support can absorb three channels at once.
Earlier choices: Any earlier path
Launch date holds, but the direct API security review becomes the critical path.Risk flags: parallel-risk
c · Let each channel onboard separately
Keep three onboarding systems. Each channel team launches when ready.
Protects: Channel team independence and speed for the first channel.
Costs: Three verification records that can drift apart, and merchants asked for the same documents three times.
Conditions: Only as a short bridge when a shared record is more than a quarter away, with reconciliation between records.
Earlier choices: Any earlier path
POS launches quickly. Within a month, 40 merchants have mismatched details between channels.Risk flags: record-drift
Stage 3 · Launch scope and go-live gates Week 20
The commercial date is four weeks away. Decide what launches, for whom, and which gates must be met first.
Situation and evidence (3 variants)
Current plan: POS first, gateway and online in wave two. The shared merchant record is stable.
Readiness
- KYC completion before activation. Gap or risk.
Required on every path under the acquirer's own policy. No option in this case activates a merchant before verification is complete.
- Channel plan. Context.
POS first, gateway and online in wave two
- Direct API security review. In place.
Not needed for wave one
- Support readiness. In place.
Runbooks for launch scope complete
- Monitoring. In place.
Activation and first-transaction dashboards live
Current plan: POS, gateway and online together. The shared merchant record is stable.
Readiness
- KYC completion before activation. Gap or risk.
Required on every path under the acquirer's own policy. No option in this case activates a merchant before verification is complete.
- Channel plan. Context.
POS, gateway and online together
- Direct API security review. Gap or risk.
Open: two findings unresolved
- Support readiness. Caution.
Runbooks for POS only
- Monitoring. In place.
Activation and first-transaction dashboards live
Current plan: Three channel teams with separate merchant records. Record mismatches are being fixed by hand.
Readiness
- KYC completion before activation. Gap or risk.
Required on every path under the acquirer's own policy. No option in this case activates a merchant before verification is complete.
- Channel plan. Caution.
Three channel teams with separate merchant records
- Direct API security review. In place.
Closed for POS only
- Support readiness. In place.
Runbooks for launch scope complete
- Monitoring. In place.
Activation and first-transaction dashboards live
What do you launch?
a · Pilot cohort with exit criteria
Launch to 50 merchants with written exit criteria: activation time, first-transaction success and review backlog.
Protects: Real evidence before scale, with a defined point to expand.
Costs: The commercial headline is smaller at launch.
Conditions: When the launch includes anything not yet run in production, such as a new channel or record.
Earlier choices: Any earlier path
The pilot meets two of three criteria. Review backlog is the one to fix before scaling.b · Full launch with named risk acceptance
Launch to the whole segment on the date. Known gaps are written down and accepted by a named executive.
Protects: The commercial date and clear ownership of the remaining risk.
Costs: Open gaps become live risks for every merchant in the segment.
Conditions: When the open gaps are operational rather than verification or security gaps, and the person accepting has the authority to do so.
Earlier choices: AA, AC, BA, BC, CA, CC
The segment launches on the date. Two operational gaps are tracked to closure by their owner.Earlier choices: AB, BB, CB
The open API findings are a security gap, so risk acceptance is challenged. Launch proceeds for POS and hosted page only.Risk flags: security-acceptance
c · Hold until every channel is ready
Move the date until all channels meet every gate.
Protects: A clean launch with nothing outstanding.
Costs: Lost commercial window and pressure on the next date.
Conditions: When a gate failure would affect verification, funds or card data, not just convenience.
Earlier choices: Any earlier path
Launch moves six weeks. The segment's largest prospect signs with a competitor for its online channel.
Risk flag notes
- Verification bypass (kyc-bypass): Activating before verification is complete is not a conversion tactic. The legitimate levers are better capture, faster explainable review and risk-based tiers approved by compliance.
- A tiering model needs an owner (model-owner): Risk tiers drift as merchant mix changes. Name an owner, document the logic and review it on a schedule.
- Stacked launch risk (parallel-risk): Three channel integrations in one window share the same people and support desk. Check whether any one of them is on the critical path.
- Divergent merchant records (record-drift): Separate records for one merchant drift. That weakens verification evidence and makes settlement and support harder.
- Security gaps are not ordinary risk acceptance (security-acceptance): Open security findings on a card data path usually need fixing or descoping, not executive acceptance.
02 Authorisation to settlement · version 2Transaction processing and financial integrityA disruption between a fictional processor and several issuers lowers authorisation success. Restore service, separate issuer business declines from technical failures, avoid duplicate processing and keep reconciliation and settlement correct.
Fictional processor, issuers and merchants. Figures are illustrative. The case runs from the first alert to the next settlement cut-off.
Stage 1 · Approvals fall, but not all for the same reason T+10 min
Authorisation success dropped from 96.9% to 89.2% in ten minutes. Part of the drop is ordinary issuer declines. Part is timeouts on one network link. Treating them the same would make things worse.
Situation and evidence (1 variant)
Timeouts are concentrated on three issuers routed through one network link. Issuer business declines such as insufficient funds are at their normal level. Merchants are calling.
Authorisation breakdown
- Duplicate processing risk. Gap or risk.
Requests with an unknown outcome may already be approved at the issuer. Resending them without an idempotency check can charge a cardholder twice.
- Authorisation success. Gap or risk.
89.2% (baseline 96.9%)
- Issuer business declines. Context.
2.6% of requests, normal range
- Technical failures (timeouts). Gap or risk.
8.1% of requests, 3 issuers, 1 link
- Alternate link capacity. Caution.
About 60% of the affected volume
- Stand-in limits. Caution.
Agreed with 2 of the 3 issuers, low-value only
What is the first response?
a · Segment by response class and reroute within capacity
Leave business declines alone. Move a bounded share of the affected issuers to the alternate link and watch its error rate.
Protects: Service for most affected traffic without overloading the alternate link.
Costs: About 40% of the affected traffic still times out until the primary link recovers.
Conditions: When you can see response codes by issuer and link, and the alternate route has headroom you have measured.
Earlier choices: Any earlier path
Success recovers to 94.8% in eight minutes. The alternate link holds at 85% of its capacity.b · Retry every failed request automaticallyUnsafe
Turn on automatic resend for all failed authorisations to recover volume quickly.
Blanket retry is unsafe on a payment path. It risks double charging and makes the outage larger.
Protects: Nothing reliable. Retries do not fix a failing link and resend business declines.
Costs: Duplicate approvals for requests that already succeeded at the issuer, extra load on a failing link, and resent declines.
Conditions: Not defensible as a blanket action. Retries are only safe for specific failure codes with an idempotency key the issuer honours.
Earlier choices: Any earlier path
Load on the failing link rises 70%. Success dips to 86.0% and the number of requests with an unknown outcome more than doubles.Risk flags: blanket-retry, duplicate-risk
c · Use stand-in for the issuers that agreed limits
Approve low-value transactions on behalf of the two issuers within their written stand-in limits. The third issuer stays on its normal route.
Protects: Cardholder and merchant experience for low-value payments.
Costs: Credit exposure within the limits, and stand-in advices that must be sent and reconciled later.
Conditions: When stand-in limits exist in writing for each issuer and advice processing is tested.
Earlier choices: Any earlier path
Low-value approvals recover for two issuers. The third issuer's timeouts continue.Risk flags: stand-in-exposure
Stage 2 · Requests with an unknown outcome T+45 min
The primary link is back. Some requests timed out after reaching the issuer, so nobody yet knows whether they were approved. Some merchants have asked customers to pay again.
Situation and evidence (3 variants)
Unknown-outcome requests: 1,240. Merchants report customers paying twice after a failed first attempt.
Outcome and integrity evidence
- Duplicate processing risk. Gap or risk.
Requests with an unknown outcome may already be approved at the issuer. Resending them without an idempotency check can charge a cardholder twice.
- Unknown-outcome requests. Gap or risk.
1,240
- Idempotency keys. In place.
Present on 97% of affected requests
- Issuer and scheme records. Caution.
Available in about 2 hours
- Merchant re-presentment. Caution.
Some merchants re-ran payments manually
Unknown-outcome requests: 2,960 (blanket retry added 1,720 resends). Merchants report customers paying twice after a failed first attempt.
Outcome and integrity evidence
- Duplicate processing risk. Gap or risk.
Requests with an unknown outcome may already be approved at the issuer. Resending them without an idempotency check can charge a cardholder twice.
- Unknown-outcome requests. Gap or risk.
2,960 (blanket retry added 1,720 resends)
- Idempotency keys. In place.
Present on 97% of affected requests
- Issuer and scheme records. Caution.
Available in about 2 hours
- Merchant re-presentment. Caution.
Some merchants re-ran payments manually
Unknown-outcome requests: 1,240. Stand-in advices for two issuers are queued for delivery.
Outcome and integrity evidence
- Duplicate processing risk. Gap or risk.
Requests with an unknown outcome may already be approved at the issuer. Resending them without an idempotency check can charge a cardholder twice.
- Unknown-outcome requests. Gap or risk.
1,240
- Idempotency keys. In place.
Present on 97% of affected requests
- Issuer and scheme records. Caution.
Available in about 2 hours
- Stand-in advices. Caution.
4,100 queued, not yet acknowledged
How do you resolve the unknown outcomes?
a · Match against issuer records using idempotency keys
Wait for issuer and scheme records, match each unknown request by key, then reverse only confirmed duplicates.
Protects: Financial correctness: only real duplicates are reversed.
Costs: Two hours of uncertainty for merchants and cardholders.
Conditions: When keys are present and issuer records will arrive before settlement cut-off.
Earlier choices: A, C
Matching confirms 180 duplicates from manual re-presentment. Each is reversed with a reference.Earlier choices: B
Matching confirms 610 duplicate approvals created by the blanket retry. Each is reversed with a reference.b · Reverse every unknown request nowUnsafe
Send reversals for all requests with an unknown outcome so no cardholder stays charged.
Blanket reversal swaps one integrity problem for another.
Protects: Speed for cardholders who were charged twice.
Costs: Valid approvals are also reversed, so merchants may ship goods they are not paid for.
Conditions: Only for a narrow set where evidence shows no approval can have reached the issuer.
Earlier choices: Any earlier path
Duplicates are cleared, but around 1,000 valid payments are reversed and must be re-collected.Risk flags: blanket-reversal
c · Let end-of-day reconciliation find them
Take no action now. The normal reconciliation will surface any duplicates.
Protects: Team focus on recovery.
Costs: Cardholders stay double charged for a day or more, and duplicates may enter settlement.
Conditions: When the volume is very small and the next reconciliation runs before settlement.
Earlier choices: Any earlier path
Reconciliation flags duplicates after the settlement file is built. Corrections must go through adjustments.Risk flags: late-detection
Stage 3 · Settlement cut-off T+5 h
The settlement file is due in three hours. Some items are still suspect. Merchants expect funds on time.
Situation and evidence (3 variants)
Confirmed duplicates are reversed. A small set of items is still unmatched.
Settlement position
- Clean items. In place.
99.6% of the batch
- Suspect items. Caution.
62 unmatched
- Dual control on adjustments. In place.
Available: preparer and independent approver
- Merchant funding SLA. Context.
Next business day
Valid payments were reversed. Finance needs to decide how to treat the re-collection.
Settlement position
- Clean items. In place.
99.6% of the batch
- Suspect items. Gap or risk.
1,000 re-collections
- Dual control on adjustments. In place.
Available: preparer and independent approver
- Merchant funding SLA. Context.
Next business day
Duplicates have not been isolated. Finance cannot say which items are clean.
Settlement position
- Clean items. Gap or risk.
Unknown
- Suspect items. Gap or risk.
Not isolated
- Dual control on adjustments. In place.
Available: preparer and independent approver
- Merchant funding SLA. Context.
Next business day
What goes into settlement?
a · Settle the clean set, hold suspects under dual control
Settle verified items on time. Hold suspects and release them through adjustments prepared and approved by different people.
Protects: On-time funding for most merchants and correct books.
Costs: Some merchants receive part of their funds a day late.
Conditions: When you can isolate suspect items and have independent approval for adjustments.
Earlier choices: AA, AB, BA, BB, CA, CB
The clean batch settles on time. Suspect items clear over two days with a full adjustment trail.Earlier choices: AC, BC, CC
Isolation takes until cut-off. The batch settles 40 minutes late but correct.b · Delay the whole settlement
Hold the entire file until every item is verified.
Protects: Absolute certainty in the file.
Costs: Every merchant is funded late, including those unaffected.
Conditions: When suspect items cannot be isolated and errors would be large or hard to recover.
Earlier choices: Any earlier path
All merchants are funded a day late. The support desk takes 900 calls.c · Settle everything and correct laterUnsafe
Send the full file on time and fix errors with chargebacks and adjustments afterwards.
Knowingly settling unverified duplicates puts financial integrity behind the deadline.
Protects: The funding date.
Costs: Known errors are paid out, and recovering funds is slower and less certain than holding them.
Conditions: Only when suspect value is trivial and recovery from merchants is reliable.
Earlier choices: Any earlier path
Funds move on time, including duplicates. Recovery takes weeks and some is written off.Risk flags: integrity-deferred
Risk flag notes
- Blanket retry (blanket-retry): Resending all failures adds load to a failing route and resends business declines. Retry only specific codes with honoured idempotency keys.
- Duplicate authorisation (duplicate-risk): A timeout does not mean the issuer did not approve. Any resend without an idempotency check can create a second approval.
- Stand-in exposure (stand-in-exposure): Stand-in approvals carry credit risk within the agreed limits and must be advised to the issuer and reconciled.
- Blanket reversal (blanket-reversal): Reversing everything with an unknown outcome reverses valid payments too. Match first, then reverse what is confirmed.
- Late detection (late-detection): Waiting for normal reconciliation can let duplicates reach settlement, where they are harder to recover.
- Integrity deferred (integrity-deferred): Settling items known to be suspect moves the problem into recovery, which is slower and lossier than holding.
03 Operations with constrained headcount · version 2Build 24/7 support and escalationA fictional payments platform needs round-the-clock support with twelve people. Decide coverage, who owns triage, diagnosis and engineering fixes, how privileged access and high-risk change are separated, and when to escalate.
Fictional organisation. The capacity model uses explicit illustrative assumptions so the arithmetic can be checked. It is not a recommendation for any real team size.
Stage 1 · Coverage that adds up Design week 1
Twelve support people. One seat staffed around the clock needs six of them under the stated assumptions. Decide what coverage you can actually sustain.
Situation and evidence (1 variant)
L1 runs triage and runbooks. L2 diagnoses and recovers. L3 is engineering squads and vendors who own code and platform fixes. Today L2 works business hours only.
Capacity model
- Illustrative assumptions. Context.
12 people, 40 contracted h/week, 80% available = 32 productive h each. 24/7 = 168 h/week per seat.
- One 24/7 seat. Context.
168 ÷ 32 = 5.25, rounded up to 6 people
- Overnight incidents, last quarter. Caution.
14, of which 9 needed L2
- Handover overlap. Gap or risk.
None today; shifts change on the hour
Which coverage model do you adopt?
a · Staff L1 and L2 around the clock
One L1 seat and one L2 seat, 24/7.
Protects: Immediate L2 diagnosis at any hour.
Costs: Uses all twelve people with no spare for handover overlap, training or absence above the assumption.
Conditions: When overnight incidents needing L2 are frequent enough to justify the cost and you can add headcount soon.
Earlier choices: Any earlier path
Coverage works on paper. The first month with two people on leave forces overtime on four shifts.Risk flags: no-slack
b · L1 around the clock, L2 extended hours plus on-call
L1 24/7. L2 staffed 07:00 to 23:00 with a paid on-call rota overnight, and two people spare for overlap and absence.
Protects: Sustainable shifts with a 30-minute handover overlap.
Costs: Overnight L2 response depends on on-call wake-up time.
Conditions: When most overnight incidents can be contained by runbooks until L2 joins within an agreed time.
Earlier choices: Any earlier path
Shifts get a 30-minute handover overlap. Overnight L2 joins within 20 minutes on average.c · Two seats at every tier, 24/7Unsafe
Double-staff L1 and L2 around the clock for resilience.
This coverage is impossible with the available people. A rota that relies on unsustainable hours is a risk, not a control.
Protects: The appearance of resilience.
Costs: Needs twice the available headcount. It only works on paper with 70-hour weeks.
Conditions: Only with roughly double the headcount. With twelve people it is not achievable.
Earlier choices: Any earlier path
The rota cannot be filled. Gaps appear in week one and overtime becomes the plan.Risk flags: impossible-coverage
Stage 2 · Ownership and separation of duties Design week 3
Recovery sometimes needs privileged access and emergency changes. Decide who holds access, who approves high-risk change, who executes it and who checks that it worked.
Situation and evidence (3 variants)
The coverage model is set. L2 currently shares a standing admin account for recovery.
Access and change evidence
- L1 triage seat, 24/7. Context.
168 h/week ÷ 32 productive h = 6 people
- L2 diagnosis seat, 24/7. Context.
168 h/week ÷ 32 productive h = 6 people
- Required vs available. Caution.
12 required of 12 available, spare 0
- Privileged access. Gap or risk.
Shared standing admin account used by L2
- Emergency change approvals, last quarter. Gap or risk.
6 of 11 approved by the person who executed them
- Post-change validation. Caution.
Done by the executor in all cases
The coverage model is set. L2 currently shares a standing admin account for recovery.
Access and change evidence
- L1 triage seat, 24/7. Context.
168 h/week ÷ 32 productive h = 6 people
- L2 seat 07:00 to 23:00, 7 days. Context.
112 h/week ÷ 32 productive h = 4 people
- Required vs available. In place.
10 required of 12 available, spare 2
- Privileged access. Gap or risk.
Shared standing admin account used by L2
- Emergency change approvals, last quarter. Gap or risk.
6 of 11 approved by the person who executed them
- Post-change validation. Caution.
Done by the executor in all cases
The rota is already short. Any design that needs extra people at night will not be staffed.
Access and change evidence
- L1 two seats, 24/7. Context.
336 h/week ÷ 32 productive h = 11 people
- L2 two seats, 24/7. Context.
336 h/week ÷ 32 productive h = 11 people
- Required vs available. Gap or risk.
22 required of 12 available, spare -10
- Privileged access. Gap or risk.
Shared standing admin account used by L2
- Emergency change approvals, last quarter. Gap or risk.
6 of 11 approved by the person who executed them
- Post-change validation. Caution.
Done by the executor in all cases
How do you set ownership and controls?
a · Time-bound access, independent approval and validation
Named, time-bound elevation for L2 with logging. High-risk changes approved by someone other than the executor, often L3. Validation by a third person or an automated check.
Protects: Every privileged action is attributable, and nobody approves or checks their own work.
Costs: Slower emergency changes and an approver rota at night.
Conditions: For any change or access that could affect funds, card data or many merchants.
Earlier choices: A, B
Elevation is logged per person. Emergency change approval moves to L3 on-call, with validation by an automated check.Earlier choices: C
The controls are sound, but nobody is free to approve at night. Approvals fall back to L3 on-call.b · Split L1 and L2 into separate teams and call it segregationUnsafe
Make L1 and L2 different teams with different managers. Keep the shared account and executor approval.
Separate teams are not separation of duties if one person can still approve and execute their own high-risk change.
Protects: A clear org chart.
Costs: No real control: the same person still holds access, approves and validates.
Conditions: Splitting teams helps capability and ownership. It is not a control on its own.
Earlier choices: Any earlier path
The structure looks tidy. The next audit finds self-approved emergency changes unchanged.Risk flags: split-not-control, self-approval
c · Move all recovery to L3
L2 diagnoses only. Every recovery action is done by engineering on-call.
Protects: Privileged access held by fewer people.
Costs: Slower recovery and engineers paged for routine runbook actions.
Conditions: When recovery actions are rare and mostly need code changes rather than runbooks.
Earlier choices: Any earlier path
Overnight recovery time rises from 25 to 55 minutes. Engineers report on-call fatigue within a month.Risk flags: l3-fatigue
Stage 3 · 03:10, an incident that does not fit the runbook 03:25
Card-not-present approvals are failing for one merchant group. The L1 runbook does not resolve it after 15 minutes.
Situation and evidence (6 variants)
L2 is on shift and starts diagnosis immediately.
Incident evidence
- Merchant impact. Caution.
1 merchant group, about 8% of night volume
- Runbook. Gap or risk.
Executed; no recovery
- Escalation triggers. In place.
Defined: 15 minutes without diagnosis, or impact above 5% of volume
- Approval path for a fix. In place.
Independent approver reachable via L3 on-call
L2 is on shift and starts diagnosis immediately.
Incident evidence
- Merchant impact. Caution.
1 merchant group, about 8% of night volume
- Runbook. Gap or risk.
Executed; no recovery
- Escalation triggers. In place.
Defined: 15 minutes without diagnosis, or impact above 5% of volume
- Approval path for a fix. Gap or risk.
The person who will execute can also approve
L2 on-call has been paged and is joining.
Incident evidence
- Merchant impact. Caution.
1 merchant group, about 8% of night volume
- Runbook. Gap or risk.
Executed; no recovery
- Escalation triggers. In place.
Defined: 15 minutes without diagnosis, or impact above 5% of volume
- Approval path for a fix. In place.
Independent approver reachable via L3 on-call
L2 on-call has been paged and is joining.
Incident evidence
- Merchant impact. Caution.
1 merchant group, about 8% of night volume
- Runbook. Gap or risk.
Executed; no recovery
- Escalation triggers. In place.
Defined: 15 minutes without diagnosis, or impact above 5% of volume
- Approval path for a fix. Gap or risk.
The person who will execute can also approve
The night shift is one person short. L1 is covering two queues.
Incident evidence
- Merchant impact. Caution.
1 merchant group, about 8% of night volume
- Runbook. Gap or risk.
Executed; no recovery
- Escalation triggers. In place.
Defined: 15 minutes without diagnosis, or impact above 5% of volume
- Approval path for a fix. In place.
Independent approver reachable via L3 on-call
The night shift is one person short. L1 is covering two queues.
Incident evidence
- Merchant impact. Caution.
1 merchant group, about 8% of night volume
- Runbook. Gap or risk.
Executed; no recovery
- Escalation triggers. In place.
Defined: 15 minutes without diagnosis, or impact above 5% of volume
- Approval path for a fix. Gap or risk.
The person who will execute can also approve
How do you escalate and command?
a · Trigger-based escalation with one incident commander
Triggers are met, so page L2 and an incident commander. L3 owner for the gateway joins. Handover notes are written as the incident runs.
Protects: One person directing, with the right owner on the problem.
Costs: Two or three people woken for an incident that may turn out small.
Conditions: When the escalation triggers have been agreed in advance and are met.
Earlier choices: Any earlier path
The L3 owner finds a certificate issue within 20 minutes. The fix is approved independently and validated against live approvals.b · L2 applies an emergency change and approves itUnsafe
To save time, the L2 engineer approves and executes a configuration change on the gateway alone.
Self-approval of a high-risk change is exactly what separation of duties exists to prevent.
Protects: Speed at 3 a.m.
Costs: No independent check on a high-risk change made by a tired person.
Conditions: Not defensible for a high-risk change. Use pre-approved standard changes if speed matters.
Earlier choices: Any earlier path
The change restores approvals but disables a fraud rule as a side effect. It is found at 09:00.Risk flags: self-approval
c · Page everyone
Wake L2, L3, vendor support and management at once.
Protects: Nobody can say they were not told.
Costs: Several people talking at once and no clear commander. Fatigue for the next day.
Conditions: When impact is severe and broad, and an incident commander is named in the first minutes.
Earlier choices: Any earlier path
Twelve people join the bridge. Diagnosis takes 45 minutes because nobody is directing.Risk flags: no-commander
Risk flag notes
- No slack in the rota (no-slack): A rota that uses every person has no room for absence, training or handover. Expect overtime and errors.
- Impossible coverage (impossible-coverage): The arithmetic does not close. A rota that depends on unsustainable hours will fail when it is needed most.
- Splitting teams is not a control (split-not-control): L1 and L2 are a capability and ownership split. Segregation of duties is about who can hold privileged access, approve high-risk change, execute it and validate it.
- Self-approval (self-approval): One person approving and executing a high-risk change removes the independent check.
- Engineering on-call load (l3-fatigue): Routing routine recovery to engineers wears down the people who own the fixes.
- No incident commander (no-commander): More people does not mean faster recovery. Name one commander and the owner of the failing service.
04 Monitoring and assisted operations · version 2AIOps and interactive monitoringAn AIOps tool groups a storm of alerts and recommends a database failover with high confidence. Separate symptoms from cause, check telemetry quality, tie signals to transaction impact and choose bounded investigation and recovery.
Fictional platform and AIOps tool. Signals and dependencies are illustrative. Nothing here runs real automation.
Stage 1 · A storm and a confident recommendation 02:07
140 alerts in six minutes. The AIOps tool says the database is the root cause with 92% confidence and offers to fail it over. Card-not-present approvals are down in one region.
Situation and evidence (1 variant)
The database alert has a history of false positives and its exporter is lagging. The token service started erroring two minutes before anything else.
Monitoring evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- AIOps recommendation. Caution.
Root cause: database. Confidence 92%. Action: fail over primary.
- 02:01 Token service. Caution.
TLS handshake errors begin, 3% of calls
- 02:03 Payment API. Gap or risk.
p99 latency 1.8 s (normal 400 ms)
- 02:04 Database. Caution.
Connection pool at 96%
- 02:06 Alert volume. Gap or risk.
140 alerts in 6 minutes across 11 services
- Payment API depends on. Context.
Token service, risk engine, database
- Token service depends on. Context.
Certificate store, HSM cluster
- Database metrics exporter. Gap or risk.
Last sample 9 minutes old
- Alert rule history. Caution.
Pool saturation alert: 70% false positive rate last quarter
- Card-not-present approvals. Gap or risk.
Down 6 points for one merchant region
- Card-present approvals. In place.
Normal
What is your first move?
a · Tie signals to customer impact before acting
Check which transactions are actually affected, order the signals in time and test whether the database is cause or symptom.
Protects: Acting on the cause, not the loudest symptom.
Costs: A few minutes of continued impact while you look.
Conditions: When impact is contained and the recommended action is disruptive.
Earlier choices: Any earlier path
The timeline shows database pressure following token retries. The database looks like a symptom.b · Accept the recommendation and fail overUnsafe
92% is high. Let the tool fail over the database.
Confidence alone does not authorise automation, especially for actions outside an approved runbook scope.
Protects: Speed, if the tool is right.
Costs: A disruptive action on stale telemetry and a noisy alert. Failover drops in-flight transactions.
Conditions: Not defensible here. Failover is outside the tool's approved action scope, and the evidence is stale.
Earlier choices: Any earlier path
Failover takes four minutes and drops in-flight requests. Approvals do not recover because the cause is elsewhere.Risk flags: confidence-not-authority, symptom-chased
c · Suppress the noise and waitUnsafe
Mute the grouped alerts for 15 minutes and wait for a clearer signal.
Do not mute unresolved customer impact. Suppress only confirmed noise, with critical signals retained, a named owner and an expiry.
Protects: Responders from alert fatigue.
Costs: Real impact continues unobserved and the signal you need may be muted.
Conditions: When alerts are confirmed false and no customer impact is visible.
Earlier choices: Any earlier path
Card-not-present approvals keep falling. A merchant reports it before the team sees it.Risk flags: muted-impact
Stage 2 · Bounded investigation 02:20
Decide how to find the cause and what the AIOps tool is allowed to do while you look.
Situation and evidence (3 variants)
The token service is the earliest signal. The tool now suggests restarting stateless token service pods, which is within an approved runbook scope.
Investigation evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- 02:01 Token service. Caution.
TLS handshake errors begin, 3% of calls
- 02:03 Payment API. Gap or risk.
p99 latency 1.8 s (normal 400 ms)
- 02:04 Database. Caution.
Connection pool at 96%
- 02:06 Alert volume. Gap or risk.
140 alerts in 6 minutes across 11 services
- Payment API depends on. Context.
Token service, risk engine, database
- Token service depends on. Context.
Certificate store, HSM cluster
- Database metrics exporter. Gap or risk.
Last sample 9 minutes old
- Alert rule history. Caution.
Pool saturation alert: 70% false positive rate last quarter
- Card-not-present approvals. Gap or risk.
Down 6 points for one merchant region
- Card-present approvals. In place.
Normal
- Correlation. In place.
Database pressure follows token retries by about two minutes
The failover did not help and approvals are worse. The tool now recommends restarting the payment API.
Investigation evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- 02:01 Token service. Caution.
TLS handshake errors begin, 3% of calls
- 02:03 Payment API. Gap or risk.
p99 latency 1.8 s (normal 400 ms)
- 02:04 Database. Caution.
Connection pool at 96%
- 02:06 Alert volume. Gap or risk.
140 alerts in 6 minutes across 11 services
- Payment API depends on. Context.
Token service, risk engine, database
- Token service depends on. Context.
Certificate store, HSM cluster
- Database metrics exporter. Gap or risk.
Last sample 9 minutes old
- Alert rule history. Caution.
Pool saturation alert: 70% false positive rate last quarter
- Card-not-present approvals. Gap or risk.
Down 6 points for one merchant region
- Card-present approvals. In place.
Normal
Impact has grown and a merchant has escalated. Alerts are unmuted.
Investigation evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- 02:01 Token service. Caution.
TLS handshake errors begin, 3% of calls
- 02:03 Payment API. Gap or risk.
p99 latency 1.8 s (normal 400 ms)
- 02:04 Database. Caution.
Connection pool at 96%
- 02:06 Alert volume. Gap or risk.
140 alerts in 6 minutes across 11 services
- Payment API depends on. Context.
Token service, risk engine, database
- Token service depends on. Context.
Certificate store, HSM cluster
- Database metrics exporter. Gap or risk.
Last sample 9 minutes old
- Alert rule history. Caution.
Pool saturation alert: 70% false positive rate last quarter
- Card-not-present approvals. Gap or risk.
Down 6 points for one merchant region
- Card-present approvals. In place.
Normal
How do you investigate?
a · Walk the dependency chain from the earliest signal
Check the token service's dependencies, refresh the stale exporter and test one hypothesis at a time.
Protects: A diagnosis you can explain and validate.
Costs: Takes a specialist and some minutes.
Conditions: When the dependency map is current and the earliest signal is clear.
Earlier choices: Any earlier path
The certificate store shows one node serving an old certificate chain after a rotation at 02:00.b · Let the tool restart pods within its approved scope
Allow the pre-approved runbook action: restart stateless pods, one at a time, with automatic stop on error.
Protects: Fast, bounded action inside an approved scope.
Costs: May mask the cause if it is not in the pods.
Conditions: When the action is inside an approved, reversible runbook scope and you keep investigating in parallel.
Earlier choices: A, C
Restarts briefly clear errors on two pods, then errors return. That points away from the pods.Earlier choices: B
Restarting payment API pods has no effect. The cause is still unknown.c · Escalate to the platform vendor
Open a priority case with the certificate and HSM vendor and wait for their analysis.
Protects: Specialist knowledge of the vendor's components.
Costs: Vendor response time while impact continues.
Conditions: When the evidence points into a vendor-owned component you cannot inspect.
Earlier choices: Any earlier path
The vendor responds in 40 minutes and asks for logs your team could have checked.
Stage 3 · Recovery and what automation may do next time 02:40
Restore service, prove it worked with measured transaction data and decide what the tool may do automatically in future.
Situation and evidence (4 variants)
The faulty node is identified. Removing it from rotation is a known runbook step.
Recovery evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- Card-not-present approvals. Gap or risk.
Down 6 points
- Database exporter. In place.
Refreshed; pool at 61%
- Certificate store node 3. In place.
Confirmed serving old chain
The cause is suspected but not confirmed in the certificate store.
Recovery evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- Card-not-present approvals. Gap or risk.
Down 6 points
- Database exporter. Caution.
Still lagging
- Certificate store node 3. Caution.
Suspected
The faulty node is identified. Removing it from rotation is a known runbook step.
Recovery evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- Card-not-present approvals. Gap or risk.
Down 9 points after failover
- Database exporter. In place.
Refreshed; pool at 61%
- Certificate store node 3. In place.
Confirmed serving old chain
The cause is suspected but not confirmed in the certificate store.
Recovery evidence
- Automation authority. Gap or risk.
A confidence score is not permission to act. Automated actions are limited to pre-approved runbook scopes; anything else needs a human decision.
- Card-not-present approvals. Gap or risk.
Down 9 points after failover
- Database exporter. Caution.
Still lagging
- Certificate store node 3. Caution.
Suspected
How do you recover and what changes afterwards?
a · Human-approved runbook step, validated by transaction data
Remove the node from rotation with on-call approval. Confirm recovery by approval rate and token success, not by alerts going quiet.
Protects: A measured recovery with a named decision.
Costs: Minutes for approval at night.
Conditions: When the action is reversible and its effect can be measured within minutes.
Earlier choices: AA, BA, CA
Approvals return to baseline within six minutes, confirmed against transaction data.Earlier choices: AB, AC, BB, BC, CB, CC
The node is removed on suspicion. Approvals recover, but the team confirms the cause only next morning.b · Expand auto-remediation to all recommendations above 90%Unsafe
Let the tool act on its own whenever confidence passes 90% in future.
A confidence threshold is not an action scope. Tonight's high-confidence call was a symptom.
Protects: Faster responses for repeat issues.
Costs: Tonight's 92% recommendation was wrong. This would have made it automatic.
Conditions: Not defensible. Expand action scopes per runbook, with limits and rollback, not by a confidence threshold.
Earlier choices: Any earlier path
The policy is drafted, then withdrawn after review finds it would have failed over the database tonight.Risk flags: confidence-not-authority
c · Manual fix, then tune correlation and alert quality
Fix by hand tonight. Afterwards, retire the noisy pool alert, add exporter staleness alerts and add certificate rotation to the dependency map.
Protects: Better signals for the next incident.
Costs: Tonight's fix depends on the engineer on call.
Conditions: When tonight's action is simple and the bigger gain is telemetry quality.
Earlier choices: Any earlier path
Recovery takes 20 minutes. The follow-up removes 40% of noisy alerts the next month.
Risk flag notes
- Confidence is not authority (confidence-not-authority): A model's confidence says nothing about the blast radius of an action. Bound automation by runbook scope, reversibility and approval.
- Symptom chased (symptom-chased): The loudest alert was a downstream symptom. Order signals in time and follow dependencies before acting.
- Muted real impact (muted-impact): Suppressing alerts without checking customer impact hides the signal you need.
05 Greenfield engineering capability · version 1Establish an insourced technology centre of excellenceA fictional payments group wants its own engineering centre instead of relying on an outsourcing vendor. Set the charter and business case, build and transition without breaking service, and decide whether to scale on evidence rather than on the label.
Fictional group with product teams at headquarters and a long-standing vendor delivering most change. The board has asked for an organisation-owned engineering centre in a second location. All names and figures are illustrative assumptions.
Stage 1 · Charter and business case Quarter 0
Before hiring anyone, decide what the centre owns, what stays with headquarters and product teams, who sponsors it and how the business case is costed. A centre with a vague mandate becomes a cost centre or a standards committee.
Situation and evidence (1 variant)
The vendor contract renews in 14 months. Finance has a salary comparison showing a large saving. Product heads at headquarters are wary of losing control of their roadmaps. The proposed location has a good engineering market but a thin pool of payments and security specialists.
Business case inputs
- Salary-only comparison. Caution.
Shows a large saving and is misleading on its own
- Fully loaded cost (fictional). Context.
About 1.6 times base salary: benefits, facilities, tools, management, leave cover
- One-off transition (fictional). Caution.
About 18k per role, so 216k for 12 roles, plus incumbent overlap fees
- Executive sponsor. Caution.
Named, but no decision rights defined yet
- Talent feasibility (fictional). Caution.
General engineers available. Payments, security and SRE specialists scarce, 4 to 6 months to fill
- Vendor contract. Caution.
Renewal in 14 months. Knowledge transfer and IP clauses need review
- Illustrative assumptions. Context.
Fictional figures. New engineers deliver about 30% in their first 3 months, 60% to month 6, 85% after. Starts depend on recruiting lead time and onboarding capacity.
What charter and scope do you launch with?
a · Phased pilot with measurable outcomes
Charter the centre for two bounded services with agreed outcomes, a fully loaded cost case and a go or no-go review at month 9.
Protects: Learning before commitment. A business case tested against real delivery, quality and retention.
Costs: Slower to show scale. The vendor contract may need a short extension.
Conditions: When the organisation has not run its own engineering centre before, or the talent market is unproven.
Earlier choices: Any earlier path
The board approves a 12 role pilot with outcome measures. Finance accepts the fully loaded cost case after the salary-only view is withdrawn.b · Launch with clear product and domain ownership
Give the centre end-to-end ownership of one product domain, with 30 roles, a named product partner at headquarters and explicit scope boundaries.
Protects: Real ownership from the start, which attracts stronger talent than a support role.
Costs: More hiring and transition risk at once. Headquarters must give up part of a roadmap.
Conditions: When a sponsor can enforce the scope boundary and the domain has few dependencies on other teams.
Earlier choices: Any earlier path
The centre owns the merchant reporting domain. Two headquarters product heads accept the boundary; one escalates and the sponsor holds the line.c · Transfer all vendor scope immediatelyUnsafe
Approve 60 roles and move every service from the vendor within the year to capture the salary saving quickly.
Overbroad immediate transfer on a salary-only business case. Headcount is not capability, and continuity depends on the incumbent you are exiting.
Protects: The headline saving and a simple story for the board.
Costs: Overbroad scope, a salary-only business case and a hiring plan the market cannot fill in time.
Conditions: Rarely defensible. Only if the vendor is exiting anyway and a large experienced team can be acquired intact.
Earlier choices: Any earlier path
The plan is approved on the salary comparison. Transition, overlap and management costs are not in the budget.Risk flags: salary-only-case, overbroad-transfer
Stage 2 · Build and transition Months 1 to 9
Hiring takes months and new engineers are not productive on day one. Decide how to sequence leadership and team hiring, how knowledge and IP move from the vendor, and what readiness means before the incumbent steps back.
Situation and evidence (3 variants)
Twelve requisitions are open for the pilot services. The vendor team is cooperative and the pilot scope is documented.
Ramp and transition constraints
- Year one target (fictional). Context.
12 engineers
- Month 3. Caution.
0 hired, 0 effective FTE
- Month 6. Context.
12 hired, 3.4 effective FTE
- Month 9. Context.
12 hired, 6.8 effective FTE
- Month 12. Context.
12 hired, 9.7 effective FTE
- Incumbent overlap needed. Caution.
About 11 months before the centre covers 70% of transferred scope
- Knowledge and IP transfer. Caution.
Shadow then reverse-shadow plan possible; contract amendment needed
- Secure environments and access. Caution.
Build environment and access model in progress
- Incumbent continuity. Context.
Vendor remains accountable for live service until exit criteria are met
- Illustrative assumptions. Context.
Fictional figures. New engineers deliver about 30% in their first 3 months, 60% to month 6, 85% after. Starts depend on recruiting lead time and onboarding capacity.
Thirty requisitions are open for one domain. The vendor team on that domain is cooperative but its contract has weak knowledge transfer clauses.
Ramp and transition constraints
- Year one target (fictional). Context.
30 engineers
- Month 3. Caution.
0 hired, 0 effective FTE
- Month 6. Context.
15 hired, 4.3 effective FTE
- Month 9. Context.
30 hired, 12.8 effective FTE
- Month 12. Context.
30 hired, 20.7 effective FTE
- Incumbent overlap needed. Gap or risk.
Beyond 12 months: the centre does not reach 70% of target capacity in year one
- Knowledge and IP transfer. Caution.
Shadow then reverse-shadow plan possible; contract amendment needed
- Secure environments and access. Caution.
Build environment and access model in progress
- Incumbent continuity. Context.
Vendor remains accountable for live service until exit criteria are met
- Illustrative assumptions. Context.
Fictional figures. New engineers deliver about 30% in their first 3 months, 60% to month 6, 85% after. Starts depend on recruiting lead time and onboarding capacity.
Sixty requisitions are open. The vendor has learned it is being exited and two key architects have left. Secure environments for the new location are not ready.
Ramp and transition constraints
- Year one target (fictional). Context.
60 engineers
- Month 3. Caution.
0 hired, 0 effective FTE
- Month 6. Context.
15 hired, 4.3 effective FTE
- Month 9. Context.
30 hired, 12.8 effective FTE
- Month 12. Context.
45 hired, 24.9 effective FTE
- Incumbent overlap needed. Gap or risk.
Beyond 12 months: the centre does not reach 70% of target capacity in year one
- Knowledge and IP transfer. Gap or risk.
No agreed plan. Key vendor staff leaving
- Secure environments and access. Gap or risk.
Not ready. Engineers would start without least-privilege access set up
- Incumbent continuity. Context.
Vendor remains accountable for live service until exit criteria are met
- Illustrative assumptions. Context.
Fictional figures. New engineers deliver about 30% in their first 3 months, 60% to month 6, 85% after. Starts depend on recruiting lead time and onboarding capacity.
How do you build and transition?
a · Leadership spine first, then a paced ramp with shadowing
Hire the centre lead, architects and engineering managers first, then about 3 starts a month. Shadow, then reverse-shadow the vendor, with written exit criteria per service.
Protects: Continuity and real knowledge transfer. Clear readiness evidence before the vendor steps back.
Costs: Slower ramp and a longer, more expensive overlap with the vendor.
Conditions: When live services are critical and knowledge sits with a few vendor people.
Earlier choices: A, B
Services move one at a time after reverse-shadowing. Exit criteria are met for the first service at month 8.Earlier choices: C
The spine is hired, but at this pace the 60 role plan cannot be met in year one. The board is told the scope must shrink or the vendor stays longer.b · Parallel hiring with a defined readiness gate
Hire leaders and teams together at about 5 starts a month. Services transfer only after a readiness review: runbooks, on-call, access, test coverage and one incident handled.
Protects: A faster ramp with a readiness gate that stops premature handover.
Costs: Managers onboard while hiring. Early teams carry more risk of inconsistent practice.
Conditions: When some experienced leaders are already available and the gate is enforced by someone independent of the centre.
Earlier choices: A, B
Two services pass the readiness gate by month 9. One fails on runbook gaps and stays with the vendor another quarter.Earlier choices: C
Hiring moves, but readiness reviews keep failing on access and knowledge gaps. Transfers slip and the vendor is extended.c · Hire fast through agencies and cut the vendor on scheduleUnsafe
Open all roles through agencies at about 10 starts a month and end the vendor contract on the original date regardless of readiness.
Ending the incumbent before exit criteria are met puts live service at risk. Headcount is not readiness.
Protects: The original date and the headline headcount.
Costs: Headcount arrives without context or mentors. Live services lose the people who know them.
Conditions: Not defensible for live payment services. A hard date is not a readiness criterion.
Earlier choices: Any earlier path
Headcount targets are reported as met. A settlement report fails in the first month after exit and nobody in the new centre has run it before.Risk flags: exit-before-ready
Stage 3 · Scale, hold or re-scope Month 12
A year in, decide whether to scale, hold or re-scope the centre based on delivery, quality, retention and business outcomes, and set the governance that keeps it an owned capability rather than a label.
Situation and evidence (9 variants)
The centre has 12 engineers and about 9.5 effective FTE. Retention is strong and quality on transferred services is stable. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. In place.
9.5 effective FTE from 12 hired (79%)
- Escaped defects on transferred services. In place.
Flat against vendor baseline
- 12-month attrition (fictional). In place.
7%
- Business outcome. In place.
Pilot services met 3 of 4 agreed outcomes
- Total cost vs case. In place.
Within the fully loaded case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 12 engineers and about 9.7 effective FTE. Retention is reasonable; one service still depends on the vendor. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. In place.
9.7 effective FTE from 12 hired (81%)
- Escaped defects on transferred services. In place.
Flat against vendor baseline
- 12-month attrition (fictional). In place.
11%
- Business outcome. In place.
Pilot services met 3 of 4 agreed outcomes
- Total cost vs case. In place.
Within the fully loaded case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 12 engineers and about 7.7 effective FTE. Attrition is high and two services had incidents after transfer. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. In place.
7.7 effective FTE from 12 hired (64%)
- Escaped defects on transferred services. Gap or risk.
Rising
- 12-month attrition (fictional). Gap or risk.
24%
- Business outcome. In place.
Pilot services met 3 of 4 agreed outcomes
- Total cost vs case. In place.
Within the fully loaded case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 27 engineers and about 15.8 effective FTE. Retention is strong and quality on transferred services is stable. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. Caution.
15.8 effective FTE from 27 hired (59%)
- Escaped defects on transferred services. In place.
Flat against vendor baseline
- 12-month attrition (fictional). In place.
7%
- Business outcome. In place.
Merchant reporting releases monthly instead of quarterly
- Total cost vs case. In place.
Within the fully loaded case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 30 engineers and about 20.7 effective FTE. Retention is reasonable; one service still depends on the vendor. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. In place.
20.7 effective FTE from 30 hired (69%)
- Escaped defects on transferred services. In place.
Flat against vendor baseline
- 12-month attrition (fictional). In place.
11%
- Business outcome. In place.
Merchant reporting releases monthly instead of quarterly
- Total cost vs case. In place.
Within the fully loaded case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 30 engineers and about 19.1 effective FTE. Attrition is high and two services had incidents after transfer. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. In place.
19.1 effective FTE from 30 hired (64%)
- Escaped defects on transferred services. Gap or risk.
Rising
- 12-month attrition (fictional). Gap or risk.
24%
- Business outcome. In place.
Merchant reporting releases monthly instead of quarterly
- Total cost vs case. In place.
Within the fully loaded case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 27 engineers and about 15.8 effective FTE. Retention is strong and quality on transferred services is stable. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. Caution.
15.8 effective FTE from 27 hired (59%)
- Escaped defects on transferred services. Gap or risk.
Rising
- 12-month attrition (fictional). In place.
7%
- Business outcome. Caution.
No outcome measures were agreed
- Total cost vs case. Gap or risk.
Over budget: overlap and management costs were not in the case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 45 engineers and about 24.9 effective FTE. Retention is reasonable; one service still depends on the vendor. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. Caution.
24.9 effective FTE from 45 hired (55%)
- Escaped defects on transferred services. Gap or risk.
Rising
- 12-month attrition (fictional). In place.
11%
- Business outcome. Caution.
No outcome measures were agreed
- Total cost vs case. Gap or risk.
Over budget: overlap and management costs were not in the case
- Centre of excellence label. Context.
A name, not evidence of maturity
The centre has 60 engineers and about 34.5 effective FTE. Attrition is high and two services had incidents after transfer. Headquarters asks whether to double the centre.
Year one outcomes (fictional)
- Effective vs hired. Caution.
34.5 effective FTE from 60 hired (57%)
- Escaped defects on transferred services. Gap or risk.
Rising
- 12-month attrition (fictional). Gap or risk.
24%
- Business outcome. Caution.
No outcome measures were agreed
- Total cost vs case. Gap or risk.
Over budget: overlap and management costs were not in the case
- Centre of excellence label. Context.
A name, not evidence of maturity
What do you decide and how is the centre governed?
a · Scale conditionally under an ownership charter
Scale by one domain at a time, each with written decision rights, funding and intake, service acceptance, risk tiers for change, security and quality guardrails, succession plans and an independent challenge role retained at headquarters.
Protects: Growth tied to evidence, with clear ownership and an independent check.
Costs: Governance effort and slower growth than headquarters wants.
Conditions: When delivery, quality and retention evidence is positive and a next domain is ready to accept.
Earlier choices: AA, AB, BA, BB
A second domain is chartered with acceptance criteria. Headquarters keeps strategy and architecture challenge.Earlier choices: AC, BC, CA, CB, CC
The charter is sound, but the evidence does not support scaling yet. Service acceptance blocks the next domain until quality recovers.b · Hold and consolidate
Keep scope as is for two quarters, fix the weakest service, close remaining vendor dependencies and build succession for key roles.
Protects: Stability and quality before more growth.
Costs: Lost momentum. Strong engineers may leave if the centre looks stalled.
Conditions: When quality or retention evidence is mixed, or a key dependency on the vendor remains.
Earlier choices: Any earlier path
Defects fall and the last vendor dependency closes. The scale decision returns in two quarters with better evidence.c · Double the centre now because it is a centre of excellenceUnsafe
Approve doubling headcount on the strength of the year one story, without service acceptance or new decision rights.
Equating the label with maturity. Scaling without acceptance criteria or decision rights multiplies the existing weaknesses.
Protects: Momentum and a visible commitment.
Costs: Scales whatever is weak today, and leaves ownership unclear between headquarters and the centre.
Conditions: Rarely. Only if every outcome measure is strong and governance is already in place.
Earlier choices: Any earlier path
Headcount doubles. Two product heads and the centre lead both claim the same roadmap and nobody resolves it.Risk flags: label-not-maturity
Risk flag notes
- Salary-only business case (salary-only-case): Insourcing cost includes benefits, facilities, tools, management, recruitment, environments, incumbent overlap and ramp time, not just salary difference.
- Overbroad transfer (overbroad-transfer): Moving every service at once concentrates transition risk and outruns the hiring market.
- Exit before readiness (exit-before-ready): The incumbent should step back only when written exit criteria are met for each service.
- A label is not maturity (label-not-maturity): Calling a team a centre of excellence says nothing about its delivery, quality or ownership. Scale on evidence.
06 Governance and launch · version 1Regulatory engagement and launch readinessA fictional acquirer plans to launch tokenised wallet acceptance in a fictional regulated market. Work out which bodies actually need to approve what, keep evidence and conditions current, and respond to gaps without trading written approvals for a deadline.
Fictional market, regulator (the Authority), payment schemes (Scheme A and Scheme B), external assessor and acquirer. No real law, regulator, deadline or engagement is described. Requirements are illustrative.
Stage 1 · Who needs to say yes Week -8
The launch is eight weeks away. The regulator, two payment schemes, an external assessor and the internal risk committee all have a role, but not the same authority. Decide how you work out what applies.
Situation and evidence (1 variant)
The product uses an existing licence but adds a new outsourced token service. Scheme certification is needed for the new acceptance flow. Internal policy requires risk committee sign-off for new outsourcing.
Authorities and roles
- Written approvals. Gap or risk.
Where a written approval or certification is required, an informal conversation does not replace it.
- The Authority (regulator). Caution.
Notification of material outsourcing may be required; applicability to be confirmed by compliance
- Scheme A and Scheme B. Caution.
Certification of the new acceptance flow before live traffic
- External assessor. Context.
Independent assurance on card data controls; reports, does not approve
- Internal risk committee. Context.
Approves new outsourcing under internal policy
How do you determine the requirements?
a · Map requirements by authority with named owners
Compliance and legal confirm what applies. Each requirement gets an owner, the evidence needed and its dependency on others.
Protects: Engaging each body on what it actually decides.
Costs: A week of mapping before visible progress.
Conditions: When a launch changes outsourcing, data flows or acceptance methods.
Earlier choices: Any earlier path
The map shows regulator notification, two scheme certifications and committee approval, with the assessor report as an input to two of them.b · Reuse the last launch's checklist
The previous launch went well. Use its approvals list.
Protects: Speed and familiarity.
Costs: The last launch had no outsourcing change, so a notification and an approval may be missed.
Conditions: When the new launch has no material differences from the previous one.
Earlier choices: Any earlier path
The checklist omits outsourcing. Nobody owns the regulator notification.Risk flags: missed-requirement
c · Take everything to the regulator
Ask the Authority to review the whole launch, including scheme and internal matters.
Protects: A feeling of maximum caution.
Costs: Asks the regulator about matters it does not decide, and delays the ones it does.
Conditions: When genuine uncertainty about applicability cannot be resolved by compliance and legal.
Earlier choices: Any earlier path
The Authority replies that scheme certification and internal approvals are not its matters. Two weeks pass.Risk flags: authority-confused
Stage 2 · Gaps against a commercial deadline Week -4
Scheme A's certification test failed on one authentication edge case. A request for more information arrives on the outsourcing arrangement. Sales has announced the date.
Situation and evidence (3 variants)
Requirements: Mapped by authority with named owners. Both gaps have an owner.
Conditions and dependencies
- Written approvals. Gap or risk.
Where a written approval or certification is required, an informal conversation does not replace it.
- Scheme A certification. Gap or risk.
Failed: one authentication edge case
- Scheme B certification. In place.
Passed, written confirmation received
- Outsourcing information request. Caution.
Received; response owner named
- Assessor report. Caution.
Draft, two minor findings
- Requirements map. In place.
Mapped by authority with named owners
Requirements: Copied from the previous launch. The outsourcing notification was never sent, so the request for information is a surprise.
Conditions and dependencies
- Written approvals. Gap or risk.
Where a written approval or certification is required, an informal conversation does not replace it.
- Scheme A certification. Gap or risk.
Failed: one authentication edge case
- Scheme B certification. In place.
Passed, written confirmation received
- Outsourcing information request. Gap or risk.
Received; notification not yet filed
- Assessor report. Caution.
Draft, two minor findings
- Requirements map. Caution.
Copied from the previous launch
Requirements: Everything routed to the regulator. Both gaps have an owner.
Conditions and dependencies
- Written approvals. Gap or risk.
Where a written approval or certification is required, an informal conversation does not replace it.
- Scheme A certification. Gap or risk.
Failed: one authentication edge case
- Scheme B certification. In place.
Passed, written confirmation received
- Outsourcing information request. Caution.
Received; response owner named
- Assessor report. Caution.
Draft, two minor findings
- Requirements map. Caution.
Everything routed to the regulator
How do you respond?
a · Fix, re-test and respond in writing
Fix the edge case and book a re-test. Send a complete written response on outsourcing. Keep a conditions register.
Protects: Every required approval on paper.
Costs: The date may slip if the re-test slot is late.
Conditions: When fixes are known and re-test slots are available within the window.
Earlier choices: Any earlier path
Re-test is booked for week -2. The written response is filed with the assessor's report attached.b · Proceed on a positive call with a contactUnsafe
A contact at the Authority sounded comfortable on a call. Treat that as clearance and launch on the date.
Informal agreement never replaces a required written approval.
Protects: The announced date.
Costs: No written outcome. The scheme failure is still open.
Conditions: Not defensible. An informal conversation does not replace a required written response or certification.
Earlier choices: Any earlier path
Legal blocks the plan. The written response is drafted in a hurry a week later.Risk flags: informal-approval
c · Descope the failing flow with written scheme confirmation
Launch without the edge-case flow, after Scheme A confirms in writing that the reduced scope is certified. Respond to the Authority in writing.
Protects: The date for most of the product.
Costs: A smaller launch and a second release later.
Conditions: When the descoped flow is not essential and the scheme confirms the reduced scope in writing.
Earlier choices: Any earlier path
Scheme A confirms the reduced scope in writing. The edge-case flow moves to a later release.
Stage 3 · Launch readiness call Week -1
Two weeks out. Decide whether to launch, under what conditions, and who holds them.
Situation and evidence (3 variants)
The re-test passed. The outsourcing response is filed and acknowledged.
Readiness register
- Written approvals. Gap or risk.
Where a written approval or certification is required, an informal conversation does not replace it.
- Scheme A. In place.
Certified after re-test
- Outsourcing response. In place.
Filed and acknowledged
- Risk committee. In place.
Approved, with two conditions
- Assessor findings. Caution.
Two minor, fixes scheduled
The written response was filed late. Its outcome is pending.
Readiness register
- Written approvals. Gap or risk.
Where a written approval or certification is required, an informal conversation does not replace it.
- Scheme A. Gap or risk.
Not certified
- Outsourcing response. Gap or risk.
Filed late; outcome pending
- Risk committee. In place.
Approved, with two conditions
- Assessor findings. Caution.
Two minor, fixes scheduled
Reduced scope is certified. The outsourcing response is filed.
Readiness register
- Written approvals. Gap or risk.
Where a written approval or certification is required, an informal conversation does not replace it.
- Scheme A. In place.
Reduced scope certified
- Outsourcing response. In place.
Filed and acknowledged
- Risk committee. In place.
Approved, with two conditions
- Assessor findings. Caution.
Two minor, fixes scheduled
What is the launch call?
a · Launch with a tracked conditions register
Launch when every required written approval is in place. Conditions and assessor findings are owned and reported to the risk committee.
Protects: A launch with clear evidence and owned conditions.
Costs: Ongoing reporting overhead.
Conditions: When every required approval is in writing and remaining items are conditions, not missing approvals.
Earlier choices: AA, AC, BA, BC, CA, CC
Launch proceeds. Conditions are reviewed monthly until closed.Earlier choices: AB, BB, CB
Scheme A is not certified, so the register shows a missing approval. Launch waits.b · Slip the date
Move the launch by four weeks to close every finding first.
Protects: A launch with nothing outstanding.
Costs: Commercial credibility after the public announcement.
Conditions: When a required approval is missing or a finding affects card data or funds.
Earlier choices: Any earlier path
Launch moves to week +4. Sales manages the change with key merchants.c · Soft launch to pilot merchants before approvals landUnsafe
Process live payments for a few friendly merchants while approvals are pending.
A small live launch is still a live launch. It needs the same required approvals.
Protects: Early operating experience.
Costs: Live transactions without the required approvals, however few.
Conditions: Only when no approval is required for the pilot scope, confirmed in writing.
Earlier choices: AA, AC, BA, BC, CA, CC
Compliance confirms the pilot scope needs the same approvals. It starts after they land.Earlier choices: AB, BB, CB
Compliance stops the pilot before it starts.Risk flags: pilot-without-approval
Risk flag notes
- Missed requirement (missed-requirement): A copied checklist misses what changed. Map requirements from the launch's actual differences.
- Authority confused (authority-confused): Regulator, schemes, assessors and internal governance decide different things. Engage each on its own matters.
- Informal approval (informal-approval): A positive conversation is not a written outcome. Required approvals must be in writing.
- Pilot is still live (pilot-without-approval): Small scope does not change whether an approval is required.
07 Distributed delivery effectiveness · version 3Improve an existing offshore engineering centreA fictional engineering centre of 60 people has stable headcount but slow, unpredictable delivery. Diagnose where time is really lost, fix decision rights across headquarters and regions, and integrate remote teams as owners rather than a ticket queue.
Fictional group with six squads (60 staff) in an engineering centre, product stakeholders at headquarters and a regional office in three time zones. Figures are illustrative assumptions, not benchmarks.
Stage 1 · Diagnose the flow Weeks 1 to 4
Headquarters says the centre is slow. Before changing anything, decide how to find out where the time goes. The cause may be waiting on decisions and handoffs, not the engineers.
Situation and evidence (1 variant)
Three headquarters product heads and the regional office send work directly to squads. Releases slip, and escaped defects are rising. The sponsor wants an early reversible improvement while the team establishes where work waits.
What is visible today
- Headcount. Context.
60, stable for 18 months
- Squad utilisation. Context.
92% reported
- Demand sources. Gap or risk.
5 stakeholders sending work directly, no shared priority
- Work in progress. Caution.
About 3 items per engineer
- Release predictability. Caution.
4 of the last 10 releases on the planned date
- Wait and queue time. Caution.
Not measured
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
How do you diagnose the problem?
a · Baseline flow, quality and experience
Measure lead and cycle time, time waiting and blocked, WIP, rework, escaped defects, predictability and a short team experience survey, using existing tool data at team level.
Protects: An honest picture of where time goes, including stakeholder decision latency.
Costs: Four weeks before visible action. Some headquarters stakeholders will see their own delays measured.
Conditions: Almost always the right start, unless an outage needs immediate action first.
Earlier choices: Any earlier path
The baseline shows a typical change spends 12 of 34 days waiting for stakeholder decisions and only 6 days in active work.b · Compare squads on utilisation and story pointsUnsafe
Rank squads by utilisation, story points delivered and commits, and review the lowest performers.
Utilisation, story points, commits and lines of code do not measure delivery or outcomes, and they push blame onto engineers.
Protects: A quick, familiar report.
Costs: Story points are not comparable across teams. Utilisation hides queues. Teams learn to game the numbers.
Conditions: Not a sound diagnosis. Activity counts can be context, never the measure of effectiveness.
Earlier choices: Any earlier path
The two lowest-ranked squads are put on improvement plans. Both are the squads with the most headquarters dependencies.Risk flags: vanity-metrics
c · Test the suspected bottleneck with a reversible change
Use existing evidence to select one suspected delay, run a bounded reversible experiment, and measure flow while it runs.
Protects: Early progress and a limited commitment that can be reversed.
Costs: The suspected delay may not be the largest. Without a baseline, improvement cannot be attributed confidently to the change.
Conditions: When the suspected cause is already well evidenced, the sponsor needs early results and the change is easy to reverse.
Earlier choices: Any earlier path
A reversible experiment starts on the suspected handoff delay. Flow measurement begins alongside it, so the team cannot establish a credible before-and-after improvement.Risk flags: experiment-without-baseline
Stage 2 · Fix decision rights and demand Months 2 to 3
Work arrives from many stakeholders with no shared priority, and decisions wait in inboxes. Decide how demand, capacity, decisions and risk approval should work across headquarters and regions.
Situation and evidence (3 variants)
The baseline is accepted. The largest delay is stakeholder decisions, then dependencies and the approval queue. Two product heads at headquarters want the same squad for conflicting priorities.
Demand and decisions
- Active work. In place.
6 days
- Waiting for stakeholder decisions. Gap or risk.
12 days
- Blocked on other teams. Gap or risk.
7 days
- Approval queue. Caution.
5 days
- Rework after review or defects. Caution.
4 days
- Lead time. Gap or risk.
34 days, 18% of it active work
- Conflicting HQ priorities. Gap or risk.
Two product heads claim squad 3 for the same quarter
- Decision SLA. Caution.
None. Questions wait for the next weekly call
- Change approval. Caution.
Every change goes through the same review, whatever its risk
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
Without a flow baseline, headquarters still believes the centre is the bottleneck. Two product heads at headquarters want the same squad for conflicting priorities.
Demand and decisions
- Wait and queue time. Caution.
Unknown. Not measured on the current dashboard
- Conflicting HQ priorities. Gap or risk.
Two product heads claim squad 3 for the same quarter
- Decision SLA. Caution.
None. Questions wait for the next weekly call
- Change approval. Caution.
Every change goes through the same review, whatever its risk
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
The reversible experiment is running and measurement has started, but there is no pre-change flow baseline. Two product heads at headquarters want the same squad for conflicting priorities.
Demand and decisions
- Wait and queue time. Caution.
Unknown. Not measured on the current dashboard
- Experiment measurement. Caution.
Measurement started with the experiment; no pre-change baseline exists
- Conflicting HQ priorities. Gap or risk.
Two product heads claim squad 3 for the same quarter
- Decision SLA. Caution.
None. Questions wait for the next weekly call
- Change approval. Caution.
Every change goes through the same review, whatever its risk
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
How do you govern demand and decisions?
a · One accountable product owner per domain, decision SLAs and risk tiers
Name a single product owner per domain, publish capacity allocation across headquarters and regions, set decision and escalation SLAs, a clear RACI, and thin risk-tiered approval with independent assurance for high-risk change.
Protects: Fast decisions with a named owner, and independent assurance where risk warrants.
Costs: Product heads give up direct access to squads. Conflicts must be resolved openly by the sponsor.
Conditions: When a sponsor is willing to arbitrate conflicting priorities and product owners have real authority.
Earlier choices: A
The squad 3 conflict is escalated and resolved by the sponsor in two days. Decision wait drops from 12 to about 5 days.Earlier choices: B, C
The model is sound, but without a baseline nobody can show it worked. Headquarters remains sceptical.b · A gate committee for every taskUnsafe
Create a weekly cross-location committee that approves every piece of work and every change.
A gate for every task slows all work without improving control where risk is real.
Protects: Everyone feels consulted.
Costs: Adds a queue to every item, low-risk changes wait as long as high-risk ones, and accountability is diluted across the committee.
Conditions: Not defensible as the standard path. A committee is only appropriate for a small number of high-risk decisions.
Earlier choices: Any earlier path
Approval queue rises from 5 to 9 days. Low-risk fixes wait behind major changes.Risk flags: gate-everything
c · Regional product owners, each with a fixed capacity share
Each region names a product owner and receives a fixed capacity share. The CIO arbitrates only conflicts that cross regions.
Protects: Fast local decisions and clear regional ownership.
Costs: Shared domains split across regions, dependencies multiply and spare regional capacity cannot automatically serve another region's priority.
Conditions: When regions map to separate business units with few shared systems and cross-region conflicts are rare.
Earlier choices: A
Regional decisions are faster: illustrative decision wait falls from 12 to 6 days, but cross-region dependency waits rise from 7 to 10 days. Fixed shares leave some capacity unused while another region queues work.Earlier choices: B, C
Local decisions appear faster, but shared work waits across regional boundaries. The model assumes 6 days of decision wait and 10 days blocked on dependencies; without a baseline these are illustrations, not measured improvement.Risk flags: regional-fragmentation
Stage 3 · Integrate remote teams Months 4 to 9
Decide how the centre works with headquarters and regional colleagues day to day: as owners in cross-location product squads, or as a team that receives tickets. Pilot first and measure before and after.
Situation and evidence (6 variants)
Decision flow is clearer. Engineers at the centre still receive tickets with little context and rarely meet the people who use the system.
Time zones and integration
- Active work. In place.
6 days
- Waiting for stakeholder decisions. Caution.
5 days
- Blocked on other teams. Gap or risk.
7 days
- Approval queue. Context.
3 days
- Rework after review or defects. Caution.
4 days
- Lead time. Gap or risk.
25 days, 24% of it active work
- Three-location overlap. Caution.
4.5 h a day when all three are in normal hours
- Centre and HQ overlap. In place.
7.5 h a day
- Meetings outside hours. Gap or risk.
Centre engineers carry most late calls today
- Access and tool parity. Caution.
Centre lacks production read access and some monitoring tools
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
Work is still queuing at the gate committee. Engineers at the centre receive tickets with little context and rarely meet the people who use the system.
Time zones and integration
- Active work. In place.
6 days
- Waiting for stakeholder decisions. Gap or risk.
10 days
- Blocked on other teams. Gap or risk.
7 days
- Approval queue. Gap or risk.
9 days
- Rework after review or defects. Caution.
4 days
- Lead time. Gap or risk.
36 days, 17% of it active work
- Three-location overlap. Caution.
4.5 h a day when all three are in normal hours
- Centre and HQ overlap. In place.
7.5 h a day
- Meetings outside hours. Gap or risk.
Centre engineers carry most late calls today
- Access and tool parity. Caution.
Centre lacks production read access and some monitoring tools
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
Regional owners make local decisions faster, but shared work crosses fixed capacity boundaries. Engineers at the centre still receive tickets with little context.
Time zones and integration
- Active work. In place.
6 days
- Waiting for stakeholder decisions. Caution.
6 days
- Blocked on other teams. Gap or risk.
10 days
- Approval queue. Caution.
5 days
- Rework after review or defects. Caution.
4 days
- Lead time. Gap or risk.
31 days, 19% of it active work
- Three-location overlap. Caution.
4.5 h a day when all three are in normal hours
- Centre and HQ overlap. In place.
7.5 h a day
- Meetings outside hours. Gap or risk.
Centre engineers carry most late calls today
- Access and tool parity. Caution.
Centre lacks production read access and some monitoring tools
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
Decision flow is clearer. Engineers at the centre still receive tickets with little context and rarely meet the people who use the system.
Time zones and integration
- Wait and queue time. Caution.
Unknown. Not measured on the current dashboard
- Three-location overlap. Caution.
4.5 h a day when all three are in normal hours
- Centre and HQ overlap. In place.
7.5 h a day
- Meetings outside hours. Gap or risk.
Centre engineers carry most late calls today
- Access and tool parity. Caution.
Centre lacks production read access and some monitoring tools
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
Work is still queuing at the gate committee. Engineers at the centre receive tickets with little context and rarely meet the people who use the system.
Time zones and integration
- Wait and queue time. Caution.
Unknown. Not measured on the current dashboard
- Three-location overlap. Caution.
4.5 h a day when all three are in normal hours
- Centre and HQ overlap. In place.
7.5 h a day
- Meetings outside hours. Gap or risk.
Centre engineers carry most late calls today
- Access and tool parity. Caution.
Centre lacks production read access and some monitoring tools
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
Regional owners make local decisions faster, but shared work crosses fixed capacity boundaries. Engineers at the centre still receive tickets with little context.
Time zones and integration
- Wait and queue time. Caution.
Unknown. Not measured on the current dashboard
- Three-location overlap. Caution.
4.5 h a day when all three are in normal hours
- Centre and HQ overlap. In place.
7.5 h a day
- Meetings outside hours. Gap or risk.
Centre engineers carry most late calls today
- Access and tool parity. Caution.
Centre lacks production read access and some monitoring tools
- Illustrative data. Context.
Fictional figures for one typical change. Flow measures describe delivery, not business outcomes.
How do you integrate the remote teams?
a · Integrated cross-location product squads, piloted
Pilot two squads with members across locations: onboarding and buddies, domain exposure visits, a planned overlap window with late slots rotated fairly, async decision records and runbooks, paired delivery, a shared definition of done, access parity and on-call with clear handovers.
Protects: Ownership and context at the centre, sustainable hours and measured results.
Costs: Travel and onboarding effort, and some headquarters engineers must change how they work.
Conditions: When work is product-shaped and leaders accept a phased pilot with before and after measures.
Earlier choices: AA
The pilot squads reduce blocked time and rework. Illustrative lead time is 20.5 days against a 34 day baseline. The before and after comparison is credible because the baseline exists. No productivity figure is promised for the rest.Earlier choices: AB
The pilot squads reduce blocked time and rework. Illustrative lead time is 31.5 days against a 34 day baseline. The before and after comparison is credible because the baseline exists. No productivity figure is promised for the rest.Earlier choices: AC
The pilot squads reduce blocked time and rework. Illustrative lead time is 26.5 days against a 34 day baseline. The before and after comparison is credible because the baseline exists. No productivity figure is promised for the rest.Earlier choices: BA, CA
The pilot squads reduce blocked time and rework. Illustrative lead time is 20.5 days. No baseline was taken, so there is nothing credible to compare it with. No productivity figure is promised for the rest.Earlier choices: BB, CB
The pilot squads reduce blocked time and rework. Illustrative lead time is 31.5 days. No baseline was taken, so there is nothing credible to compare it with. No productivity figure is promised for the rest.Earlier choices: BC, CC
The pilot squads reduce blocked time and rework. Illustrative lead time is 26.5 days. No baseline was taken, so there is nothing credible to compare it with. No productivity figure is promised for the rest.b · Keep the centre as a ticket queue with tighter specifications
Headquarters writes more detailed tickets; the centre implements them and hands back for review.
Protects: A familiar model and clear task boundaries.
Costs: More handoffs, less context, more rework and little ownership at the centre.
Conditions: For short, well-defined work with little domain knowledge needed.
Earlier choices: AA, BA, CA
Rework rises and blocked time grows. Illustrative lead time is 27 days. Retention at the centre falls.Earlier choices: AB, BB, CB
Rework rises and blocked time grows. Illustrative lead time is 38 days. Retention at the centre falls.Earlier choices: AC, BC, CC
Rework rises and blocked time grows. Illustrative lead time is 33 days. Retention at the centre falls.Risk flags: ticket-factory
c · Require every engineer to be available around the clockUnsafe
Ask all centre engineers to stay reachable across all three time zones so nobody waits.
Time zone coverage should come from rotas and handovers, not from every engineer being available 24/7.
Protects: Short waits for answers in the short term.
Costs: Burnout, attrition and errors. It is not real coverage.
Conditions: Not defensible. Use a planned on-call rota with handovers for real out-of-hours needs.
Earlier choices: Any earlier path
Response times improve for a month, then three engineers leave and on-call incidents rise.Risk flags: always-on
Risk flag notes
- Activity metrics (vanity-metrics): Utilisation, story points, commits and lines of code are not delivery measures and invite gaming. Monitoring individuals also damages trust without diagnosing where work waits.
- Experiment without a baseline (experiment-without-baseline): A reversible experiment can start learning early, but measurement started alongside it cannot establish a credible before-and-after improvement.
- Gate for every task (gate-everything): Risk-tiered approval keeps control where risk is real without queuing every change.
- Regional capacity boundaries (regional-fragmentation): Fixed regional shares speed local decisions but split shared domains, increase dependencies and can leave capacity unused while another region waits.
- Ticket factory (ticket-factory): A remote team that only receives tickets has no context and no ownership; quality and retention suffer.
- Always-on is not coverage (always-on): Sustainable coverage comes from rotas, overlap windows and handovers, not from everyone being reachable.