Hansel
← The village

No. 03 · The observatory

Methods

What each score and claim in the Legacy Software Atlas can and cannot say.

Status
Open
Scored
2026-10-05
Reported
2026-10-09
Sections
definitions, scoring, workflows, evidence, limitations
The observatory in the gingerbread village

How to read

This appendix explains how the study was done and how much to trust it.

Sections cover the terms, the steps, the map, the factors and the gaps.

Each page of the site links back to the evidence it rests on.

The terms

Legacy software is a pre-2005 core that is slow or costly to replace.

A category is one job software does for a buyer, like core banking or payroll.

A core system is the system of record holding the official data.

A workflow is a chain of tasks one person can run end to end.

The study lists six workflows per category, 210 in all.

An AI-native challenger is a company coded as built around AI from the start.

Legacy spend is what buyers pay yearly for systems still legacy.

Buyable spend is the part held by segments that normally buy software.

ERP runs finance, orders, inventory and production in one system.

Critical and important

A category is critical when failure stops the business and the firm competes on it.

It is important when only the first is true.

Twenty-two categories are critical and thirteen important.

Getting on the map

The study seeded 38 candidate categories and added eight Asia-Pacific checks.

Four cited signs mark a legacy category: pre-cloud core, slow replacement, workarounds, shrinking skills.

Admission needs one strong sign, and strong plus half the weak signs at least 2.5.

Thirty-five of the 46 candidates passed; eleven were dropped.

Each kept category names three incumbents and one challenger as examples.

Those are selected examples, not a census or the largest three vendors.

How inputs score

Pain, lock-in and crowding inputs rank categories into fifths.

The bottom fifth scores 1 and the top fifth scores 5.

For app rating the order flips: a lower rating scores higher.

Size instead bins the spend midpoint into fixed bands.

AI fit averages feasibility over twelve weighted tasks.

The workaround sign and regulator answer score directly.

Pain, lock-in and crowding average their input scores to one decimal.

A median is the middle value of a sorted list.

The method

Step 1 kept categories showing at least one strong cited legacy sign.

Thirty-five of 46 candidates passed; a fact-check held 232 of 256 citations.

Step 2 sized each category by legacy spend, using the geometric-mean midpoint.

Step 3 scored five factors; step 6 scored 210 workflows inside them.

Steps 4 and 5 mapped challengers, wrote cases and drew the playbook.

The formula

The total is 35 plus 5 times the signed factor sum.

Utility billing scored 35 plus 5 times (3 + 3.6 + 4 - 2.0 - 1.7), or 69.5.

All five factors weigh equally, and so do inputs inside a factor.

Totals move in half points because three factors keep one decimal.

The factors

Pain averages keep-alive awards, failure reviews, ratings, failed replacements and workarounds.

Lock-in averages contract length, non-competitive share, replacement time and regulator tests.

Crowding averages AI-native traction, funding per billion and YC companies.

A missing input leaves the average; that differs from an observed zero.

Size

Global spend times a judged legacy share sets each range.

The geometric mean of its bounds gives the midpoint.

Utility billing works out to the root of 1.5 times 4.2, about 2.5.

Bins run under $0.1B, under $1B, under $5B, under $20B, else 5.

Fourteen categories score 4, eighteen score 3, three score 2, none 5.

Buyer counts allocate spend across segments; they do not set it.

Pain

Five inputs score pain: keep-alive awards, failure reviews, inverted rating, failed replacements, workarounds.

Keep-alive needs ten relevant awards and reaches it in 13 categories.

Backend-failure needs 50 reviews and reaches it in 18.

The inverted rating needs three apps and reaches it in 30.

Failed or overrun replacements need three cases and reach it in all 35.

The workaround sign has no analogous sample threshold; all 35 carry it.

Fifteen categories grade on three inputs or fewer; POS uses two.

Utility billing averages (4 + 4 + 4 + 3 + 3) over five, or 3.6.

Licensing scores 4.4; trade finance and mortgage score 1.7 on thinner evidence.

Lock-in

Four inputs average: contract median, non-competitive share, replacement median, regulator answer.

Minimum observations are ten awards, ten awards and three replacements.

The ledger collected 1,922 awards and labelled 1,919 of them.

423 awards record a maintain, extend or replace decision.

After attribution, 281 carry durations and 272 carry procedures.

All 35 categories hold at least three replacement durations, 228 in total.

Twenty-five categories score lock-in on just two inputs.

Student information averages (4 + 5 + 4 + 1) over four, or 3.5.

AI fit

Twelve core tasks per category make 420 task scores in total.

Each task scores feasibility 0 to 4, shifted up one.

Scores weight by time share at 3, 2 and 1, then round.

Twenty-four categories score 4, ten score 3, warehouse management 2.

The mean is 2.499, an ordinal ranking, not a percentage automated.

Tax scores 3.46 and card payments 3.52; the rounding cliff separates them.

Crowding

Three inputs rank: AI-native traction rows, funding per billion and YC companies.

Traction means named customers or reported revenue, not a marketing claim.

Ninety-two rows carry traction; six categories show none at all.

The YC input maps 316 companies out of 1,939 labelled.

Nine categories score 4 or more on every crowding input.

Telecom averages (3 + 1 + 1) over three, or 1.7.

Its funding input is $40M over $18.6B, about $2.1M per billion.

Sensitivity

Without the workaround input, manufacturing and utility tie at 70.5.

Without pain at all, mainframe leads the map.

Without the regulator input, utility lock-in rises to 2.3 and fails.

Under that same cut telecom leads at 74.

Without the YC input, the crowding leader changes.

Five borderline regulator calls can move totals by ten points.

The map

Nine categories sit top tier at 62.5 or above, sixteen mid, ten bottom.

Ties order by legacy midpoint.

Penalty factors correlate -0.78 with totals; positive factors 0.45.

Crowding sorts the map most and size sorts it least.

The rank order shifted with method versions, correlation 0.28.

The regulator rule

One written rule codes all 35 categories on regulator approval.

It scores yes when a US, EU or UK regulator must approve.

Advance notice to a regulator also scores yes.

Cost reviews, general audits and data-protection notices score no.

Seventeen categories answer yes and eighteen no; five calls sit borderline.

Rescoring with the rule moved airline reservations and government tax a tier.

Rounds and scopes

The rounds list holds 298 rows: 284 category-specific, 8 company-wide, 6 exits.

Only the 284 count toward funding per billion, in money and round count.

The company-wide raises total $2.0B; exits carry no disclosed amounts.

Category-specific rounds total $9.0B of the $11.0B disclosed across all rows.

Labelling and audit

Bulk labelling was done by Jev, a non-generative classifier sold by TypeSafe.

All Jev labelling cost $0.67 over 5,023 calls, inside an $8 cap.

This round labelled awards, app reviews and YC lists for $0.26 over 1,704 calls.

Blind audits used separate model labels, not human labels, with an 80 percent bar.

Award category agreement ran 92 percent.

App failure agreement reached 87 percent at the used cut.

Jev rated tasks about half a point more automatable than the blind labels did.

AI fit therefore ranks categories; it is never an absolute claim.

Mainframe and COBOL

Mainframe scores 69, half a point below utility billing.

It misses only on lock-in, at 2.5 against the 2.0 line.

Its $14.5B of legacy spend yields just $0.6B buyable.

Most buyers own their code, so the money goes to platforms and integrators.

A better opening needs faster documented replacement lowering lock-in.

Low competition must also survive fuller counting.

Large enterprises buying migration outcomes would make much of $13.8B buyable.

That changes the build-versus-buy view, not the score or rank.

These are conditional inferences, not findings that any condition holds.

Workflow openings

The category search found one narrow pass; the workflow search finds more.

Six workflows per category are ranked on labour, AI usage, pain and crowding.

Workflows that touch the system of record carry a large penalty.

How workflows are scored

Labour value multiplies US employment, mean wage and time share per occupation.

Employment and wages come from the US occupational survey; tasks come from O*NET.

Unmatched occupation codes fall back through broader occupation groups.

AI usage averages Claude task penetration across the workflow’s tasks.

Crowding counts AI-native rounds mapped to that workflow.

Pain borrows the category score; touches-core is a research judgment.

Fifty-one of 210 workflows record zero usage; a recorded zero is not missing.

The opportunity index

Each workflow ranks against the other 209 on labour, usage and pain.

It adds those three ranks and subtracts the crowding rank.

A missing value ranks zero; absent usage borrows the category AI fit.

Touching the core adds a penalty equal to the pre-penalty base-index spread.

The index runs from -644 to 498 and ranks, so it has no pass mark.

The stricter cut

Fifty-five non-core workflows show no mapped round, exits excluded.

Counting exits in, the zero-round non-core count is 53.

Sixteen pass the strict cut; fourteen do when exits count.

The cut needs category pain 3 or more and usage at median 0.18 or above.

Company-wide rounds stay mapped; removing them also frees telecom customer care.

Five of the strict sixteen are collections and arrears work.

Twenty-one crowded workflows hold 40 percent of all mapped rounds.

Reading the leads

Ninety-six workflows have no mapped AI-native round.

Missing mapped funding records coverage, not the absence of competition.

Labour value sums to $6.3T because whole occupations count many times.

Taxpayer contact alone lists $85.1B, roughly 1.8M FTE of customer-service time.

Nearly all is 70% of working time for every US customer-service representative.

That spans every industry; tax examiners add only a small share.

Those are labour pools, not software market sizes.

One lead is 3270 mainframe screen transaction entry, flagged non-core.

Entering transactions on those screens writes to the core, so the flag is generous.

The leads point at work beside the core, not at proven demand.

Evidence status

All 35 categories carry accepted provider and pain records.

Provider records hold 105 category-company incumbent slots.

The ledger holds 864 accepted receipts out of 869 checked.

Three excluded and two disputed receipts remain recorded for history.

The snapshot dates are 2026-10-05 for scoring and 2026-10-09 for the report.

What acceptance means

Evidence acceptance is structural: receipted, typed and bound to recorded sources.

A source date is a record, not a live check of the source.

Validation proves the recorded bindings, not that a source still says the same.

What the counts rest on

The measures rest on 1,922 collected public awards and 2,718 app reviews.

They also use 438 apps, 353 documented replacements and 298 funding rows.

A further 1,939 Y Combinator companies were labelled for the crowding input.

The 353 replacements split: 116 late or over, 42 cancelled, 90 on time, 105 ongoing.

The ongoing share keeps that ledger from reading as a failure rate.

App symptoms reflect end customers and cannot prove core causality.

A keep-alive award can mark a working system, not only pain.

Limitations

This page collects the gaps the study could not close.

Each limit sits with the finding it qualifies.

Measurement limits

No direct buyer-pain survey exists anywhere in the study.

Fifteen categories are graded on three pain inputs or fewer.

Keep-alive and backend-failure inputs reach their minimum in 13 and 18 categories.

Of 1,922 collected awards, 1,457 come from EU sources.

App-review inputs measure end customers, not the paying buyer.

Eighteen of 35 spending ranges cross a size-bin line.

The legacy share inside each range is a judgment call.

Twenty-five of 35 lock-in scores rest on two inputs each.

Five regulator calls sit borderline; one answer moves a total by ten.

Switching rates are still not measured anywhere.

Contract-length and non-competitive inputs hit their floor in 8 and 9 categories.

Coverage limits

The failure ledger undercounts: only 19 of 436 rows are stalled or failed.

Known failures including we.trade and Marco Polo sit outside that table.

Crowding excludes non-AI modern competitors and incumbent AI features.

Twenty-four of the 284 counted rounds carry no disclosed amount.

The company-search display parser is broken and remains unfixed.

The regional split follows buyer counts, not spending.

India’s $11.9B hospital-records segment rests on facility counts.

A 369-record regional pass remains unincorporated in the published split.

Several job and review sources stayed unreachable.

The replacement ledger favours famous projects and lists some twice.

A gap in the study is not necessarily a gap in the market.

Labelling limits

AI-fit labels were produced and audited by models, not humans.

Two labelling choices were made after seeing an audit.

App and YC agreement figures overweight positive labels.

Jev scores about half a point high on task order, not category order.

Award-category agreement ran 92% on a 60-item audit sample.

Action agreement ran 70% on five choices and 83% once collapsed.

No human-label check is scheduled.

Workflow limits

Labour values count whole occupations; they rank work, not markets.

Workflow pain borrows the category score, and touches-core is a judgment.

A zero AI-usage score records no Claude usage, not no AI use.

Workflow crowding still counts every mapped round.

Playbook limits

Eight cases cannot rank strategies causally or give success probabilities.

Company figures are unaudited unless stated, and some steps are inferred.

All five factors weigh equally; that is a choice, not a measurement.

Strategy outcomes are observed patterns, not measured effects.

The playbook’s thresholds are decision guides, not forecasts.

Row counts are not company counts: 436 rows carry 423 distinct names.

Age and category explain much of the outcome variation.

Exit-type splits and every test line are judgment calls.

Scores are relative to this 35-category map, not absolute.