Core Engineering Metrics

Data Rules49 rulesdeviations: 2

re-verify the laws live → Verify

Transparency as a mechanism: this is the LIVE registry of every transformation between the raw Jira/ADO/BambooHR data and a number on screen — every rule, filter and fix; nothing happens silently. Each rule shows what controls it (fixed in code / a calculation policy / a tunable value / an owner tool) and its current per-project state. A deviation from the document standard glows red here, badges Metrics & Rules, and Verify recomputes the conservation laws on every load. If a number “doesn't eyeball-match Jira”, the cause is almost certainly one of these rows.

DOCa requirement of the source document/meeting — the spec says soFIXa correctness fix — without it the numbers would simply be technically wrongHONESTYan honesty rule — it discloses a gap instead of papering over itOWNERan owner tool — manual, reversible, always disclosedⓘ — hover the icon on any rule: a plain-words explanation + an example

Ingest (what changes during sync)

What happens at the moment data is downloaded from Jira / Azure DevOps / BambooHR into the local database. These rules fix the raw material (locales, API paging, stuck records) — before any math.

FIXAccept-Language: en on every Jira request — status/type names don't depend on the account locale (the “Chinese statuses” incident).ialways · in code
FIXStatus category is resolved by status ID from the changelog (name is only a fallback): Jira names are not unique.ialways · in code
FIXTicket upsert heals issueType, key and projectKey — moves/renames/locale poisoning don't freeze in.ialways · in code
DOCPR↔ticket link: jira_keys extracted by regex from PR title + source branch, uppercased.ialways · in code
FIXPR commits are fetched in a continuation loop ($top=500 per page, until exhausted) — the earliest commit survives for a PR of ANY size.ialways · in code
FIXBuilds and release deployments are fetched in continuation loops (builds used to be cut at 200).ialways · in code
HONESTYA deployment without completedOn/status is skipped — an unfinished run is not an event.ialways · in code
HONESTYDeployments upsert with onConflictDoNothing — a re-sync can never wipe manual failure flags.ialways · in code
FIXDisabled (isDisabled) ADO repos are skipped — their PR API errors out.ialways · in code
FIXEmployee country: full name → ISO-2 via dictionary; a holiday's country comes from its name prefix (“US …”).ialways · in code
FIXsync_runs rows stuck “running” for >2h are closed as error “interrupted”.ialways · in code

Cohort (THE single entry for ticket metrics)

The cohort = which tickets count as “done in the window” at all. Every ticket metric (Throughput, Lead, Cycle, TTFP…) reads exactly this one list — that's why the counts always reconcile across pages.

DOCdone_at = the LAST entry into a done status inside the window; the current status must still be done (re-opened tickets drop out).idoneStatusesADS: Done, Resolved, ClosedEVO: Done, Resolved, Closed
DOCAll windows are half-open [from, to) — no double counting on boundaries.ialways · in code
DOCExcluded issue types (Epics) never enter any metric.iexcludedIssueTypesADS: EpicEVO: Epic
OWNERManual exclusions — ONE shared mask: a ticket, an epic/feature WITH all descendants, an ADO repo (its PRs leave PR/TTFP/LtC) or a pipeline (its runs leave DF/CFR); disclosed by the “✕ N” chip and in Data quality.iowner toolcurrently excluded: 0
DOCwho-filter: assignee ∈ AI-agent roster / not; deployments, PRs, bugs and IMC incidents are who-blind.ialways · in code

Lead / Cycle / Active

The time metrics: how long a ticket lived (Lead — created→done), how long the work took (Cycle — started→done), and how much of it was hands-on work (Active — minus waiting, weekends, vacations).

DOCLead = created → done_at in calendar days (weekends included — the document standard).iincludeNonWorkingDaysADS: ONEVO: ON
DOCCycle start = the FIRST transition into the In-Progress category (≤ done_at); never started → cycle = null, not 0. Empty list = the canonical category rule.icycleStartStatusesADS: EVO:
DOCTime after a bounce back into a “new”-category status is SUBTRACTED from Cycle (the meeting's own “bug to fix”).isubtractBounceBackADS: ONEVO: ON
FIXCycle/TTFP are clamped at Math.max(0, …).ialways · in code
DOCActive: non-passive statuses only, weekdays only, minus vacations (BambooHR) and country holidays, capped per day; SHOWN in a selectable unit (calendar ÷24 · working days ÷8/9 · hours), chart always calendar.ipassiveStatusesADS: On Hold, Blocked, Failed on QA, Ready To Test, Ready To MergeEVO: On Hold, Blocked, Failed on QA, Ready To Test, Ready To MergeworkingHoursPerDayADS: 8EVO: 8
HONESTYActive for people not linked to BambooHR is computed without vacations (a “no HR” badge discloses it).ialways · in code

TTFP / LtC / PR

The code-side metrics: how fast the first pull request appears (TTFP), how long code travels to production (Lead Time for Changes), and PR throughput/acceptance.

DOCTTFP = the first PR opened AT/AFTER work start (spec M5); bounce-back time is subtracted (“same rules as Cycle”).ialways · in code
DOCTickets whose ONLY PRs predate the start → a separate disclosed prBeforeStart bucket (not 0 and not “no PR”); bucket sums = cohort (a law).ittfpAnomalyBucketADS: ONEVO: ON
DOCLtC = first commit → the FIRST successful prod deploy AFTER merge; changes not yet in prod are honestly not counted.ialways · in code
DOCPR metrics: only terminal (completed/abandoned) PRs closed in the window; acceptance = merged/(merged+abandoned).ialways · in code
DOCAgent PR authorship: author_id ∈ the roster (adoUserId).ialways · in code

Deployments (DF / CFR)

The DORA deployment metrics: how often we ship to production (Deployment Frequency) and what share of those shipments breaks (Change Failure Rate).

FIXEnvironment tier by name: LOWER tokens are checked FIRST (substring guard: “pre-production” is not prod); unrecognized → unknown, kept out of prod and disclosed as a count.ipreProdEnvNamesADS: pre-prod, preprod, pre-production, staging, stage, canary, alpha, beta, qa, test, dev, sandbox, uat, featureEVO: pre-prod, preprod, pre-production, staging, stage, canary, alpha, beta, qa, test, dev, sandbox, uat, featureprodEnvNamesADS: production, prod, prd, live, stableEVO: production, prod, prd, live, stable
FIXBuild pipelines marked as deploying: tier by the DEFINITION NAME; unrecognizable name = PROD (the mark itself means “this deploys”).ialways · in code
DOCCFR denominator = ELIGIBLE runs (success ∪ failure); canceled/notDeployed are excluded and disclosed (ineligibleProd).icfrEligibleOnlyADS: ONEVO: ON
FIXThe manual flag OVERRIDES the result: a flagged succeeded run is a failure and NOT a success (succeeded+failed=total — a law).imanualFlagOverridesResultADS: ONEVO: ON
DOCDF = successful prod deploys; rates per day / week / month (month = monthDays).isuccessResultsADS: succeededEVO: succeededmonthDaysADS: 30EVO: 30
DOCIMC candidates NEVER change CFR automatically; a human flags the specific deploy run (C5: rollback/hotfix for a P0/P1 from shipped code).ialways · in code

Bugs (Quality)

Quality: how many bugs appear and how severe they are; split into production / pre-production when the tickets carry environment labels.

DOCA bug = Bug/Incident CREATED in the window.iunplannedIssueTypesADS: Bug, IncidentEVO: Bug, Incident
HONESTYseverity NULL → “Unspecified” (never dropped); first matching severity field wins.iseverityFieldsADS: customfield_10050, customfield_10511, customfield_10093EVO: customfield_10050, customfield_10511, customfield_10093
DOCPartial env labeling: prod + preProd + unknown = all (a law); headline = prod once ANY label exists, but the unlabeled remainder is always disclosed.ialways · in code
HONESTYEnvironment comes from ticket labels; no labels → the prod/pre-prod split is honestly “unavailable”, ALL bugs still shown + which labels are needed.iprodEnvLabelsADS: production, prodEVO: production, prodpreProdEnvLabelsADS: pre-prod, preprod, pre-production, staging, qaEVO: pre-prod, preprod, pre-production, staging, qa
DOCIMC incidents map to teams by the Engineering Teams field (exact array-element match), computed at read time — never stored.iimcTeamTokensADS: Ad TechEVO: EVO

Status breakdown / Bottlenecks / CFD

Where the time actually goes: per-status time totals, what is stuck in work right now (Aging WIP) and where queues grow week over week (the Cumulative Flow Diagram).

DOCStatus Breakdown: intervals from start to done; done statuses excluded; the “new” category is shown but marked “excluded from Cycle”.ialways · in code
HONESTYLaw: Σ of non-new buckets ≤ Σ cycle; the gap = re-opened tickets' time inside done (measured and disclosed on Verify).ialways · in code
FIXWorkflow strip: one pill per status NAME (dominant category by transition count).ibreakdownMergeSameNamePillsADS: ONEVO: ON
HONESTYAging WIP: only the In-Progress category (backlog rot like “Removed, 777 d” is not WIP).iagingWipOnlyInProgressADS: ONEVO: ON
HONESTYCFD: work stages only, top-8 by peak (the backlog and done bands drowned the queues that matter).icfdWorkStagesOnlyADS: ONEVO: ON
OWNERSuspect thresholds (180/365/730/custom) are view-level — they never change data.ialways · in code

Display level (stored data untouched)

How numbers LOOK on screen — rounding, medians, one abbreviation. Nothing here changes the stored data.

DOCround1 for display; MEDIANS for “averages” (robust to outliers — a spec decision).ialways · in code
FIX“Product Backlog Item” → “PBI” (full name kept in the title attribute).ialways · in code
HONESTYNames/categories/keys are NEVER rewritten for display — the PBI shortening is the only one.ialways · in code

Owner / admin (visibility, not data)

What the owner can switch on/off by hand — and the honesty boundary that no switch can cross.

OWNERAdmin flags hide pages/elements for everyone; the owner key reveals all; rule and exclusion writes are key-gated.iowner tool
HONESTYHiding never touches the honesty disclosures: the “✕ N excluded” chip and Data quality stay visible always.ialways · in code