Metrics & Rules

The Lead Time should measure the time between when the ticket was created until it is done. The purpose is to measure the full end-to-end customer experience. Rules: • Done aligned with Team's DoD • Waiting time should not be excluded. Waiting time can be: – Non working days (Weekends, bank holidays and vacations) – On Hold/Blocked states • Present time in days Open the source page in Confluence ↗

The Cycle Time should measure the time from when the ticket is put in progress until it is done. The purpose is to measure the actual work that was done. Active work is usually the term used for cycle time; however, the word active gets easily misinterpreted. Cycle time is supposed to include waiting time that many times appears after it started. Effective hours/days spent on concrete engineering work without any waiting time is typically something else that's not traditional cycle time (please check Active Work Time). Rules: • Same rules as Lead Time • In Jira, active work usually starts when setting a ticket to the next states: In Progress / Any other? • If the ticket goes back to New, the time it remains in New should not be included when the ticket goes past In Progress again. Open the source page in Confluence ↗

Active Work Time tracks only when the team is actively working, removing all waiting time from the cycle time. It accurately reflects the work required by the Engineering teams. Besides the waiting time already covered on the Cycle Time and Lead Time there are 2 other layers of waiting time that happen every day. State transitions — All Jira statuses and how they are meant to be used either represent active time, passive time (wait time), or a mix of both. Examples: • In Progress — mostly active • Code Review — a mix of active and passive. Most of the time, moving a ticket for review means waiting on someone else to review a PR. The active work on that state is the actual time each engineer spent reviewing the PR. • On Hold/Blocked — passive time as the team can't proceed • Ready To Test — passive time; there is no active time here • In Testing — similar to In Progress and is mostly active time • Ready to merge — mostly passive; normally a residual active time • Release on Stage / Release on Live — mix of active and passive Working hours — The Engineering teams don't work 24/7 and per definition this metric already excludes non-working days and should also exclude non-working hours on working days, typically 8 or 9 hours of work. We are currently not tracking the time spent on each ticket in each working day. It's possible that someone is actually working on multiple tickets or even involved in meetings and other activities that aren't trackable or not worth it. So this is an attempt of standardizing 8/9 hours of work per day. Rules: • Done following the same rules as Lead Time and Cycle Time • All waiting time defined in Lead Time and Cycle Time should be excluded – Vacations can potentially benefit from an integration with Bamboo HR – On Hold/Block assume such states need to be mapped • Exclude the following states defined in the State Transitions section: On Hold/Blocked; Failed on QA; Ready To Test; Ready To Merge; Any other? • Implement a logic to comply with the Working Hours section so that non-working hours are excluded. This doesn't need to be perfect, but at least on cycle times of multiple days, it will allow removing most of the time when people are not actually working. • Present time in days Open the source page in Confluence ↗

The Time to First PR measures the time between when a ticket was put in progress until an associated PR is opened. The intention is to measure how quickly engineers can turn a ticket into a reviewable implementation. Rules: • Same rules as Cycle Time Open the source page in Confluence ↗

Lead Time for Changes is a core DevOps Research and Assessment metric that measures the time it takes for a committed code change to successfully run in production. The intent is to measure the actual responsiveness of our delivery system, not just the active engineering effort. Essentially the Lead Time for Changes will behave like the Cycle time, the only difference is that it starts to count when an associated commit is pushed instead of the ticket being set to In Progress. Rules: • Same rules as Cycle Time (From the meeting: "the definition was not… the hours from pull request merge to deployment to production… it's the time of the first commit until that commit lands in production. It's different from the merge.") Open the source page in Confluence ↗

Prod Bug Rate — The Prod Bug Rate measures the number of bugs that were reported in the production environment. On top of it, it should also break down this number per severity. These bugs can be created by the Incident Management process or any team member. The important part is that it should account for production bugs. This metric should also offer a way to look at the number of bugs in a rate fashion, like the average number of bugs per week. Pre-Prod Bug Rate — This is essentially the same as the Prod Bug Rate but for all bugs found on non-productive environments. Open the source page in Confluence ↗

Change Failure Rate (CFR) is a core DORA metric that measures the percentage of software deployments to production that result in a degraded service, impairment, or outage and subsequently require immediate remediation. It evaluates the quality and stability of your delivery pipeline, ensuring that speed (measured by Lead Time) does not come at the expense of stability. A deployment is a failure if it requires: • An emergency rollback to a previous version. • A hotfix or urgent "fix-forward" patch shipped outside the normal cadence, for example, a major incident (P0 or P1) triggered by the Incident Management process but only if the cause was effectively from the code that was shipped. Rules: • Change Failure Rate is a percentage metric calculated as the (number of failed deployments / total number of production deployments) x 100 • Failed deployments obtained as mentioned before Open the source page in Confluence ↗

The Throughput should measure the number of completed tickets within a timeframe as well as the average number of completed tickets per week. Usually the latter is more useful to explain what's the usual flow of the team on a week basis. Rules: • If Epics are excluded from delivery, they should not count for the throughput. (From the meeting: "you split this by agents and humans, which is pretty cool"; "having an average of how many tickets per week… we think more of weekly basis".) Open the source page in Confluence ↗

The Deployment Frequency measures how often a team/project successfully releases code to the production environment. It serves as a primary indicator of the team's overall throughput, agility and batch size. Like Throughput it should show the total number of successful production deployments for a given timeframe as well as a rate basis. There should be 3 types of rate: • Daily • Weekly • Monthly (From the meeting: "even if it said 50 deployments in those 30 days, it doesn't tell me what usually I think about, which is number of deploys per week or per day… per day, per week, per month — it's really interesting to have those numbers right there.") Open the source page in Confluence ↗

The Planned vs Unplanned Work measures how the flow of a team can be impacted by unexpected work. The distinction between planned and unplanned work separates value-generating progress from unpredictable operational friction. Unplanned work is any unexpected task, disruption, or emergency that breaks the current iteration or sprint focus. It represents the operational tax paid for system instability, technical debt, or shifting priorities. Maintenance vs Unplanned Work — Usually, teams should reserve capacity for maintenance work (per sprint, per week), which means that if there is unplanned work, like an urgent issue to fix, this should be treated as planned work if the effort falls under the reserved capacity. If more capacity is needed for the same or other tasks, then it becomes Unplanned work. Rules: • If the team does not make any capacity planning for maintenance, any kind of deviation, anything that wasn't planned, should be interpreted as unplanned work (bugs, support requests, stakeholders requests, …). For this scenario, capture unplanned work by: Status Bug, Incident; Label: unplanned_work • If the team reserves capacity for maintenance, then unplanned work should only be accountable after the reserved capacity has been maxed out. For this scenario, capture unplanned work by: Label: unplanned_work Open the source page in Confluence ↗

PR Throughput — The PR Throughput should measure the number of merged PRs within a timeframe as well as the average number of merged PR per week. PR Acceptance Rate — The PR Acceptance rate should measure the percentages of merged PRs vs the ones that were abandoned. Open the source page in Confluence ↗

The AI Participation Rate rate should measure the percentages of tickets done that had an AI Agent Assigned. Open the source page in Confluence ↗