back to notes

Agility Health Radar TeamHealth 2.5 Quantitative Questions

TeamHealth 2.5 Quantitative Questions

Scale = PRE-CRAWL: 1-2 | CRAWL: 3-4 | WALK: 5-6 | RUN: 7-8 | FLY: 9-10

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Predictable Delivery***
Predictability: How predictable is your team? What % of your target work items (or SLA if you are a support team) do you meet within your time-box (ex: iteration/week)?
Pre-Crawl (1-2): We don’t set targets OR we meet less than 50% of our planned items / SLA.
Crawl (3-4): We meet 50%+ of our planned items (we miss 5 out of 10).
Walk (5-6): We meet 70%+ of our planned items (we miss 3 out of 10).
Run (7-8): We meet 80%+ of our planned items (we miss 2 out of 10).
Fly (9-10): We meet  90%+ of our planned items (we miss 1 out of 10).

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Time to Market***
Deployment Frequency: How often DOES your team deploy to production?
Pre-Crawl (1-2): We deploy every 3+ months.
Crawl (3-4): We deploy every month.
Walk (5-6): We deploy every 1-2 weeks.
Run (7-8): We deploy and release on demand.
Fly (9-10): We deploy continuously and release on demand.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Time to Market***
Deployment Frequency: How often COULD your team deploy to production?
Pre-Crawl (1-2): We could deploy every 5+ weeks.
Crawl (3-4): We could deploy every 3-4 weeks.
Walk (5-6): We could deploy every 1-2 weeks.
Run (7-8): We could deploy once per day.
Fly (9-10): We could deploy multiple times per day or on demand.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Time to Market***
Lead Time to Deploy/Change: On average, how long does it take from final code commit to code successfully running in production?
Pre-Crawl (1-2): 1+ month.
Crawl (3-4): Within 1 or 3 weeks.
Walk (5-6): Within 2+ days.
Run (7-8): Within 1 day.
Fly (9-10): Less than 1 hour or within minutes.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Value Delivery***
Throughput Increase: How much improvement have you had over your baseline (example: first month of Agile adoption) in throughput? You can measure your number of work items OR number of story points delivered.
Pre-Crawl (1-2): We have improved by 0-20%.  
Crawl (3-4): We have improved by 20-40%.
Walk (5-6): We have improved by 40-60%.
Run (7-8): We have improved by 60-80%.
Fly (9-10): We have improved by >80%.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Value Delivery***
Business Outcomes: How well does this team deliver on the planned business outcomes or goals?
Pre-Crawl (1-2): We don’t define measurable business outcomes/metrics or understand the value/impact of our work to our customers.
Crawl (3-4): We deliver 50% of our planned outcomes.
Walk (5-6): We deliver 60%+ of our planned outcomes.
Run (7-8): We deliver 75%+ of our planned outcomes.
Fly (9-10): We deliver 95+ of our planned outcomes.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Value Delivery***
Feature Throughput: Of the targeted feature work, how many does your team release every 10-12 weeks on average?
Pre-Crawl (1-2): We don't track the number of features delivered OR We deliver 1-2 features.
Crawl (3-4): We release 3-4 features.
Walk (5-6): We release 5-6 features.
Run (7-8): We release 7-8 features.
Fly (9-10): We release 9-10+ features.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Quality***
MTTR (Mean Time To Recover): On average, how long does it take to recover from a high severity production incident?
Pre-Crawl (1-2): >= 8 hrs.
Crawl (3-4): < 8 hrs.
Walk (5-6): < 6 hrs.
Run (7-8): < 2 hrs.
Fly (9-10): Minutes or seconds.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Quality***
Change Failure Rate: What % of your production releases over the last quarter have resulted in high severity incidents? For example, ones that lead to unplanned outages, severe performance degradations, or require a hotfix, roll-back, etc.
Pre-Crawl (1-2): ~ 40%+ (4 in every 10 releases).
Crawl (3-4): ~ 30% (3 in every 10 releases).
Walk (5-6): ~ 20% (2 in every 10 releases).
Run (7-8):  ~ 10% (1 in every 10 releases).
Fly (9-10): ~ 5% or less  (1 in every 20 releases).

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Quality***
Test Automation: How well have we matured towards our test automation goals?
Pre-Crawl (1-2): We don’t have any test automation goals OR We've automated 25% or less of our target goals.
Crawl (3-4): We've automated 60% of our target goals.
Walk (5-6): We've automated 80% of our target goals.
Run (7-8): We've automated 95% of our target goals.
Fly (9-10): We've fully automated 99%+ of our target goals.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Quality***
Defect Ratio: What is the average ratio of escaped defects to stories / work items in your backlog? Use either count of items or points.
Pre-Crawl (1-2): Our defect ratio is at 50%+ (example: We have 5 escaped defects for every 10 stories).
Crawl (3-4): Our defect ratio is at 30%+.
Walk (5-6): Our defect ratio is at 20%+.
Run (7-8): Our defect ratio is at 10%+.
Fly (9-10): Our defect ratio is 5% or less.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Responsiveness***
Story Cycle Time: On average, how long does it take us to deliver a story to production once it’s pulled by the team? If you don’t use Stories, then measure a work item.
Pre-Crawl (1-2): within 1 month+.
Crawl (3-4): within 3 weeks.
Walk (5-6): within 2 weeks.
Run (7-8): within 1 week.
Fly (9-10): Less than 3 days.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Responsiveness***
Feature Cycle Time: On average, how long does it take us to deliver a feature to production once it’s pulled by the team?
Pre-Crawl (1-2): within 3+ months.
Crawl (3-4): within 8 weeks.
Walk (5-6): within 6 weeks.
Run (7-8): within 4 weeks.
Fly (9-10): within 2 weeks.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Responsiveness***
Impediments: What % of your work is blocked by factors outside of your control? Example: Waiting for approvals, decisions, other teams, etc.
Pre-Crawl (1-2): We are usually blocked on 7 out of 10 stories. 
Crawl (3-4): We are usually blocked on 5 out of 10 stories.
Walk (5-6): We are sometimes blocked on 3 out of 10 stories.
Run (7-8): We are sometimes blocked on 2 out of 10 stories.
Fly (9-10): We are rarely blocked on my stories and can often complete them with minimal wait.

Dimension - Performance, Sub-Dimension - Measurements, Competency - ***Responsiveness***
Dependencies: Typically, how many teams does it take to complete a feature?
Pre-Crawl(1-2): We don’t track the number of teams needed to complete one feature.
Crawl (3-4): 5+ teams.
Walk (5-6): 3-4 teams.
Run (7-8): 2 teams.
Fly (9-10): 1 team.