back to notes

Agile Maturity

Testing
Acceptance testing in lieu of all other tests
No unit testing framework, ad hoc testing
Unit testing, manual functional testing; application has a testable architecture
Unit testing integrated into build process, testing automated as much as reasonable given application
Developers write unit tests before writing functional code
Test are identified and produced as part of a story creation
Automated functional testing (e.g.. GUI testing); stories remain in development until all bugs are fixed or deferred


SCM
Traditional schemes based on file-locking and time-sharing
SCM supports version of code
IDE(s) integrate with SCM; developers include meaningful comments in commits
SCM support merging; optimistic check-ins
SCM support atomic commits
All development collateral is in SCM
SCM is transparent to delivery team, behind build process


Collective Code Ownership
Knowledge held by specific team members; people work in isolation
No pairing, people work alone, some informal process for keeping people informed
Pairing; no code locking
Pairing scheme ensures rotation
Team signs up for functionality rather than assignment to individuals
Within an sprint, functionality delivery is signed up for just in time
Old (bugs) and new functionality is queued, development team pops the stack in real time


Collaboration
Regular progress updates from management to the delivery team (as opposed to the other way around); irregular team meetings
Project collaboration tools (wiki, mailing list, IM) in place and used throughout project team; project status published and visible to all
Daily stand-ups and sprint meetings; problem solving is bottom-up as opposed to top-down; team sets sprint objectives and agrees on estimates.
integrated, continuous build process, with build status notification to the team and collective responsibility for state of the build
Business is part of the team, stakeholders accept working software at reviews in lieu of other tracking or progress metrics
Frequent (near-real time) prioritization of old and new functionality
Build automatically deploys to QA environment available to any interested party;


Responsive to Business
Frozen specification, unresponsive to business value
Team tries to accommodate new requirements by ad hoc change requests ("pile it on")
Iterative process with sprints of length short enough to respond to business change
Showcases per sprint; business prioritizes functionality per sprint
Continuous involvement of the business on the team: business certifies story complete out of dev; involved in (rather than approval of) story definition and acceptance tests
Business writes high-level stories for all requests; organizational story queue exists; prioritization decisions made from full story backlog
Development process is integral to business initiatives; development teams work as a part of the business unit rather than as a service to the business unit


Assurance and Governance
Status reports document progress; schedule acts as plan
Concept of release planning is introduced
Sprint planning is introduced
Plan becomes communication and tracking tool rather than an exception report
Every sprint review examines value delivered and assesses next sprint by need, priorities (and delivery)
Organizational adoption: Introduction of Portfolio Planning and global optimization of resources (use of metric and standard measures)
Business review of value and return helps teams on regular basis based upon status


Story Formation
Development tasks extracted from voluminous, frozen requirements artifacts
Production of lightweight artifacts (e.g. panel flows) to drive high-level requirements development
Mapping granular requirements (e.g., use cases) in high level user requirements
Marshalling use cases into discreet statements of functionality that can be delivered in time boxed development sprints
Stories are an expression of end-to-end functionality to be developed, including testable acceptance criteria and a statements of value
Spontaneous story development provided by business for in-flight projects; stories not derived from extant sources but are immediate expressions of customer demand/need
Global repository of functional requirements for stories developed by business for any business requirements, including a formal measure of business value delivered


Simplicity
Big up-front design that attempts to accommodate all potential future needs
Application of fundamental design patterns
Structural refactoring to decouple application architecture
Doing only what needs to be done
Aggressive and constant refactoring to improve code quality/simplicity extant code
Spiking solution with each release to introduce new ideas and challenge architectural decision
Technical design decisions taken with each story


Build
Ad-hoc build requests; scripting and component marshalling performed manually
Consistent, repeatable build process with durable artifacts executed manually with each release
Build is automated, executed on a timed basis
Unit tests are integrated with the automated build; team is constantly notified of the status of the build; build triggered by SCM updates
Test and metrics (e.g., code quality, complexity, check style, etc.) are integrated as gatekeeper events; build data archived and reported in build portal/dashboard
All product components advertise dependencies; master repository established
Product integration tests included; build process executes automatic deployment to QA testing environment


Practices/Ways of Working
Stories sufficiently elaborated prior to the team beginning development
User Stories are created, easy to read and understand
Acceptance Criteria accompanies each of the user stories and is complete
Story Boards exist and are available for all to view
The entire team is present at release planning meetings
Release cadence established and communicated
Work item planning meeting attended and effective
Daily standup are on time, fully attended and effective communication occurs
Team retropsectives attended, participation is active and effective
Testing is done on a work item as a condition of "Done"
The team practices coding standards
Pair Prgramming is used regularly
Automated testing occurs and is figured into commitments
Refactoring happens regularly and not just when there is free time or a problem arises
Behavior-driven Development is practiced
There is a high commitment to technical excellence
Impediments are actively removed by a dedicated team member
Automated Test Scripts exist, represent core functionality and are used
Team defines, estimates, and selects their own work (work items and tasks)
Team discusses acceptance criteria during work item planning
The entire Team performs testing (QA as well as Developers)
What stage is the Team in? (Forming = 1, Storming = 2, Norming = 3-4, Performing = 4-5)
The Team actively participates in the Demos/Reviews


Clarity and Market
There is one single person or "wringable neck" responsible for the backlog priority even if there are multiple stakeholders
Everyone on the team knows who the owner is
Product backlog identified
Backlog prioritized and ranked by business value
Backlog estimated (if applicable)
There is defined acceptance criteria for stories/work items created by those who understand business value
Product Owner(s) participate in Release Planning
Product Owner(s) participate in team Retrospectives
Product Owner collaboration with team is continuous
Demos and Presentations exist and reflect team progress
Story success criteria is created (Definition of done)
Epics and Features are created instead of projects and programs
UI/UX/CX is not just a consideration but is baked into the product lifecycle
There is shared responsibility between IT and business on clarity and market
Portfolios exist and there is cross-portfolio roadmapping that occurs at least annually
Business does not just consider agility or agree with it, but agility has become the main driver to organizational/holistic agility
Products are maintained in lifecycles that sunrise and sunset as the need arises, with high quality being a differentiator
Evolutionary design is practiced
There is a commitment to design excellence
Release Plans are created, maintained and reported on


Tools
The Kanban/Story board is visible to the team and stakeholders
Resource/Capacity Planning - Tracking is performed
Resource/Capacity Planning - Forecasting is facilitated
Reporting - Burn-up Chart (Release Plan) - Baseline is performed
Burn-up Chart (Release Plan) with Targeted Schedule is updated
The Burn-up Chart (Release Plan) with Actual Schedule with Projection is maintained
The Burn-up Chart (Release Plan) with Committed Points vs Accepted Points is maintained
The Burn-up Chart (Release Plan) with Milestones/Interim Releases is maintained
The Burn-up Chart (Release Plan) with Dependencies is maintained
Burn Down Charts (Sprint Plan) with Targeted Schedule are created
Burn Down Charts (Sprint Plan) with Actual Schedule with Projection are created
Burn Down Charts (Sprint Plan) with Committed Points vs Accepted Points are created
Key metrics such as wait time, lead time, and cycle time are tracked
Source control system exists
Continuous build with 100% successful builds
Developers integrate code multiple times per day
Prioritization tools exist and are used regularly
Tools exist for managing the User Stories
Tools exist for managing the code
Metrics are captured and used for reviews and retrospectives


Leadership and Strategy
There is a dedicated individual who knows Agile, is a emergent leader, and serves the team
Servant-leadership is exercised in all levels of leadership
There is predictability across individual teams and programs supported by leaders
Leadership training in agility has occurred and is regularly refreshed
Leaders encourage MVP/MMF concepts in order to achieve value faster
Architecture is emergent, led and mentored by architects who enable and empower teams
Agile training is integrated into onboarding
Resource management is by team and based on the amount of work the team has gotten done in the past
Value streams are well defined, staffed, aligned, and cultured
Lean product management is practiced at the portfolio and business layers inside the organization
The culture is one that not only embraces change, but creates an environment where change occurs
Agile begins at the strategic layer in the organization, focusing on changing priorities, value determiniation, and key internal and external drivers.
Funding occurs at the value stream level rather than at the project level.
The environment (seating, floor space, etc.) has been developed to enrich communication and foster collaboration.
There is an architectural runway that is driven from and works in tandem with the feature/epic backlog


Ways of Thinking
The Team agrees on the definition of "Done" for work items, it is visible, and is close to "ready to ship."
The Team leads communication - not managed
The Team self-polices and reinforces use of agile practices and rules
The Team inspects and adapts (continuous improvement) the overall process
The Team has an effective channel for obstacle escalation
Team members complete commitments
Team throughput measured and used for planning
Team develops and manages the work item Backlog
Daily progress tracked by a Kanban/Story board
The Team works at a sustainable pace
Work In Progress (WIP) limits are established for each lane on the Kanban/Story board by the team
Work In Progress (WIP) limits are enforced by the team
Team manages their own interdependencies and constraints
Violation of Work In Progress (WIP) limits prompt discussion and corrective action
Work item defects are fixed before the work item is placed into production
Unit tests written before development of a work item
Team communication is effective
The team stays on Task
The team stays on Schedule
Simplicity - the art of optimizing the amount of work not done - is practiced regularly
Team members consistently look for ways to improve their abilities and competencies


Culture and Organization
The Team works in a physical environment that fosters collaboration
The Team is cross-functional with an integrated PO (BA), Dev, and QA
The Team is 100% dedicated (no time-slicing)
The Team is fewer than 10 people
The Team is co-located
Team has administrative access to their own workstations
Team has administrative control over their development environment
Team is permitted to refactor anywhere in the code base
Communities of Practice are setup and thriving
Refactoring is continuous
Does agile have executive sponsorship/support?
Does the business support agile?
Does the business have a basic understanding of agile and its benefits and challenges?
Does the company culture support an agile environment?
Business leaders and stakeholders engaged and participate
Modern organizational change management practices are employed
The team is insulated from external Issues and distractions
There is facilitated negotiation between the team and the PO, and Equitable Exchange is practiced
There is a learning culture that desires to improve and change for the better constantly
The organization is taking agility past IT and into other areas such as finance, marketing, etc.
There is little perceptable difference between business and IT team members
Teams are self-organizing


Observation/ Interactions Things to look for and questions to ask yourself during the observations:
Iterations/Development Cycles How long are the iterations? Less than 4 weeks?
Iterations/Development Cycles Does the team have a place they can work together?
Iterations/Development Cycles Is there unplanned or uncommitted work being brought into the iteration/cycle?
Iterations/Development Cycles Are people asking for the aforementioned change or is it emergent design?
Iterations/Development Cycles Does the team practice equitable exchange?
Iterations/Development Cycles Is the team team staying non-distracted?
Iterations/Development Cycles How many work items are in-progress by the team and by individuals at any given time?
Iterations/Development Cycles Is testing a part of the definition of done for each work item?
Iterations/Development Cycles Is testing relegated to only one individual on the team?
Iterations/Development Cycles Are stakeholders or business contacts available to the team with limited noticed?
Iterations/Development Cycles Are the teams checking in and integrating their code continuously during a iteration?
Iterations/Development Cycles Is the team performing?
Iterations/Development Cycles Are they doing the development best practices?
Iterations/Development Cycles Are distractions being kept away from the team?
Iterations/Development Cycles Are they working together and collaborating.
Iterations/Development Cycles Is there a "single ringable neck" who can answer questions (SME) and make decisions on behalf of the customer/stakeholder?
Daily Communication Do they occur?
Daily Communication What is the duration?
Daily Communication Does everyone on the team participate?
Daily Communication Do others participate that are not on the team?
Daily Communication Do they use a Board?
Daily Communication Where do they conduct them (designated space)?
Daily Communication Is there someone facilitating and keeping the daily standup focused and timeboxed?
Daily Communication Does it feel more like a status meeting or more of an activity meeting?
Daily Communication Do they discuss user stories or only reference them?
Daily Communication Are impediments being looked for?
Daily Communication Do they take anything offline?
Daily Communication Are there notes or action items that come out of the standup?
Demos/Reviews Does the team set aside time to prepare?
Demos/Reviews Is the time included as part of the definition of done for the iteration/release or just an afterthought?
Demos/Reviews Who conducts the demo/review?
Demos/Reviews Do people outside of the team attend? PS/SMEs? Others like management or stakeholders? Other teams?
Demos/Reviews Do they show what was committed to and what was achieved? (Story Points)
Demos/Reviews Does this align with the Definition of Done by the team?
Demos/Reviews Do they show progress with a burn-up/burn-down chart?
Demos/Reviews Do they show what is coming in the next iteration or cycle?
Demos/Reviews Does anyone attending ask questions or interact?
Demos/Reviews Any compliments or excitement?
Demos/Reviews How does the team react?
Demos/Reviews How does the audience react?
Retrospectives Do they conduct them?
Retrospectives Is there someone facilitating and keeping the Retrospective focused and not a finger pointing exercise?
Retrospectives Does business attend?
Retrospectives Does each person on the team contribute/participate?
Retrospectives Are there any action items coming out of it?
Retrospectives Do they talk about the good as well as the not good?
Retrospectives Is it positive and open?
Retrospectives Do people feel able to honest and voice their ideas and opinions without being criticized?
Planning Does the team participate?
Planning Do they use a certain method when estimating? Planning Poker, T-Shirt sizes, Powers of 2, Fibonacci sequence, or simply small?
Planning Is there business availability to answer questions and provide clarification prior to estimating work?
Planning Does each team member ask questions or discuss the user story?
Planning How much time do they spend? Is it calculated as part of the iteration or work item?
Planning Are questions prevalent? Is it an inquisitive event?
Planning Is it timeboxed to 4 hours for a 4 week iteration and less depending on iteration/release length?
Planning Is it facilitated?
Planning Is commitment real or cursory to the final plan?
Planning Does planning include QA, design, UX/UI, architecture and everyone necessary to provide good information?
Planning Is refactoring and technical debt brought up each time? Is it incorporated into the sprint and estimated?
Planning Is documentation included in the estimates?
Backlog Refinement Do they do it as a separate function?
Backlog Refinement Is there a backlog or just in time work items? Is lead time non-existent?
Backlog Refinement How often is the backlog grooming performed?
Backlog Refinement How large is it (enough for several iterations)?
Backlog Refinement Who participates in it?
Backlog Refinement Does the backlog contain Themes and Epics only?
Backlog Refinement Is the backlog prioritized?
Backlog Refinement Is it sized or estimated?
Backlog Refinement Who participates in the estimating?
Backlog Refinement Does what is being groomed align with the overall product roadmap or vision?
Backlog Refinement Does what is being groomed align with the release planning?
Release Planning Do they do it?
Release Planning Who participates?
Release Planning How often is it performed?
Release Planning How long was it?
Release Planning Does it sync with the iterations?
Release Planning Is a method to prioritize used?
Release Planning Are pre-set release dates used?
Release Planning Or is it customer or feature driven?
Release Planning Is it done at the story level or higher such as Epics or Themes?
Release Planning Are they using the team(s) velocity in their planning?
Release Planning Are the features/functions prioritized and dependencies discussed?
Release Planning How often are releases scheduled for?
Release Planning Are hardening iterations or phases added as part of the planning?
QA Testing Who does it?
QA Testing Is it done as part of the definition of done of the work item?
QA Testing If not developers, are they dedicated to the team?
QA Testing Do they troll each day during daily standups for things to test?
QA Testing When is it performed?
QA Testing Is testing/QA part of the story board?
QA Testing Are there only manual scripts?
QA Testing Any automated testing being done?
QA Testing Are these regression tests?
QA Testing Is developing acceptance criteria seen and understood as part of the testing/QA process?
QA Testing Is testing seen as something that everyone does or an act by a single individual?
Team Interactions/Dynamics & Make-up How does the team interact with business and any servant-leadership?
Team Interactions/Dynamics & Make-up How do they view or interact with stakeholders during reviews/demos?
Team Interactions/Dynamics & Make-up How do they react during Retrospectives?
Team Interactions/Dynamics & Make-up Is there a core team? Do they stay together or is it more part-time people coming in and out of the team?
Team Interactions/Dynamics & Make-up Does the team like working together? Do they pair program?
Team Interactions/Dynamics & Make-up Do they work together in the same room?
Team Interactions/Dynamics & Make-up Is there good communication and camaraderie between team members?
Team Interactions/Dynamics & Make-up Do they socialize?
Team Interactions/Dynamics & Make-up Is there passion on the team?
Team Interactions/Dynamics & Make-up Is there a sense of ownership?
Planning Activities (NA if project is inflight) Is this a new team?
Planning Activities (NA if project is inflight) What activities did they do? Architecture? Features?
Planning Activities (NA if project is inflight) Did they write stories and estimate?
Planning Activities (NA if project is inflight) How did they perform? Did they make their target points?
Planning Activities (NA if project is inflight) Are they being coached and mentored?
Planning Activities (NA if project is inflight) Is there a definition of “Done”?
Planning Activities (NA if project is inflight) Is there a vision or roadmap shared with the team?


Artifacts/ Activities Things to look for and questions to ask yourself during inspections:
Boards (Kanban, Story, Task) Are they using one?
Boards (Kanban, Story, Task) Where is it? Is it easily visible to the team as well as visitors?
Boards (Kanban, Story, Task) Are stories on cards or sticky notes and placed on the board? Color coded? How? Why?
Boards (Kanban, Story, Task) Does it show the backlog as well as “Done”?
Boards (Kanban, Story, Task) What other steps/phases are shown?
Boards (Kanban, Story, Task) Are there a lot of stories in-flight (WIP) but few that are in testing or done?
Boards (Kanban, Story, Task) Can someone come in and look at the board and see where the team is at?
Boards (Kanban, Story, Task) Is it updated each day?
Boards (Kanban, Story, Task) Are dependencies noted or stories grouped?
User Stories Are they easy to read and understand?
User Stories Does the team know why they are building it?
User Stories Is there acceptance criteria?
User Stories Can the team(s) estimate work?
User Stories Have they been decomposed sufficiently?
User Stories Does the team suggest alternatives?
User Stories Does every story have point estimation? Does it ever change or is updated?
Acceptance Criteria Who provided the criteria? Is it approved by the team?
Acceptance Criteria Does the developer understand the criteria and it has been reviewed explained by the author (say an Architect) of the story?
Acceptance Criteria Did QA assist in defining and documenting the criteria? Did QA review with the developer what they could test and how?
Automated/ Manual Test Cases/Scripts Do they have automated testing? Is it part of their normal testing practice? Regression?
Automated/ Manual Test Cases/Scripts Who does the testing?
Automated/ Manual Test Cases/Scripts Do they match to the acceptance criteria?
Automated/ Manual Test Cases/Scripts Where are they kept? In a tool?
Automated/ Manual Test Cases/Scripts Are defects tracked and reported on?
Metrics Are they being captured?
Metrics Are they being used in reviews? How?
Tools What tools do they use?
Tools Does everyone use the tools? Do they know how?
Tools How easy are they? Do they help or hurt?
Reports/ Presentations Does the demo show what was committed to AND what was completed? Features/Functions as well as Story Points?
Reports/ Presentations Does it show where they are at in regards to the progress for the release (above or below the line)? The overall schedule?
Reports/ Presentations When showing the features and functions both technical and non-technical does it explain why it is important to the business and customers?
Reports/ Presentations Does it show what’s up next?
Reports/ Presentations Does it show impediments and resolutions/action plans?
Release Plans/Schedules Are there any?
Release Plans/Schedules Are they published and reviewed frequently? Are they visible to all?
Release Plans/Schedules Are the metrics applied and accounted for?
Release Plans/Schedules How much time is allocated to do release planning?
Engineering Practices Architecture, platform and tools defined and stable?
Engineering Practices Coding standards/naming conventions defined?
Engineering Practices Continuous integration?


Topic Questions
Business/Organization Change Management Why do [did] you want to make the transition to Agile?
Business/Organization Change Management What were you hoping to achieve?
Business/Organization Change Management Does the business know this?
Business/Organization Change Management How is (was) that communicated and to whom?
Business/Organization Change Management Is (was) the business driving this or IT?
Business/Organization Change Management Do (did) you have an Agile Adoption Roadmap?
Business/Organization Change Management Where do you think you are at? What do you think you are doing well or not well?
Business/Organization Change Management What do (did) you think are (were) the limiting factors for Agile Adoption in your organization?
Business/Organization Change Management Why?
Business/Organization Change Management Have you done any pilot project(s) using Agile?
Business/Organization Change Management If yes, what were they (app dev, BI, etc.) and how long were they (3,4,5,6 months or longer)?
Business/Organization Change Management How big were the team(s)?
Business/Organization Change Management What have been the results?
Business/Organization Change Management How is (was) that communicated and to whom?
Business/Organization Change Management Are they providing budget/funding for additional efforts?
Business/Organization Change Management What do you see are the challenges/reasons that are preventing agile adoption?
Ownership/Accountability Who “owns” the application requirements, is there a business owner or product owner?
Ownership/Accountability Who has this role?
Ownership/Accountability Who “owns” the application architecture, is there an architecture owner?
Ownership/Accountability Who has this role?
Ownership/Accountability Does the business owner have a structure underneath them?
Ownership/Accountability Business Analysts on each team?
Ownership/Accountability Are they surrogates?
Ownership/Accountability How much authority do they have?
Ownership/Accountability How often does the PO countermand some requirement decision by a BA?
Ownership/Accountability Are there multiple products & multiple business owners/POs?
Ownership/Accountability How do they integrate?
Ownership/Accountability Is there a combined/coordinated roadmap?
Visibility Has communication improved through the adoption of agile?
Visibility Can you describe what’s going well and what isn’t?
Visibility Tell me about your relationship with dev, QA and the product owner
Tools and Practices Have you adopted any agile lifecycle management tools such as rally, version one, etc.?
Tools and Practices What is going well and where are you having issues?
Managing Do you have a PMO?
Managing How are they integrating your projects into their analysis and reporting?
Managing Who manages the team(s)?
Managing How many business owners or Product Owners does the individual team work with?
Managing Are your projects finishing with the “big push” at the end to meet the changes that were not known until the very end of the project?
Managing How is that impacting your development team?
Budget and Approval What is the project approval process?
Budget and Approval What kinds of budget estimates are necessary to get a project approved (particularly by project phase)?
Budget and Approval How is scope creep managed?
Budget and Approval How is cross-application cross-platform integration managed in design/development/testing/deployment?
Work Structure Do (did) you have iterations?
Work Structure If so, how long are (were) they and do (did) your users/stakeholders get to review what was completed at the end of each iteration?
Work Structure How are you allocating time for the team for iteration and release planning?
Work Estimation How are you estimating product requirements?
Work Estimation Are the estimates matching up with actuals?
Work Estimation If no, why do you think that is occurring?
Work Requirements How is the organization adopting conveying requirements?
Work Requirements Do you write user stories?
Work Requirements If so how are you doing with user story definition versus traditional requirements?
Work Requirements Where are you having issues and what is going well?
Work Requirements How are you leveraging your BAs?
Work Requirements Do you have a prioritized backlog, how is the business prioritizing the backlog?
Work Requirements Is the team including other stakeholders in the prioritization?
Work Requirements Are they providing budget/funding for additional efforts?
Work Testing How is your testing organization adapting?
Work Testing Are they part of the Agile team or still separate?
Work Testing Are they able to keep up with the development team?
Work Testing How do you conduct your testing?
Work Testing Do you leverage any automated testing strategies?
Work Testing Do you want to?
Work Testing How are the testers writing acceptance criteria?
Work Testing Is this impacting estimation? How?
Work Scope How much do the User Stories evolve after planning?
Work Scope Is this impacting your velocity or cycle time?
Work Scope How does the Dev team rate the incoming User Stories?
Work Impediments Are production support issues preventing you for realizing your goals?
Work Impediments How are you adapting to meet your goals?


Behavior
Topic Comments Best Match A B C
Agile Values and Principles The team is aware overall and some can speak to a point or two, but overall the team is only aware. Aware Understands the Agile values and the principles, can speak to them As a team, has internalized the Agile values and principles, can speak to them, and does their best to live by them
Working Agreements The team has static documents located on their SP site and knows where to locate them. Aware Created static documents that typically include items like: DoD, DoR, meeting etiquette, etc. The team updates working agreement based on lessons learned during retrospectives and during the normal course of working together
Role-based training The team is completing their training and the PL and PO have completed their training. Aware Team members are actively completing required Agile training for their role. The PL has completed all required Agile training. PO has completed all recommended Agile training for their role.
Accountability Overall, the team is accountable and doesn't need reminders constantly, but occasionally reminders are necessary. Aware Team needs support to be accountable (e.g., task updates, running demos, needs reminders, …) Team feels accountable and is proactive (proactively updates tasks, plans for demos, …)
Responsibility The team is between individual work and team collaboration. Aware Individuals take responsibility for their tasks Team shares responsibilities
Empowerment The team is empowered, but at times needs leadership support to help make decisions. Aware The team follows the Agile @ American framework, feeling empowered to make decisions, but sometimes requires decision support from a leader Team understands the Agile @ American framework and follows it, but is now empowered to adapt the guidelines as needed for their unique situation and is making decisions and helping drive decisions in the larger context (release, roadmap)
Transparency The team communicates constantly as issues arise, for help, information radiators, etc. Aware Team provides visibility and assesses progress at required intervals (e.g., daily standup, demo, …) The team communicates openly and provides 24/7 access to up-to-date progress (Rally, info radiators, etc.)
Safety There are times when team members feel they may need to choose their words wisely, but overall have a safe environment. Aware Team members speak up, but are careful in choosing their words Team members feel safe in meetings, they openly and courageously discuss items
Collaboration The team communicates effectively within one another in the team, but across teams is more of a producer/consumer model vs. collaboration with outside teams. Aware Team is collaborating effectively during ceremonies Team is collaborating effectively during ceremonies and across other teams
Conflict Management The team can at times handle their own conflict, but at other times must request help from a leader or manager to solve the problem. Aware The team handles conflicts occasionally, but at times needs to escalate conflict to a leader/manager for resolution The team works together to resolve conflicts, rarely needs to have a manager/leader to assist
Communication The team quickly escalates issues to stakeholders and does not only do it during ceremonies. Aware Communicates at regular working sessions and ceremonies (planning meetings, etc.), provides basic info at the end of each iteration, release, etc. Communicates proactively, provides info to stakeholders, team knows when to escalate
Decision Making The team makes recommendations, but require a leader or manager to make the decision. Aware The team may make recommendations, but they are dependent on leaders/managers to make the decisions Team makes decisions and reports frequently to key leaders and stakeholders


Product Management
Topic Comments Best Match A B C
Product Owner The PO is adequately knowledgeable in the domain and available as much as posible for the team. Aware The Product owner is adequately knowledgeable in the domain and available when needed The Product Owner is part of the team, a subject matter expert, fully engaged on a daily basis, and willing to help meet team goals
Backlog Management The team has the separate backlogs that are prioritized and sized. Aware Product backlog is separate from the Release backlog, release backlog is identified, sized & prioritized Release backlog mapped back to roadmap and a Product backlog to manage against
Backlog Grooming The team consistently has about 1 iteration worth of stories ready via standing planning sessions. Aware Scheduled session at least once per iteration preparing stories for the next iteration The scheduled sessions will look beyond the next iteration; actively look for stories to decompose with risk and effective delivery in mind
Prioritization Prioritization exists by at least one factor per the business. Aware Simple ordering 1..n based on estimation of perceived business value (one dimensional) Multi-dimensional prioritization techniques (e.g. feature value, cost of development, market impact, organization impact, cost of not providing, process improvements, risk,...)
User Stories & Acceptance Stories have good structure and acceptance criteria and meet INVEST criteria for each story Aware Format of User Stories is correct and includes Acceptance Criteria Stories demonstrate high INVEST properties, the quality of the "so that" - is high and speaks to the true business value the story is delivering
Features Features are identified on roadmap, but not necessarily right sized features. Aware Team identifies features during roadmap planning Teams decompose large features down into smaller features in order to right-size them into a single release, user stories are linked to these right-sized features
Product Portfolio Management The PO is currently wearing multiple hats for multiple projects and must split time between them. Aware PO focuses primarily on the current project, with only cursory views to other products or projects Product owner looks for opportunities to prioritized/combine efforts for a product to get the most ROI; engages customers to assist in defining prioritization


Planning and Review
Topic Comments Best Match A B C
Vision The document exists and is static sitting on SharePoint Aware Team has a static vision document Living document and can speak to it, updated at scheduled intervals
Roadmap The document exists and is static sitting on SharePoint Aware Team has a static roadmap document Living document and can speak to it, updated at scheduled intervals
Iteration 0 The meetings are often pushed aside due to resource constraints. Aware Team schedules and conducts Iteration 0 Planning sessions for each project kickoff, including an Agile mentor/coach Teams do their own Iteration 0s without the need for an Agile mentor or coach
Release The team creates a release plan, but does not track and share KPIs against progress. Aware Team has a release plan Burndown charts and KPIs are used to make changes to release plan. Plan updates are made at regular intervals.
Iteration The team leverages capacity planning and story points for planning, but does not consistently deliver 80% or better against plan. Aware Team iteration planning is based on both story point velocity and available hours capacity, team commitment Team achieves 80% or more of planned scope per iteration consistently, holds themselves accountable to the commitment
Tasking Tasking is done by EOD and all tasks are fewer than 6 hours. The team may be spending too much time here in detail and being overly prescriptive. Aware Team tasks out all user stories by end of the 1st day of the iteration Team collaborates on creating tasks, instead of individual tasking, tasks are sized appropriately (typically less than 6 hours)
Daily Standup The team has daily standups, but due to business demands/freezes there are times when work is on hold. Daily stand ups tend to be 30 min or longer. Aware The team holds daily standups and each member answers the three questions Conversations around what got completed and plans for today, keeps conversations specific to tasks within stories, uses 16th minute for additional discussions
Demos and Reviews Demos are often pushed off due to time constraints. Also, not all stakeholders are present during demos. Aware Conducts a demo at the end of each iteration, team attends, stakeholder attendance may be hit or miss Regular demos, with consistent stakeholder attendance and feedback
Retrospectives These are regularly scheduled and regularly skipped. :) Aware The team uses one way of conducting retros and holds them regularly, needs an external facilitator or the PL Always conducts Retros, multiple ways used, sees tangible improvements from retrospective action follow ups
Estimation and Sizing Velocity is being used, but not necessarily communicated so everyone clearly understands how and why planning is being done or at capacity. Aware Estimates stories in the release backlog prior to the start of the release, is not consistent in using velocity with estimates to determine scope for a release Consistently estimates stories prior to the start of a release, ensures all stories in the backlog are estimated as soon as possible when changes may occur


Product Delivery
Topic Comments Best Match A B C
Tracking Progress The team is aware of this, but no burndown charts are used or leveraged during stand ups. Aware The team reviews iteration burn-down on a regular basis Actively monitors iteration burn-down and release burn-up charts to take action when required
Testing The team is good at putting this info in ahead of development in the user story. Aware Stories are accepted by running the associated tests successfully against the acceptance criteria Test scenarios based on the acceptance criteria are delivered with the user story and are used to validate story acceptance criteria, User Acceptance Testing (UAT) occurs when applicable
KPI's The KPIs are being logged, but info not being used for future planning sessions. Aware Logs the KPI's on a regular basis Uses the KPI's to adapt team approach; addressed in retrospective and/or iteration planning
Managing Issues No defects or issues are left and all are fixed during the iteration. Aware When testing finds an error it is corrected before a story is accepted by the PO Issues that are found in the iteration for stories that were previously accepted are fixed within the iteration if possible, has basic defect reports
Change Mgmt CRs are not being done or if they are they are very informal. Aware Project Lead and Product Owners understand the official AA change management process as it relates to Agile @ American In addition to column B, team mitigates risks and reduces potential impact by proactively responding to substantial impacts related to scope, schedule and/or costs
Team Roles The team does most work by role. Aware All key roles are represented - PO, PL, Team, with other roles (QA, BA, DBA,…) as needed Team members can help out with other tasks when needed and the team as a whole is appropriately dedicated
Test Automation No automation currently being done. Aware Basic level of test automation Automated regression tests are run regularly
Developing user stories The team does not have priority moving into the iteration, but once in iteration planning they work them in order based on what makes sense for the development team. Aware Stories are developed in priority order and are accepted after the story has completed successful testing Team delivers stories quickly within the iteration (about 1/4 to 1/2 of the iteration length) and team members swarm as needed to meet iteration goals
Risk Mgmt Risks are assessed and documented and placed in the backlog. No risk log exists, just what is noted within Rally. Aware Identifies risks during planning Regularly reviews and updates risk lists
Technical Stories Spikes are done prior to development, but not tracked in Rally as spikes. Aware Uses technical stories to reduce risks and better prepare for business value user stories Uses technical stories to increase learning and reduce technical debt, technical stories have a 0 story point estimate
Product Deployment Due to business need/demand as far as frequency. Aware The team deploys to production once per release. The team deploys to production multiple times per release.


Technical Excellence - App
Topic Comments Best Match A B C D
App - Code Reviews Aware Code reviews are done on a regular basis Code reviews are performed on a daily basis, and fast reaction is taken by the code author Code reviews are performed multiple times a day, or the team uses pair programming
App - Environments Aware Has clearly defined environments - Dev, QA, Staging, Int QA, etc. The team uses promotion and deployment processes that mimic production deployment processes - run books are updated prior to completing a release The team uses promotion and deployment processes that support continuous deployment
App - Architectural Patterns and Reference Architectures Aware Team is aware of architectural patterns and uses them in the design of solutions, the architect is engaged and providing potential solutions and architectural guidance Team modifies and adapts architectural patterns and reference architectures to design optimal solutions Team actively creates new architectural patterns and reference architectures in order to design specific solutions, these patterns are reused within their domain
App - Coding Standards Aware Teams follows informal coding standards most of the time Standards exists and are strictly followed for all production code Standards are evolved as needed by collaborating with the broader development community
App - Unit Testing and CI Aware Team checks in code on a frequent basis & has an automated build machine that runs most of the time The team writes unit tests as part of their coding, and uses continuous integration (CI) to obtain quick status on the code build Build center metrics (test coverage, complexity, etc.) are reviewed each iteration - team goals are established to improve code quality. The team learns from broken builds and implements improvements to reduce future broken builds.
App - Technical Debt Aware Technical Debt is typically addressed later when specific issues need to be fixed or improved Team address technical debt as they develop Team demonstrates a history of addressing technical debt quickly, technical debt (cost of change) is measurably decreasing for the system
App - Design Patterns Aware Team applies common design patterns when developing new software Team uses common Design Patterns in developing new software, team handles technical debt by refactoring existing code towards common Design Patterns Uses common Design Patterns by adapting them into new Design Patterns which improve the development process and the quality of deliverables
App - Code Ownership Aware Team members typically only work in their code areas Team values collective code ownership Team practices collective code ownership and each team member can fix issues in all areas
App - Testing Aware Unit tests are developed along with new code Automated unit tests are written before code is implemented Automated functional tests are written before code is written
App - Refactoring Aware Refactor is done when specific issues need to be fixed or improved Refactoring is performed proactively in anticipation of future issues Refactoring is part of the design and development process
App - Development Skills Aware Team learns new technical skills as they are needed Team learns new technical skills beyond what is required for the current need Team exhibits continuous learning of technical skills, team develops standards and frameworks that are shared with other teams
App - Deployment Aware Team deploys when needed using a combination of manual processes and automated scripts Team deploys when needed using automated scripts Team use continuous deployment, feature-level promotion, and rollback of functionality using automated scripts


Technical Excellence - Infrastructure
Topic Comments Best Match A B C D
Infra - Architectural Patterns and Reference Architectures Aware Team is aware of common infrastructure-related architectural patterns and uses them in the design of solutions. Team refactors existing configurations towards common infrastructure related architectural patterns Team can create appropriate architectures based on requirements and generally accepted good practices, patterns and reference architectures the team creates are reusable by other teams to solve related problems
Infra - Architectural Upgrades Aware Team can carry out significant architectural upgrades of infrastructure elements Team can carry out significant architectural upgrades of infrastructure elements by applying smaller upgrades in a well-planned way Team routinely upgrades architectural components
Infra - End of Life Aware Team can retire systems and infrastructure components gracefully on a case-by-case basis, using manual methods Team can retire systems and infrastructure components gracefully on a case-by-case basis, using automated methods End of life for infrastructure components is planned in advanced based on the age of assets, announced termination of vendor support, capacity planning, advances in technology, cost management, and other factors - end of life does not come as a surprise
Infra - Automated Configuration and Deployment/Rollout Aware Server configurations and standard deployments are performed manually Routine server configurations and standard deployments are scripted Team normally works out the automation of configuration and deployment steps in a lab or test environment before beginning any configuration or deployment in a production environment, scripts are maintained under version control in the same way as application teams manage their source artifacts
Infra - Automated Verification Aware Server configurations and standard deployments are verified using manual methods Server configurations and standard deployments are verified using automated tools Team normally begins by developing verification scripts and then using the verification scripts to guide the creation of configuration and deployment scripts, all scripts are maintained under version control, deployment scripts are subjected to automated regression testing
Infra - Production Issues, Automated Monitoring Aware Team develops repeatable ways to address recurring production issues and provides documentation of corrective actions in a well-known location Team develops repeatable ways to address recurring production issues and provides documentation of corrective actions in a well-known location, team reviews patterns in production issues and acts to remove root causes as appropriate Team's applications and third party packages hook into the standard business activity monitoring facilities to provide early warning or automated prevention of production issues


Roles
Product Owner (PO)
Clearly defined Product Owner (PO)
PO is empowered to prioritize
PO has knowledge to prioritize
PO has direct contact with team
PO has direct contact with stakeholders
PO speaks with one voice (in case PO is a team)
Scrum Master
Team has a Scrum Master (SM)
SM sits with the team
Scrum Team
Team members sit together
Max 9 people per team
Development Team
Team has all skills needed to bring backlog items to Done
Team members also do tasks from other team members
Events
Sprint Planning Meetings
Have sprint planning meetings
PO participates
PO brings up-to-date PBL
Whole team participates
Results in a sprint plan
Whole team believes plan is achievable
PO satisfied with priorities
Iterations
Timeboxed iterations
Iteration length 4 weeks or less
Always end on time
Team not disrupted or controlled by outsiders
Team usually delivers what they forecast
Daily Scrum
Daily Scrum happens
Whole team participates
Problems & impediments are surfaced
Sprint Demo
Demo happens after every sprint
Shows working, tested software
Feedback received from stakeholders and PO
Sprint Retrospective
Retrospective happens after ever sprint
Results in concrete improvement proposals
Some proposals actually get implemented
Whole Scrum team participates
Artifacts
Sprint Backlog
Team has a sprint backlog
Highly visible
Updated daily
Owned exclusively by the team
Product Backlog
PO has a product backlog (PBL)
Top items are prioritized by business value
Top items are estimated
Estimates written by the team
Top items in PBL small enough to fit in a sprint
PO understands purpose of all backlog items
Definition of Done
Have Definition of Done (DoD)
DoD achievable within each iteration
Team respects DoD
Definition of Ready
Have Definition of Ready (DoR)
DoR achievable before each iteration
PO respects DoR

Impediments
Whole team knows top 1-3 impediments
SM has strategy for how to fix top impediment
SM focusing on removing impediments
Escalated to management when team can’t solve
Iterations that are doomed to fail are terminated early
Estimations
Everyone on the team participates in estimating
PO available when team is estimating
Estimate relative size (story points) rather than time
Velocity
Velocity is measured
All items in sprint plan have an estimate
PO uses velocity for release planning
Velocity only includes items that are Done
Daily Scrum
Daily Scrum is every day, same time & place
PO participates at least a few times per week
Max 15 minutes
Each team member knows what the others are doing
Sprint Burndown Chart
Team has a sprint burndown chart
Highly visible
Updated daily
PBL (Items) / Tasks
PO has product vision that is in sync with PBL
PBL and product vision is highly visible
PBL items are broken into tasks within a sprint
Sprint tasks are estimated
Estimates for ongoing tasks are updated daily

You have a Chief Product Owner (if many POs)
Dependent teams do Scrum of Scrums
Dependent teams integrate within each sprint
There is one master Product Backlog

Having fun! High energy level
Overtime work is rare and happens voluntarily
Discussing, criticizing, and experimenting with the process

Meetings
Within five minutes you set up a video call in a meeting room
Everyone has a working webcam and headset
Dedicated meeting rooms for your team meetings
Team
Everyone sees each other in real several times a year
Every location has different roles
Everyone has access to the same data, servers, shares, etc.
All information is available to every location



Agile assessment
checklist:
1. The team knows, for sure, that at any given time they are working on deliverables
that have the greatest value for the business.
2. When the implementation team claims to be Done with something, the business
stakeholder usually agrees that it is, in fact, done and Accepts it.
3. When something is Accepted, it is sufciently well-built and well-tested that it
would be safe to deploy or ship it immediately.
4. The team delivers Accepted product increments at least monthly.
5. When the product increments are shipped or deployed, the users and customers
are generally satised.
6. If the business stakeholder changes the priorities or the requirements, the
implementation team can adapt easily, switching gears to deliver according to
the updated business needs within the next iteration.
7. The business stakeholders express condence that they will get the capabilities
they need in a timely manner. 8. The business can recognize real value from the deliverables: each product
increment ultimately has a positive impact on the bottom line.
9. The team has been working at the same pace, delivering roughly the same
amount every iteration, for a while.
10. The people on the implementation team agree that they could keep working at
the current pace indenitely.

Do
The
Simplest
Thing
That
Could
Possibly
Work
The model follows the principle of “the simplest thing that could possibly work.” To that end,
the documentation for the model is intentionally sparse. Application of this principle also means
that only the minimum number of practices that are needed at any given time should be applied.
Remove
Dependencies
Look for ways to create loose coupling rather than tight coupling. Three examples:
Rather
than
tying
backlog
grooming
to
iteration
planning,
move
as
much
as
possible
to
a
short
weekly
meeting
that
is
independent
of
iterations.

Rather
than
doing
detailed
planning
and
analysis
for
a
whole
release,
focus
on
the
first
MVI
for
a
release,
thus
moving
to
a
loose
coupling
between
a
release
and
the
MVIs
that
make
it
up
(if
more
than
one).
Make
each
team
responsible
for
as
much
of
its
own
destiny
as
possible
rather
than
depend
on
centralization.
For
instance,
an
SOA
architecture
where
each
team
can
release
independently
is
at
one
end
of
this
spectrum.

Current Level Target Level Impeded (0) In transition (1) Sustainable (2) Agile (3) Ideal (4)
Team Dynamics Being Agile 0 No understanding of the spirit of Agile Doing the mechanics 80% of the team can explain the benefits of Agile, believe in the benefits of Agile, understand the spirit of Agile. The team is making improvements on a regular basis Working in an Agile manner Actively pursuing new ways of working in an Agile manner
Morale 0 Blame game, finger pointing, denial, anger, shouting, backstabbing, passive aggressive, turn-over and other behaviors on a regular basis. Desire to go back to the old ways, active resistance to change, scapegoating. There is churn or people are frequently making references to quitting or how much they dislike their work or work environment. There are still elements of previous state, but there is steady progress away from those behaviors, problems are being actively addressed, and there is a general feeling that morale is improving For the most part people are getting along and happy at work. There is very little if any talk about "going back" and it is generally believed that things are either better than before or at least not worse The team generally believes that their work life is significantly better than before. They are happy, engaged, productive, and genuinely enjoy working together. Most team members feel like this is one of the best teams they have ever worked on, they are excited to come in to work and are looking forward to the next day when they leave.
Tuckman Stage 0 Forming Storming Norming Have been performing consistently for at least 8 weeks Have been performing consistently for the past 6 months
Working agreement 0 Non-existent Some defacto team norms that are generally recognized, but haven't yet been written down and agreed on by the team. Written down, agreed on by the team, clearly visible in a public area such as the team room. Followed by the team Followed naturally, very short list, highly visible, exceptions are quickly identified and addressed.
Team Structure Team size 0 >20 people on team It is recognized that a smaller team size is needed and there is either a near term plan or the team is actively being reduced in size. < 20 people on the team < 10 people on the team 7 +/- 2 people on the team
Dedicated resources 0 Most team members are on multiple teams or working on multiple projects Most people are 50% dedicated to the team. Nobody is less than 30% dedicated to the team. Most people are 70% dedicated to the team. Nobody is less than 50% dedicated to the team. Most people are 90% dedicated to the team. Nobody is less than 70% dedicated to the team. Most people are 100% dedicated to the team, nobody is less than 60% dedicated to the team.
Continuity / Standing team 0 Constant churn of people on the team and/or team was formed for a single release or a single major initiative and will be disbanded after shipping. There is an understanding that this is important, progress is being made, and further steps are being taken to get to the next stage 50%+ of the team is constant over the past 9 months and team has made multiple production releases or worked on multiple major initiatives without being reformed each time. More than 70% of the team is constant over the past 9 months and team has made multiple production releases and worked on multiple major initiatives without being reformed each time. More than 90% of the team has been constant over the past 12 months
Cross functional 0 A significant portion of what is needed to get the stories to done exists outside of the team Some of the skills necessary to get the stories to done exists outside of the team All of the necessary skills for performing the work exist on the team All of the necessary skills for performing the work exist on the team and there is some cross training of skills All of the necessary skills for performing the work exist on the team and most of the team is cross trained on most of those skills
Collocation 0 Team members have very little proximity to each other. Plans are in place to move team members as close to each other as is currently feasible. Most team members are accessible to any other team member within 30 seconds Most team members sit within hearing distance of each other Most team members are sitting in a team area together.
Self organization 0 Most people do not have the ability to choose what they work on, estimates are not determined by the team. Team does not feel like it can make decisions on its own. Some members just want to be told what to do. Some of the behaviors from the next stage are being discussed, encouraged, or tried Teams are pulling work from the product backlog themselves, doing their own team-based estimation, choosing what to work on themselves, and using the definitions of ready and done to guide interaction with those outside the team. The roles and responsibilities of the Scrum Master are shared by the entire team and the need for a designated and/or dedicated Scrum Master is significantly reduced. When some members of the team are not present, the team is able to adjust and continue getting stories done. The team is self organized
Sustainable pace 0 People are tired, irritable, burnt out, working overtime on a regular basis. Current situation is considered business as usual. There is a recognition that the current pace is not sustainable and steps are being taken to improve the situation. Consensus is that the team is working at a pace that is close to sustainable indefinitely, though the workload is still inconsistent with bursts of heavy work loads Consensus is that the team is working at a pace that is sustainable indefinitely, though there is still occasional crunch time Steps are actively taken by the organization and the team to make sure that the team has a high morale, works no more than 40 hours per week, takes all vacation days and is a high performing team
Multi-team synchronization 0 Synchronization with other teams that are working together toward a common deliverable is either ad-hoc or done primarily through up-front planning and setting iteration content towards a planned integration phase Team synchronizes with other teams using at least one of the methods in level 4 Team uses as least 2 of the methods from level 4 and integrates their work with the whole product at least every 4 weeks Team uses at least 3 of the methods in level 4 or equivalent, and there is little or no pre-planning of iteration content towards a planned integration phase (integration is continuous) Team synchronizes with other teams by having common iteration start/stop dates (or use Kanban), standup-of-standups (or similar), retro-of-retros (or similar), and participates in a whole-product review on a frequent basis.
Impediments 0 Invisible and/or ignored. Fear of reprisals. Reluctance to raise impediments. Impediments that are raised are not resolved. Raising impediments is actively encouraged and is frequently done. Some impediments are resolved. The team is beginning to see the benefits of this practice and feel comfortable practicing it. Raising impediments is becoming routine and there is a high degree of comfort in doing it. Impediments are usually resolved. Root cause analysis is sometimes performed and there is a growing recognition of the value of raising impediments. Impediment raising and resolution are a cultural norm. Individual and team impediments that can be addressed at those levels are addressed. Root cause analysis is frequently performed and acted on. Root cause analysis and resolution is a cultural norm
Product Shippability 0 No stories shippable in less than four weeks from ready to done or not measured Shippability is measured and visible Team strives for shippability 60% of story points go from ready to done in less than four weeks 90% of story points go from ready to done in less than two weeks
End-to-end cycle time 0 A year or more from concept to ready to release Can get from concept to ready to release in 6 months Can get from concept to ready to release in 3 months Can get from concept to ready to release in weeks Days from concept to ready to release
Product vision 0 Not defined It is written down somewhere or the product owner or similar person knows what it is There is a written definition which is accurate and well known by everyone involved There is a compelling product vision which can be clearly articulated by the product owner or similar person Simple, clear, compelling, everyone involved can articulate it well.
Stories follow INVEST 0 No knowledge of INVEST Team understands INVEST and is starting to follow parts of it on some stories. Following most of INVEST on many stories Following INVEST for most stories Following INVEST for all stories
Definition of ready 0 Does not exist There is an understanding of the need for a definition of ready and/or there is a tacit agreement for the content of one There is a fairly good definition of ready which resulted from the collaboration between multiple members of the team. Definition of ready includes existence of acceptance criteria There is a strong, clear, comprehensive (yet simple) definition of ready which resulted from the collaboration of most of the members, agreement and input from all, and it is publicly posted In place, comprehensive, periodically reviewed and updated, strictly followed
Definition of done 0 Does not exist There is an understanding of the need for a definition of done and/or there is a tacit agreement for the content of one There is a fairly good definition of done which resulted from the collaboration between multiple members of the team There is a strong, clear, comprehensive (yet simple) definition of done which resulted from the collaboration of most of the members, agreement and input from all, and it is publically posted In place, comprehensive, periodically reviewed and updated, strictly followed
Story size 0 Random The team is starting to see the relationship between small stories and success. Team has a rule of thumb encouraging small stories Most stories can be done in a week or less Most stories shippable in 1-3 days
Backlog grooming 0 Stories are rarely ready to be worked on prior to the team starting to work on those stories It is understood that consistent and frequent grooming is an important goal and steps are being taken to get there. 60%+ of the time there are stories ready when needed There are usually just enough stories ready There are always more than enough stories ready
Stories use vertical slices 0 No knowledge of vertical slices or they can't be done due to external constraints Using vertical slices for an increasing percentage of stories Using vertical slices for 50%+ of stories Using vertical slices for 70%+ of stories Using vertical slices for 90%+ of stories
Work in progress 0 Amount of WIP unknown. No knowledge of one piece flow (e.g. small batch size) WIP is tracked and visible. One piece flow is understood and there is interest in doing it. Most of the time, members are working on 2 or more stories at a time. One piece flow is actively being pursued, WIP limits are set, most of the time members are working on at most 2 stories and usually only one. Sometimes, multiple members are working on the same story. WIP limits are set and respected. Most of the time members are only working on one story and frequently more than one member is working on the same story. Only as much work that can be done simultaneously without increasing the cycle time of any of the work in progress. Most of the time multiple members are working on the same story.
Agile Process Mechanics Standups, check-ins, huddles, or similar. 0 Not being held Being held regularly and on their way to stage 2. 80% of the team participates on a regular basis, the main meeting is less than 20 minutes, real impediments are raised on a regular basis, the focus is on the stories for this team, the team understands that the meeting is for them. Daily, short, effective. Runs well with or without somebody officially responsible for the meeting. Team does an on-the-spot analysis of progress towards shippability and takes corrective action if needed. Positively adapted to the needs of the team
Retrospectives 0 Not being held Held, but not regularly or not frequently enough Held regularly, well attended, produces action items. Action items are frequently acted on Held regularly, well attended, enjoyable, produces action items that are recorded and generally acted on Creatively run, format varied from time to time, forward looking, often produces breakthrough ideas that are acted on and produce results
All work based on user stories 0 Not being followed It is understood that it is important to use user stories for all work and steps are being taken to get there. User stories exist for 50%+ of the work, but still using other artifacts for some work or translating some user stories to other artifacts for some work. User stories exist for 80%+ of work, but still using other artifacts for some work or translating some user stories to other artifacts for some work. All work based on user stories
Estimation 0 Ad-hoc, done by a few people, based on hours, or entirely task-based Done on a regular basis The whole team participates in estimation, real story points are used. Most team members no longer thinking in hours. 90+% of the time estimation involves the whole team thinking in story points. Consistently done at least weekly by the whole team thinking in story points.
Progress tracking 0 Not implemented Progress is tracked and known using burnup, burndown, CFD or similar method and sometimes iinfluences behavior of the team. Progress is tracked and frequently influences the behavior of the team Progress information usually influences the behavior of the team The team proactively uses progress information to head off potential problems
Reviews 0 Not happening, not happening on a regular basis, or happening less often than once in 6 weeks Happening at least once every six weeks, but some or all of the following are happening: not reviewing all stories, ill-prepared to do the review, trying to "sell" what was done as opposed to finding missed expectations and encouraging feedback Happening at least once every four weeks, most stories are reviewed, team is fairly well prepared, feedback is encouraged and incorporated into future stories Reviews are a cultural norm. Every story is reviewed and the team is very well prepared. Active feedback is encouraged, the reviews are well attended and perceived as valuable to stakeholders. The team proactively involves stakeholders on a regular basis and frequently delights stakeholders during reviews. The team and stakeholders work closely together and often discover unexpected value as a result of that interaction.
Agile Engineering Practices Architecture 0 Primarily done by designated architects up-front prior to implementation Team starting to work with architects and architects starting to delegate more decisions to the team 50% of architectural decisions made by the team. 50% of architectural decisions made just-in-time 80% of architectural decisions made by the team. 80% of architectural decisions made just-in-time Primarily done on a just-in-time basis by the team in consultation with the architecture team.
Proximity of testing to implementation 0 Long after implementation Within eight weeks Mostly within four weeks Mostly within two weeks and mostly before the next story is started For software projects, TDD with UI-based testing done immediately after story is coded
Holistic testing (sw projects) 0 Different kinds of testing (unit, functional, integration, etc.) all done without coordination There is a recognition that holistic testing is a good thing and steps are being taken to move towards it. For 50%+ of user stories, the developers and testers coordinated their testing efforts For 80%+ of user stories, the developers and testers coordinated their testing efforts All testing coordinated ahead of coding and based around user stories
Test automation (sw projects) 0 Not being used 30%+ code coverage via test automation and plans are in place to increase this level 50%+ code coverage for all new user stories via test automation 50%+ code coverage via test automation 90% + code coverage via test automation
Continuous Integration (sw projects) 0 Not implemented Set up, but manually run. Failures not fixed right away. Run every hour. Failures fixed fairly quickly. Run every 10 minutes. Drop everything on failures until fixed. Run on every check-in.
Unit testing (sw projects) 0 Not being used Some coding involves unit testing. There is an understanding that unit testing produces better code and reduces overall effort All new stories involve some amount of unit testing All new stories involve the responsible amount of unit testing. Unit testing of stories included in the definition of done. Hard to imagine a shop that is better at unit testing. Deep knowledge of the latest unit testing techniques, using mock objects, etc.
Refactoring (sw projects) 0 Not understood and/or not being done Some understanding of single responsibility principle (SRP) and open/closed principle. Some amount of refactoring done as needed when implementing stories. Refactoring around SRP and O/C principle. Doing the appropriate amount of refactoring with most user stories Deep understanding of refactoring. True refactoring is a cultural norm. Hard to imagine a shop that is better at refactoring. Deep knowledge of the latest refactoring techniques. Refactoring to patterns.


Nokia Test as updated by Citrix and Scrum Inc. v. 2013-7-31
• As an agile citizen, I can assess a team’s behavior and compare it to current Scrum best practices, so I can consider changes that might increase productivity.

Instructions
• Each person on a team should have a piece of paper
• There are 10 assessments
• Each assessment has a score from 0 to 10
• In each assessment, sum the “Acceptance Tests” scores that pass.
• Total score will range from 0 to 100
• Average the scores for team members to get the team score

Assessment 1: Iteration
As a team, before we commit to a Sprint, we know its duration, so we deliver better rhythmic, synchronized value.
Acceptance Tests:
Variable, 4 < duration ≤ 6 weeks: 2
Variable, duration ≤ 4 weeks: 4
Constant for last 3 iterations, duration = 1 month: 5
Constant for last 3 iterations, duration = 4 weeks: 6
Constant for last 3 iterations, duration = 3 weeks: 8
Constant for last 3 iterations, duration ≤ 2 weeks: 10

Assessment 2: In-Sprint Testing
As a team, we take joint responsibility for all testing, so our Sprint product has sufficient quality to be immediately deployable.
Acceptance tests (sum):
Team creates some unit tests in-sprint: 1
Team creates unit tests for each story in-sprint: 1
Team tests each story prior to Sprint Review: 2
Team tests each story immediately after coding: 2
Team automates feature tests for each new story: 2
Build system packages, deploys to stage or live, and runs all automated feature tests at least every 24 hours: 2

Assessment 3: Sprint Stories
As a team, we commit to work only when backlog items conform to a Definition of Ready, so we generate business value fast.
Acceptance Tests (sum):
Sprint requirements are documented: 1
Requirements are independent, well-prioritized stories: 1
Stories start with this: “As a <stakeholder>, I can <do something>, so <business gains value>”: 2
Stories have externally verifiable acceptance tests: 2
Team has a written, enforced Story Definition of Ready: 2
Team has a written, enforced Story Definition of Done: 2

Assessment 4: Product Owner
As a team, a single Product Owner helps the team understand and prioritize value, so we generate profits long-term.
Acceptance tests (sum):
A single external person (PO) prioritizes work: 2
PO interrupts team work only during Scrum meetings: 2
PO attends all Planning, Grooming, Review and most Standups: 2
PO creates a product backlog, with stories estimated by the team before Sprint Planning: 1
PO maintains a velocity-aware release roadmap: 1
PO motivates team to reduce technical debt: 2

Assessment 5: Product Backlog
As a team, we have a value-ranked backlog, so we can focus on work that will generate the most business value for the least effort.
Acceptance Tests (sum):
Team serves multiple prioritized Product Backlogs: 1
Team serves a single prioritized Product Backlog: 2
PO regularly discusses release burndown with team, and adjusts backlog priorities based on historic velocity: 1
Stories more than 3 months out trend larger in effort: 1
Team can explain the ROI of each story: 1
PO assesses value (NPV, buy-a-feature) to rank stories: 2
PO prioritizes cheap prototypes to test value early: 2

Assessment 6: Estimation
As a team, our estimates are largely free of statistical bias, so stakeholders can reley on release forecasts and make more money.
Acceptance Tests (sum):
Team agrees to estimates before committing: 1
PO, SM and non-developers do not supply estimates: 1
Team carefully avoids anchor bias before estimation: 1
Representatives or actual team creates poker estimates: 1
Actual team creates poker estimates: 2
Teams use reference stories to make estimates: 2
Actual velocity is < +/-20% of estimated velocity: 2

Assessment 7: Sprint Burndown
As a team, we know our progress toward completion of backlog items, so members can help with high-priority work-in-progress.
Acceptance Tests (sum):
Burndown exists, team knows where it is: 1
Team reviews, adjusts tasks and burndown daily: 1
Tasks have hour or point estimates estimates (or team makes tasks about the same size): 2
Tasks burn down only after whole task is done: 2
Stories burn down (no tasking) after whole story is done: 2
All team members know team’s historic Velocity: 1
Team commits to sprint backlog at or below Velocity:1

Assessment 8: Retrospection
As a team, we review our processes, so we can sustainably improve productivity.
Acceptance Tests (sum):
Team conducts retrospectives at least every 2 months: 2
Team conducts retrospectives after each Sprint: 2
Team limits retrospective participation to team and SM.
Team optionally invites PO and others or uninvites SM: 2
Team uses sticky-notes/other tools to ensure all members participate and tracks followup: 2
Team puts top process improvement in the backlog for next sprint with acceptance tests: 2

Assessment 9: Scrum Master
As a team, the Scrum Master (SM) competently enforces process, removes impediments, and provides transparency, so we can focus well.
Acceptance Tests (sum):
SM understands Scrum and agile concepts deeply: 2
SM performs no tasks in the Sprint: 1
SM enforces rules established by the team: 1
SM sees impediments early, and handles for the team: 2
SM maintains and uses a prioritized impediments list: 1
SM makes team’s progress transparent to outsiders: 2
SM communicates well with team, other teams, managers, stakeholders and PO: 1

Assessment 10: Team
As a team, we work together effectively to releasing our software, so we can get software to users earlier and adapt rapidly.
Acceptance tests (sum): 3 ≤ team size without counting SM or PO ≤ 7: 2
Team members volunteer (are not assigned) to tasks: 2
At least 2 members can independently finish each task: 2
Team collectively commits to Sprint goal and backlog: 1
Team collectively fights impediments in-sprint: 1
Team reduces technical debt every sprint: 2


An Example Checklist for ScrumMasters
Michael James
(mj4scrum@gmail.com)
14 September 2007
(Revised 2 Feb 2016)
A Full Time Facilitator?
An adequate ScrumMaster can handle two or three teams at a time. If you're content to limit your role to
organizing meetings, enforcing timeboxes, and responding to the impediments people explicitly report, you can
get by with part time attention to this role. The team will probably still exceed the baseline, pre-Scrum expectation
at your organization, and probably nothing catastrophic will happen.
But if you can envision a team that has a great time accomplishing things no one previously thought possible,
within a transformed organization, consider being a great ScrumMaster.
A great ScrumMaster can handle one team at a time.
We recommend one dedicated ScrumMaster per team of about seven when starting out.
If you haven't discovered all the work there is to do, tune in to your Product Owner, your team, your team's
engineering practices, and the organization outside your team. While there's no single prescription for everyone,
I've outlined typical things I've seen ScrumMasters overlook. Please mark each box with √, Δ, ?, or N/A, as
described on the last page.
Part I -- How Is My Product Owner Doing?
ScrumMasters improve Product Owner effectiveness by helping them find ways to maintain the Product Backlog
and release plan. (Note that the Product Owner is the one responsible for the prioritized backlog.)
☐ Is the Product Backlog prioritized according to his/her latest thinking?
☐ Are requirements and desirements from all stakeholders captured in the Product Backlog? Remember: the
backlog is emergent.
☐ Is the Product Backlog a manageable size? To maintain a manageable number of items, keep things more
granular towards the top, with general epics at the bottom. It's counterproductive to overanalyze too far past
the top of the Product Backlog. Your requirements will change in an ongoing conversation between the
developing product and the stakeholders/customers.
☐ Could any requirements (especially those near the top of the Product Backlog) be better expressed as
independent, negotiable, valuable, estimable, small, and testable user stories1?
☐ Have you educated your Product Owner about technical debt and how to avoid it? One piece of the puzzle
may be to write automated test and refactoring into the definition of "done" for each backlog item.
☐ Is the backlog an information radiator, immediately visible to all stakeholders?
☐ If you're using an automated tool for backlog management, does everyone know how to use it easily?
Automated management tools introduce the danger of becoming information refrigerators without active
radiation from the ScrumMaster.
1 http://xp123.com/articles/invest-in-good-stories-and-smart-tasks/
Copyright © 2007-2016 Michael James. All Rights Reserved.
☐ Can you help radiate information by showing everyone printouts?
☐ Can you help radiate information by creating big visible charts?
☐ Have you helped your Product Owner organize backlog items into appropriate releases or priority groups?
☐ Does everyone know whether the release plan still matches reality? You might try showing everyone Product/
Release Burndown Charts after the items have been acknowledged as “done” during 2 every Sprint Review
Meeting. Charts showing both the rate of PBIs actually completed and new ones added allow early discovery
of scope/schedule drift.
☐ Did your Product Owner adjust the release plan after the last Sprint Review Meeting? The minority of Product
Owners who ship adequately tested products on time re-plan the release every Sprint. This probably requires
deferring some work for future releases as more important work is discovered.
Part II -- How Is My Team Doing?
While you are encouraged to lead by the example of collaborating with team members on their work, there is a
risk you will get lost in technical tasks. Consider your primary responsibilities to the team:
☐Is your team in the state of flow? Some characteristics of this state3:
• Clear goals (expectations and rules are discernible and goals are attainable, aligning appropriately with
one's skill set and abilities).
• Concentration and focus, a high degree of concentration on a limited field of attention.
• A loss of the feeling of self-consciousness, the merging of action and awareness.
• Direct and immediate feedback (successes and failures in the course of the activity are apparent, so that
behavior can be adjusted as needed).
• Balance between ability level and challenge (the activity is neither too easy nor too difficult).
• A sense of personal control over the situation or activity.
• The activity is intrinsically rewarding, so there is an effortlessness of action.
☐ Do team members seem to like each other, goof off together, and celebrate each other's success?
☐ Do team members hold each other accountable to high standards, and challenge each other to grow?
☐ Are there issues/opportunities the team isn't discussing because they're too uncomfortable?4
☐ Have you tried a variety of formats and locations for Sprint Retrospective Meetings?5
☐ Has the team kept focus on Sprint goals? Perhaps you should conduct a mid-Sprint checkup to re-review the
acceptance criteria of the Product Backlog Items committed for this Sprint.
2 Mike Cohn, Agile Estimation and Planning. (2005).
3 Mihaly Csikszentmihalyi, Flow: The Psychology of Optimal Experience (1990).
4 Marshall Rosenberg, Nonviolent Communication: A Language of Life: Life-Changing Tools for Healthy Relationships (2003).
Also consider enlisting a professional facilitator who can make uncomfortable conversations more comfortable.
5 Derby/Larson Agile Retrospectives: Making Good Teams Great (2006).
Copyright © 2007-2016 Michael James. All Rights Reserved.
☐ Does the Sprint taskboard reflect what the team is actually doing? Beware the “dark matter” of undisclosed
tasks and tasks bigger than one day’s work. Tasks not related to Sprint commitments are impediments to
those commitments.
☐ Does your team have 3-9 people with a sufficient mix of skills to build a potentially shippable product
increment?
☐ Is your team's taskboard up to date?
☐ Are the team self-management artifacts visible to the team, convenient for the team to use?
☐ Are these artifacts adequately protected from meddlers? Excess scrutiny of daily activity by people outside
the team may impede team internal transparency and self management.
☐ Do team members volunteer for tasks?
☐ Has the need for technical debt repayment been made explicit in the definition of done, gradually making the
code a more pleasant place to work?
☐ Are team members leaving their job titles at the door of the team room, collectively responsible for all aspects
of agreed work (testing, user documentation, etc.)?
Part III -- How Are Our Engineering Practices Doing?
☐ Does your system in development have a "push to test" button allowing anyone (same team or different team)
to conveniently detect when they've caused a regression failure (broken previously-working functionality)?
Typically this is achieved through the xUnit framework (JUnit, NUnit, etc.).
☐ Do you have an appropriate balance of automated end-to-end system tests (a.k.a. "functional tests") and
automated unit tests?
☐ Is the team writing both system tests and unit tests in the same language as the system they're developing?
Collaboration is not enhanced by proprietary scripting languages or capture playback tools that only a subset
of the team knows how to maintain.
☐ Has your team discovered the useful gray area between system tests and unit tests6?
☐Does a continuous integration7 server automatically sound an alarm when someone causes a regression
failure? Can this feedback loop be reduced to hours or minutes? ("Daily builds are for wimps." -- Kent Beck)
☐ Do all tests roll up into the continuous integration server result?
☐Have team members discovered the joy of continuous design and constant refactoring8, as an alternative to
Big Up Front Design? Refactoring has a strict definition: changing internal structure without changing external
behavior. Refactoring should occur several times per hour, whenever there is duplicate code, complex
conditional logic (visible by excess indenting or long methods), poorly named identifiers, excessive coupling
between objects, etc. Refactoring with confidence is only possible with automated test coverage. Neglecting
6 http://blogs.collab.net/agile/junit-is-not-just-for-unit-testing-anymore
7 http://www.martinfowler.com/articles/continuousIntegration.html
8 Martin Fowler, Refactoring: Improving the Design of Existing Code (1999).
Copyright © 2007-2016 Michael James. All Rights Reserved.
refactoring makes it hard to change the product in the future, especially since it’s hard to find good developers
willing to work on bad code.
☐ Does your definition of "done" for each Product Backlog Item include full automated test coverage and
refactoring? Learning Test Driven Development (TDD) increases the probability of achieving this.
☐ Are team members pair programming most of the time? Pair programming may dramatically increase code
maintainability and reduce bug rates. It challenges people's boundaries and sometimes seems to take longer
(if we measure by lines of code rather than shippable functionality). Lead by example by initiating paired
workdays with team members. Some of them will start to prefer working this way.
Part IV -- How Is The Organization Doing?
☐ Is the appropriate amount of inter-team communication happening? “Scrum of Scrums” is only one way to
achieve this, and rarely the best.9
☐ Are teams independently able to produce working features, even spanning architectural boundaries?10
☐ Are your ScrumMasters meeting with each other, working the organizational impediments list?
☐ When appropriate, are the organizational impediments pasted to the wall of the development director's office?
Can the cost be quantified in dollars, lost time to market, lost quality, or lost customer opportunities? (But
learn from Ken Schwaber's mistakes: "A dead ScrumMaster is a useless ScrumMaster."11)
☐ Is your organization one of the few with career paths compatible with the collective goals of your teams?
Answer "no" if there's a career incentive12 to do programming or architecture work at the expense of testing,
test automation, or user documentation.
☐ Has your organization been recognized by the trade press or other independent sources as one of the best
places to work, or a leader in your industry?
☐ Are you creating a learning organization?
Conclusion
If you can check off most of these items and still have time left during the day, I’d like to hear from you.
There’s no canned formula for creating human ingenuity. This paper lists points which may, or may not, help in
your situation.
Once you start to realize what you could do to make a difference, you may find yourself afraid to do it. This is a
sign you're on the right track.
9 See http://less.works/less/framework/coordination-and-integration.html for alternatives.
10 http://FeatureTeamPrimer.org/
11 Ken Schwaber, Agile Project Management with Scrum (2004)
12 Alfie Kohn, Punished By Rewards: The Trouble with Gold Stars, Incentive Plans, A's, Praise, and Other Bribes (1999)
Copyright © 2007-2016 Michael James. All Rights Reserved.
Organizational Impediment Form
Surface issue:
Root cause (Use five times “Why?”):
Business Impact:
Emotional Impact:
Clear, actionable request:
Organizational Impediment Form
Surface issue:
Root cause (Use five times “Why?”):
Business Impact:
Emotional Impact:
Clear, actionable request:
Copyright © 2007-2016 Michael James. All Rights Reserved.
Organizational Impediment Form
Surface issue:
Root cause (Use five times “Why?”):
Business Impact:
Emotional Impact:
Clear, actionable request:
Organizational Impediment Form
Surface issue:
Root cause (Use five times “Why?”):
Business Impact:
Emotional Impact:
Clear, actionable request:
Copyright © 2007-2016 Michael James. All Rights Reserved.
Organizational Impediment Form
Surface issue:
Root cause (Use five times “Why?”):
Business Impact:
Emotional Impact:
Clear, actionable request:
Organizational Impediment Form
Surface issue:
Root cause (Use five times “Why?”):
Business Impact:
Emotional Impact:
Clear, actionable request:
Copyright © 2007-2016 Michael James. All Rights Reserved.
INSTRUCTIONS
If you have received this checklist as a training assignment and your current
(or most recent) employer has been attempting anything like Scrum, please
apply this to what you’ve seen there. Mark each item with one of the
following:
√ (for “doing well”)
Δ (for “could be improved and I know how to start”)
? (for “could be improved, but how?”)
N/A (for “not applicable” or “would provide no benefit”)
Or, if your current (or most recent) employer has not been attempting
anything like Scrum, mark each item with one of the following:
√ (for “doing well” or “would be easy to do well”)
Δ (for “would be a challenge and I know how to start”)
? (for “would be a challenge and I don’t know how to start”)
N/A (for “not applicable” or “would provide no benefit”)
When all items are marked, declare 2-6 organizational impediments on the
attached Organizational Impediment Forms, whether or not they’re derived
from this checklist. Choose impediments you have at least 1% hope of
changing.
Copyright © 2007-2016 Michael James. All Rights Reserved.