Working the Core Domain: Delivering Business Value by Shortening QA Feedback Cycles

By Robert Reppel

QA Feedback Cycles and Project Success

One of the truisms of delivering software intensive projects is that fixing a defect or changing the way a feature works is much more costly if done late during development.  Changes to systems which are already live are most expensive of all.  A key objective for planning such projects is therefore to shorten the time between a feature being specified and the time when QA and usability feedback about it is available.  Getting feedback as early as possible has many advantages:

• It maximizes opportunities for risk mitigation.
• It minimizes re-work because it keeps the development team from spending effort on refining functionality which will later turn out to require changes or replacement.
• Delayed feedback means that the code base with which developers have to work in order to incorporate changes will be larger than would otherwise be the case.  If a need to rework some business logic is identified three months after it has been specified it is likely that a lot more code will have been written in the meantime than would be the case if only a week had passed.  Working with a larger code base increases the risk of causing side effects in other parts of the system.  These side effects and the dependencies between system components which cause them are what make applications expensive to maintain and enhance. 

Approaches

Shortening the QA feedback cycle means frequent releases, ideally hand in hand with automated acceptance and unit tests which enable early detection of broken functionality.  In projects which involve a lot of legacy code automated testing can be very hard to do:  It often requires a considerable amount of refactoring to break dependencies before it is possible to start writing tests for individual business rules, etc.  Choosing what to refactor and developing meaningful tests can be a very worthwhile activity, leading to significantly better maintainability.  However, doing so in a way which maximizes the benefits is not easy:

• It requires a great deal of knowledge about test driven design and refactoring technique.
• Writing tests first and writing code to make them pass afterwards requires a very different mindset from the one we are used to in more “traditional” development.  The understanding of how to do this right (especially in the context of working with legacy systems) is still evolving, with lively debates and experimentation in the developer community.
• Because of the need to choose what to refactor and what to refactor it towards, moving to test-driven development requires time, commitment and ongoing involvement by business stakeholders. Direct contact between users of the system and developers (frowned upon in many organizations as “wasting developer time”) is starting to be recognized as particularly important for mitigating the most frequent cause of software project failure: Building the wrong thing.

If moving to full test-driven development is seen as impractical (if only in the short term), it may be possible to reap some of its benefits by re-thinking the software delivery process in order to achieve more frequent releases to manual QA.  Arguably, this represents a preparatory step towards test driven development because it will provide impetus to simplify deployment processes and break up large releases into smaller ones. It can also lead towards streamlined QA processes through better test plans and doing at least some automated regression testing through script based tools which simulate actions in the user interface, e.g. mouse clicks in browsers.

Identifying the Core Domain – The Role of Social Interactions and Organizational Culture

“In designing a large system, there are so many contributing components, all complicated and absolutely necessary to success, that the essence of the domain model, the real business asset, can be obscured and neglected.” – Eric Evans ¹

Shortening the QA feedback cycle requires decisions about the order in which features are to be developed.  Often this can be done using a combination of two strategies²:

• Develop highest risk and highest priority functionality first. Example: If integrations with third party applications are required, develop and test them early in the project on infrastructure as close to the live system(s) as possible. Likewise, focus first on features the users can’t possibly do without.
• Set-based design:  Leave critical, expensive-to-change decisions until the last possible moment in order to have more opportunity to evaluate alternatives. Example: You know that you have to receive notifications from and send content to a content management system.  To give stakeholders more time to select the best CMS product, you use mock objects and / or stubs to abstract and simulate communication with the future CMS while continuing to develop the rest of the system.

Developing highest risk/priority functionality first and leaving hard to change decisions for as long as possible is well and good, but how do you decide what those things are?  Leave it to technicians and your system integrations may be top of the list. Ask product managers and getting early feedback whether a user registration workflow is easy to use can turn out to be much more important. 

An additional complication is that the most senior technical talent naturally gravitates towards the infrastructure side of an application: From a developer’s point of view, coding business logic is relatively straightforward.  Getting plumbing such as database access strategies, use of various APIs and the technical details of a modern user interface right is much more complex.  Considering the usual hiring criteria for technical resources in the industry, it is also good for the resume: “Deep expertise with Windows Workflow Foundation and .NET 3.5” will beat “Thorough understanding of reinsurance industry practices and business rules” almost every time.

Building a shared understanding of what the core domain of a system is and delivering value for it as early as possible is clearly crucial.  Hard and fast answers for the best way to make the tradeoffs necessary to do it are hard to come by, but experience has shown that removing as many barriers to communication between business, users and developers is a key success factor.  The not always practical ideal is a co-located, cross functional team. This is because of Conway’s Law:

“…organizations which design systems … are constrained to produce designs which are copies of the communication structures of these organizations”  ³

Originally postulated in 1968 and often seen as somewhat tongue-in-cheek, the law has nonetheless turned out to have a distressing amount of predictive power.
Examples of Conway’s law in action:

• An organization has a Database / API Team and a Front End Team.  Getting frontend and backend integrated turns out to be challenging because the APIs almost, but not quite support what the front end developers need.
• When additional functionality for an existing class is needed, the original developer of the class is more likely to re-use it.  A new developer is more likely to re-implement it.

In both cases, what appear to be technical considerations are actually symptoms of how social interactions between people operate in the context of organizations and projects.

Summary

Shortening the time interval between specifying a feature and getting QA feedback on whether it has been successfully implemented is a key consideration for project success.

Test-driven development and the agile practices which enable it are considered to be an ideal approach for shortening the QA feedback cycle.  They are hard to do, requiring commitment and deep technical and business domain expertise.  One way to start realizing the benefits of timelier QA is to remove communications barriers between business stakeholders, end users and developers, enabling them to grow a shared understanding of the core business domain and value proposition of the system.