The process of creating a software product is also a process of learning. If for example you are creating a mobile application that allows nurses to gather information about patients, you will eventually learn:
-
Whether the nurses, hospital or other stakeholders find the application useful or a waste of time.
-
Whether your architecture scales to tens or hundreds or thousands of users.
-
Whether the nurses understand the user interface and can use it correctly.
Feedback is a key ingredient in learning. Without feedback you will have a hard time learning what specific application design, architecture and user interface will succeed. If the feedback is positive then all is well and you can continue on your current path. If, as is more likely, the feedback is negative you will need to assimilate the new information, learn from your mistakes and make the necessary changes.
Slow feedback restricts you to learning slowly. If it takes you two years to learn that your architecture doesn’t scale then you have a problem. You might be able to fix things at some expense, or perhaps the project will fail. What’s more, you’ve just spent two years only to learn one single issue with scaling. At this rate you will only learn about 20 things about scaling in your professional lifetime.
Fast feedback allows you to learn quickly. Imagine if you could spend a day’s effort when the architecture is first conceived to check if it will scale. You could then come up with an alternative architecture, test each in turn, learn from the results and repeat until you’ve found the solution. Not only have you saved the project in advance from one form of failure, you’ve also learned far more in a far shorter period of time.
Release early, release often
Under certain restricted but common circumstances you can receive early and rapid feedback by releasing your software early and often. Feedback received from real usage is the most valuable as it is feedback specifically about the usage you care about most. By releasing your software as quickly as possible, and then continuously updating as you address feedback and try new things, you can quickly learn what works and what doesn’t.
Early releases require:
-
A product that is usable even when a subset of functionality is missing or broken. For example, a video game may ship with some features disabled or areas locked: later updates can add the content when it’s ready. On the other hand the software controlling a rocket’s takeoff needs to be fully functional: failed rocket launches are expensive and potentially fatal. Rapid releases can make this condition easier to meet.
-
Knowledge that scaling will not be an issue. For example, if the application can’t handle the expected user load then there’s no point in releasing it. Some applications only run on a single device and therefore have no scaling requirements. Others need to scale but are sufficiently standardized that the technologies for scaling are widely available and understood, e.g. content-distribution networks for websites where only read-only queries need to scale. If scaling may be an issue and the architecture is not well understood, restricted sign-ups (or "closed beta") can allow the application to launch even in the face of uncertainty.
-
Willingness by user to accept change. A software library that guarantees backwards compatibility might suffer from permanent design flaws if you release it too early and users rely on its original design.
Rapid releases require:
-
Technical infrastructure that allows for rapid updates. A website can build on continuous deployment infrastructure to automatically update the website every time developers commit a change to the source code. In contrast computer server vendors upload firmware updates to their website and then rely on voluntary human intervention by their customers to apply them. Of course if you need to implement custom infrastructure for rapid releases this may make it impossible to release early.
-
A development process designed for rapid updates. A development team that does four weeks of manual quality assurance before releases cannot have more than one release a month.
If you can release early and often the chapter on Real-World Testing will guide you further in making the most out of the feedback you receive.
Prototyping and modeling
While not all applications meet the requirements for early and rapid releases all is not lost. You can still speed up your feedback process, and therefore learn more quickly, by building models and prototypes. These terms are sometimes used interchangeably but here we’ll use them to mean two distinct modes of investigation.
-
A model is a simplified and abstracted aspect of the design, allowing you to reason about it in isolation. For example, you can build a performance model with a tool like PDQ to determine the scalability of a design. The model will only focus on processing speed and connectivity of the different parts of the architecture, omitting all other aspects of the design.
-
A prototype is a minimal implementation of at least part of the design. You can prototype the user interface with sketches and run them by users. You can code just enough of the logic to understand whether or not a design will support the requirements. You can implement a proof-of-concept and demonstrate it to potential customers to see if they might purchase it.
Why should you consider prototyping and modeling?
Who is it for: Organization, development team and users can all benefit.
Strengths: Faster feedback allows the organization and development team to learn faster, and reduces the chance of discovering design mistakes late in the development cycle, when fixing it will be costly. Prototyping and modeling may also result in a product that better meets user needs, as the development team uses them as a filter to reject bad designs.
Weaknesses: Feedback from prototyping and modeling might be inaccurate, since prototypes and models are never a complete picture of the final result.
Costs: Prototyping and modeling add additional time to delivery. The development team should therefore scale their use to the scope of the project, e.g. up to half a day for a two week project and weeks for a year-long project.
Example
Let’s return to our hypothetical example of a mobile data entry application for nurses and see how prototyping and modeling can help ensure a good design upfront. The initial idea is that you will give each nurse a personal tablet that they will carry with them at all times.
To test out this idea you prototype the user experience: you find a small number of volunteer nurses from different parts of the hospital, hand them a tablet and ask them to enter some patient data into a website form every 15 minutes. You then follow them at a distance and observe their usage. It turns out carrying a tablet is going to be a problem: the nurses often need two hands, and end up putting down the tablet and then forgetting it. Data entry is also a problem since data is apparently often copied from charts; the nurses need one hand to hold the chart, one hand to hold the tablet and a hypothetical third arm to enter the data. Finally, the nurses report problems with the WiFi network at the hospital; it appears to have bad reception in much of the hospital.
Half a day’s observation has ruled out your initial design. The next idea to consider is placing a tablet in each room used by the nurses, attached to the wall so they don’t have to hold it. Beyond additional prototyping of the user experience this also suggests the need for some cost modeling: some back of the envelope calculations of the cost of upgrading the WiFi network vs. using other forms of networking made possible by a fixed physical location. Threat modeling is also needed to flesh out the design. As the tablet will now be physically accessible to patients, visitors and other staff when the nurses are elsewhere you need to the security implications of this changed design and how you might deal with them.
Continual prototyping
Existing projects can also benefit from modeling and prototyping. Every time you implement significant new feature it’s worth considering if these techniques will be useful.
Prototyping a new feature by implementing just enough logic to demonstrate its feasibility is a great way to learn how to build something new. For example, a product I worked on originally allowed moving a filesystem from one machine to another by copying the data. We wanted to extend it to support environments where moving a filesystem would not involve copying the data, but rather detaching a network based volume from one machine and re-attaching it to the destination. A team of two developers prototyped this feature over the course of a few days as quickly as possible, cutting every corner they could. Their implementation demonstrated the idea was possible and found some of the problems involved: choice of libraries, unexpected slowness in cloud environment operation, and more. We then threw this code away, and based on what they’d learned re-implemented this feature the right way: a suitable architecture, error handling, good test coverage and so on.
Modeling can also be helpful in determining where to focus development efforts. For example, a developer needed to implement fuzzy search for company names: a search for "Godwin" should bring up the entry for "Goodwin Procter LLP". Playing around with an online comparison of similarity metrics she found that the Jaro-Winkler distance algorithm was well-suited to this problem. This algorithm requires pair-wise comparison between two strings which would require iterating over all company names in the database. An estimate of the number of companies in the database and the number of fuzzy searches suggested this would not be a significant performance bottleneck, and so she chose the simple solution of fetching all company names from the database to the application server.
Conclusion
Rapid feedback both reduces wasted resources and improves your ability to learn. When possible rapid and early releases are the ideal form of feedback, as they allow you to learn from actual real-world usage. For all projects prototyping and modeling can provide fast feedback that is less accurate, but also more focused on specific dimensions of the design problems you face.