When it comes to feature testing, you\u2019re in a bind.<\/p>\n\n\n\n
On the one hand, you need real-world data and feedback from real-world users.<\/strong> You know that every new feature you develop is, at best, an educated guess about what your real-world users want from you. No matter how educated that guess might be, and no matter much internal validation you perform, you can only generate meaningful data and feedback on each new feature you develop by releasing it to real-world users to test out in their real-world environments.<\/p>\n\n\n\n On the other hand, it\u2019s risky to give real-world users an unproven feature. <\/strong>You know that every new feature you release might have something wrong with it. Maybe there\u2019s a technical issue you missed during development. Maybe it just doesn\u2019t align to user expectations as closely as you hoped. No matter the issue, releasing an unproven feature can cause real harm to your brand\u2019s user relationships.<\/p>\n\n\n\n This is a tricky problem and one that is never going to be fully solved. But, thankfully, there are methods you can follow to minimize the problem, and collect real-world data and feedback while mitigating the impact when something (inevitably) goes wrong. <\/p>\n\n\n\n In this piece, we\u2019ll explore one of these methods\u2014 rollbacks. <\/p>\n\n\n\n It\u2019s a simple practice, with powerful implications.<\/p>\n\n\n\n When you perform a rollback, you take some code out of a live environment. Back in the day rollbacks could be truly massive. Software products used to be updated in giant new releases that could include a wide range of changes\u2014 including multiple new features and significant changes to existing features. If one of these huge releases had some fatal bugs in it, or just wasn\u2019t well-received by users, then the entire thing might need to be rolled back (even if the issues were contained within just a few elements of the release).<\/p>\n\n\n\n All of this has changed with the adoption of Agile methodology. Releases keep getting smaller and more incremental, and so do rollbacks. Most modern Product Managers have adopted phased release plans, where they only release a single new or upgraded feature at a time\u2014 and often only to individual segments. And when modern Product Managers do release multiple new or upgraded features at once, the different features are separate from each other.<\/p>\n\n\n\n This evolution has changed the way rollbacks happen. After a new release, Product Managers can now isolate the individual feature(s) that have proven unfit for live usage and perform a targeted rollback on them, and them alone. The whole rollback process is now much faster, much nimbler, much more precise\u2014 and delivers much greater benefits.<\/p>\n\n\n\n When a Product Manager properly structures and deploys rollbacks, they improve their ability to test new features in a real-world environment<\/a> with real-world users with a minimal level of risk. An imperfect feature is no longer the end of the world. If a feature has development issues or poor alignment with user requirements, you can perform a rollback and remove it from a live environment in real-time with just one click.<\/p>\n\n\n\n For Product Managers, this changes the game. The more mature your rollback capability, the more you can afford to make mistakes. Your risk shrinks, giving you the freedom to test more features with more users earlier in the development cycle, ultimately leading you to iterate your products faster and faster.<\/p>\n\n\n\n Now, rollbacks are not a silver bullet. They don\u2019t absolve you from doing everything you can to develop the highest-quality features possible before you test them. But rollbacks allow you to test new features with greater confidence and reduced concerns about creating problems for your users. <\/p>\n\n\n\n For most Product Managers, there are two common use cases why you might need to perform a rollback.<\/p>\n\n\n\n This first use case is pretty self-explanatory.<\/p>\n\n\n\n You might have the most robust and thorough QA and testing processes in the world. It\u2019s still highly likely your new features will still have one or more bugs in them when you release them into a live environment. Maybe they\u2019re issues you just didn\u2019t think to search for or didn\u2019t know how to search. Maybe they\u2019re issues that only show up in live environment after hundreds of real-world users tool around with the feature.<\/p>\n\n\n\n Regardless of the reason, if significant technical issues pop up in your new feature, then you\u2019ll likely want to perform a rollback on that feature to fix it. With the right rollback process, you can react to these errors in near-real-time and remove the feature\u2014and maybe even fix it\u2014in minutes before it impacts too many users. <\/p>\n\n\n\n This second use case is a little more sophisticated.<\/p>\n\n\n\n Essentially, after you release a new feature you monitor how users respond to it, and how well it\u2019s hitting your business KPIs. If your new feature is not performing as expected, and is generating negative usage data and user feedback, then you can perform a rollback to remove it from its live environment. If it isn\u2019t hitting\u2014or at least tracking towards\u2014its business goals, then it might not be worth keeping live. With the right rollback process, you can also review and respond to the usage data and user feedback you receive in near-real-time, and prevent too many users from getting too disgruntled about receiving a feature that misses the mark.<\/p>\n\n\n\n In one word: speed. <\/p>\n\n\n\n In both use cases, rollbacks are most effective\u2014and mitigate the most risk\u2014when you are able to first monitor feature performance in real-time, to then translate that performance into a quick \u201cyes\/no\u201d decision to rollback (or not), and finally to execute on that rollback decision as rapidly as possible. <\/p>\n\n\n\n The faster you can go through this entire process, the lower the chance that you will create a prolonged negative user experience. In some scenarios, the decision to perform a rollback and the execution of that rollback need to happen in minutes. <\/p>\n\n\n\n It\u2019s a daunting mandate, but here are a few tips to help you meet it.<\/p>\n\n\n\n It\u2019s challenging to decide\u2014in the moment\u2014whether or not to rollback a feature. Even the best feature release can be complex and chaotic.<\/p>\n\n\n\n There are multiple moving parts to monitor\u2026<\/p>\n\n\n\n There are many different data and feedback points to take into consideration\u2026<\/p>\n\n\n\n And there\u2019s a lot of emotion at play\u2026<\/p>\n\n\n\n You and your team just spent weeks, maybe even months, pouring your blood, sweat, and tears into designing and developing the new feature that you\u2019re testing<\/a>. If your users love it, then you get that sugar high of knowing you just completed a job well done, and you can just sit back and watch the good data and feedback roll in. But if your users don\u2019t immediately respond as positively as you hoped, then it\u2019s easy to experience an emotional crash and to want to rollback the feature before you even know if the bad response is consistent, let alone what you should do to fix your errors in the next iteration. <\/p>\n\n\n\n For these reasons, and many more, it\u2019s hard to make the right rollback decision in the moment during a feature release test. Instead, it\u2019s better to make your rollback decisions before<\/em> you release your new features into the wild. <\/p>\n\n\n\n Here\u2019s what we mean.<\/p>\n\n\n\n Basically, before you release any new features to any real-world users, you first decide what success and failure looks like for this feature in objective, data-driven terms. <\/p>\n\n\n\n Then, you decide how much data your release will need to generate before you can make an accurate call about whether the feature is a success or a failure.<\/p>\n\n\n\n Finally, you use these parameters during your release to make objective \u201cyes\/no\u201d decisions about whether or not you should rollback your feature at any point. Instead of getting caught up in the moment, you just monitor the performance metrics that you decided were most important, and once they hit the thresholds you set prior to release you simply follow the plan and you either rollback the feature or you don\u2019t\u2014 no real-time agonizing required.<\/p>\n\n\n\n In the past, it was near-impossible to perform a rollback quickly from a technical perspective. You needed to have a technical team standing by, waiting to dig into the code to turn off live features, or to revert to a prior state of the entire platform. The entire process was slow, it was labor-intensive, and it took your technical teams away from their valuable development work. <\/p>\n\n\n\n Software has solved all of these problems. With our own feature management platform<\/a>, you can roll back a feature in real-time by just toggling a single field<\/a> with just one click. You don\u2019t need any technical expertise to do so. You don\u2019t need to develop and test a complex rollback process prior to feature release. You don\u2019t even need to think about the technical details\u2014 you can save all of that thinking to create the right strategic decision trees that we outlined in the prior section.<\/p>\n\n\n\nWhat is a Feature Rollback?<\/strong><\/h2>\n\n\n\n
Why Should Product Managers Perform Rollbacks?<\/strong><\/h2>\n\n\n\n
When Should You Perform a Rollback? Two Common Use Cases<\/strong><\/h2>\n\n\n\n
Rollback Use Case 1: Your Feature has a Bug<\/strong><\/h3>\n\n\n\n
Rollback Use Case 2: Your Feature is Poorly Received<\/strong><\/h3>\n\n\n\n
After you roll back your feature, you can either utilize the data and feedback you collected to fix the feature and help it better align to user expectations and business requirements, or you can decide that the feature was fundamentally misguided and just needs to be retired. <\/p>\n\n\n\nWhat Do These Two Use Cases Have in Common?<\/strong><\/h3>\n\n\n\n
How to Make Faster Rollback Decisions<\/strong><\/h3>\n\n\n\n
How to Execute Rollbacks Faster<\/strong><\/h3>\n\n\n\n