Climbing lessons for IT: Testing as safety anchor

In this article, I intend to delve deeper into the topic of risk management, which I have previously touched upon in this prior piece:

I will focus on drawing parallels between climbing protection systems and various methodologies employed in Information Technology (IT) testing. This comparative analysis aims to shed light on two crucial yet distinct aspects of effective risk management:

1. Depth represents mastery of the tools and techniques used to manage risks. Just as a seasoned climber must thoroughly understand the function, strengths, and limitations of each piece of climbing protection gear, individuals tasked with IT Testing need to master the tools and techniques available to them. This depth of understanding ensures that you can deploy these tools effectively and efficiently to mitigate risks.
2. Width: This emphasizes the importance of using a variety of tools suitable for the specific context or situation at hand. For instance, a climber will use different kinds of anchors for Trad, Sport, or Ice Climbing routes; similarly, an IT professional would use a combination of Testing approaches that best fit the project context.

Lastly, the goal is to integrate these two dimensions cohesively, creating a seamless approach to Testing. Much like a skilled climber relies on basic tools such as the climbing rope, carabiners, and harness to use the different types of anchors, the IT Professional will combine multiple Test types into seamless DevOps pipelines.

Depth dimensions: Unit testing example

To illustrate the depth dimension in risk management, let's examine the fundamental case of Unit Testing, drawing a parallel with Sport Climbing on routes safeguarded by fixed bolts. Here, each test is likened to a bolt anchor capable of arresting a fall, thereby preventing adverse outcomes.

Both approaches offer significant advantages by minimizing risks, thereby facilitating superior results. In climbing, they enable more challenging ascents and the creation of tougher routes. In IT project delivery, they enhance speed, quality, and predictability. However, expertise and training are essential for safe utilization and awareness of residual risks.   For example:

• Back Clipping:   A dangerous mistake that is too common among beginner climbers is to clip the rope so that it travels from the outside into the cliff.  In a fall, the rope can cause the carabiner gate to open and unclip from the carabiner, leading to longer and more dangerous falls. Analogously, in Unit Testing, incorrect assertions that remain valid even in error conditions render tests ineffective during failure scenarios. "Make it fail, Make it pass," a technique in Test-Driven Development, helps mitigate this issue.
• Z-clipping: closely spaced bolts are misused (e.g., clipped in the wrong order), leading to rope drag that could impede or cause a climber's fall. Likewise, unit tests with superfluous or concealed dependencies may slow down or halt test execution entirely.
• Rusted Bolts: These climbing anchors, left in the rock and exposed to varying weather conditions, gradually deteriorate, losing strength, and need to be evaluated critically by the climber when used. Similarly, older tests may become obsolete, continuing to assess irrelevant code elements or failing to update to newer testing framework versions, thus missing out on advanced features and improvements.

Improving the depth dimension also implies a Growth Mindset approach for continuously learning and adopting new tools and techniques, such as new Belay devices in climbing (e.g., Petzl recently release a new device, the Neox, that while similar to the widely used GriGri 2, has some different features and need specific training to be used correctly for safety ). In IT testing, the same learning approach applies to adopt new testing tools that might also rely on innovative Gen AI approaches.

Width dimension: Beyond Unit testing

To illustrate the width dimension in risk management, let's go beyond sport climbing and look at other contexts.

When climbing traditional routes (Trad Climbing or BigWalls) with no pre-placed fixed bolts in place,  alternative anchors such as Pitons, Nuts, and Friends are used instead. These require specific and more complex skills for the placement and evaluation of the residual risks, making this a specialized premium skill.

Ice Climbing, typically done in winter on waterfalls or steep mountain couloirs, is yet another context with additional equipment and skills. Here placing an ice screw for safety requires specific strength, technique and knowledge of ice quality to minimize the risks.

Other types of climbing, such as Sport Climbing, are more competitive in nature and focus on performances. Even here, specific safety measures are used, like automated top-rope automated belaying devices officially used in Speed Climbing Events (that have become more popular with the latest Tokyo and Paris Olympics).

Similarly, when testing IT Projects, a variety of testing techniques can be used, each one providing benefits in different contexts and requiring specific tools and skills. As an example, refer to the following article that categorizes the many kinds of possible software testing strategies:

To effectively navigate between the many types of possible tests, I suggest focusing on the three following axis of evaluation.

The first axis is the type of Requirements that are being tested. Typically, these are split into:

• Functional: Related to the functional behavior of the application, such as service calls, business logic, processes, and UI navigation.
• Non-Functional: Related to quality features of the solution, such as performance (response times, traffic load), scalability, resilience (failover and disaster recovery), and security

Another important axis is the Scope of the tests. The scope can range from a single method or function to an entire component for a unit test, to a set of related interconnected components for an integration test, up to an entire system for end-to-end testing.

As the third axis, I would evaluate the trade-offs between test repeatability (for test regression purpose) and test creation and maintenance cost. The following spectrum of tests is typical:

• Manual tests require the least amount of development effort but are slow and potentially not consistently repeatable
• Automated tests that need to be developed and maintained but can provide a fast way to execute a large number of repeatable tests concurrently in a short amount of time
• Continuous tests are executed at each compilation, build, or deployment. These tests are even more complex and expensive because they need to be optimized to not slow down the software development lifecycle, but in exchange, they can be executed frequently to further accelerate the finding and resolution of problems.

Combining the approaches: The "Swiss Cheese" principle

The width and depth approaches can (and should!) be combined to create the most effective solution to the problem at hand. A good way to describe this approach is the Swiss Cheese Model, a risk management and safety concept that offers a visual representation of how multiple, independent layers of defense must fail simultaneously to result in an accident. Each layer, depicted as a slice of Swiss cheese, has its own specific vulnerabilities or weaknesses, symbolized by holes in the cheese. When the holes align, they create a pathway for the hazard to penetrate all defenses, leading to an incident. The model emphasizes the importance of redundancy to better prevent and identify problems.

Yet again, we can compare the security approach in climbing with IT Testing:

• Redundancy is used by having multiple anchors when needed (e.g., a Bolt every few meters or using a redundant double placement of Friend or Nuts in a sketchy situation), while an IT Project might increase the number of tests in specific test suites.
• Mixing anchor types whenever applicable: A sport route might have a section of crack climbing that can be better protected with additional Trad equipment, or a mixed Ice Route might have a rock climbing section that is protected with fixed bolts. Similarly, an IT project uses a mix of test types  and strategies (Functional vs Non-Functional, Unit vs End-to-End, Manual vs Automated) to minimize the overall risk

Conclusion

The overall goal in both scenarios is maximizing the benefits of risk reduction while minimizing the cost and effort to achieve this.

In both scenarios, we need to combine everything to work seamlessly together:

• In Climbing, we rely on the basic skills for using standard equipment such as ropes and harnesses
• In Software Development and Testing, the DevOps approach provides a way to handle the end-to-end strategy

I hope that this in-depth comparison provides (in a lightweight and funny way) some useful thinking points, even if not 100% accurate and precise.