Discovering a methodology and creating a tool to take advantage of this underutilized testing approach

Have you ever heard of the advantages and benefits of exploratory testing but weren’t encouraged enough to implement it? Well, a few months ago, we started working on developing a methodology and tools for managing exploratory testing. This exciting project we are doing is part of a thesis project at the University ORT Uruguay, lead by Gabriela Sanchez (one of our QA consultants) and Miguel Sanchez (who is also heavily involved in the testing community).

exploratory
Photo credit: Jose Muñoz. via Visual hunt / CC BY-SA

The motivation and main vision for the project was to make it easier to take advantage of this beneficial, yet underutilized approach and to evangelize it as an agile testing method.

In this post I intend to present the project and thus give rise to a series of articles that we will write this year which will delve into different aspects of performing and managing exploratory testing. We’ll also be sharing our analyses of existing tools on the market, real experiences, good and bad practices, etc.

For a good introduction to exploratory testing, I suggest reading this blog post by James Bach, one of the greatest thought leaders and proponents of exploratory testing.

Project Overview

Testers are facing a great challenge as we are immersed in a state where agile development increasingly takes place. Testing (now agile testing) is integrated into the development process where it is the responsibility of the entire team. How can we go along with these changes while maintaining a level of excellence in testing?

The approach that seems best suited for this new reality is exploratory testing. This testing approach consists of design, execution and simultaneous learning. However, its implementation difficulties are well known and date back to its origins. How can we sell the approach to our customers (who are accustomed to measuring our results based on “test cases”)? How can we still show them what we were doing during all the hours we spend testing? How can we start managing exploratory testing and be able to answer questions like the one above without changing its intrinsic characteristics? How can we place structure over a practice that emphasizes individual freedom without sacrificing that freedom and without bureaucratizing it?

In 2000, the Bach brothers, James and Jonathan, developed a methodology for conducting exploratory testing, known as session-based exploratory testing. A new way to perform exploratory testing (probably the most structured of all the ways), it provides information that helps us to make this activity more intellectually challenging and enriching and above all, information to be able to “sell it” better. Read more here.

Still, today we face many difficulties in its implementation. Some of those difficulties include processing the resulting information from the sessions to obtaining interesting data that easily allows us to answer questions like:

How much progress was made today?

How much is left to test?

How many more days of testing do we need?

There are numerous tools that assist us in conducting exploratory testing, some of which are very good indeed. However, at present, we have not found an open source tool that enables us to manage exploratory testing in a centralized way. We’d like to see one that provides simple and streamlined processing of information generated during the sessions, rapid attainment of relevant metrics that allow us to plan according to our current context, a data collection environment, etc.

It is also said that exploratory testing is ideal for the experienced tester, and many times we have seen that even experienced testers (particularly perhaps those used to working with test cases) are not sure how to start using this approach.

Three Key Objectives

For these reasons, this project was born with the mission to achieve these three key objectives:

1. Methodology

Research and analyze the different ways of performing and managing exploratory testing in order to learn more about it so as to develop a methodology that is best suited for agile development.

2. Tools

Design a tool to manage the process of exploratory testing from start to finish, from initial planning and implementation sessions until further processing and obtaining metrics and indicators to improve testing from cycle to cycle (sprint to sprint). We also plan for the the tool to guide a beginner exploratory tester through the whole process so he or she can apply it efficiently.

3. Dissemination

Achieving a greater dissemination of exploratory testing in the community of testers, as we believe it is an excellent approach and is being very underutilized (at least that is what was revealed by a small survey that we conducted in 2015 among the TestingUY community).

 

As a part of our third objective, we’ll be be publishing posts to the Abstracta blog related to our progress and results of the project objectives. We’ll then show the benefits of exploratory testing, the best way to use it (the methodology) and the tools on the market today, and the opensource tool that we will develop as the end result of this project. These posts will mainly be coming from Gabriela Sanchez and Miguel Sanchez, the testers in charge of the project.

We appreciate any and all feedback or input that anyone has to offer, and if you are interested in collaborating, just let say the word by writing to us at hello@abstracta.us 


Project Participants

Gabriela Sanchez

Gabriela Sanchez is a consultant at Abstracta, a company fully dedicated to testing, risk control, development, and commissioning of software production. For years, Abstracta has been giving companies the support to produce quality software, sharing its know-how of different QA activities and helping software companies to ensure the quality of their products. Gabriela is also a Professor of the Project Management and Software Engineering II subjects at the ORT University in Montevideo, Uruguay. She holds the title of a testing leader at CES. With over 5 years of experience in the area of IT, QA, and QA management, a couple of her main interests related to testing are teaching and conducting academic research.

Twitter : @Gabyeke

Pablo Miguel Sánchez

Pablo Miguel Sánchez is a Professor of Software Engineering II at the University ORT in Uruguay, where he studied system engineering. He is an IT specialist at CCC Medical Devices, where he performs tasks related to software development and testing. His main interests include QA, Development and Teaching.

Twitter: @PabloMiguelSa


Check out these related posts

Yoda’s “The Way of the Jedi Tester”: A Guide for Agile Testing 
[Infographic] The State of the Software Testing Profession 2015-2016

Tags: