Agile Testing and the Role of the Testing Team

Abstract

Agile testing is a software testing process that implements the principles set forth in agile software development. Agile testing employs the iterative methodology in which requirements are not finalized in one phase but gradually implemented to get a final product. This paper seeks to explore the different aspect in agile testing including the role of the testing team. This paper will also look into how software testing factors into some of the different agile methodologies, namely Scrum, Extreme programming and Kanban. Finally, we will look at agile testing methods and popular tools used in agile testing.

Testers role in Agile testing

Many software companies are adopting the Agile methodology for their software development process in order to accelerate their delivery of products to their clients. The agile methodology ensures that the company delivers the right product at the right time as quickly and efficiently as possible. Agile methodology has four core values in which a tester should adhere to.

What a tester is required to do when working on projects using Agile methodologies is different from the requirements of a traditional software project. Testers on Agile projects must have great testing acumen, they must be effective communicators, constantly interact with colleagues and business representatives, and work as part of a whole team responding frequent changes and feedback. To work as part of an Agile software team, testers must understand the principles and values underpinning the Agile development approach. These are:[ https://searchcio.techtarget.com/definition/Agile-Manifesto]

1. Individuals and interactions over processes and tools;
2. Working software over comprehensive documentation;
3. Customer collaboration over contract negotiation; and
4. Responding to change over following a plan.

In order to function effectively as an agile methodology tester, the tester first needs to get acquainted with Agile software development. “Agile development, which covers a range of development models, is based on an ‘iterative’ approach to creating software. In this way, the software product is built in progressive small chunks with each new piece adding to the features developed in the preceding iteration.” [2]

In light of the four core principles of agile methodologies, testers in agile testing can adapt to the principles in the following ways

• Individuals and interactions:Testers in an Agile project should regard themselves as an integral part of the development team. In agile testing there is no demarcation between testing and development team as both work hand in hand. By working together as a single team, delivery goals are met effectively.[2] When the testers regard their role with the whole-team approach, testers will also work closely with developers. Testers can work with developers to create unit tests. When a project is using continuous integration, tester will have a more beneficial role to play to help the developers implement the unit tests.
• Working software: The testers will not burden themselves working on detailed requirements specifications design documents and test plans and so on, the main goal is to implement a working solution for the customer.
• Customer collaboration; The testing team will work closely with the customer, to deliver the software on a regular basis. The Agile testing team should get constant feedback from the customer and implement any change requests that can be easily be added before too much progress is done in the sprint. Having good communication with the business clients helps to create suitable user acceptance tests because the clients will know how exactly they wish the system to function.
• Responding to change: In software development, changing requirements are inevitable so being able to respond to change quickly and effectively can be the difference between project success and failure. These requirement changes and last-minute requests can affect the scope of testing which can frustrate testers. Testers should provide adequate information about what tests have been run and which part of the application hasn’t been tested so that an informed decision on whether to release the feature or not can be made. [3]

Agile software development approaches

Scrum

“SCRUM is an agile development method which concentrates specifically on how to manage tasks within a team-based development environment. Basically, Scrum is derived from an activity that occurs during a rugby match. Scrum believes in empowering the development team and advocates working in small teams (say- 7 to 9 members).” [4] In scrum there are some testing activities that can be done. Sprint meetings where features of the software being developed are picked up from backlogs. The work is estimated and prioritized. During the sprint test execution can be done where both the tester and the developer work together. Defect are recorded in a Defect Management tool and tracked.[5] Defects can be discussed and analyzed during the scrum meetings.  In daily scrum meeting testers get the information about tasks that have been completed and any tasks that need to be given back to the developer from fixes. On a daily basis the tester will perform acceptance testing, system testing. The tester will also aid the developers in crafting the unit test and integration test. [4]

Extreme programming

“Extreme Programming is a software development methodology that is designed to improve the quality of software and its ability to properly adapt to the changing needs of the customer or client.” [6] Extreme programming prioritizes consistent customer collaboration and delivery cycles that are iterative. The most  important aspect of Extreme programming that we should know is that it was designed to deliver large amounts of software within the shortest time possible . That is why extreme programming has really high client involvement into the process of project realization. Unlike other Agile methodologies, in Extreme programming the customer takes part at all team meetings personally. When it comes to agile testing, extreme programming fits right in. In extreme programing, testers and developers are required to right tests before writing any code. Extreme programming works in a way where a meeting is held with the customer or client to discuss the requirements with the team at the beginning of every iterative cycle. Therefore, the team knows the acceptance requirements for the code. The developers in conjunction with the testers can use test driven development to test and develop the product. This is a repetitive process if testing, developing and refactoring until we have a refined accepted product. As you can see, in this methodology, testing and code development are tightly knit since the product feature has to be delivered in the shortest time possible.

Kanban

Kanban is a visual-driven workflow management method that is designed to help teams work in a manner that maximizes efficiency and also helps the be more agile.  In Agile testing, Kanban mainly prioritizes visualizing the status of the project to understand what is in backlog, what is currently in progress and what is completed status.[7] This helps people involved in the testing process to get a clear view of the status of the tasks at any point in time. This transparency makes communication more efficient and issues can be resolved much quicker.

When using Kanban in agile testing it is important to define a workflow steps that will be used to go through the testing effort. A white board can be used or even Kanban software. Using the Kanban methodology has the following advantages:

1. The visual transparency helps resolve issues in a much quicker manner as team members can easily pinpoint the source of the issues.
2. Prioritization of the testing tasks can be done much more easily.
3. The improved planning and prioritization that comes with using Kanban in software testing leads to faster product deliveries. Since the testing tasks are more visual, dependence of specialized skills can be spread through the team so that there is no bottleneck because there is dependence on a specific tester.

3 AGILE TESTING METHODS, TECHNIQUES AND TOOLS

Test-driven development

TDD is a method whereby unit tests are created, in small incremental steps, and, in the same small incremental steps, the code is created to meet those tests. Metaphorically, think of how a bush’s shape can be made orderly and in conformance with desired behaviour by the use of a lattice-like frame (i.e. a trellis). These unit tests allow software developers to verify whether their code behaves according to their design, both as they develop the unit and after making any changes to the unit. This provides a level of confidence that leads many developers to stay with TDD once they have become accustomed to the process. However, other developers find the process too tedious or cumbersome, and instead create and run their unit tests after coding.TDD involves first writing a test of the expected low-level functionality, and only then writing the code that will exercise that test. When the test passes, it is time to move to the next piece of low-level functionality. As you grow the number of tests and code base incrementally, you also need to refactor the code frequently. If you do not refactor, you may end up with ‘spaghetti’ code that is neither maintainable nor understandable. Some people would rather do lots of architectural design first, and you can do that as well, to some extent, to avoid too much refactoring. However, the strength of TDD is that you only develop the minimum code that is required to pass the existing unit tests, and only then move on to the next piece of test and code. This way you avoid unnecessary code. Your code will be lean, fast and maintainable. Debugging will be very easy as you have only a small code increment to look at (Lehtonen et al., 2014).The tests need to be automated, usually in a test framework such as JUnit, CppUnit or any of the other xUnit family of frameworks (where x stands for any programming language). We’ll discuss these unit testing frameworks further in Section 3.4. Without automation, repeating the tests all the time is not viable, which makes refactoring and other code changes more likely to result in undetected regression. These automated tests also should be part of a continuous integration framework, so you will know that your whole system works as you include more new code. The process is highly iterative:• You write a new test and expect it to fail.• You write just enough code to pass that test and keep running the test and changing or adding code until the test passes.• You refactor code for maintainability and run the new test and earlier tests, again repeating the actions if the tests do not pass.Graphically, this process is shown in Figure 3.1.The tests you create in this way are unit tests for the code. They can even work as a replacement of some of the technical documentation or technical design. Typically, you would try to achieve at least 100 per cent decision (branch) coverage with unit tests. You can also apply TDD to integration tests and systems tests, although this is seldom done in practice.

Acceptance test-driven development

When doing ATDD as part of an Agile practice, tests are created in an iterative way, starting before, and continuing during, the implementation of a user story (see more about user stories in Section 1.2).User stories need to include acceptance criteria and those in turn can be turned into (drafts of) acceptance tests. All this happens through business representatives, developers and testers working together in a specification workshop (Adzic, 2009). The workshop is not only about creating acceptance tests. The real goal is to collaboratively understand what the software should and should not do. ATDD strives to encourage and improve the communication between the business, the developers and the testers. Acceptance tests are a by-product, or, if you will, an exposition of this discussion. The (drafts of) acceptance tests should next be detailed enough that code can be exercised against them. In other words, they tend to be more towards the concrete test end of the logical test–concrete test spectrum. They could be, and usually are, also automated for ease of use. The acceptance tests are continuously refined with additional details about how to interpret the user story and how to show whether it satisfies the expressed user needs or not. ATDD tests are higher-level tests than unit tests, but still quite small tests, each exercising one or more acceptance criteria of a user story. They belong to quadrant two of the Agile testing quadrants, which we’ll discuss later in this section. The ATDD process is described in detail in Section 3.3.ATDD makes it possible to:• test the code quickly at the user story level, system level or acceptance level;• verify that acceptance criteria are met;• find higher-level defects early in user story development;• automate regression test sets, or at least create automatable elements for such sets;• improve collaboration between all stakeholders.ATDD as a methodology does not require test automation, but it is usually coupled with a powerful test automation framework that can take input from specification workshops (with the help of the test automation expert). Some acceptance tests need to access databases, some need to access application program interfaces (APIs) and some the user interface, so usually a framework is also needed from a technical point of view. Frameworks can then use multiple test execution tools.

Behaviour-driven development

BDD starts from a point of view that behaviours of software are easier for stakeholders to understand when taking part in test creation than the specific tests themselves (Chelimsky et al., 2010). The idea is, together with all team members, business representatives and possibly even customers, to define the behaviours of the software. This definition can happen in a specification workshop, very much like ATDD. Tests are then based on the expected behaviours, and the developer runs these tests all the time as she develops the code.BDD promotes immediate test automation even more than ATDD. One of the leading ideas is to use very clear English (or another spoken language), so that both business users participating in a workshop and a test execution tool can understand the behaviour and the test that verifies that behaviour. Some test automation frameworks are adapted (such as Robot Framework) and some even created (such as Cucumber and JBehave) to work specifically with BDD methodology.Participants of the specification workshop are instructed to think how a system will behave. One format is the Given-When-Then syntax, part of the Gherkin format of the Cucumber tool family:• given some initial context;• when an event occurs;• then ensure some outcomes.

The test pyramid

The concept of the test pyramid was introduced by Mike Cohn (Cohn, 2005), originally with three levels: unit, service and user interface (UI), but we refer to Janet Gregory and Lisa Crispin’s popularisation of the concept here (Gregory and Crispin, 2014). Often the test pyramid is referred to as the test automation pyramid, as the basic idea is that all test levels in the pyramid can be automated. Manual or automated, the shape of the pyramid refers to the fact that there are more tests at the lower levels, thus ensuring earlier testing and cheaper defect removal. Figure 3.2 shows the test pyramid with typical test levels.

Testing quadrants, test levels and testing types

If you had to choose one concept only to describe what testing is like in Agile software development, it would be the Agile testing quadrants, or just testing quadrants for short. The concept of testing quadrants was invented by Brian Marick, and popularised by Janet Gregory and Lisa Crispin (2014). Testing quadrants transfer the traditional test levels and testing types into an Agile context, giving reasons for the existence of various different types of testing.As seen in Figure 3.4, there are four Agile testing quadrants:• Q1, which is Technology-Facing and Supporting the Team;• Q2, which is Business-Facing and Supporting the Team;• Q3, which is Business-Facing and Critiquing the Product;• Q4, which is Technology-Facing and Critiquing the Product.On the left half of the figure are tests that are Supporting the Team. Gregory and Crispin (2014) have renamed these tests as tests that Guide Development, which is much more fitting. As the ISTQB® Agile Tester syllabus (and equivalent exam) currently refer to terms in the previous version of Crispin and Gregory’s picture (2008), we are also going to do so to provide consistency. These tests can be seen as verification tests, since they check whether we are building what we defined. These are very much needed in the quick cycle of Agile development. When you add some new, small piece of code into continuous integration, you want to reduce the risk that you may have broken something. Like all regression tests, these tests should not find defects most of the time. They should usually pass and let you know that you are on the right track.On the right half, you have tests that Critique the Product. These can mostly be seen as validation tests, since they check whether we are building what was needed. These tests are crucial for finding those defects that certainly lurk in any software. Many Agile teams first start with the tests that Support the Team, only to realise that they also need tests that Critique the Product.On the bottom horizontal axis are the Technology-Facing tests. These are the technical tests that are usually and most easily done by developers or other technical people. They are often structural in nature, i.e. white-box tests.On the top horizontal axis you have Business-Facing tests. These are often behavioural in nature, i.e. black-box tests. You need these also. To put it in simple terms, you cannot do unit testing only, you also need to test from the customer perspective.

TOOLS IN AGILE PROJECTS

Task management and tracking tools: Jira

Communication and information sharing tools: slack

Software build and distribution tools:Jenkins

Configuration management tools: git

Automated test execution tools: junit

[2] https://learning.oreilly.com/library/view/agile-testing-foundations/9781780173368/12_chapter03.xhtml#s3 for most paper.

[3] https://www.testingexcellence.com/agile-testing-challenges-qa-agile-projects/

[4] https://www.amarinfotech.com/roles-responsibilities-agile-tester.html

[5] https://www.guru99.com/scrum-testing-beginner-guide.html

[6] https://hygger.io/blog/tests-in-extreme-programming/

[7] https://leankit.com/learn/kanban/how-to-use-kanban-with-software-testing/