Maintenance Testing

Software that is in production periodically requires a maintenance release. As new needs emerge or defects are discovered after deployment, maintenance is performed.

When Maintenance Testing is Done?

Once delivered, Software and Systems are maintained for various changes like:

Adding new functionality
Deleting existing functionality
Altering existing functionality
Fixing any defect
Improve or preserve Non-Functional characteristics, including compatibility, reliability, security, portability, and performance efficiency.

Maintenance testing is performed when any changes are made as a part of maintenance. Maintenance Testing ensures that changes are made successfully and there is no possible side effect (e.g., regression) on the part of the system that is not changed. Maintenance releases are sometimes scheduled in advance with defined feature sets or may also take the form of emergency fixes (hotfixes) with little planning (if any).

Scope of Maintenance Testing

Maintenance Testing is required at different Test Types and Levels based on its scope. The scope of maintenance testing depends on three factors:

Degree of Risk of the changes (for example, the degree to which the changed area of software communicates with other components or systems.)
Size of the Existing System
Size of the Changes

Triggers of Maintenance Testing

For Planned and Unplanned changes, maintenance testing takes place for several reasons. The Triggers for maintenance testing can be classified into two types:

1. Modification

Modifications are required:

For planned release-based enhancement
Corrective Changes
Emergency changes
Operational environment changes (OS or database upgrades)
COTS software upgrades
Patches for defects and vulnerabilities

2. Migration:

Migration from one platform to another require operational testing of the new environment, changed software, and data conversion.

An application is at the retirement stage when it reaches the end of life. During retirement, testing is done for data migration or archiving if long data-retention periods are required.
Testing restores/retrieve procedures after archiving for long retention periods may also be needed.
Regression testing is also required to ensure functionality still in service is working.

Maintenance testing may be triggered for the Internet of Things when completely new or modified things, such as hardware devices or software services, are introduced to the overall system. For such a system, maintenance testing emphasizes integration testing at different levels, including application and network levels. Maintenance Testing also focuses on the security aspects, in particular, those relating to personal data.

Impact Analysis for Maintenance

Impact Analysis is determining how the existing system may be affected by changes. Maintenance testing is performed to test the impact of changes to an operational system or its environment. For maintenance releases, impact analysis is done to evaluate the changes to identify the intended consequence as well as expected and possible side effects. The analysis also identifies areas of the system that will be affected by the change.

Impact analysis also helps to identify the impact of the change on existing tests. The side effects and affected areas in the system need to be tested for regressions, possibly after updating any existing tests affected by the change. Impact analysis may be done before a change is made to help decide if the change should be made based on the potential consequences in other areas of the system.

Impact analysis can be difficult if:

Specifications (e.g., business requirements, user stories, architecture) are outdated or missing.
Test cases are not documented or are out of date.
Bi-directional traceability between tests and the test basis has not been maintained.
Tool support is weak or non-existent.
The people involved do not have domain and/or system knowledge.
Insufficient attention has been paid to the software's maintainability during development.