Non-Functional Testing

Non-functional testing is the testing of "how well" the system behaves. It is a type of testing performed to test the non-functional characteristics or requirements of the system. It answers questions such as how many concurrent users the system supports, web page load time, number of transactions that can take place per second, etc.

• Requirement Gathering: Non-functional requirements are often poorly specified or even non-existent. It is advisable to obtain multiple viewpoints when capturing non-functional requirements. In agile software development, non-functional requirements may be stated as user stories or added to the functionality specified in use cases as non-functional constraints.
• Test Levels: Non-Functional Testing should start as early as possible and should be performed at all test Levels. Failure to plan for non-functional testing or late discovery of Non-Functional defects like security defects is extremely dangerous for the project. A common mistake is waiting until all functional tests are complete before starting non-functional tests, which can result in the late discovery of critical non-functional defects. Many non-functional tests are expensive, so the Test Manager must select which non-functional tests to perform based on risk and constraints. 
• Technique: Black-box techniques may be used to derive test conditions and test cases for non-functional testing. For example, boundary value analysis can be used to define the stress conditions for performance tests. For example, a system supports 10,000 concurrent users but not 10,001.
• Types: Some commonly known Non-Functional Testing types are Load Testing, Performance Testing, Usability Testing, Stress Testing, Security Testing, and Recovery Testing.
• Coverage:  The thoroughness of non-functional testing can be measured through non-functional coverage. Non-functional coverage is the extent to which tests have exercised some type of non-functional element. It is expressed as a percentage of the type(s) of the element being covered. For example, using traceability between tests and supported devices for a mobile application, the percentage of devices that are addressed by compatibility testing can be calculated, potentially identifying coverage gaps. 
• Knowledge Requirement: Non-functional test design and execution may involve special skills or knowledge, such as knowledge of the inherent weaknesses of a design or technology (e.g., security vulnerabilities associated with particular programming languages) or the specific user base (e.g., the personas of users of healthcare facility management systems). It also requires knowledge of tools like JMeter for Load Testing or a tool like Burp Suite for security testing.

7 Non-Functional Characteristics

The ISO 25010 standard [ISO 25010:2011] introduces a system and software product quality model that categorizes product quality properties.

1. Compatibility:  Sub characteristics are Co-existence and Interoperability.
2. Maintainability: Sub characterstics are Modularity, Reusability, Analyzability, Modifiability, and Testability
3. Performance efficiency: Sub characteristics of Performance efficiency are Time behavior, resource utilization, and capacity. An example is a slow response to user input.
4. Portability: Sub characteristics are Adaptability, Installability, and Replaceability.
5. Reliability: Sub characteristics are Maturity, Availability, Fault tolerance, and Recoverability.
6. Security: Sub characteristics of Security are Confidentiality, integrity, non-repudiation, accountability, and authenticity.
7. Usability: It is performed to determine the extent to which the software product is understood, easy to learn, and operate. Sub characteristics of usability testing are Appropriateness recognizability, learnability, operability, user interface aesthetics, user error protection, and accessibility. An example is a difficulty in understanding fields and screens.