Lesson: Does It Work? Test and Test Again

Contributed by: IMPART RET Program, College of Information Science & Technology, University of Nebraska-Omaha

Photo shows a person typing on a laptop, which displays codes, in various text colors.
Students get a taste of programming code
copyright
Copyright © 2012 Almonroth, Wikimedia Commons {PD} http://commons.wikimedia.org/wiki/File:Typing_computer_screen_reflection.jpg

Summary

Testing is critical to any design, whether the creation of new software or a bridge across a wide river. Despite risking the quality of the design, the testing stage is often hurried in order to get products to market. In this lesson, students focus on the testing phase of the software/systems design process. They start by exploring existing examples of program testing using the CodingBat website, which contains a series of problems and challenges that students solve using the Java programming language. Working in teams, students practice writing test cases for other groups' code, and then write test cases for a program before writing the program itself.
This engineering curriculum meets Next Generation Science Standards (NGSS).

Engineering Connection

The testing process is a crucial step of the engineering design process. One of the most critical areas in software development is to adequately test the code written by programmers. Software engineers must thoroughly test all aspects (cases) of a program to see if it stands up to the rigorous demands that consumers place on the application. Programs that are not properly tested may not be well received by the public and might end up costing more money for a business in order to fix problems after the fact. In the lesson and its associated activity, students design test cases for a program to fully test it, see the value of testing and learn how it is a valuable part of the larger engineering design process.

Pre-Req Knowledge

Students must know Java Programming basics. Although their knowledge need not be advanced, students must be able to develop code that needs to be tested in order to design test cases as part of a larger design process.

Learning Objectives

After this lesson, students should be able to:

  • Explain the role that testing plays in the software/systems design process.
  • Select a set of test cases for a section of program code.
  • Evaluate test cases for completeness and code coverage.
  • Analyze program code for defects based on the results of test cases.

More Curriculum Like This

Using JUnit

Students focus on the testing phase of the design process by considering how they have tested computer programs in the past and learning about a new method called JUnit to test programs in the future. JUnit is a testing method that is included with NetBeans (Java) installs or can be downloaded from ...

High School Lesson
Curiosity Killed the App

Students gain experience with the software/system design process, closely related to the engineering design process, to solve a problem. The lesson culminates in a hands-on experience with the design process as students simulate the remote control of a rover.

Middle School Lesson
Program Analysis Using App Inventor

In this lesson, students learn about, design and create flow charts for different scenarios, including a game based on the Battleship® created by Hasbro©. In the associated activity, Flow Charting App Inventor, students apply their knowledge from this lesson and gain experience with a software appli...

Python Calculus

Students analyze a cartoon of a Rube Goldberg machine and a Python programming language script to practice engineering analysis. In both cases, they study the examples to determine how the different systems operate and the function of each component.

High School Lesson

Educational Standards

Each TeachEngineering lesson or activity is correlated to one or more K-12 science, technology, engineering or math (STEM) educational standards.

All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN), a project of D2L (www.achievementstandards.org).

In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics; within type by subtype, then by grade, etc.

  • Use a computer simulation to model the impact of proposed solutions to a complex real-world problem with numerous criteria and constraints on interactions within and between systems relevant to the problem. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Troubleshooting is a problem-solving method used to identify the cause of a malfunction in a technological system. (Grades 6 - 8) Details... View more aligned curriculum... Do you agree with this alignment?
  • The design process includes defining a problem, brainstorming, researching and generating ideas, identifying criteria and specifying constraints, exploring possibilities, selecting an approach, developing a design proposal, making a model or prototype, testing and evaluating the design using specifications, refining the design, creating or making it, and communicating processes and results. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Evaluate the design solution using conceptual, physical, and mathematical models at various intervals of the design process in order to check for proper design and to note areas where improvements are needed. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Identify questions and concepts that guide scientific investigations. Students should formulate a testable hypothesis and demonstrate the logical connections between the scientific concepts guiding a hypothesis and the design of an experiment. They should demonstrate appropriate procedures, a knowledge base, and conceptual understanding of scientific investigations. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
Suggest an alignment not listed above

Introduction/Motivation

A circular diagram presents the typical steps in software development. The process is circular in nature and includes the following five steps: Requirement Analysis, Design, Implementation, Testing, and Evolution. In the center of the diagram is the title "SDLC Software/System Development Life Cycle."
Figure 1. The software development cycle.
copyright
Copyright © 2012 Cliffydcw, Wikimedia Commons {PD} http://commons.wikimedia.org/wiki/File:SDLC_-_Software_Development_Life_Cycle.jpg

The software/system design process — a specific use of the engineering design process — is cyclical in nature and has five steps that are necessary for the development of quality solutions (see Figure 1). In computer science classes, you are taught to analyze the problem that you wish to solve by designing a software (computer) program. After your initial analysis, you next design the code by making choices about algorithms, control structure, and more. In the implementation phase, you put your design into practice by using the proper syntax for the computer language.

One critical component of the design process that is often overlooked, and is also difficult to do well, is the testing step. In this lesson, you will learn to put a computer program, written in Java, through its paces (that is, explore all possibilities) and thoroughly test it, with the ultimate goal of learning how to properly design and implement test cases for your code. The design and implementation of test cases is a miniature software/systems process inside of the larger engineering design process.

In this lesson, you will explore the testing tool, CodingBat. Let's start by navigating to the CodingBat website (http://codingbat.com). Once there, try to write solutions to one or more of the Java problems on the website.

Note to teacher: Use the information provided in the next section [Lesson Background & Concepts for Teachers] to lead a class discussion on how CodingBat works and what different types of test cases need to be written.

Lesson Background and Concepts for Teachers

Understanding CodingBat: CodingBat is a free website with live coding problems that can be used to build coding skills in Java (and other languages). The coding problems allow users to gain a deeper understanding of coding concepts as they practice solving problems. Instructional videos about individual topics are provided as well. The website can be used by the teacher to provide homework for students, a lab setting, or to provide live lecture examples.

A unique feature of the website is that the problems are short, similar to test questions, with instant feedback, so that learning can be immediate and successful. The website provides an opportunity for students to learn how to write Java method code in an environment that helps them concentrate on coding with fewer distractions.

To properly test a program, you need to develop three different test cases. These include normal cases, edge cases and error cases. A "normal" test case consists of a specific input for a program and the expected results for that input. A normal test case must include both specific inputs and the expected results so that it can verify if the test case passed. An "edge" test case tests input near the boundary where a program changes from one behavior to another. This type of testing might be to verify if a game advances to the next level if all the criteria for the level the user is on are met, or if the level is based on points and the "level" up number of points is achieved, if the function performs correctly. An "error" test case has an output were an error is expected. For instance, if a program does computations to see if the program handles division by zero, which is an invalid operation, it should produce an error, rather than crashing the entire program. Together these test cases form a testing suite, and when implemented properly, they evaluate the performance and correctness of the software application.

Note to teacher: Depending on their Java experience, you may wish to pre-select specific problem(s) for students to solve. During this coding experience, guide a class discussion about the testing phase of the design process. Example questions:

  • How does the CodingBat website determine whether submitted code is correct? (Answer: It runs the code several times, with different input values, and checks the output.)
  • Is the submitted code correct or incorrect if some, but not all, tests run incorrectly? (Answer: It is incorrect. Even one incorrect test case demonstrates a problem in the code.)
  • How do you think the creators of the CodingBat website decide which cases to test? (Answer: For a thorough testing phase, all cases [normal, edge and error] are tested.)

CodingBat is strongly recommended for computer science classes that teach the Java programming language, or classes that incorporate Java. It can be used by teachers and students to develop skills in writing test cases for programs.

In the associated activity, Testing the Edges, students develop test cases for a provided problem.

Vocabulary/Definitions

edge case: A test case whose input is near a "boundary," where the program changes from one behavior to another. For example, if a game allows a character to hold four items at a time, then edge cases would include collecting the fourth item (the last item that the character can hold) and collecting the fifth item (an item that would exceed the character's carrying capacity).

error case: A test case whose expected result is an error. For example, for a program that divides two numbers, dividing by zero is an error case. A well-written program should have a plan in place to handle unexpected data in a reasonable manner; error cases test this plan.

test case: A specific set of input for a program or part of a program, along with the expected results of running the program with that input. Note that a test case must contain both the input and the output. A test can only help to verify a program's correctness if the expected output can be compared to the actual output of the test case.

test suite: A collection of test cases that is intended to fully "exercise" a section of a program. A complete test suite should include edge cases and error cases as well as a representative selection of "normal" cases.

test-driven development: The software designer creates test cases for a section of code first, before writing the program itself. While this seems "backward" to many programmers, writing tests first helps the programmer to focus on what the code should do. By considering expected results in advance, most programmers write better code with fewer errors.

Associated Activities

  • Testing the Edges - Students apply the software/systems design process with open-ended solutions, specifically focusing on the testing phase. Working in small groups, they create Java functions and test cases to determine whether a Java function correctly implements the requirements of the problem.

Attachments

Assessment

Pre-Lesson Assessment

Class Discussion: Ask students what they know about the software development cycle (design process).

  • How many steps are in the software development cycle/design process? (Answer: The software development cycle has five steps or parts.)
  • What are the steps? (Answer: The steps are: problem analysis, design, implementation, testing and evolution.)
  • What does each step entail? (Answer: In problem analysis, the problem is framed and constraints are evaluated. In design, possible solutions are brainstormed that address the problem and constraints identified in the first step. Implementation is putting a design into practice; or implementation means that code is written. The fourth step is testing, which is critical for the development of software. The last step, evolution, is thinking about how the product can be improved or applied differently.)

Summative Assessment

Prompted Writing Assignment: Assign students to answer the questions on the Writing Assessment handout. (Note: Alternatively, write one/all of the following prompts on the classroom board for students to answer in their engineering journals.)

  • Explain how testing fits into the software/systems design process. (Example answer [answers will vary]: Testing is a critical component of the design process. To properly test software, a mini-design process must be completed to fully understand how to test and create the specific tests needed to adequately test the designed program.)
  • Explain why testing is critical to the successful completion of a quality result. (Example answer [answers will vary]: Testing enables software designers to see if a newly designed solution really addresses the identified problem; it is absolutely essential that the design be adequately tested to minimize errors in the future.)
  • Explain why tests for a section of a program should include normal cases, edge cases, and error cases. (Example answer [answers will vary]: For normal cases: these test that the code actually does what it is supposed to do. For edge cases: these cases are where the code is most likely to contain mistakes, so they should be tested more thoroughly than normal cases. For error cases: the test should confirm that the program does something reasonable [which might be just giving an error message] rather than trying to return a nonsensical answer.)
  • How do these fit into "designing" a quality software test? (Example answer [answers will vary]: These cases are part of the analysis phase of the design process. In the analysis phase, the software designer must decide what cases are required to adequately test the software. Without thorough testing, the software is not adequately prepared for large scale consumer use.)

Contributors

Ryan Stejskal, Brian Sandall, Janet Yowell

Copyright

© 2013 by Regents of the University of Colorado; original © 2012 Board of Regents, University of Nebraska

Supporting Program

IMPART RET Program, College of Information Science & Technology, University of Nebraska-Omaha

Acknowledgements

The contents of this digital library curriculum were developed as a part of the RET in Engineering and Computer Science Site on Infusing Mobile Platform Applied Research into Teaching (IMPART) Program at the University of Nebraska-Omaha under National Science Foundation RET grant number CNS 1201136. However, these contents do not necessarily represent the policies of the National Science Foundation, and you should not assume endorsement by the federal government.

Last modified: September 5, 2017

Comments