SummaryStudent groups use the Java programming language to implement the algorithms for optical character recognition (OCR) that they developed in the associated lesson. They use different Java classes (provided) to test and refine their algorithms. The ultimate goal is to produce computer code that recognizes a digit on a scoreboard. Through this activity, students experience a very small part of what software engineers go through to create robust OCR methods. This software design lesson/activity set is designed to be part of a Java programming class.
This hands-on activity provides students with experience in using the engineering design process (more specifically, the software design process; a specific application of the engineering design process) in a controlled environment. Design is a critical part of engineering. Engineers of all disciplines follow the steps of the design process to develop and improve solutions to real-world problems. Software engineers perform exactly the same process that students complete in the activity when they develop the software packages we use everyday. They define a problem, design solutions, implement those designs, test the implementation, and evolve the solutions.
Students and teachers must have a strong understanding and experience with Java programming basics.
After this activity, students should be able to:
- Explain software testing and challenges involved in software testing.
- Apply optical character recognition as a Java method.
- Determine the reliability of their OCR implementations.
More Curriculum Like This
Students explore the concept of optical character recognition (OCR) in a problem-solving environment. They research OCR and OCR techniques and then apply those methods to the design challenge by developing algorithms capable of correctly "reading" a number on a typical high school sports scoreboard....
Students focus on the testing phase of the design process by considering how they have tested computer programs in the past and learning about a new method called JUnit to test programs in the future. JUnit is a testing method that is included with NetBeans (Java) installs or can be downloaded from ...
In this lesson, students learn about, design and create flow charts for different scenarios, including a game based on the Battleship® created by Hasbro©. In the associated activity, Flow Charting App Inventor, students apply their knowledge from this lesson and gain experience with a software appli...
Students gain experience with the software/system design process, closely related to the engineering design process, to solve a problem. The lesson culminates in a hands-on experience with the design process as students simulate the remote control of a rover.
Each TeachEngineering lesson or activity is correlated to one or more K-12 science,
technology, engineering or math (STEM) educational standards.
All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN),
a project of D2L (www.achievementstandards.org).
In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics;
within type by subtype, then by grade, etc.
Each TeachEngineering lesson or activity is correlated to one or more K-12 science, technology, engineering or math (STEM) educational standards.
All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN), a project of D2L (www.achievementstandards.org).
In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics; within type by subtype, then by grade, etc.
- The design process includes defining a problem, brainstorming, researching and generating ideas, identifying criteria and specifying constraints, exploring possibilities, selecting an approach, developing a design proposal, making a model or prototype, testing and evaluating the design using specifications, refining the design, creating or making it, and communicating processes and results. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment? Thanks for your feedback!
- Design problems are seldom presented in a clearly defined form. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment? Thanks for your feedback!
- Analyze characteristics and properties of two- and three-dimensional geometric shapes and develop mathematical arguments about geometric relationships (Grades Pre-K - 12) Details... View more aligned curriculum... Do you agree with this alignment? Thanks for your feedback!
- use geometric ideas to solve problems in, and gain insights into, other disciplines and other areas of interest such as art and architecture (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment? Thanks for your feedback!
- Create geometric models to visualize, describe, and solve problems using similar triangles, right triangles, and trigonometry. (Grades 9 - 11) Details... View more aligned curriculum... Do you agree with this alignment? Thanks for your feedback!
- Characteristics: Students will identify and describe geometric characteristics and create two- and three-dimensional shapes. (Grade 12) Details... View more aligned curriculum... Do you agree with this alignment? Thanks for your feedback!
Each group needs:
- computer (with Internet access)
- OCR algorithm, in flow chart, pseudocode or similar paper format; created in the associated lesson
- Java programming environment; install a program such as NetBeans (free) or download software such as: http://introcs.cs.princeton.edu/java/mac/ for Mac OS X (free) or https://textpad.com/ for Windows (trial copy)
- PictureEdit.java, DigitReader.java, ScoreMaker.java; students need these three .java files to get their programs running; they provide a framework and simplify the process of creating OCR software; they are provided in the ScoreMaker.zip
- (optional) access to a digital camera, to take a few images of school scoreboards in use
- Pre-Activity Quiz, one per student
- Post-Activity Quiz, one per student
In this activity, you will obtain valuable hands-on experience with the steps of the engineering design process. All engineers follow some type of design process as they develop solutions for problems. As you play the part of a team of software engineers, your challenge is to create a Java program that can perform optical character recognition (OCR) on one digit of a scoreboard. This is a small part of OCR, but is a good place to start.
How do software engineers approach a new challenge? (Listen to student ideas; expect them to know this.) They first define the difficulties and constraints that affect the problem and then create design solutions that address the problem and those constraints. You did this in the associated lesson when you created your algorithms on paper. Now, you will see if what you designed will work. You will implement, test and refine your algorithms, which are the next steps in the engineering design process.
Engineers work collaboratively in teams to develop software, so you will not work alone. Remember that teams benefit from the different resources and ideas that each person brings. That means that teams have the potential to solve problems more easily than people working alone. As you work in your group, build on each other's ideas and expertise to improve your design solutions.
Java: An object oriented computer programming language.
optical character recognition: The mechanical or electronic conversion of handwritten, typewritten or printed text into machine-encoded text. (OCR)
ScoreMaker software: A java-based software designed by the author and provided as an attachment to this activity.
test oracle: In software testing, a set of known input data and a corresponding set of known or expected output data.
Optical character recognition (OCR) is the process of having a machine recognize characters (printed, scripted, etc.) and turn them into machine characters that can be manipulated on a computer (or other machine) in some way. To create a program that covers all aspects of OCR is nearly impossible due to the different nature of typefaces, scripts and individual handwriting. This means that to thoroughly test an OCR software package for all possible cases it might encounter is impossible. As part of this activity, students think about and explore the nature of this impossibility in testing. Have students consider the following questions as they work through the activity:
- How will you handle the impossibility of testing every possible case that might ever be applied to the algorithm?
- What must an algorithm do in order to perform well enough? How will you know?
Before the Activity
- Make copies of the Pre-Activity Quiz and Post-Activity Quiz, one each per student.
- Arrange for enough computers—at least one for each student group.
- Make sure each computer has a Java programming environment available, as well as the ScoreMaker software and the three Java classes (provided in ScoreMaker.zip) so students can test their OCR implementations.
- If possible, have student groups each take two or three different digital photographs of suitable scoreboards in use at the school's sports venues. Alternatively, or in addition, find images on the Internet (see Figure 2). These images will be used for "real-world" testing of their programs, after use of the three images packaged with the ScoreMaker software in ScoreMaker.zip, and using the Pixel Editor program mode to simulate scoreboard images for initial testing purposes.
- Make sure student groups have handy the OCR algorithms they created in the associated lesson.
With the Students
- Administer the pre-quiz, as described in the Assessment section.
- Present the Introduction/Motivation content to the class.
- Have students sort into their same groups as in the associated lesson, groups of two to three students each. Have each group assemble at a computer with its already-developed OCR algorithm.
- Have groups begin by using a Java programming environment to implement the OCR algorithm they created in the associated lesson.
- Review with students the engineering challenge: To read ONE digit from a typical high school scoreboard, such as one you might find on a football field or in a gym. These digits are generally set up in a seven-segment array (see Figure 1).
- Work with the class to help students think through the creation of their Java programs. Some example topics and points to make:
- How does a computer "see" a scoreboard? With images, the computer only sees a cluster of pixels, so we must use the colors of each pixel to determine what we are looking at.
- As a class, start to process the scoreboard images and discuss some pre-processing techniques like turning color images into black and white images. Doing this makes it easier for the next step—to determine the digit in the image.
- Talk about the seven-segment display that is common to most digital scoreboards. Work through how it might be determined whether a segment was on or off. At this point, expect groups to begin to take more independent approaches. How will your group's program determine if the top segment was on or off?
- Give teams some time to test and modify their Java programs with the aim to achieve the most reliable readers possible. Direct them to test them using the ScoreMaker software package (the zip file).
- Have students use the Lights Editor to precisely control what images look like and to see how the software works on similar-looking numbers, such as 9 compared to 4, digits that are very similar except for the top segment.
- Have students test thoroughly with many different scoreboard images, such as the three available with the ScoreMaker software and simulated scoreboard images made with the Pixel Editor program mode.
- After testing in the Lights Editor, have students perform "real-world" tests by using their own digital photographs of in-use school scoreboards and images from the Internet. Expect the testing of these individual numbers to reveal situations in which groups' algorithms fail, leading them to iterate their designs further.
- After additional testing and revising, expect all groups to have some level of optical character recognition software developed. As a class, test the different solutions alongside each other using the "Select OCR" feature; this enables testing to see how each group's software works in as many different situations as can be accomplished.
- After each group has optimized its program to what they deem is its best potential, have each group present to the class how its program functions, its algorithm logic, how they tested, what steps its algorithm takes to find the number, and how well it works using the ScoreMaker software. Then, as a class, test several images using each OCR algorithm design in turn. Have groups explain how they decided if a segment was on or off so that everyone can see the varying levels of success with each approach.
- As a cumulative effort, have all the groups work together to create a class "final version" of the OCR software that builds on the class best practices.
- To conclude, hold a class discussion to talk about how the lack of a testing oracle affects the testing of an OCR implementation and what this means for a final product. Is it guaranteed to work in all cases? If not, what is good enough? How do you know? Since you cannot anticipate and test every possible scenario, you may not be certain that the software will work in all situations. What other types of software might be susceptible to this kind of uncertain results?
- Ask students if they know how scientists and engineers generally differ in their work. (Listen to student ideas.) Make the point that scientists and engineers both contribute to the world of human knowledge, but in different ways.
- Scientists follow the steps of the scientific method to make testable explanations and predictions about what they observe and want to discover in the world. Scientists ask questions and develop experiments, or sets of experiments, to answer questions.
- Engineers follow the steps of the engineering design process to create solutions to problems and to create things that have never existed before, such as products, websites, structures and processes. Engineers identify specific needs and then create solutions that meet those needs. They think: Who need(s) what because why?
- Administer the post-quiz, as described in the Assessment section.
Pre-Quiz: Before starting the activity, administer the three-question Pre-Activity Quiz. The pre-quiz ask students to consider challenges in designing and testing software that does not allow for full testing. Example answers are provided on the Pre-Activity Quiz Answer Key.
Activity Embedded Assessment
Observations: As students are engaged in the activity, ask yourself the following (or similar) questions:
- Do students grasp the idea that many software programming situations do not lend themselves to be absolutely, fully tested?
- Are students able to identify techniques to test software when no oracle or known answers are available for every question?
Presentations: After groups have polished their programs, have each team present to the class how its program functions, its algorithm logic, its testing discoveries and how well it works using the ScoreMaker software. Then, as a class, test several images using each OCR algorithm design in turn, so everyone sees the varying levels of success with each approach. Use this opportunity to assess group accomplishments in maximizing testing results to build robust and accommodating algorithms.
Post-Quiz: At activity end, administer the Post-Activity Quiz. Through three essay questions, assesses students individually on their understanding of software that is not able to be fully tested for all scenarios and resulting implications. Example answers are provided on the Post-Activity Quiz Answer Key.
Automatic Number Plate Recognition. Last updated April 14, 2014. Wikipedia, The Free Encyclopedia. Accessed April 16, 2014. http://en.wikipedia.org/wiki/Automatic_number_plate_recognition
Comparing the Engineering Design Process and the Scientific Method. Science Buddies. Accessed April 15, 2014. http://www.sciencebuddies.org/engineering-design-process/engineering-design-compare-scientific-method.shtml
Copyright© 2014 by Regents of the University of Colorado; original © 2013 Board of Regents, University of Nebraska
Supporting ProgramIMPART RET Program, College of Information Science & Technology, University of Nebraska-Omaha
The contents of this digital library curriculum were developed as a part of the RET in Engineering and Computer Science Site on Infusing Mobile Platform Applied Research into Teaching (IMPART) Program at the University of Nebraska-Omaha under National Science Foundation RET grant number CNS 1201136. However, these contents do not necessarily represent the policies of the National Science Foundation, and you should not assume endorsement by the federal government.
Last modified: September 26, 2018