Hands-on Activity: Java Programming of OCR

Contributed by: IMPART RET Program, College of Information Science & Technology, University of Nebraska-Omaha

Three images of the license plate number 4YCH428. The first image shows black letters on a gray background. Next, the background is white. Then, each letter/number has a red box around it.
Automatic number plate recognition uses OCR to read vehicle plates. This license plate image has been de-skewed, sized, adjusted for brightness and contrast, and its individual characters segmented so it is ready for OCR.
copyright
Copyright © 2006 Achim Raschka, Wikimedia Commons http://commons.wikimedia.org/wiki/File:California_license_plate_ANPR.png

Summary

Student groups use the Java programming language to implement the algorithms for optical character recognition (OCR) that they developed in the associated lesson. They use different Java classes (provided) to test and refine their algorithms. The ultimate goal is to produce computer code that recognizes a digit on a scoreboard. Through this activity, students experience a very small part of what software engineers go through to create robust OCR methods. This software design lesson/activity set is designed to be part of a Java programming class.

Engineering Connection

This hands-on activity provides students with experience in using the engineering design process (more specifically, the software design process; a specific application of the engineering design process) in a controlled environment. Design is a critical part of engineering. Engineers of all disciplines follow the steps of the design process to develop and improve solutions to real-world problems. Software engineers perform exactly the same process that students complete in the activity when they develop the software packages we use everyday. They define a problem, design solutions, implement those designs, test the implementation, and evolve the solutions.

Pre-Req Knowledge

Students and teachers must have a strong understanding and experience with Java programming basics.

Learning Objectives

After this activity, students should be able to:

  • Explain software testing and challenges involved in software testing.
  • Apply optical character recognition as a Java method.
  • Determine the reliability of their OCR implementations.

More Curriculum Like This

Do You See What I See?

Students explore the concept of optical character recognition (OCR) in a problem-solving environment. They research OCR and OCR techniques and then apply those methods to the design challenge by developing algorithms capable of correctly "reading" a number on a typical high school sports scoreboard....

High School Lesson
Using JUnit

Students focus on the testing phase of the design process by considering how they have tested computer programs in the past and learning about a new method called JUnit to test programs in the future. JUnit is a testing method that is included with NetBeans (Java) installs or can be downloaded from ...

High School Lesson
Curiosity Killed the App

Students gain experience with the software/system design process, closely related to the engineering design process, to solve a problem. The lesson culminates in a hands-on experience with the design process as students simulate the remote control of a rover.

Middle School Lesson
Program Analysis Using App Inventor

In this lesson, students learn about, design and create flow charts for different scenarios, including a game based on the Battleship® created by Hasbro©. In the associated activity, Flow Charting App Inventor, students apply their knowledge from this lesson and gain experience with a software appli...

Educational Standards

Each TeachEngineering lesson or activity is correlated to one or more K-12 science, technology, engineering or math (STEM) educational standards.

All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN), a project of D2L (www.achievementstandards.org).

In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics; within type by subtype, then by grade, etc.

  • Use geometric shapes, their measures, and their properties to describe objects (e.g., modeling a tree trunk or a human torso as a cylinder). (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • The design process includes defining a problem, brainstorming, researching and generating ideas, identifying criteria and specifying constraints, exploring possibilities, selecting an approach, developing a design proposal, making a model or prototype, testing and evaluating the design using specifications, refining the design, creating or making it, and communicating processes and results. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Design problems are seldom presented in a clearly defined form. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Analyze characteristics and properties of two- and three-dimensional geometric shapes and develop mathematical arguments about geometric relationships (Grades Pre-K - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • use geometric ideas to solve problems in, and gain insights into, other disciplines and other areas of interest such as art and architecture (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Create geometric models to visualize, describe, and solve problems using similar triangles, right triangles, and trigonometry. (Grades 9 - 11) Details... View more aligned curriculum... Do you agree with this alignment?
  • Characteristics: Students will identify and describe geometric characteristics and create two- and three-dimensional shapes. (Grade 12) Details... View more aligned curriculum... Do you agree with this alignment?
Suggest an alignment not listed above

Materials List

Each group needs:

  • computer (with Internet access)
  • OCR algorithm, in flow chart, pseudocode or similar paper format; created in the associated lesson
  • Java programming environment; install a program such as NetBeans (free) or download software such as: http://introcs.cs.princeton.edu/java/mac/ for Mac OS X (free) or https://textpad.com/ for Windows (trial copy)
  • PictureEdit.java, DigitReader.java, ScoreMaker.java; students need these three .java files to get their programs running; they provide a framework and simplify the process of creating OCR software; they are provided in the ScoreMaker.zip
  • (optional) access to a digital camera, to take a few images of school scoreboards in use
  • Pre-Activity Quiz, one per student
  • Post-Activity Quiz, one per student

Introduction/Motivation

Animated gif of a seven-segmented display (shown in Figure 1) that lights up different combinations of the segments in bright green to form the 0-9 numbers and A-F letters, one at a time, continuously.
A seven-segmented LED displays digits zero through nine, and then rotates through A, b, C, d, E, F.
copyright
Copyright © 2013 INeverCry, Wikimedia Commons http://commons.wikimedia.org/wiki/File:Seven_segment_display-animated.gif

In this activity, you will obtain valuable hands-on experience with the steps of the engineering design process. All engineers follow some type of design process as they develop solutions for problems. As you play the part of a team of software engineers, your challenge is to create a Java program that can perform optical character recognition (OCR) on one digit of a scoreboard. This is a small part of OCR, but is a good place to start.

How do software engineers approach a new challenge? (Listen to student ideas; expect them to know this.) They first define the difficulties and constraints that affect the problem and then create design solutions that address the problem and those constraints. You did this in the associated lesson when you created your algorithms on paper. Now, you will see if what you designed will work. You will implement, test and refine your algorithms, which are the next steps in the engineering design process.

Engineers work collaboratively in teams to develop software, so you will not work alone. Remember that teams benefit from the different resources and ideas that each person brings. That means that teams have the potential to solve problems more easily than people working alone. As you work in your group, build on each other's ideas and expertise to improve your design solutions.

Vocabulary/Definitions

Java: An object oriented computer programming language.

optical character recognition: The mechanical or electronic conversion of handwritten, typewritten or printed text into machine-encoded text. (OCR)

ScoreMaker software: A java-based software designed by the author and provided as an attachment to this activity.

test oracle: In software testing, a set of known input data and a corresponding set of known or expected output data.

Procedure

Background

Optical character recognition (OCR) is the process of having a machine recognize characters (printed, scripted, etc.) and turn them into machine characters that can be manipulated on a computer (or other machine) in some way. To create a program that covers all aspects of OCR is nearly impossible due to the different nature of typefaces, scripts and individual handwriting. This means that to thoroughly test an OCR software package for all possible cases it might encounter is impossible. As part of this activity, students think about and explore the nature of this impossibility in testing. Have students consider the following questions as they work through the activity:

  • How will you handle the impossibility of testing every possible case that might ever be applied to the algorithm?
  • What must an algorithm do in order to perform well enough? How will you know?

Before the Activity

  • Make copies of the Pre-Activity Quiz and Post-Activity Quiz, one each per student.
  • Arrange for enough computers—at least one for each student group.
  • Make sure each computer has a Java programming environment available, as well as the ScoreMaker software and the three Java classes (provided in ScoreMaker.zip) so students can test their OCR implementations.
  • If possible, have student groups each take two or three different digital photographs of suitable scoreboards in use at the school's sports venues. Alternatively, or in addition, find images on the Internet (see Figure 2). These images will be used for "real-world" testing of their programs, after use of the three images packaged with the ScoreMaker software in ScoreMaker.zip, and using the Pixel Editor program mode to simulate scoreboard images for initial testing purposes.
  • Make sure student groups have handy the OCR algorithms they created in the associated lesson.

With the Students

A diagram generally looks like a figure eight (two stacked circles), but it is formed by the close placement of seven trapezoid-shaped segments labeled A through G. The segments are arranged as a rectangle of two vertical segments on each side with one horizontal segment on the top, middle and bottom. The seventh segment bisects the rectangle horizontally. Next to the "figure 8" sits a dot with the letters "DP" in it, for "decimal point," the eighth segment.
Figure 1. A seven-segment display number with labels.
copyright
Copyright © 2006 H2g2bob, Wikimedia Commons http://en.wikipedia.org/wiki/File:7_segment_display_labeled.svg

  1. Administer the pre-quiz, as described in the Assessment section.
  2. Present the Introduction/Motivation content to the class.
  3. Have students sort into their same groups as in the associated lesson, groups of two to three students each. Have each group assemble at a computer with its already-developed OCR algorithm.
  4. Have groups begin by using a Java programming environment to implement the OCR algorithm they created in the associated lesson.
  5. Review with students the engineering challenge: To read ONE digit from a typical high school scoreboard, such as one you might find on a football field or in a gym. These digits are generally set up in a seven-segment array (see Figure 1).
  6. Work with the class to help students think through the creation of their Java programs. Some example topics and points to make:
  • How does a computer "see" a scoreboard? With images, the computer only sees a cluster of pixels, so we must use the colors of each pixel to determine what we are looking at.
  • As a class, start to process the scoreboard images and discuss some pre-processing techniques like turning color images into black and white images. Doing this makes it easier for the next step—to determine the digit in the image.
  • Talk about the seven-segment display that is common to most digital scoreboards. Work through how it might be determined whether a segment was on or off. At this point, expect groups to begin to take more independent approaches. How will your group's program determine if the top segment was on or off?
    A photograph shows a scoreboard with many rectangles of light bulbs, some lighted, some not, thereby creating the desired numbers. The information indicates that the home team is winning 13 to 7 with 12 minutes and 28 seconds to go.
    Figure 2. An example of a typical high school football scoreboard.
    copyright
    Copyright © Everbrite http://www.sportspectator.com/fancentral/football/scoreboard.html
  1. Give teams some time to test and modify their Java programs with the aim to achieve the most reliable readers possible. Direct them to test them using the ScoreMaker software package (the zip file).
  • Have students use the Lights Editor to precisely control what images look like and to see how the software works on similar-looking numbers, such as 9 compared to 4, digits that are very similar except for the top segment.
  • Have students test thoroughly with many different scoreboard images, such as the three available with the ScoreMaker software and simulated scoreboard images made with the Pixel Editor program mode.
  • After testing in the Lights Editor, have students perform "real-world" tests by using their own digital photographs of in-use school scoreboards and images from the Internet. Expect the testing of these individual numbers to reveal situations in which groups' algorithms fail, leading them to iterate their designs further.
  1. After additional testing and revising, expect all groups to have some level of optical character recognition software developed. As a class, test the different solutions alongside each other using the "Select OCR" feature; this enables testing to see how each group's software works in as many different situations as can be accomplished.
  2. After each group has optimized its program to what they deem is its best potential, have each group present to the class how its program functions, its algorithm logic, how they tested, what steps its algorithm takes to find the number, and how well it works using the ScoreMaker software. Then, as a class, test several images using each OCR algorithm design in turn. Have groups explain how they decided if a segment was on or off so that everyone can see the varying levels of success with each approach.
  3. As a cumulative effort, have all the groups work together to create a class "final version" of the OCR software that builds on the class best practices.
  4. To conclude, hold a class discussion to talk about how the lack of a testing oracle affects the testing of an OCR implementation and what this means for a final product. Is it guaranteed to work in all cases? If not, what is good enough? How do you know? Since you cannot anticipate and test every possible scenario, you may not be certain that the software will work in all situations. What other types of software might be susceptible to this kind of uncertain results?
  5. Ask students if they know how scientists and engineers generally differ in their work. (Listen to student ideas.) Make the point that scientists and engineers both contribute to the world of human knowledge, but in different ways.
  • Scientists follow the steps of the scientific method to make testable explanations and predictions about what they observe and want to discover in the world. Scientists ask questions and develop experiments, or sets of experiments, to answer questions.
  • Engineers follow the steps of the engineering design process to create solutions to problems and to create things that have never existed before, such as products, websites, structures and processes. Engineers identify specific needs and then create solutions that meet those needs. They think: Who need(s) what because why?
  1. Administer the post-quiz, as described in the Assessment section.

Attachments

Assessment

Pre-Activity Assessment

Pre-Quiz: Before starting the activity, administer the three-question Pre-Activity Quiz. The pre-quiz ask students to consider challenges in designing and testing software that does not allow for full testing. Example answers are provided on the Pre-Activity Quiz Answer Key.

Activity Embedded Assessment

Observations: As students are engaged in the activity, ask yourself the following (or similar) questions:

  • Do students grasp the idea that many software programming situations do not lend themselves to be absolutely, fully tested?
  • Are students able to identify techniques to test software when no oracle or known answers are available for every question?

Post-Activity Assessment

Presentations: After groups have polished their programs, have each team present to the class how its program functions, its algorithm logic, its testing discoveries and how well it works using the ScoreMaker software. Then, as a class, test several images using each OCR algorithm design in turn, so everyone sees the varying levels of success with each approach. Use this opportunity to assess group accomplishments in maximizing testing results to build robust and accommodating algorithms.

Post-Quiz: At activity end, administer the Post-Activity Quiz. Through three essay questions, assesses students individually on their understanding of software that is not able to be fully tested for all scenarios and resulting implications. Example answers are provided on the Post-Activity Quiz Answer Key.

References

Automatic Number Plate Recognition. Last updated April 14, 2014. Wikipedia, The Free Encyclopedia. Accessed April 16, 2014. http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Comparing the Engineering Design Process and the Scientific Method. Science Buddies. Accessed April 15, 2014. http://www.sciencebuddies.org/engineering-design-process/engineering-design-compare-scientific-method.shtml

Contributors

Derek Babb

Copyright

© 2014 by Regents of the University of Colorado; original © 2013 Board of Regents, University of Nebraska

Supporting Program

IMPART RET Program, College of Information Science & Technology, University of Nebraska-Omaha

Acknowledgements

The contents of this digital library curriculum were developed as a part of the RET in Engineering and Computer Science Site on Infusing Mobile Platform Applied Research into Teaching (IMPART) Program at the University of Nebraska-Omaha under National Science Foundation RET grant number CNS 1201136. However, these contents do not necessarily represent the policies of the National Science Foundation, and you should not assume endorsement by the federal government.

Last modified: August 29, 2017

Comments