Lesson: Do You See What I See?

Contributed by: IMPART RET Program, College of Information Science & Technology, University of Nebraska-Omaha

The digits zero through nine in the OCR-A typeface, which looks like a stereotypical "computer" typeface with simple, thick strokes.
Engineers developed optical character recognition technology to scan and recognize digits and text.
copyright
Copyright © 2008 John Sauter, Wikimedia Commons http://commons.wikimedia.org/wiki/File:OCR-A_char_digits.svg

Summary

Students explore the concept of optical character recognition (OCR) in a problem-solving environment. They research OCR and OCR techniques and then apply those methods to the design challenge by developing algorithms capable of correctly "reading" a number on a typical high school sports scoreboard. Students use the structure of the engineering design process to guide them to develop successful algorithms. In the associated activity, student groups implement, test and revise their algorithms. This software design lesson/activity set is designed to be part of a Java programming class.

Engineering Connection

Like software engineers, students are challenged to design a new piece of software to perform a task. In the development of any new design, engineers follow the steps of the design process as they create and refine possible solutions. This is true in all engineering disciplines, for example, as mechanical engineers develop new vehicles or rollercoasters, or as chemical engineers experiment with new combinations of chemicals or new applications for existing chemicals. Through the lesson and its associated activity, students experience the entire software/systems design process. They perform the requirement analysis and design phases during the lesson, and the implementation, testing and evolution phases during the associated activity. The lesson is a hands-on performance-based project in which groups work through the same design process steps that real-world engineering teams do in order to accomplish goals.

Pre-Req Knowledge

Students must have an understanding of Java programming basics. Although their knowledge need not be advanced, students must be able to develop code that needs to be tested in order to be able to design test cases as part of the larger design process.

Learning Objectives

After this lesson, students should be able to:

  • Apply some steps of the design process to solving a problem.
  • Describe the engineering design process and the software design process.
  • Explain the importance of using a design process to develop a solution to a problem.
  • Explain optical character representation (OCR) and techniques that can be used by OCR.
  • Develop an algorithm for OCR.

More Curriculum Like This

Java Programming of OCR

Student groups use the Java programming language to implement the algorithms for optical character recognition (OCR) that they developed in the associated lesson. They use different Java classes (provided) to test and refine their algorithms. The ultimate goal is to produce computer code that recogn...

High School Activity
Curiosity Killed the App

Students gain experience with the software/system design process, closely related to the engineering design process, to solve a problem. The lesson culminates in a hands-on experience with the design process as students simulate the remote control of a rover.

Middle School Lesson
Making the Connection

Students learn and apply concepts and methods of graph theory to analyze data for different relationships such as friendships and physical proximity. They are asked about relationships between people and how those relationships can be illustrated.

High School Lesson
Program Analysis Using App Inventor

In this lesson, students learn about, design and create flow charts for different scenarios, including a game based on the Battleship® created by Hasbro©. In the associated activity, Flow Charting App Inventor, students apply their knowledge from this lesson and gain experience with a software appli...

Educational Standards

Each TeachEngineering lesson or activity is correlated to one or more K-12 science, technology, engineering or math (STEM) educational standards.

All 100,000+ K-12 STEM standards covered in TeachEngineering are collected, maintained and packaged by the Achievement Standards Network (ASN), a project of D2L (www.achievementstandards.org).

In the ASN, standards are hierarchically structured: first by source; e.g., by state; within source by type; e.g., science or mathematics; within type by subtype, then by grade, etc.

  • Use geometric shapes, their measures, and their properties to describe objects (e.g., modeling a tree trunk or a human torso as a cylinder). (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • The design process includes defining a problem, brainstorming, researching and generating ideas, identifying criteria and specifying constraints, exploring possibilities, selecting an approach, developing a design proposal, making a model or prototype, testing and evaluating the design using specifications, refining the design, creating or making it, and communicating processes and results. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Design problems are seldom presented in a clearly defined form. (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Analyze characteristics and properties of two- and three-dimensional geometric shapes and develop mathematical arguments about geometric relationships (Grades Pre-K - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • use geometric ideas to solve problems in, and gain insights into, other disciplines and other areas of interest such as art and architecture (Grades 9 - 12) Details... View more aligned curriculum... Do you agree with this alignment?
  • Create geometric models to visualize, describe, and solve problems using similar triangles, right triangles, and trigonometry. (Grades 9 - 11) Details... View more aligned curriculum... Do you agree with this alignment?
  • Characteristics: Students will identify and describe geometric characteristics and create two- and three-dimensional shapes. (Grade 12) Details... View more aligned curriculum... Do you agree with this alignment?
Suggest an alignment not listed above

Introduction/Motivation

(Have available computers with Internet access, one per group, so students can conduct their own background research on optical character recognition.)

In this lesson, you will work as if you were an engineer challenged to develop a solution to a problem. You will use the software/system design process (see Figure 1) to solve a problem that is very complex. Your task is to learn about and then design a computer program to perform optical character recognition (OCR). The creation of OCR is VERY complex, so to begin, it will be greatly simplified. You will learn about OCR and then create an algorithm that can be used to "read" a number off a typical high school scoreboard.

A circular diagram shows the five fundamental steps of software design: 1) requirement analysis (software specification), 2) design, 3) implementation (coding), 4) testing (validation), 5) evolution (maintenance and further development). It shows that the process is circular in nature.
Figure 1. The steps of the software development life cycle.
copyright
Copyright © 2012 Cliffydcw, Wikimedia Commons http://commons.wikimedia.org/wiki/File:SDLC_-_Software_Development_Life_Cycle.jpg

In the associated activity, you will implement your algorithm in a Java programming environment and test/redesign it with the goal of getting it to function correctly. To complete this task, you will follow the steps that all engineers use to solve problems—the engineering design process. All types of engineers perform these same steps: determine the problem requirements and constraints, come up with ideas and design the best solution, create/implement that solution, test and refine it and then see where the solution takes them over time (evolution). Software engineers do exactly this when they develop new software. They have a problem in mind for the software to solve while meeting certain requirements. Next, a design is created, implemented and tested. When (not if) the software application fails in some aspect, engineers return to earlier steps of the design process in order to improve the product.

The software/system design process has some similarities to the scientific method, with which you may be familiar. Both are problem solving processes that guide scientists and engineers in a structured way to contribute to the world of human knowledge, but they do so in different ways. The steps of the scientific method include: formulate a question, make a hypothesis, make predictions, test, and analyze. Scientists make testable explanations and predictions about the world to find out how nature works while engineers create solutions to problems, generating new things such as products, websites, environments and experiences. The design process that engineers use serves them well to develop solutions to complex problems. To learn the process, you must practice in controlled situations. This lesson and activity gives you a "playground" to explore.

To begin, let's start with researching optical character recognition (OCR) processes. Your goal is to learn about the challenges of getting a computer to read text, which is not an easy task. (Write the following questions on the classroom board.) Use these questions to help frame your research:

  • What are some uses of optical character recognition (OCR)?
  • What kinds of people (jobs) would use OCR or find it useful?
  • What are some challenges in developing OCR software?
  • How can computers learn to read human writing?

In your research, look at different OCR methods and take notes on what you learn. (Write the following topics on the board.) Some example OCR methods and techniques are:

  • Image pre-processing
  • Image de-skewing
  • Focusing of the window on the relevant data
  • Seven-segment specific testing

When you have gained some understanding of OCR and its methods through research, you'll focus on the more specific problem of reading a digit from a sports scoreboard.

(Continue by delivering the content in the Lesson Background section.)

Lesson Background and Concepts for Teachers

Implementing the Lesson

A diagram generally looks like a figure eight (two stacked circles), but it is formed by the close placement of seven trapezoid-shaped segments labeled A through G. The segments are arranged as a rectangle of two vertical segments on each side with one horizontal segment on the top, middle and bottom. The seventh segment bisects the rectangle horizontally. Next to the "figure 8" sits a dot with the letters "DP" in it, for "decimal point," as the eighth segment.
Figure 2. A seven-segment display number with labels.
copyright
Copyright © 2006 H2g2bob, Wikimedia Commons http://en.wikipedia.org/wiki/File:7_segment_display_labeled.svg

Divide the class into groups of two or three students each, and present the engineering challenge: To read ONE digit from a typical high school scoreboard, such as one you might find on a football field or in a gymnasium. These digits are generally set up in a seven-segment array (see Figure 2). To do this, you will perform the first two steps of the system/software design process: identify requirements of the problem and create an algorithm (your "design solution") on paper that will later be implemented and tested. Your algorithm can be in the form of a flow chart, pseudocode or something similar. An example might be: "Look at the seven major points on Figure 2 and decide if the points are lighted or not. After gathering the "lighted" or "not lighted" information, determine what digit you have." This is an example of pseudocode. The seven points in question are the seven segments that make up scoreboard numbers.

When each group has what it thinks is a viable algorithm, have groups share their ideas and discuss each algorithm's perceived strengths and weaknesses.

Then lead a class discussion, going through the following research questions to give students the opportunity to share what they learned about OCR:

  • What are some uses of OCR? (Example answer: To turn printed text into digital text. To turn a photograph of a license plate into a known license plate identifier composed of digital letters and numbers.)
  • What kinds of people (jobs) would use OCR or find it useful? (Example answers: People who work at the post office use OCR for automatic routing of mail. Scholars who want to make the writing in old manuscripts and out-of-print books searchable. )
  • What are some challenges in developing OCR software? (Example answer: Letters have more than one way of being represented, such as capitals vs. lowercase, script vs. printed/block, handwritten vs. typed.)
  • How can computers learn to read human writing? (Answer: Machine learning, testing and examples.)

Background & Concepts

The testing of software without an oracle—situations in which it is impossible to test all possible cases—is very difficult. The lesson is a hands-on design-based project in which students are faced with developing a software application in which testing is very difficult. They are challenged to design, implement, test and redesign in order to correctly address the problem. They are able to test in "real-world" situations by using photographs of scoreboards from the Internet or school sports venues. Expect students to find that their algorithm projects do not function correctly all the time. Attaining correctness is part of the challenge. However, another part of the activity is how students handle the design process and the vagueness of the project. Consider all this when assessing students.

Testing Software without an Oracle: Software is tested to make sure that the code does what the programmers and designers intend. Some programs can be easily tested with a set of known input data and a corresponding set of known or expected output data. This "test oracle" is used to see if the software passes or fails a series of tests. Unfortunately, it is far more common that software, once in use, will be subjected to a wide array of input data such that it is impossible or impractical to test every case in the software design process. In these cases, because the exact set of input data is unknown, oracles cannot be created to test the programs.

Several manufacturers are embarking on an exciting field with an enormous potential impact on society—the automation of the driving process by creating driverless cars. Much like optical character recognition, it is not possible to ever say that a driverless vehicle is "perfect," thus software engineers must test as much as they can because the existence of so many potential scenarios makes it impossible to test every one. Thus, the algorithm must be adaptive enough to handle situations that were not expected or tested.

Optical Character Recognition: OCR is an emerging computer science technology in which an image of text is "fed" to a computer and processed to convert the image into a text document. (Note: "Fed" is the general term for getting data into the computer where it can be used. Images might include those retrieved from the Internet or captured with a digital camera as photographs.) In image form, letters and numbers are only pixels of color. Once identified via OCR as specific letters and numbers, the text is able to be manipulated, searched, indexed or otherwise modified. OCR is a prime example of a case in which every possible data input cannot be tested. Computer scientists must create a robust algorithm to determine what letter is being displayed in an assemblage of pixels in an image. A variety of methods and techniques have been developed to help prepare and identify pertinent text for OCR processing.

Image Pre-Processing: To make images more readable, we eliminate unnecessary data. One way is to convert color images to black and white images to improve the contrast. This can be done so the image only uses black or white, no grays, resulting in a starkly contrasted image that is easier to work with.

Image De-Skewing: Often, images are captures at angles, instead of head-on. By identifying the corners of the information we are interested in, we can adjust the image to make the text appear on straight lines even if the photo was taken at an angle.

Focusing of the Window on the Relevant Data: Since scoreboards do not move and cameras can be mounted in static locations, we can manually identify sub-areas within larger scoreboard images where individual digits are expected to appear. By only looking at known, single digits, we can focus our algorithms.

Seven-Segment Specific Testing: Each number in a display is comprised of seven segments. To read letters, we would need more than seven data points. The benefit of narrowing our OCR to scoreboards is that if we have knowledge of these seven segments and their on/off statuses, we can determine which numbers are being displayed.

Machine Learning and Training: Machine learning is a technique of enabling computers to decide what aspects of a letter are important. For example, a letter is fed into a computer and the computer is told that the letter is an "A." More examples of the letter "A" are fed into the computer until it creates enough of its own rules so as to be able to identify what an "A" is, at which point a random letter could be correctly identified as an "A" by the computer based on criteria unknown to the programmer but understood by the computer.

Applications: Currently, OCR is used to digitize books and other writing created before the advent of computers. This enables the whole of human writing to be stored in digital format so that it can be protected, indexed and searched. The U.S. Postal Service uses OCR as a time- and cost-saving measure to automatically route mail without human intervention. Automatic number plate recognition uses OCR to read vehicle registration plates for use at toll roads, toll bridges, red light cameras, speed monitoring cameras, police surveillance, etc.

Vocabulary/Definitions

engineering design process: A series of steps engineers use as they design solutions for problems and challenges. The steps include: defining a problem, brainstorming, researching and generating ideas, identifying criteria and specifying constraints, exploring possibilities, selecting an approach, developing a design proposal, making a model or prototype, testing and evaluating the design, refining the design, creating or making it, and communicating processes and results.

Java: An object-oriented computer programming language.

optical character recognition: The mechanical or electronic conversion of handwritten, typewritten or printed text into machine-encoded text.

pseudocode: A high-level description of how a computer program or other algorithm functions.

test oracle: In software testing, a set of known input data and a corresponding set of known or expected output data.

Associated Activities

  • Java Programming of OCR - Using a Java programming environment, student groups create the algorithms for OCR that they developed in the lesson. They continue with the steps of the engineering design process as they implement, test and revise their algorithms. Ultimately, students create programs that can read numbers on a scoreboard.

Attachments

Assessment

Pre-Lesson Assessment

Questions: Obtain a sense of students' base knowledge of the engineering design process by asking them the following questions:

  • Describe any experience you have had with the engineering design process or scientific method.
  • What were the steps in the design process?
  • How did the steps function together in a cyclical process?

Post-Introduction Assessment

Observations: As students are engaged in the lesson research and algorithm development, ask yourself the following (or similar) questions:

  • Do students understand what the design process is and how to apply it?
  • Are students able to explain what OCR is and how it works?
  • Can students verbalize how different OCR techniques work?

Lesson Summary Assessment

Post-Quiz: Administer the Do You See What I See? Post-Quiz, composed of three essay questions. Review students' answers to gauge how much they know about the software design process, OCR and how the design process plays a role in developing a successful algorithm. Example answers are provided in the Do You See What I See? Post-Quiz Answer Key.

Additional Multimedia Support

The source of the seven-segment display number graphic, which may be helpful to show students as an overhead projector image or handout, is available at Wikimedia Commons at http://en.wikipedia.org/wiki/File:7_segment_display_labeled.svg.

See the Science Buddies' Comparing the Engineering Design Process and the Scientific Method for good explanations and flow charts of the steps of both processes at http://www.sciencebuddies.org/engineering-design-process/engineering-design-compare-scientific-method.shtml.

References

Automatic Number Plate Recognition. Last updated April 14, 2014. Wikipedia, The Free Encyclopedia. Accessed April 16, 2014. http://en.wikipedia.org/wiki/Automatic_number_plate_recognition

Comparing the Engineering Design Process and the Scientific Method. Science Buddies. Accessed April 15, 2014. http://www.sciencebuddies.org/engineering-design-process/engineering-design-compare-scientific-method.shtml

Contributors

Derek Babb

Copyright

© 2014 by Regents of the University of Colorado; original © 2013 Board of Regents, University of Nebraska

Supporting Program

IMPART RET Program, College of Information Science & Technology, University of Nebraska-Omaha

Acknowledgements

The contents of this digital library curriculum were developed as a part of the RET in Engineering and Computer Science Site on Infusing Mobile Platform Applied Research into Teaching (IMPART) Program at the University of Nebraska-Omaha under National Science Foundation RET grant number CNS 1201136. However, these contents do not necessarily represent the policies of the National Science Foundation, and you should not assume endorsement by the federal government.

Last modified: September 5, 2017

Comments