Final Project Report Barcode Scanner on a XScale PXA27x Grace Chen {
[email protected]} Philip Cheng {
[email protected]} December 04, 2008 Abstract Modern hand held devices such as smart phones and PDAs have become increasingly powerful in recent years. Dramatic breakthroughs in processing power along with the number of extra features included in these devices has opened the doors to a wide range of commercial possibilities. In particular, most cell phones regularly include cameras, processors comparable to PCs from only a few years ago, and internet access. However, even with all these added abilities, there are few applications that allow much passing or decryption of environmental information. As mobile devices become more like PCs they will come to replace objects we tend to carry around such as checkbooks, credit cards, cameras, planners, mp3 players, etc. In short, we will be using them to accomplish our daily tasks. One application that falls into this category is barcode reading. Current barcode readers often offer decryption of data but few have been applied to online commerce. Specifically, we will be using an Intel Xscale PXA27x device to simulate a barcode scanner with the added ability of adding items to a commercial shopping cart. Index Terms— cross-compile, Intel XScale PXA27x DVK, Linux, LINKS, UPC, Zebra, ifconfig. I. Introduction A. Motivation A barcode is a machine-readable strip of data printed in parallel lines, used to represent a multitude of information. Traditionally, a barcode scanner is used by retailers to keep track of inventory and speed up data entry. Due to their heavy commercial and industrial sector usage, barcode scanning applications have been producer-centric, focusing on improving the efficiency, accuracy, and productivity of supply management. In contrast, we want to create an application that is consumer-centric. This application would function as a personal shopper, assisting the user by allowing him to add items to an online database or "wish list". With browser integration, this application could also be extended to search for product reviews or even price comparisons. The uses for a barcode reader in the marketplace are numerous and there are endless ways to tie it to the consumer to make exciting and innovative applications. For convenience, our device should also be easy to carry and use. We are not expecting a user to haul around a bulky device with limited functionality. Instead, we wanted the application to be loadable onto a small, multifunctional embedded device like a cell phone or PDA. The Intel PXA27x device was the perfect instructional board for this prototype, as it is a general purpose device capable of hosting a web browser, camera, and other useful applications. Using the PXA27x, a barcode reader can be utilized to combine with other applications, providing the customer with state of the art interaction. With this project we hope to add the functionality of a barcode reader into a handheld device, and place it into the palms of an everyday consumer. B. State of the Art Techniques There are many different barcode types that exist for many different purposes. We can split these into 1D and 2D barcodes. 1D barcodes are what most people think barcodes are: columns of varying width lines that are imprinted on the back of products. Within the 1D barcode realm we have: EAN-13/UPC-A, Code 128, Code 39, EAN-8 etc. [3] 1D barcodes are space inefficient, the barcode gets longer as more data is encoded and grows horizontally. Thus, 2D barcodes were invented to encode data not just horizontally, but also vertically in order to save space. [5] Within
the 2D barcode realm we have: DataMatrix, QR code, EZ code, Smart code, etc. Each type of barcode is called a symbology, and there is a corresponding standard that defines the symbol and how to decode and encode the symbol. 1D barcodes are mostly used when space isn't an issue and messages are short. They are cheaper since the technology to decode them is easier, which is why they are more commonplace than 2D barcodes. Barcodes are much cheaper than employing another technology, such as RFID. RFID requires purchase of tags, whereas barcode technology requires just inking costs. Thus from a cost point of view, it makes sense to use barcodes. Additionally, barcodes are already widely available in the marketplace. Since our main use case was to be able to add items to a shopping list, we decided to focus on a 1D barcode reader/scanner. Currently, UPC labels are located on the back of most consumer products whereas 2D barcodes are not as readily available. This means we can find many more sample products to use as test cases. There are several libraries available to decode barcode symbologies. A select few of these libraries are open source. One prominent open source library is the ZXing project. [2] It has the capability to read not just 1D barcodes but also 2D barcodes and is based on the Android platform. Although this library is widely used and has a tremendous amount of technical support, we decided not to use it because of the large framework (Android) that we would need to install and the risk that came with this. We vetoed the JJIL project [9] for a similar reason; it required the installation of J2ME. Also, we wanted a library that was much more portable and light weight, without requiring us to install additional platforms. Libraries that fell into this category included: libdatamatrix, libdecodeqr, and QrCode but these were limited by the barcodes they could actually read, which were only 2D formats. Zebra turned out to be the best library because it is a C/C++ library that doesn't require a special run-time environment to be installed over Linux , was high speed with small memory footprint and code size (< 1k lines of code) , and had the ability to read a wide range of 1D barcodes that are essential to complete the main use case. II. Project Description A. Project Goals Our project has three main goals. The first goal is to replicate a commercial barcode scanner on the Intel PXA27x device. Milestones include configuring the on-board camera and installing image manipulation and barcode decoding libraries. Our second goal is to decode the barcode images in real-time, quickly and efficiently. This involves improving the image decoding algorithms and choosing efficient image manipulation and barcode libraries. Our final goal is to connect to the web to access an online database. Milestones include installing the browser, configuring proper network settings, and finally creating an online shopping application and database. Our main use case will involve adding items to a Christmas shopping list. The user will pick up an item that he desires in the store and then take a picture of its barcode. Afterwards, the barcode contents will be decoded and the product name is found within a UPC database. The user will then be redirected to an online shopping list, with the current item added. There are many possibilities for what could be stored in a barcode. We could denote movie information, website links, a save state or item in a video game, etc. Whatever the case may be, it can be thought of as linking information using photos taken on a common handheld device. Thus, we have streamlined the transfer of data in a simple and elegant manner. B. Project Implementation Our system architecture can be organized into five main components: A camera, an image manipulation library, a barcode decoder, a web browser, and an online shopping application. To capture camera images, we used the camera available on board and modified the ci-capture program. We installed the image manipulation library, ImageMagick [10] to adjust the taken images for quality and convert into desired format. For the barcode decoding libraries, we used Zebra [1]. We also installed a free open source graphics and text mode browser called LINKS[8][7]. Finally, we created a web application which simulated an online shopping list.
The use case sequence, in relation to the five main components, is as follows: A user will utilize the modified cicapture program to take a picture of a barcode, which is held in front of the camera to capture. Using ImageMagick libraries, this saved .yuv is converted into a .gif, cleaned up, and passed to the barcode decoder library. The decoding result is passed to the web browser, and a textual page rendered to simulate adding the item to a user's shopping list. The typical use case is displayed in Figure 2.
Figure 2: Typical use case. The project work involved evaluating the various libraries to find the most efficient solution. We finally settled on installing the following libraries: Zebra, ImageMagick, LINKS, zlib, pkg-config, libpng, IJG libjpeg, and TIFF. In addition, the ci-capture program was modified to save image capture in .yuv format. We further optimized the barcode decoding algorithm by making multiple passes over the image to assemble data in parts. ImageMagick API's was then used for the post-processing work on the image. This included converting the image to grayscale and improving contrast, resulting in a cleaner image with higher chance of successful decoding. The decoding API’s were called to scan the image. A HTML page and a corresponding backend database was created to function as the shopping list. This resulting UPC was then parsed in code and sent to this page. The main software components residing on the Intel PXA270 platform can be seen in Figure 3
.
Figure 3: Software components.
III. Challenges and Limitations Described below are some of the major challenges we had to deal with when building an application for our device. A. Camera Issues One major challenge we had to deal with was camera constraints. Most barcode decoders expect a camera with autofocus and relatively high resolution. Our included camera has neither of these. Additionally, the captured image may need to be filtered to have the correct color, brightness, or quality. Any misalignment, skew, or distortion could cause the barcode to be unreadable. Although Zebra's algorithms try and remedy these problems, these uncertainties cause unreliable results.
Figure 4: Side-by-side comparison. In Figure 4, we have taken a picture of a barcode and tried to pass that image to the scanner to decode. However, it fails to properly decode the image, citing improper contrast, brightness, and size as potential reasons for failing, even after trying to improve the image quality via ImageMagick. There is a stark difference in contrast between the two pictures and the background in our webcam photo is very dark. This is due to the fact that our camera lacks both auto-focus and a flash. The straightforward solution to this problem turned out to be exhaustively testing our size, lighting, and orientation conditions. Beginning with a 10" x 3" barcode and a 45 watt lamp, we were able to remove our autofocus and flash limitations. Slowly, we reduced the amount of light and size of the barcode until we reached our base condition. At the same time, we thoroughly tested skew and orientation of our camera and barcode. It turns out we were able to completely remove the need for additional light and reduce the size of our barcode to 2"x1", a much more reasonable size. In practice, however, there are still some usability issues since the image must be very clear and large. This results in the barcode photo working roughly fifty percent of the time in the worst condition. This could be considered unacceptable depending on our requirements. B. File Format Issues Formatting issues can often render data unusable if they cannot be understood. The initial problem was simply saving our image in a usable format. Firstly, the ci-capture code does not come with much documentation. The file type used in ci-capture is a .yuv file, which is a color representation that takes into account human perception as opposed to the Red-Green-Blue (RGB) color space. YUV is typically used when interfacing with analog or digital photographic equipment. Because ci-capture interfaces with a webcam, this is not unexpected. Our webcam records the data as video and takes the last two frames and stores it into .yuv a file. Unfortunately, Zebra does not recognize this file format. To resolve this issue we installed the ImageMagick library which could convert from .yuv files to .gif files that are usable by our Zebra Image decoder. C. Tool chain Issues Another problem we encountered occurred during installation of the Zebra barcode libraries. Zebra has several dependencies: GTK+, QTV, Python, ImageMagick, and GThread. Each of these libraries had their own set of dependencies, and it was a big roadblock trying to get them installed. We found the OS install to be lacking; many libraries that were considered to be standard weren't present. Cross-compiling added an additional complexity, as some scripts did not provide a cross-compilation option or did not support the armv5 architecture. This was finally resolved by experimenting with the configure file to disable certain dependencies, hacking the Makefiles with hardcoded paths, and manually setting environment variables. D. Networking Issues Although the LINKS browser was able to run on the target, we unfortunately ran out of time when setting up
PCMCIA support. We were able to rebuild the Linux kernel and recognize the card with our controller, but unable to bring up the wireless interface with ifconfig. With a little more time, this problem would have been solved but our priority was getting the camera and decoder to recognize our barcodes efficiently. For our experiments, the addition of a wireless connection was not given much precedence since it is something we already know works. Additionally, a database and webpage were set up to replicate an online shopping cart, but because we were unable to set up our wireless connection they could not be used by our application. IV. Future Work There is a lot of future work that can be done with this project. The user experience could be improved upon. It would greatly increase our success rate if the user were given visual or auditory clues on how to align the barcode image for best image capture and barcode scan. We would like to add a better camera with higher resolution, autofocus, and or macro lens to obtain accurate and fast decoding. Besides adding new hardware, we could also improve the image capture of the camera. For example, we could combine multiple low-resolution, blurred image captures from the camera into one super high resolution image. This improved image would be sharper and cleaner, thus allowing us to decode the barcode despite image restrictions. Beyond the camera, we have challenges such as efficiency of algorithms and power consumption for the device. The barcode reader requires a lot of image processing and file input and output. This requires a huge amount of heap memory to filter and alter the image. Currently we are using BLOB (binary large object representation) which allocates a large buffer for the image. In the future, we would optimize the buffer size for this during image manipulation and barcode scanning. We would also perform benchmarking during maximum memory utilization (ie: several worst case memory consuming applications) to make sure we don't overstep the bounds on memory allocation. V. Conclusion The barcode scanner works very close to how we expected. Because the provided camera has neither autofocus nor flash, it was often difficult to determine the exact cause of problems. However, after extensive testing we were able to produce a working scanner with good results. Thanks to efficient image conversion and decoding processes, the entire process runs predictably within reasonable time constraints; roughly a few seconds. As far as resources are concerned, waste is kept low thanks to our pipelined data processing approach. Although decoding often takes up the majority of the process, the time it takes is usually constant. Therefore, the network connection can be considered the major bottleneck since it may produce uncontrollable results. Barcode decoding with mobile devices is definitely possible with the correct implementation. Yet in practice, results are dependent on the quality of the camera as well as the speed of the network connection. VI. References 1. Zebra. http://zebra.sourceforge.net/ 2. Zxing. http://code.google.com/p/zxing/ 3. Rigal, Daniel. Barcode. http://en.wikipedia.org/wiki/Barcode 4. Information Technology—Automatic Identification and Data Capture Techniques—QR Code 2005 Bar Code Symbology, ISO/IEC 18004, Int'l Organization for Standardization, 2006. 5. Understanding 2D Symbologies: Detailed Information on Barcodes, Japan Automatic Identification Systems Assoc., 2004. 6. T.-D. Han et al., Machine Readable Code Image and Method of Encoding and Decoding the Same, US Patent 7,020,327, US Patent and Trademark Office, 2006. 7. Ihmig, M. 2005. Porting Linux to Intel's Glencoe Platform. Web site: http://www.cs.cmu.edu/~rtmach/mainstone/glencoe.pdf. 8. LINKS. http://links.sourceforge.net/ 9. John's Java Imaging Processing Library. http://code.google.com/p/jjil/w/list 10. ImageMagick. http://www.imagemagick.org/script/index.php