Abstract

Abstract. Computer vision systems relaying on infrared (IR) and hyperspectral cameras are being increasingly used in a wide range of applications including surveillance, thermal imaging, and analysis and classification of goods. As the cost of solid-state IR and hyperspectral image sensors decreases, distributed intelligent imaging sensors become a feasible model in industry and defense. This model imposes severe constraints on the size, cost, and power consumption of the individual sensors. The implementation of low-level image restoration algorithms and application-level tasks such as object recognition and classification, must take place locally at the sensor in order to avoid the excessive consumption of communication bandwidth and power demanded by live video streaming. These algorithms need also to operate in real-time, with sufficient performance to process high-resolution images at high frame rates. Under these constraints, dedicated hardware solutions become attractive because they can exploit directly on the digital hardware the parallelism typically available in image processing algorithms, and can achieve also a much better price/performance/power tradeoff than traditional microprocessors or digital signal processors. In this proposal, we intend to develop customizable hardware architectures, low-level image processing algorithms, and application-level algorithms for infrared and hyperspectral cameras. We will develop and adapt algorithms that are optimized not only for performance but also for efficient hardware implementation. Our goal is to design both reconfigurable and custom VLSI implementations of these algorithms and integrate them with the image sensor to perform the aforementioned tasks in real-time, with at least an order of magnitude less power consumption and die area than those of traditional software solutions implemented on microprocessors, simultaneously achieving a much lower cost. Custom VLSI solutions will achieve the best price/power/performance tradeoff, but reconfigurable architectures will allow a degree of flexibility in the implementation which is traditionally associated with software solutions. This proposal comprises work in four main areas: 1. Design of low-level and application-level algorithms for IR image processing, such as nonuniformity correction, stabilization, face recognition, and food quality control. 2. Design and implement VLSI arrays to efficiently execute these algorithms, and integrate them with the image sensors. 3. Design a reconfigurable architecture capable of implementing a class of image processing algorithms and offering flexibility to the programmer after fabrication. 4. Extend these solutions to hyperspectral images and adapting them to efficiently deal with the large dimensionality of hyperspectral data. Throughout this work, we will collaborate in the design of the algorithms and their implementations. We will use mathematical and computational analysis to optimize the performance of the algorithms while minimizing their implementation cost. We will use logic simulation and synthesis to design our VLSI arrays, but will also use rapid prototyping on FPGA boards to experimentally verify their performance in the laboratory, and interface them with real IR and hyperspectral cameras. Finally, we will design and build a VLSI implementation of our arrays, and explore the tradeoffs in reconfigurability and programmability.