Abstract

Abstract. Recent advances in infrared (IR) imaging sensor technology have made it possible build IR cameras that feature increasing sensitivity, resolution, and frame rates. These cameras, when integrated into a compact system with communication and computation capabilities, can feed a personal IR visualization device and/or be part of a distributed intelligent IR computer vision system, thus enabling and expanding such diverse applications as border surveillance, noninvasive early tumor detection, preventive maintenance of industrial equipment, spectrography, nondestructive material testing, waste sorting, food inspection, search-and-rescue operations, and art examination, among many others. However, for the applications listed above to be feasible in a portable or distributed environment, the IR camera must complain with severe restrictions in size, cost, weight, and power consumption. Moreover, IR image visualization and analysis require the computation of image correction, image processing, and computer vision algorithms that impose very high requirements in computational throughput in order to be executed in real time. The algorithms must be executed at the camera in order to avoid the use of external or remote computers that increase the cost and size of the system, or require very high communication bandwidth and power consumption for live video streaming. Traditional embedded computing solutions based on digital signal processors can not achieve such high throughput at low power, while custom digital image processing circuits are di!cult to design and lack the flexibility that enables the rapid development of new applications. In this proposal, we aim to develop a Smart Infrared Camera (SmIRC) system, which integrates an IR imager, a specialized processor, and local communication and visualization interfaces. The core of the system is the processor, which features an architecture that combines traditional embedded processors with a reconfigurable computing engine to achieve high performance with low power in a compact package. We will approach the research in three main fronts: We will design the architecture of the specialized processor, defining the granularity of the reconfigurable fabric and the interface between hardware and software. We will design a programming model for the processor based on a streaming architecture that combines dataflow-centered hardware modules and multithreaded software components. We will validate our results by mapping applications to the architecture and programming model, including image correction, processing, and distributed computer vision algorithms. We will target standalone applications running on the camera, such as nonuniformity correction, temperature stabilization, super resolution, and face detection. We will also implement distributed algorithms that integrate information from multiple SmIRCs and distribute the computation between their specialized processors, such as multi-camera registration and multispectral image synthesis. The three tasks of architecture design, programming model, and application development, will be carried out largely concurrently because findings on one front influence design decisions on the other two, leading to an iterative design process. As a result of the project, we aim to build a prototype of our SmIRC processor architecture, integrate it with commercial IR camera cores, and test our applications with real-world data. We will support our programming model with a tool flow that integrates custom-built tools with existing hardware and software compilers.