Imagine a pen-sized device to check your skin for melanoma. You skim the surface of your skin, and, if necessary, the pen advises you to see your physician to have a closer look at a certain spot. Such a pen would scan your skin, and detect if skin cancer is developing, even in an early stage. It would distinguish between healthy and suspicious spots, even if you can see no difference.
Future melanoma pen (artist impression
The technology to detect what our eyes can’t see has been available for more than a decade. Called hyperspectral imaging (HSI), it enables to collect and process information from across the electromagnetic spectrum. This is unlike the human eye, which just sees the narrow spectrum of visible light. Every surface and material gives off a specific HSI fingerprint. This enables to distinguish between surfaces that look totally alike to the human eye: regular or fake banknotes, coffee beans or small stones, healthy skin spots or melanoma.
So today already, there are HSI cameras that can check your skin for melanoma. However, the machinery that you’d need is bulky and difficult to set up, and the cost would be prohibiting.
Imec has challenged itself to overcome the HSI hurdle set by the size, cost, and slow speed of current systems. We want to enable microsized hyperspectral imaging. HSI that, ultimately, fits into a pen. We want to make it much faster, so that it can be used to monitor real-time events. And last, we want to make it so cheap that everyone could afford the pen.
Applications for HSI abound, but the technology isn’t yet up to the task
The potential applications of HSI abound, and it has been hailed over time as a revolutionary technique. In food sorting, HSI cameras could scan for unwanted items (stones, other organic material, unripe or too ripe fruits). The medical world is also interested in HSI, especially in the property that a HSI camera can look below the surface of tissues. A HSI scan could, for example, minimize invasive surgery needed for tissue sampling. Or it can be used to oversee labs and hospitals for contaminants. In automotive applications, HSI cameras could be used to scan the condition of the road ahead. HSI can also be used for air surveillance, looking at tree or crop health, or searching for oil and other resources.
However, the full potential of hyperspectral imaging has not yet been realized. Current HSI solutions are mainly used in research centers, with a few systems installed in satellites. They are bulky, expensive optical systems, costing in the order of 80k USD. Moreover, they are slow, analyzing at most 180 lines per second. For food sorting, to replace current systems, you’d need to scan at least 8000 lines per second. And the storage and analysis of the captured data is even slower, making real-time analysis all but impossible. This is because hyperspectral images are large multi-dimensional datasets, potentially exceeding hundreds of megabytes.
The challenge: microsized, superfast, and cheap
What we envision are industrial application-specific tools, with a much smaller, monolithically integrated design. They should be able to scan more than 8000 lines/second, so that they can be used in food sorting applications. And they should analyze the scanned data in real time, which will call for application-optimized solutions.
State-of-the-art HSI cameras use expensive and bulky optics, taking pictures through a moving slit, allowing them to analyze line per line. The image of such a line is then decomposed using optical components such as gratings or liquid crystal tunable filters. Last, the data of this decomposition are stored and analyzed. Each component of these cameras is state-of-the-art, but the overall design is suboptimal in many ways.
Our solution will use Fabry-Pérot filters. These are resonant filters consisting of 2 mirrors. Depending on the distance between the mirrors, only light of a certain wavelength passes through the filter. By adapting the distances in discrete steps, the spectrum may be sampled. These filters can be micromachined and integrated on an image sensor. Ultimately, we’re envisaging a sensor where each pixel of the image sensor has its own Fabry-Pérot filter. This would allow analyzing complete images at a time, instead of lines.
Microsizing will be only part of the solution. Another huge gain in efficiency will come from co-design: designing system, hardware, software all together with a view of optimizing each layer with the help of the other layers. An example from the HSI sensor is the size of the filter steps, where we can use the software to compensate for larger tolerances. This somewhat relaxes the specs and fabrication challenges for the Fabry-Pérot filters.
Imec’s research into next-generation vision systems
Hyperspectral imaging is only one example of the vision systems that we have in the pipeline, either as ongoing R&D or as ideas that will be further developed in the near future. With our program, we want to create next generation vision systems by combining our expertise in systems design and multimedia with our CMORE process technology (www.imec.be/cmore). Precisely with the combination of that expertise, we see ways to create innovative solutions. Solutions that, through their added functionality and microsize, will create new applications, or make existing applications much wider used. Think of the melanoma-pen with embedded hyperspectral camera and image analysis.