New Human-Machine Interface Enables Scientists to Visualize and Interpret Large Data

Lehigh is executing a $3-million institutional investment in a multidisciplinary research initiative that ensures to change the way human beings harness and then interact with data and with the refined instruments of scientific discovery.

The Nano/Human Interface Presidential Engineering Research Initiative intends to produce a human-machine interface that is capable of improving the ability of scientists to visualize and also understand the huge amounts of data — mostly in the terabytes, or trillions of bytes — that are produced by scientific research.

Martin P. Harmer, professor of materials science and engineering, is the leader of the initiative. Other senior faculty leaders are Jeffrey M. Rickman, professor of materials science, engineering and physics; Anand Jagota, department chair of bioengineering; Daniel P. Lopresti, department chair of computer science and engineering and director of Lehigh’s Data X Initiative; and Kate Arrington, associate professor of psychology.

Lehigh President John D. Simon and Provost Patrick V. Farrell approved this initiative.

“With this initiative,” the group wrote in its proposal, “we are poised to lead the next revolution in the conduct of science, namely drawing inferences from vast datasets whether experimentally or computationally generated.

Harmer and Rickman stated the initiative will focus on enabling researchers to:

• Ask “what-if” questions in real time
• Redefine machine learning
• Find correlations in huge databases
• Obtain data based on in situ analytics
• Incorporate simulation and experimentation “on the fly”
• Visualize and interact with multidimensional, complex sets of data

“The development of this human-machine element,” the group wrote, “will require a partnership with behavioral psychologists and graphic designers, who understand the most effective, visual human learning strategies.”

The Lehigh initiative brings into line two of the six “big ideas” for research acknowledged by the National Science Foundation in the next decade. These are work at the human-technology frontier and collecting data for 21^st-century engineering and science.

The group stated that electron microscopes, spectroscopes and various other tools that describe the properties of materials, including computers capable of simulating materials’ behavior, produce terabytes of data. To be useful, this data will have to be coupled with analytical tools that organize and then display it in a manner such that humans will be able to comprehend and understand. Instead, much of the data is discarded or under-used.

The group is working along with Library and Technology Services (LTS) in order to create a network of spaces on campus where researchers will work together with an artificial intelligence tool named Buddy. This tool will support researchers by learning from the data it attains, similar to online retailers who learn to anticipate the preferences and probable actions of consumers browsing websites.

We will talk to Buddy like we talk to our phones, buddy will remember what it has done before and will learn from its interactions with us. It will learn what we’re interested in, anticipate what we want to do and give us a potential solution.

Martin P. Harmer, professor of materials science and engineering

According to the group, this potential to “interrogate” data will allow researchers to define, in real time, the importance of their results and to recognize which additional data, and how much, is required for solving a problem.

“Our proposed active learning approach will constitute an artificial neural network to permit acquisition tools to learn in real time…[and to] reduce the complexity of a problem,” the group wrote.

The group mentioned two areas of research where progress has decelerated because of the inability to completely visualize and understand huge amounts of data. These refer to the generation of nanomaterials and the development of medical applications for carbon nanotubes (CNTs) wrapped with single DNA strands.

“Nanomaterials have been studied for decades and continue to outperform conventional materials in research labs,” the group wrote. “The critical challenge is to scale up nanomaterials to industrial parts. To accomplish this, scientists and engineers need to characterize and manipulate tens of thousands of nanoscale interfaces.

“While current characterization tools are highly advanced, they produce vast amounts of [data] where meaningful information is obscured. We need an analytical toolbox that can identify emerging patterns in this data and permit visual interaction in a comprehensive, informative and intuitive way.”

The group highlighted that DNA-wrapped CNTs are currently produced for medical imaging and sensing and also for drug delivery. They also pointed out that the functionality of these hybrids relies on their structure and, eventually, on the sequence of the DNA.

“The space of DNA sequences is vast and the search for ordered hybrids involves sifting through massive data streams both from computer simulation and optical spectroscopy,” the group wrote. “We need the ability to analyze and identify a small number of functional hybrids (in the hundreds) from among the [trillions of] possibilities.”

The initiative comprises of two external advisers. Uma Chowdhry is Chief Science and Technology Officer emeritus at DuPont. Alton D. Romig Jr. ’73 ’75 M.S. ’77 Ph.D. is Executive Officer of the National Academy of Engineering, former Vice President and General Manager of Lockheed Martin Aeronautics Company Advanced Development Programs (also known as Skunk Works) and former Executive Vice President of Sandia National Laboratories.

The group is making plans to host an international workshop in 2018. Ohio State University and the Army Research Laboratory have expressed a great level of interest in collaborating with the Nano/Human Interface researchers.