The human brain is rather good at recognizing a large variety of objects: a chair, a ball, a hang-glider, etc. Recognition is usually almost instantaneous, assuming one learned at some point in the past what each of these objects is called.

When we are faced with an object we have never seen before, things are a bit different. The brain will quickly sort through known objects and may find something that looks similar, but just because two things look similar doesn’t mean that they are the same or even related.

That is precisely the situation we are faced with in the field of materials characterization. Each of the instruments in our Materials Characterization Facility at Carnegie Mellon University produces images – sometimes at a truly astounding rate –
and the majority of those images represent objects or features that we have never before seen. Thus, we have all these multimillion-dollar instruments producing images, and we (including our students) must essentially start from the beginning and learn what it is we are seeing.

To teach our students how to interpret the images they acquire with electron microscopes, we offer several courses at CMU, both at the undergraduate and graduate levels. Since 1998, we have taught these courses in our Digital Microscope Classroom (see figure) – a large circular room located at the center of the facility. This classroom, which is quite unique in the world, is hard-wired for remote operation of scientific instruments, including scanning and transmission electron microscopes (SEM and TEM), X-ray diffractometers (XRD) and scanning probe microscopes (SPM).

Materials characterization is so fundamental to the materials engineering profession that we require that our sophomore students learn to operate several of these tools. At the graduate level, we offer courses on SEM and TEM, not only to cover the manual skills of actually operating the instruments but also to cover the underlying theory of how the images and spectra are formed.

The theoretical portion of these courses is where the students learn to interpret what they are seeing. In some cases, this is relatively easy. We have all seen SEM images of insect eyes, spider legs or plant pollen, and we can generally interpret them correctly without having to take an entire course.

For images acquired with a TEM, on the other hand, our intuition and life experience are simply inadequate and offer no help when we try to understand what we are looking at. Fortunately, modern computers allow us to predict what the image should look like. All it takes is having some idea of what might be present in the material (inclusions, lattice defects, pores, etc.) and then applying the image formation theory to generate a simulated image, which can then be compared to the observed image.

This simulation approach is a bit of a roundabout way to gain insight, but it works well, and it has helped generations of students and researchers figure out what they are looking at. At CMU, we have built up significant expertise in the ability to predict microscopy images, thus enabling state-of-the-art materials research.

 

A Deeper Dive into Professor De Graef’s Research

For the past two years, professor De Graef’s group has received funding from the Department of Defense (DoD) in the form of a Vannevar Bush Faculty Fellowship. This research program focuses on the prediction of electron microscopy images, covering both SEM and TEM instruments.

We have created new algorithms to automatically index diffraction patterns, and we are currently studying whether or not machine-learning approaches might be helpful in this area. Our algorithms can handle truly massive data sets. The largest data set we have successfully analyzed is a 3-D stack of slices through a nickel-based superalloy microstructure consisting of more than 110 million diffraction patterns.