Samuel Berthelot
Back to selected work

Hyperspectral reconstruction from photon-starved electron microscopy

SEM-EDS maps a meteorite section as a cube of X-ray spectra, one per pixel, and most of its voxels record zero or one photon. I reconstructed cubes from data this sparse, with the sharp electron image as a guide.

Role
Student researcher
Where
LRE, EPITA Research Laboratory
When
Jan to Jul 2026
Stack
Python, PyTorch on GPU, NumPy
Output
Technical report, talk at the EPITA SCIA seminar 2026

The measurement

A scanning electron microscope sweeps a focused electron beam across a polished section of the meteorite. Where the beam lands, it knocks electrons out of the inner shells of atoms, and the atoms answer with X-rays at energies that identify each element. An energy-dispersive spectrometer (EDS) counts these photons and sorts them by energy, so every beam position yields a spectrum.

The spectra of a whole scan form a hyperspectral cube, with two spatial axes and one energy axis. The chondrite cube I worked on has 1024 × 1024 pixels and 2048 energy channels: about 2.1 billion voxels, or 8.6 GB in single precision.

The hyperspectral cubeA cube with two spatial axes, x and y, of 1024 pixels each, and an energy axis of 2048 channels. One column of the cube, running along the energy axis from a single pixel, is highlighted: that column is the pixel's spectrum.its spectrumx: 1024 pixelsy: 1024 pixelsenergy: 2048 channelsOne pixel's spectrumA stem plot of photon counts against energy channel, from channel 0 to 2047. Almost every channel holds 0 photons; a few channels hold 1 photon. Drawn for illustration.One pixel's spectrumphotons101 photon0 photons02047energy channel
The cube and one of its columns. Each pixel holds a spectrum along the energy axis; with so few photons, it is almost all zeros with a few ones. Drawn for illustration, not measured data.

Most of those voxels hold zero or one photon. That is typical of EDS maps: the beam spends only a short time on each pixel, few of the electrons that hit the sample produce an X-ray that reaches the detector, and the few photons a pixel collects are spread over 2048 channels.

Two effects shape the data. Counts are whole numbers: a voxel whose expected count is 0.3 photons records 0 most of the time, sometimes 1, never 0.3. And photon arrivals are random: the count follows a Poisson distribution, whose standard deviation is the square root of its mean, so at about one photon the noise is as large as the signal. I framed the reconstruction as an inverse problem under this quantization and Poisson noise: recover the underlying cube from the counts.

Why the usual data term fails here

Many reconstruction methods minimize the sum of two terms:

minimize D(x, y) + λR(x)

Here x is a candidate cube and y the recorded counts. The data-fidelity term D measures how well x explains the counts, for example as a squared error or as the negative log-likelihood of the noise model. The regularizer R encodes what a plausible cube looks like, for example that neighboring spectra are similar. The trade-off weight λ sets the balance between them: too low and the noise passes through, too high and real detail is smoothed away. It has to be tuned, usually against a ground truth that real data does not have.

In this photon-starved regime, I showed that the classical data-fidelity term is degenerate, so I took it out of the objective.

A constraint instead of a trade-off

In its place, I put a hard box constraint on the unknowns: each value of the cube must stay between a lower and an upper bound consistent with the recorded counts. The problem becomes:

minimize R(x)
subject to l ≤ x ≤ u

The regularizer is convex and so is the box, so the whole problem is convex: any local minimum is a global one. With a single term left in the objective, there is no trade-off weight to tune.

I solved it by projected gradient. Each iteration takes a gradient step on the regularizer, which may push some values out of the box, then projects back onto the box. For a box, the projection is clipping: a value above its upper bound is set to that bound, a value below its lower bound is set to the lower bound, and the rest stay as they are.

xk+1 = clip(xk − τ∇R(xk), l, u)

One iteration of projected gradient on a boxTwo unknowns, x1 and x2, drawn as a point in the plane. The box holds the values between the lower bounds l1, l2 and the upper bounds u1, u2. From the current point x k inside the box, a gradient step on the regularizer lands outside the box, past the upper bound of x1. Clipping sets x1 back to its upper bound and leaves x2 unchanged, which gives the next point x k plus 1 on the edge of the box.x1x2l1u1l2u2the boxxkxk+1after the stepclipgradient stepon the regularizer
Projected gradient with two of the unknowns. A gradient step on the regularizer can leave the box; clipping brings each coordinate that left back to its nearest bound and leaves the others as they are. Illustration.

The step size τ affects how fast the iterations converge, not which problem they solve. Both steps only involve each voxel and its neighbors, which suits a GPU: I ran the method in PyTorch on the real chondrite cube.

Letting the electron image guide

Along with the X-ray counts, the microscope records a backscattered-electron image of the same area. That signal is far stronger, so the image is sharp, and its gray level follows the mean atomic number of the material: mineral grains show up as regions of nearly even gray with clear edges.

I designed a bilateral regularizer guided by that image. It penalizes differences between the spectra of neighboring pixels, and weights each pair by how similar the two pixels look in the electron image. Neighbors with the same gray level, most likely in the same grain, are pulled together; neighbors across an edge are barely linked. Smoothing stays inside grains and stops at their boundaries.

Neighbor links guided by the electron imageA grid of 7 by 5 pixels shaded with two gray levels from the electron image, one per mineral grain, separated by a staircase boundary. Each pixel is linked to its neighbors. Links between pixels of the same gray level are thick: they are smoothed together. Links that cross the boundary are thin: smoothing barely acts across them, so the edge stays sharp.strong link: similar gray levelsweak link: across a grain boundary
A patch of pixels shaded by the electron image, with two grains. Inside a grain, neighbors look alike and their link is strong, so their spectra are smoothed together; across the boundary the link is weak, so the edge stays sharp. Illustration.

Simulated data

0.50 photons per pixel
  • Mg
  • Si
  • Fe
Simulated data. A made-up section imaged one X-ray photon at a time: raw counts left of the line, and on the right a reconstruction guided by a simulated electron image. It illustrates the idea, not the lab's method or data. Drag the line or change the exposure.

Results

PSNR and the spectral angle compare a reconstruction with a known answer, so they were measured on a synthetic cube with Poisson noise. Spectral dispersion between neighboring pixels was measured on the real cube, which has no ground truth.

Results on real and synthetic data
MetricDataResult
PSNR against the ground truthSynthetic, Poisson noise+1.5 dB over raw counts
Spectral angle (SAM)Synthetic, Poisson noise22.3° to 18.9°
Spectral dispersion between neighboring pixelsReal chondrite cube65% lower

PSNR, the peak signal-to-noise ratio, compares the reconstruction with the ground truth through their mean squared error, on a logarithmic scale relative to the peak value: higher is better.

The spectral angle (SAM) treats each spectrum as a vector and measures its angle to the true spectrum, so it compares the shape of the spectrum regardless of its overall scale: lower is better.

Spectral dispersion between neighboring pixels measures how much the spectra of neighboring pixels differ from one another. Any smoothing lowers it, so on real data it shows how much noise was removed, while PSNR and the spectral angle on synthetic data measure accuracy.

What stays private

The code and report stay with the lab, which still uses the work. I am happy to walk through the method in an interview.