Episode 4: Automation
of SPM with AI
Episode 4: Automation of SPM with AI
In this edition, Prof. Sergei Kalinin, Professor of Materials Science and Engineering, at the University of Tennessee, USA, speaks about his highly relevant work using artificial intelligence (AI) and machine learning models (ML) in the automation of scanning probe microscopy (SPM) for the development of advanced materials.
He sheds light on his work using microscopic techniques, such as AFM, STM, piezo response microscopy, multimodal imaging, and electrochemical strain microscopy, to study electrochemical phenomena in advanced materials, and goes into the development of automated ML models, optimized workflow design, and implementing autonomous experiments.
TO LEARN MORE
Automating Microscopy with AI: A Conversation with Professor Sergei Kalinin
Artificial intelligence has become part of our daily lives. Whether it is when large language models help us write emails or when image filters create new realities, AI has become an integral part of many everyday tasks. But while most of us simply use AI, research laboratories develop and apply AI in entirely new territories.
One such field is microscopy, particularly microscopy in material sciences, where AI is applied to analyze microscopy data, but also to help optimize the experiments themselves and how data are acquired.
To learn more about AI in microscopy, we spoke with Prof. Dr. Sergei Kalinin in episode 4 of the Conversations on AFM podcast. Prof. Dr. Kalinin has devoted his career to scanning probe microscopy (SPM) and, in recent years, integrated machine learning with automated experimentation. His work in material sciences has provided novel insights into batteries and semiconductors [1], [2].
Why Quantitative Imaging Matters
Many microscopy methods create beautiful imagery, but at this point in time, we are often beyond just acquiring images that are proof of concept or solely qualitative data. We seek numbers, insights, and knowledge. Thus, rather than visual images, quantitative imaging and post-acquisition processing are needed.
For example, batteries and fuel cells are ionic materials that have strong electromechanical responses. Electrochemical Strain Microscopy (PFM) - a variant of SPM – can be used to measure the electrochemical strain [3].
The fact that this type of data themselves are quantitative was foundational for Kalinin’s work, enabling experiments that move beyond image-taking to true measurement of material properties at the nanoscale. It also paved the way for the integration of AI, because algorithms thrive on data that can be reliably compared and modelled.
The Meaning of Automated Experimentation
Acquiring data on a microscope isn’t as simple as pushing a button, but it is filled with hundreds of micro-decisions: where to scan, when to zoom in/out, and which region to test spectroscopically. Most of these decisions are guided by human intuition, experience, and prior knowledge. But some of these decisions can be automated and shifted from human operators to AI agents [4].
“So rather than a human making all the decisions, it should be the AI agent that has access to the information streaming from the instrument and making these decisions and implementing these decisions.”
– Prof. Dr. Sergei Kalinin
Automated experimentation introduces algorithms that observe data streams and take over some of those choices. By automating choices, experiments are thought to be more reproducible and less prone to human bias [5]. The result is not science fiction. AI agents today can already perform basic workflows: mapping a large sample, adjusting imaging parameters, or carrying out repetitive measurements faster and for longer than a human could.
However, Prof. Dr. Kalinin is clear that in complex, unfamiliar environments - the real playground of scientific discovery - human expertise remains irreplaceable. AI can currently mimic the actions of a beginner or mid-level operator, but the creativity and adaptability of an expert microscopist remain out of reach.
Interestingly, Kalinin emphasises that one of the biggest hurdles to automating microscopy is variability in human workflow design. Different operators, even working on the same instrument and the same sample, often run experiments in completely different ways. This can depend on many factors, such as training level, field of research, sample preparation, scientific question, etc. Encoding such diverse, highly variable, and often tacit knowledge into an AI agent is a highly complex task [6].
Data: Big, Small, and Smart
A natural question is how much data is required to train these systems. While many AI approaches rely on “big data”, Kalinin is cautious. Scientific microscopy rarely generates the kind of vast, standardized datasets found in medical imaging or consumer applications. Instead, progress may depend on “small data” approaches that reflect how humans learn efficiently from limited examples.
That said, fields with established workflows - such as semiconductor manufacturing - are well-positioned to benefit from big-data-driven AI. For exploratory science, the challenge is harder.
Looking Ahead: Rewards and Objectives
When asked about the future, Kalinin points to the central role of reward functions. In reinforcement learning, a reward is what drives the system to improve its behaviour. In the context of scientific microscopy, rewards might be sharper resolution, confirmation of a hypothesis, or discovery of a structure–property relationship.
Humans are remarkably good at juggling between multiple imaging aims, switching between curiosity, optimisation, and hypothesis testing. But AI, for now, requires clearly defined reward functions. Where the two approaches meet, however, lies enormous potential [7]. Optimisation algorithms can efficiently pursue specific rewards, while reasoning systems may help researchers frame objectives and articulate the “right” reward functions.
Human and Machine: A Complementary Future
What becomes clear is that AI will not replace scientists at the microscope (yet). Instead, it will become a collaborator: handling repetitive tasks, rapidly adapting/optimising parameters, and freeing researchers to focus on exploration, insight, and developing the big questions.
However, it is upon the scientists and the community to establish the research direction, connect non-adjacent dots, develop global scientific priorities, and establish new experimental/microscopy approaches. The ideal outcome is a partnership in which both machine and human excel at what they do best [8].
You can also download our resource collection to learn more about AFM technologies and their application.
References
- B. N. Slautin et al., “Materials discovery in combinatorial and high-throughput synthesis and processing: A new Frontier for SPM,” Appl. Phys. Rev., vol. 12, no. 3, p. 031321, Aug. 2025, doi: 10.1063/5.0259851.
- Y. Liu et al., “Autonomous scanning probe microscopy with hypothesis learning: Exploring the physics of domain switching in ferroelectric materials,” Patterns, vol. 4, no. 3, Mar. 2023, doi: 10.1016/j.patter.2023.100704.
- A. Raghavan et al., “Evolution of Ferroelectric Properties in SmxBi1–xFeO3 via Automated Piezoresponse Force Microscopy across combinatorial spread libraries,” ACS Nano, vol. 18, no. 37, pp. 25591–25600, Sept. 2024, doi: 10.1021/acsnano.4c06380.
- S. V. Kalinin et al., “Automated and Autonomous Experiments in Electron and Scanning Probe Microscopy,” ACS Nano, vol. 15, no. 8, pp. 12604–12627, Aug. 2021, doi: 10.1021/acsnano.1c02104.
- S. V. Kalinin et al., “Designing workflows for materials characterization,” Appl. Phys. Rev., vol. 11, no. 1, p. 011314, Mar. 2024, doi: 10.1063/5.0169961.
- R. B. Canty et al., “Science acceleration and accessibility with self-driving labs,” Nat Commun, vol. 16, no. 1, p. 3856, Apr. 2025, doi: 10.1038/s41467-025-59231-1.
- Y. Liu et al., “Machine Learning-Based Reward-Driven Tuning of Scanning Probe Microscopy: Toward Fully Automated Microscopy,” ACS Nano, vol. 19, no. 21, pp. 19659–19669, June 2025, doi: 10.1021/acsnano.4c18760.
- S. Lo et al., “Review of low-cost self-driving laboratories in chemistry and materials science: the ‘frugal twin’ concept,” Digital Discovery, vol. 3, no. 5, pp. 842–868, 2024, doi: 10.1039/D3DD00223C.
Interested in learning more about this topic?
You can find detailed applications and technical notes, expert-led webinars, and on-demand instrument and measurement demonstrations in our online resource library. Get instant, full-length access to all resources related to this podcast using the form below.
This resource collection includes:
- 2 full-length e-books
- 1 full length application note
- 1 full length on-demand webinar recording
FULL-LENGTH E-BOOK
AFM for Materials E-Book
Download our in-depth introductory guide to atomic force microscopy and its materials characterization capabilities.
FULL-LENGTH E-BOOK
The Definitive AFM Modes Handbook
Everything you need to understand, select, and apply AFM techniques in materials research.
FULL-LENGTH APPNOTE
PeakForce Magnetic Force Microscopy
This application note describes what PeakForce MFM is and its value for exploring magnetic properties at the nanoscale with atomic force microscopy.
ON-DEMAND WEBINAR RECORDING
Hyperspectral Characterization of Advanced Materials
In this webinar, Bruker speakers give an overview of Bruker’s DataCube modes and perform a live demonstration illustrating their capabilities. Hear about the hyperspectral DataCube concept, DataCube Contact Resonance PFM for accurate measurements of piezo electric properties and correlative chemical and mechanical information by Resonance Enhance Force Volume Mode (REFV).
