Tutorial on
From Quantum Agents to Self-Programming Quantum AI: Reinforcement Learning, Architecture Search, and Adaptive Memory
Instructor
|
Samuel Chi-Yen Chen
Wells Fargo
United States
|
|
|
|
Short Bio
Samuel Yen-Chi Chen is a Lead Research Scientist in Quantum AI at Wells Fargo and was previously an Assistant Computational Scientist at Brookhaven National Laboratory. He received a Ph.D. in Physics, an M.D., and a B.S. in Physics from National Taiwan University. His research focuses on quantum artificial intelligence, quantum machine learning, quantum reinforcement learning, adaptive sequential models, quantum architecture search, and self-programming quantum architectures. His research spans variational quantum circuits for reinforcement learning, quantum recurrent and long short-term memory models, quantum reservoir computing, automated quantum architecture design, differentiable quantum architecture search, and quantum fast-weight programming. His recent work investigates adaptive and self-programming Quantum AI systems that integrate learning, memory, architecture adaptation, and sequential decision making. Chen has published broadly across quantum computing, machine learning, artificial intelligence, signal processing, and scientific applications. His research has received the IEEE QTC Distinguished Best Technical Paper Award at IEEE QCE 2025, a 2nd Best Paper award in the QML Track at IEEE QCE 2025, and the Best AI Paper Award at IEEE ICTC 2024. He regularly delivers tutorials and invited lectures on quantum machine learning, quantum reinforcement learning, quantum architecture search, and Quantum AI at major IEEE, ACM, and international conferences. He also organizes workshops and special sessions in Quantum AI and serves as an Associate Editor for IEEE Circuits and Systems Magazine and IEEE Transactions on Network and Service Management.
|
Abstract
Quantum artificial intelligence is increasingly moving beyond static prediction toward adaptive, sequential, and self-programming learning systems. This tutorial presents a unified introduction to this emerging direction through the perspective of intelligent agents. Beginning with variational quantum models for reinforcement learning, we develop a progression from quantum learning models to quantum agents, automatically designed architectures, memory-enabled quantum systems, and ultimately self-programming quantum AI.
The tutorial first introduces the foundations of hybrid quantum-classical learning and explains how parameterized quantum circuits can represent value functions, policies, and other components of reinforcement-learning agents. We then examine major approaches to quantum reinforcement learning, including value-based, policy-based, recurrent, distributed, and reservoir-based methods. The tutorial next addresses a fundamental question: can quantum learning systems automatically design and adapt their own architectures? This motivates quantum architecture search using reinforcement learning, evolutionary optimization, and differentiable architecture search.
The second half focuses on sequential intelligence and adaptive memory. We discuss quantum recurrent models, quantum long short-term memory, reservoir computing, and fast-weight approaches that enable quantum models to process temporal information and modify their effective computation dynamically. These developments lead naturally toward self-programming quantum architectures in which model structure, memory, measurements, or computational behavior can adapt during learning.
Throughout the tutorial, emphasis will be placed on practical implementation, benchmarking, trainability, scalability, simulation, noise, classical baselines, and reproducibility. The tutorial is intended to connect the intelligent-agent perspective of ICAART with emerging research in quantum machine learning and adaptive quantum systems.
Keywords
Quantum Artificial Intelligence; Quantum Reinforcement Learning; Quantum Agents; Variational Quantum Circuits; Quantum Architecture Search; Differentiable Architecture Search; Quantum Sequential Learning; Quantum Memory; Self-Programming AI; Hybrid Quantum-Classical Learning
Aims and Learning Objectives
The tutorial aims to provide participants with a coherent understanding of how quantum machine learning models can evolve from static predictors into adaptive intelligent agents and self-programming learning systems.
By the end of the tutorial, participants should be able to:
1. Understand the basic principles of variational quantum circuits and hybrid quantum-classical learning in the context of intelligent agents.
2. Formulate sequential decision-making problems using quantum reinforcement-learning models.
3. Distinguish value-based, policy-based, recurrent, reservoir-based, and distributed approaches to quantum reinforcement learning.
4. Explain why automated architecture design is important for variational quantum models.
5. Compare reinforcement learning-based, evolutionary, and differentiable approaches to quantum architecture search.
6. Understand how recurrent networks, quantum reservoirs, and fast-weight mechanisms can provide memory and adaptive computation.
7. Recognize emerging concepts of self-programming quantum AI in which architecture, memory, measurements, or computational behavior can adapt during learning.
8. Critically evaluate scalability, trainability, noise, benchmarking methodology, classical baselines, and reproducibility in current Quantum AI research.
Target Audience
The tutorial is intended for researchers, practitioners, Ph.D. students, advanced graduate students, and technically oriented professionals working in artificial intelligence, autonomous agents, reinforcement learning, machine learning, quantum computing, or hybrid quantum-classical algorithms.
A particular goal is to make emerging Quantum AI research accessible to members of the ICAART community who have strong backgrounds in AI or intelligent agents but limited previous exposure to quantum computing. The tutorial should also be useful to quantum computing researchers seeking a structured introduction to reinforcement learning, adaptive architectures, sequential models, and self-programming learning systems.
Prerequisite Knowledge of Audience
Basic knowledge of machine learning and linear algebra is expected. Familiarity with neural networks or reinforcement learning is helpful but not required.
No previous research experience in quantum computing is required. The tutorial will introduce the quantum-computing concepts necessary to understand parameterized quantum circuits, measurements, hybrid quantum-classical optimization, and Quantum AI models.
The emphasis will be on conceptual understanding, algorithmic structure, and practical modeling rather than on advanced quantum-information theory.
Detailed Outline
Part I — From Machine-Learning Models to Quantum Agents — approximately 25 minutes
Motivation for Quantum AI; the intelligent-agent perspective; basic quantum computing concepts; parameterized and variational quantum circuits; data encoding, measurements, and hybrid quantum-classical optimization; moving from supervised quantum models toward sequential decision-making agents.
Part II — Quantum Reinforcement Learning — approximately 45 minutes
Markov decision processes and reinforcement learning fundamentals; quantum representations of value functions and policies; variational quantum deep Q-learning; policy gradient and actor-critic approaches; recurrent quantum reinforcement learning; reservoir-based quantum reinforcement learning; distributed and multi-agent perspectives; practical training considerations.
Part III — Agents that Design Themselves: Quantum Architecture Search — approximately 40 minutes
Limitations of manually designed variational circuits; the quantum architecture search problem; architecture representation and search spaces; reinforcement learning-based quantum architecture search; evolutionary architecture optimization; differentiable quantum architecture search; joint optimization of model parameters and architecture; resource-aware and hardware-aware design.
Part IV — Quantum Agents with Memory — approximately 30 minutes
Why memory matters for intelligent agents; sequential learning and partial observability; quantum recurrent neural networks; quantum long short-term memory; quantum reservoir computing; temporal representations; efficiency and scalability considerations for quantum sequence models.
Part V — Toward Self-Programming Quantum AI — approximately 35 minutes
Fast weights and programmable computation; quantum fast-weight programming; adaptive circuit behavior; self-modulating quantum models; bounded and controlled memory; programmable measurements and adaptive observables; connections between architecture search, memory, and self-programming systems; emerging directions toward adaptive quantum intelligence.
Part VI — Practical Lessons, Demonstrations, and Open Problems — approximately 25 minutes
Simulation and hardware considerations; software workflows; benchmarking against classical models; barren plateaus and optimization; noise and reproducibility; resource requirements; interpreting claims of quantum advantage; practical demonstrations; open research problems in quantum agents, adaptive architectures, memory, and self-programming Quantum AI.