Integrating Machine Learning and Agent-Based Models

Day 3 | WISE Workshop — Addis Ababa 2026

Sean Sylvia, Ph.D. & Sachiko Ozawa, Ph.D.

2026-02-04

Learning Objectives

By the end of this session, you will be able to…

  1. Understand why ML and ABM are complementary methodologies

  2. Identify five approaches for integrating ML with agent-based models

  3. Apply ML demand forecasts as ABM agent parameters

  1. Analyze the Beer Game case study and RL-based ordering policies

  2. Connect Days 1-2 forecasting work to Day 4 simulation design

Section 1: Introduction

Why Integrate ML and ABM?

Machine Learning

  • Strength: Pattern recognition
  • Learns from historical data
  • Excellent at prediction
  • “Black box” — limited interpretability

Agent-Based Models

  • Strength: Mechanistic understanding
  • Simulates individual behavior
  • Captures emergence
  • Transparent — clear causal mechanisms

Together: ML provides data-driven predictions that inform mechanistic simulations

The Integration Challenge

ML and ABM: How do we bridge predictions to simulation parameters?

Five Integration Approaches

ML-ABM Integration Taxonomy

Overview of Today

Section Topic Time
1 Introduction (this section) 10 min
2 Integration approaches in depth 20 min
3 Case study: Supply chain + RL 15 min
4 Reinforcement learning concepts 10 min
5 LLM agents (emerging methods) 10 min
6 Application to WISE project 10 min
7 Health economics applications 10 min
8 Wrap-up and discussion ~5 min

Section 2: Integration Approaches

Integration Workflows Overview

Data flow for each integration approach

Approach 1: ML Predictions as Agent Parameters

The Idea

Train ML on real-world data → use predictions to parameterize agent decisions

\[\text{Real Data} \xrightarrow{\text{Train ML}} \text{Model} \xrightarrow{\text{Predict}} \text{Agent Parameters}\]

Example: Cholera ABM (Zhang et al., 2019)

Training:

  • Bayesian Network trained on household survey
  • Learns: P(use river water | income, distance, quality)

In ABM:

  • Each agent queries BN for water source decision
  • Epidemic dynamics emerge from individual choices
  • More realistic than fixed probability rules

When to Use ML as Agent Parameters

Good Fit

  • Have training data for agent behaviors
  • Want data-driven heterogeneity
  • Need realistic decision distributions
  • Your WISE forecast → ordering behavior

Challenges

  • ML model becomes part of ABM
  • Need to handle prediction uncertainty
  • May require re-training for new contexts

Approach 2: ML for Calibration

The Problem

  • ABM parameters are unknown or hard to estimate
  • Traditional calibration is computationally expensive
  • Need to match simulation outputs to real-world data

The Solution

Use ML to learn the inverse mapping:

\[\text{Observed Data} \xrightarrow{\text{ML Model}} \text{ABM Parameters}\]

ML Calibration Workflow

ML learns the inverse mapping from outputs to parameters

Example Results

Neural calibration achieves:

  • 195-390x speedup over traditional methods
  • Comparable accuracy to Bayesian inference
  • Automatic uncertainty quantification

When to Use ML Calibration

Good Fit

  • Many unknown parameters
  • Rich observational data
  • Need fast calibration
  • Want uncertainty estimates

Challenges

  • Requires many ABM runs for training
  • May not generalize to new scenarios
  • Identifiability issues (multiple parameter sets → same output)

Approach 3: ML-Driven Agent Behaviors

The Problem

  • Agent decision rules are complex or unknown
  • Rules may change over time (adaptive behavior)
  • Hard to specify rules from first principles

The Solution

Let ML/RL learn agent decision rules from data or experience:

\[\text{Agent State} \xrightarrow{\text{ML Model}} \text{Agent Decision}\]

ML-Driven Behaviors in Practice

Agents with different decision-making approaches interacting in ABM

Example

Cholera ABM (Zhang et al., 2019):

  • ML derived behavioral rules from household survey data
  • Rules adapt based on water quality perceptions
  • More realistic than fixed decision rules

When to Use ML-Driven Behaviors

Good Fit

  • Complex decision-making
  • Behavioral data available
  • Adaptive agents needed
  • Policy optimization

Challenges

  • Need training data or environment
  • Computational overhead per agent
  • Interpretability concerns

Approach 4: Surrogate Models

The Problem

  • ABMs are computationally expensive
  • Can’t run thousands of scenarios for sensitivity analysis
  • Real-time decisions need fast predictions

The Solution

Train ML to approximate ABM input→output mapping:

\[\text{ABM Inputs} \xrightarrow{\text{Neural Network}} \text{ABM Outputs (Approximate)}\]

Surrogate Model Speedup

Runtime comparison: Full ABM vs ML Surrogate

Training a Surrogate Model

Three-phase surrogate training: Generate data → Train model → Deploy fast predictions

When to Use Surrogate Models

Good Fit

  • Sensitivity analysis
  • Policy optimization
  • Real-time applications
  • Uncertainty quantification

Challenges

  • Approximation error
  • Extrapolation risks
  • Need representative training scenarios
  • May miss rare events

Approach 5: Hybrid / Differentiable ABMs

The Problem

  • Can’t optimize ABM end-to-end
  • Gradient-based optimization needs derivatives
  • Traditional ABMs use discrete, non-differentiable operations

The Solution

Make ABM differentiable — enable gradient descent:

\[\frac{\partial \text{Loss}}{\partial \text{Parameters}} \xrightarrow{\text{Gradient Descent}} \text{Optimal Policy}\]

GradABM: A Differentiable Epidemic Model

End-to-end differentiable ABM with backpropagation through simulation

COVID-19 Application

  • Validated UK vaccine prioritization strategy
  • Explored counterfactual policies
  • 100x faster than traditional calibration

When to Use Hybrid/Differentiable ABMs

Good Fit

  • Policy optimization
  • End-to-end learning
  • Complex interventions
  • Counterfactual analysis

Challenges

  • Implementation complexity
  • Requires differentiable operations
  • Not all ABMs can be made differentiable
  • Specialized frameworks needed

Summary: Choosing an Approach

Approach Use When Key Benefit
Calibration Parameters unknown, have observational data Automated parameter estimation
ML Behaviors Decision rules complex, have behavioral data Realistic agent decisions
Surrogate Need many fast runs, computational bottleneck 1000x+ speedup
Hybrid/Diff. Need end-to-end optimization Gradient-based policy learning
LLM Agents Cognitive realism, no training budget Natural reasoning, zero-shot

Often combine approaches: e.g., ML calibration + surrogate for sensitivity analysis

Section 3: Case Study — Supply Chain

The Beer Game: A Classic Supply Chain Problem

Setup

  • 4-echelon supply chain:
    • Retailer → Wholesaler → Distributor → Manufacturer
  • Each node places orders to upstream supplier
  • Information delays (orders take time)
  • No global visibility

The Challenge

Manage inventory with only local information

Four-echelon supply chain

The Bullwhip Effect

What Happens

Small demand fluctuations at retail level amplify up the supply chain.

Order variance amplifies upstream in supply chains

Why It Matters

  • Excess inventory (waste)
  • Stockouts (shortages)
  • Inefficient resource use
  • Direct relevance to pharmaceutical supply chains

Bullwhip Effect: Before and After RL

Order variance amplification with traditional vs DQN-learned policies

Deep Q-Network (DQN) for Ordering Policy

The Integration

  1. ABM Environment: Multi-echelon supply chain simulation
  2. RL Agent: Deep Q-Network at each node
  3. State: Local inventory, orders, backlog
  4. Action: How much to order
  5. Reward: Minimize holding + backlog costs

Key Result

DQN learns to smooth orders even without global information

  • Reduces bullwhip effect by ~60%
  • Matches centralized optimal within 1%
  • Generalizes to demand patterns not seen in training

ML-Enhanced Inventory Management

Connecting ML Forecasts to Inventory Policy

Demand forecast driving inventory reorder decisions

The ML → Inventory Pipeline

Forecast uncertainty drives safety stock and reorder decisions

ML → Inventory Pipeline: Key Insight

Forecast uncertainty (prediction intervals) directly informs safety stock levels

  • Higher uncertainty → more safety stock
  • Facility-specific forecasts → tailored inventory policies

GradABM: Epidemiology Example

Differentiable Epidemic Simulation

Application

  • COVID-19 vaccine prioritization
  • UK national strategy evaluation
  • Counterfactual policy analysis

Method

  • Neural networks encode transmission
  • Gradients flow through simulation
  • Learn optimal intervention timing

Results

  • Validated age-based prioritization
  • Showed timing matters more than coverage
  • 100x faster than Bayesian calibration

Takeaway: Differentiable ABMs enable rapid policy optimization

Section 4: Reinforcement Learning Concepts

RL + ABM: The Big Picture

Core Idea

ABM serves as the environment in which an RL agent learns

RL-ABM Interaction Loop

The RL Framework

Key Components

Component In RL In Supply Chain ABM
State Current observation Inventory levels, pending orders
Action Decision Order quantity
Reward Feedback signal -(holding cost + stockout cost)
Policy Decision rule Ordering strategy

Learning Process

Agent explores actions → observes rewards → updates policy → improves over time

RL Learning: Policy Improvement Over Time

RL agent learning to optimize inventory policy

When to Use RL with ABMs

Good Fit

  • Sequential decisions: Actions now affect future
  • Complex dynamics: Hard to derive optimal policy
  • Uncertain environment: Need adaptive behavior
  • Policy optimization: Finding best intervention

Examples

  • Inventory management
  • Epidemic interventions
  • Resource allocation
  • Traffic control

Advanced RL

Multi-Agent RL (MARL)

  • Multiple learning agents interact
  • Example: AI Economist — agents + government learn together
  • Challenge: Non-stationary environment

Inverse RL

  • Learn decision rules from observed behavior
  • “What reward function explains this behavior?”
  • Useful for calibrating agent rules from real data

These are active research areas — resources provided at end for those interested

Deep Dive: The AI Economist

Two-Level Reinforcement Learning

Inner Loop (Agents):

  • Each agent maximizes own utility
  • Learns: work, trade, consume decisions
  • Adapts to tax policy environment

Outer Loop (Planner):

  • Government maximizes social welfare
  • Learns: optimal tax schedules
  • Adapts to agent behavior changes

Supply Chain Analogy:

AI Economist Supply Chain
Citizens Facility managers
Tax policy Reorder policies
Utility Minimize costs
Govt Central planner

Key Result

AI-designed progressive tax outperformed Saez optimal by +8% on equality-productivity tradeoff

Section 5: LLM-Powered Agents

A New Paradigm: LLMs as Agent Brains

The Idea

  • Replace hand-coded rules with language models
  • Agents “think” via natural language
  • Zero training required — prompt-based

Key Innovation

  • Cognitive realism without manual specification
  • Handles novel situations
  • Interpretable reasoning chains

LLM Agent with Persona, Memory, and Reasoning

AgentSociety: Large-Scale LLM Simulation

Landmark Achievement (Piao et al., 2025)

Scale:

  • 10,000 LLM-powered agents
  • 5 million interactions
  • Validated against real-world patterns

Validated Phenomena:

  • Information polarization
  • Policy response patterns
  • Collective emergency behavior

10,000 LLM agents with emergent social behaviors

LLM vs. RL Agents: Key Tradeoffs

Dimension RL Agents LLM Agents
Training Expensive (many simulations) None (prompt-based)
Consistency High (learned policy) Variable (stochastic sampling)
Reasoning Implicit (in weights) Explicit (interpretable)
Novel situations Limited to training distribution Can handle novel scenarios
Cost Training upfront Per-inference API calls
Behavioral realism Optimized Cognitively plausible

Tradeoff: RL gives optimal behavior; LLM gives realistic behavior

Supply Chain Application: LLM Agents

Potential for Pharmaceutical Supply Chains

Agent Personas:

  • Facility manager (risk-averse)
  • Regional coordinator (balancer)
  • Central planner (optimizer)

Memory Contents:

  • Past stockout events
  • Supplier reliability history
  • Seasonal patterns observed

Why LLM Agents?

  • Handle qualitative factors
    • “Supplier X has been unreliable”
    • “Rainy season affects roads”
  • Natural heterogeneity
    • Different reasoning styles
    • Realistic decision diversity
  • Explainable decisions
    • “I ordered extra because…”

Future direction for WISE project?: LLM agents for scenario exploration

Section 6: Application to WISE Project

WISE Workshop Integration Pipeline

Project workflow from data to policy insights

Integration Strategy

What We’re Doing

Day Component Output
Days 1-2 ML for demand forecasting Predictions by facility, medication
Day 3 Integration design (today) Interface specification
Day 4 ABM simulation Supply chain model
Day 5 Policy analysis Intervention recommendations

Integration Type

Approach 2 (ML-driven behaviors) + elements of Approach 1 (calibration):

  • ML forecasts → Agent ordering parameters
  • Uncertainty bounds → Safety stock rules

The Interface: ML → ABM

# ML Model Output (from Days 1-2)
forecast = {
    'facility_id': 'F001',
    'medication': 'Amoxicillin 500mg',
    'period': '2026-03',
    'predicted_demand': 450,
    'lower_bound': 380,      # 95% CI
    'upper_bound': 520
}

# ABM Agent Input (for Day 4)
agent_params = {
    'facility_id': 'F001',
    'reorder_point': 520,    # upper_bound for safety
    'order_quantity': 450,   # predicted demand
    'lead_time': 14,         # days
    'review_period': 30      # days
}

Hands-On Preview

In the notebook exercise, you will:

  1. Load your trained ML model from Day 2
  2. Generate demand forecasts for multiple facilities
  3. Calculate inventory parameters from forecasts
  4. Format outputs for ABM input
  5. Visualize the forecast → policy connection

Key Questions to Consider

  • How does forecast uncertainty affect safety stock?
  • What happens when forecasts are wrong?
  • How should we update forecasts over time?

Discussion: Connecting Your Forecasts to ABM

From Day 2

You built ML models to predict antibiotic demand by facility

  • Time series features
  • Facility characteristics
  • Seasonal patterns
  • Prediction intervals

For Day 4

We’ll build a supply chain ABM where agents make ordering decisions

  • Facility agents place orders
  • Lead times vary
  • Stockouts have consequences
  • Demand is uncertain

Discussion Questions (5 min with neighbor)

  1. How should your demand forecasts inform agent ordering behavior?
  2. Should agents use point predictions or uncertainty intervals?
  3. What happens when forecasts are systematically wrong (biased)?
  4. How might we validate that the ABM produces realistic outcomes?

Section 7: Health Economics Applications

When to Use Each Approach

Decision Framework

Your Situation Recommended Approach
Have behavioral data, want realistic agents ML as Parameters (Approach 1)
Unknown parameters, need calibration ML Calibration (Approach 2)
Complex decisions, adaptive behavior needed ML/RL Behaviors (Approach 3)
Need fast scenario sweeps Surrogate Models (Approach 4)
Policy optimization, gradient-based Hybrid/Differentiable (Approach 5)
Cognitive realism, no training budget LLM Agents (emerging)

WISE Project: Approach 1 (ML forecasts → ordering parameters) + potential Surrogate for sensitivity

Health Economics Application Examples

VBC Simulation

Problem: Evaluate shared savings contracts

Integration: - ML predicts patient risk - ABM simulates care decisions - Surrogate enables contract optimization

Outcome: Optimal risk corridor design

Clinical AI Adoption

Problem: Will clinicians use AI tools?

Integration: - LLM agents simulate physician reasoning - ABM captures workflow constraints - RL learns optimal nudge policies

Outcome: Adoption pathway recommendations

Epidemic Response

Problem: Vaccine prioritization strategy

Integration: - ML calibrates transmission from data - Differentiable ABM enables gradient-based optimization - Fast counterfactual analysis

Outcome: Age-stratified priority schedules

Policy Impact Analysis Workflow

Four-Phase Approach

Phase 1: Calibrate

  • Fit ABM to historical data
  • Use ML for parameter estimation
  • Validate against held-out periods

Phase 2: Surrogate

  • Generate 10K+ scenarios
  • Train neural network approximator
  • 1000x speedup for sweeps

Phase 3: Sweep

  • Grid search policy space
  • Uncertainty quantification
  • Identify Pareto-optimal policies

Phase 4: Validate

  • Run full ABM on candidates
  • Check surrogate accuracy
  • Stress-test edge cases

This workflow combines multiple integration approaches for robust policy analysis

Section 8: Wrap-Up

Key Takeaways

1. Five Integration Approaches

  • Calibration: ML estimates ABM parameters
  • Behaviors: ML/RL drives agent decisions
  • Surrogates: ML approximates ABM for speed
  • Hybrid/Diff.: End-to-end differentiable optimization
  • LLM Agents: Natural language reasoning (emerging)

2. Choose Based on Your Need

  • Unknown parameters → Calibration
  • Complex decisions → ML/RL behaviors
  • Computational bottleneck → Surrogate
  • Policy optimization → Hybrid/RL
  • Cognitive realism, no training → LLM Agents

3. WISE Project Approach

ML forecasts → ABM parameters → Policy simulation

Resources

Key Papers

Topic Reference
Integration review Zhang et al. (2021). IEEE TNNLS
Differentiable ABM Chopra et al. (2023). AAMAS
Surrogate modeling Angione et al. (2022). PLOS ONE
Neural calibration Gaskin et al. (2023). PNAS
Beer Game DQN Oroojlooyjadid et al. (2021). M&SOM
LLM agent society Piao et al. (2025). arXiv

Tools

Looking Ahead

This Afternoon (1:30-5:00)

Supply Chain Simulation Foundations

  • Introduction to ABM concepts
  • Building your first Mesa model
  • Hands-on: Simple supply chain agent

Day 4

Full ABM Development

  • Multi-facility supply chain model
  • Connecting ML forecasts
  • Policy experiments

Questions & Discussion

Discussion Prompts

  1. For the WISE project supply chain, which integration approach seems most promising? Why?

  2. What data would we need to train an RL agent for inventory management?

  3. How might we validate that our ML → ABM integration produces realistic results?

Let’s discuss before the hands-on session!

References

  • Angione, C., et al. (2022). Using ML as a surrogate model for ABM. PLOS ONE.
  • Chopra, A., et al. (2023). Differentiable Agent-based Epidemiology. AAMAS.
  • Gaskin, T., et al. (2023). Neural parameter calibration for ABMs. PNAS.
  • Oroojlooyjadid, A., et al. (2021). Deep Q-Network for the Beer Game. M&SOM.
  • Piao, R., et al. (2025). AgentSociety: Large-scale simulation with LLM agents. arXiv.
  • Zhang, L., et al. (2021). Synergistic integration between ML and ABM. IEEE TNNLS.