Advancements in computing capabilities have enabled machine learning algorithms to learn directly from large amounts of data. Deep reinforcement learning is a particularly powerful method that uses agents to learn by interacting with an environment of data. Although many traders and investment managers rely on traditional statistical and stochastic methods to price assets and develop trading and hedging strategies, deep reinforcement learning has proven to be an effective method to learn optimal policies for pricing and hedging. Machine learning removes the need for various parametric assumptions about underlying market dynamics by learning directly from data. This research examines the use of machine learning methods to develop a data-driven approach to derivatives pricing and dynamic hedging. Nevertheless, machine learning methods like reinforcement learning require an abundance of data to learn. We explore the implementation of a generative adversarial network-based approach to generate realistic market data from past historical data. This data is used to train the reinforcement learning framework and evaluate its robustness. The results demonstrate the efficacy of deep reinforcement learning methods to price derivatives and hedge positions in the proposed systematic GAN-based market simulation framework.
01
A Wasserstein GAN trained on historical equity data that generates arbitrarily many synthetic and realistic forward return paths, capturing stock generic properties such as fat tails, volatility clustering, and autocorrelation structure.
02
A deep RL agent that learns to price derivatives without parametric assumptions, operating directly from market state observations, removing any human-defined processes or models such as Black-Scholes.
03
A multi-agent framework that learns optimal discrete-time delta-hedging strategies under realistic market frictions, including transaction costs and spreads, and outperforming classical model-based benchmarks.
The proposed framework couples a GAN-based market simulation module with a multi-agent deep reinforcement learning system. Historical equity and options data trains a generative model that produces statistically faithful synthetic market scenarios. These scenarios form the training environment in which RL agents learn optimal pricing and hedging policies without ever imposing parametric assumptions on the underlying price process.
Reinforcement learning agents require massive amounts of environment interaction data. Historical financial time series are scarce — decades of daily data amount to only a few thousand observations. To overcome this bottleneck, we train a Wasserstein GAN with gradient penalty (WGAN-GP) on historical equity prices and implied volatility surfaces to synthesize a virtually unlimited supply of realistic market scenarios.
The generator maps a latent noise vector to sequences of market states (spot prices, returns, volatilities, and option prices). The discriminator distinguishes generated trajectories from real historical data. WGAN-GP training provides stable convergence and well-calibrated gradient feedback. The resulting generator reproduces key stylized facts of financial returns:
The derivatives pricing and hedging problem is formulated as a Markov Decision Process (MDP). At each discrete time step the agent observes the current market state and selects an action to maximize a long-run risk-adjusted reward.
State Space 𝒮
Current spot price, time-to-expiry, implied volatility, current portfolio Greeks (Δ, Γ, Θ, Vega), normalized P&L, and cash position.
Action Space 𝒜
Continuous hedge ratio Δ ∈ [−1, 1] and, for the pricing agent, an option mid-price offset from theoretical value.
Reward 𝑟
Risk-adjusted P&L net of proportional transaction costs, with a variance penalty to encourage stable hedging over time.
Agents are trained using Proximal Policy Optimization (PPO) with an actor-critic architecture. The actor network outputs the hedging action distribution; the critic estimates the state-value function used to compute advantages. Multiple agents are instantiated across different option strikes and maturities to form the multi-agent system, enabling generalization across the volatility surface.
Classical Black-Scholes delta hedging assumes continuous rebalancing and zero transaction costs — assumptions that are violated in every real market. Our RL hedging agent learns a discrete-time rebalancing policy that directly accounts for bid-ask spreads and proportional transaction fees, optimizing a mean-variance objective over the realized hedging P&L distribution.
The learned policy adapts dynamically to changing market regimes: when transaction costs are high relative to the option's Gamma exposure, the agent learns to under-hedge and let the position drift — behavior consistent with optimal bandwidth-based hedging theory but derived purely from interaction with the simulated market environment.
The framework is evaluated on European call and put options on equity underlyings, with GAN-generated market scenarios validated against held-out historical data. Performance is benchmarked against Black-Scholes delta hedging and binomial tree pricing under equivalent transaction cost assumptions.
Reduced Hedging Error
The RL hedging agent achieves lower P&L variance than Black-Scholes delta hedging when transaction costs are incorporated, particularly for high-Gamma near-expiry options.
GAN Fidelity
Synthetic price paths produced by the WGAN-GP match the return distribution, autocorrelation function, and volatility clustering of real market data across multiple equity underlyings.
Assumption-Free Pricing
The RL pricing agent learns competitive derivative prices without parametric assumptions, generalizing across strikes and maturities on GAN-generated out-of-sample scenarios.
Full text available via MIT DSpace — open access.
@mastersthesis{qian2023thesis,
author = {Qian, Samson},
title = {Multi-Agent Deep Reinforcement Learning and {GAN}-Based Market Simulation
for Derivatives Pricing and Dynamic Hedging},
school = {Massachusetts Institute of Technology},
year = {2023},
url = {https://dspace.mit.edu/handle/1721.1/150206},
}