Monte-Carlo Tree Search (MCTS) is a workhorse for planning in uncertain environments, but it typically compresses the outcomes of simulated rollouts into a single scalar mean. That works fine when the goal is to maximize expected return, but it discards information about rare, high-impact events. For applications where downside risk matters — such as finance, robotics, or safety-critical systems — a mean-only view can lead to policies that look good on average but fail badly in the tail.
A new paper on arXiv introduces Categorical Thompson Sampling with Optimism, a distributional extension of MCTS. Instead of storing only the average return, the algorithm maintains a categorical representation of the return distribution for each state–action pair. This allows the planner to reason about quantile-based risk measures like conditional value-at-risk (CVaR), which captures the expected loss in the worst-case tail of the distribution.
The authors position the method as a way to make MCTS risk-sensitive without abandoning its tree-search structure. By combining categorical distributions with Thompson sampling and an optimism bias, the algorithm balances exploration with a focus on tail outcomes. The work is still at the preprint stage, but it points toward a practical path for bringing distributional thinking into one of the most widely used planning frameworks.