Deep Reinforcement Learning for Aging Cheese Inventory Management Under Correlated Prices

In aging cheese inventory management, producers must jointly consider auto-regressive and correlated prices for raw milk and for age-differentiated cheese products. Purchasing decisions determine the youngest inventory additions, while production and issuance decisions balance immediate revenues against further maturation. Issuance flexibility arises because the cheese age label represents a minimum maturation time, allowing deliberate excess.

We develop a generic Markov decision process model for ameliorating inventory systems that captures auto-regressive and correlated purchase and sales prices. Because the state and action spaces grow exponentially with the number of age classes, we solve the problem using deep reinforcement learning (DRL). However, standard actor-critic DRL algorithms struggle with the high-dimensional action spaces induced by issuance flexibility. We therefore introduce and compare three strategies to address this challenge. First, we consider reducing the action space dimensionality by curtailing issuance flexibility. The second approach reduces dimensionality via actor pipelining, allowing issuance actions to be determined by simple decision rules. Third, we analyze multi-agent actor designs that distribute learning across multiple neural networks.

Our results show that actor pipelining provides the best trade-off between policy quality and computational effort, outperforming multi-agent designs. Compared to a rolling-horizon planning benchmark, our DRL policies increase profits by 16.7% on average. Moreover, modeling cheese-specific price dynamics yields a 7.1% profit improvement relative to policies based on simplified price assumptions. Full factorial experiments reveal that, in contrast to prior work, the benefit of issuance flexibility is limited under cheese-specific price processes and becomes substantial only in highly volatile, fast mean-reverting environments.