Reward Optimizing Recommender Systems

Readers of recent books and recommender systems papers will learn a great deal about a plethora of machine learning techniques used in recommender systems including collaborative filtering, content based recommender systems, click models, off policy estimation, reinforcement learning and contextual bandits. They will also learn about A/B testing, but it will be somewhat an afterthought.

This book is different, it makes optimizing for reward at A/B test time the central point of study. The methodology of A/B testing is described in detail, highlighting both its significant strengths and potential limitations. The remainder of the book is developed from this point of view. This foundational nature will require a more comprehensive discussion of inferential and causal theory than other treatments, providing a stronger understanding about how the disparate machine learning techniques for recommendation fit together.

Until now, it has required a great deal of industry experience to identify why approaches based on (say) a direct implementation of reinforcement learning are likely to fail in production. Readers of this book should gain this intuition with less need to make painful mistakes in the wild.