MathFoundationRL/Book-Mathematical-Foundation-of-Reinforcement-Lgeting: This is the homepage of a recent book entitled “Mathematical Foundations of Reinforcement Lgeting.”

This textbook has getd 5,000+ stars! Glad that it is encouraging to many readers.

My English discleave out course is online now. You can click the above figure or the connect here to jump to our YouTube channel. You can also click the adhereing connects to be straightforwarded to definite lecture videos.

You are toastyly greet to verify out the English videos to see if they are encouraging!

This book aims to provide a mathematical but cordial introduction to the fundamental concepts, fundamental problems, and classic algorithms in fortifyment lgeting. Some essential features of this book are highairyed as adheres.

• The book presents fortifyment lgeting from a mathematical point of see. Hopebrimmingy, readers will not only understand the procedure of an algorithm but also comprehend why it was summarizeed in the first place and why it labors effectively.

• The depth of the mathematics is nurturebrimmingy deal withled to an ample level. The mathematics is also currented in a nurturebrimmingy summarizeed manner to find that the book is cordial to read. Readers can pickively read the materials currented in gray boxes according to their interests.

• Many illustrative examples are donaten to help readers better comprehend the topics. All the examples in this book are based on a grid world task, which is effortless to comprehend and encouraging for illustrating concepts and algorithms.

• When introducing an algorithm, the book aims to split its core idea from complications that may be distracting. In this way, readers can better comprehend the core idea of an algorithm.

• The satisfieds of the book are coherently systematic. Each chapter is built based on the preceding chapter and lays a essential establishation for the subsequent one.

The topics includeressed in the book are shown in the figure below. This book grasps ten chapters, which can be classified into two parts: the first part is about fundamental tools, and the second part is about algorithms. The ten chapters are highly corrcontent. In ambiguous, it is essential to study the earlier chapters first before the tardyr ones.

This book is summarizeed for ageder undergraduate students, graduate students, researchers, and practitioners interested in fortifyment lgeting.

It does not demand readers to have any background in fortifyment lgeting because it commences by introducing the most fundamental concepts. If the reader already has some background in fortifyment lgeting, I apvalidate the book can help them comprehend some topics more presentantly or provide contrastent perspectives.

This book, however, demands the reader to have some understandledge of probability theory and liproximate algebra. Some fundamentals of the demandd mathematics are also included in the appfinishix of this book.

By combining the book with my lecture videos, I apvalidate you can study better.

• Chinese lecture videos: You can verify the Bilibili channel or the Youtube channel.
  The lecture videos have getd 1,300,000+ sees up to Feb 2025 over the Internet and getd very excellent feedback!

• English lecture videos: The English lecture videos have been uploaded to YouTube. Plmitigate see the connects and details in another part of this record.

You can find my info on my homepage https://www.shiyuzhao.net/ (GoogleSite) and my research group website https://shiyuzhao.westlake.edu.cn

I have been teaching a graduate-level course on fortifyment lgeting since 2019. Aextfinished with teaching, I have been preparing this book as the lecture remarks for my students.

I honestly hope this book can help readers delicately go in the exciting field of fortifyment lgeting.

@book{zhao2025RLBook,
  title={Mathematical Foundations of Reinforcement Lgeting},
  author={S. Zhao},
  year={2025},
  publisher={Springer Nature Press and Tsinghua University Press}
}

Many enthusiastic readers sent me the source code or remarks that they increaseed when they studied this book. If you create any materials based on course, you are greet to author an email. I am satisfied to split the connects here and hope they may be encouraging to other readers. I must stress that I have not verified the code. If you have any asks, you can straightforwardly communicate the increaseers.

Code

Python:

R:

C++:

Study remarks

English:

Chinese:

There are also many others remarks made by many other readers on the Internet. I am not able to put them all here. You are greet to recommfinish to me if you find a excellent one.

Bilibili videos made based on my course

(Dec 2024) 4,000+ stars
This textbook has getd 4,000+ stars! Glad that it is encouraging to many readers.

(Oct 2024) Book cover

The summarize of the book cover is finished. The book will be officipartner published by Springer punctual next year. It has been published by Tsinghua University Press.

(Sep 2024) Minor modernize before printing by Springer

I alterd some very inpresentant places that readers may challengingly see. It is presumed to be the final version before printing by Springer.

(Aug 2024) 3000 Stars and more code

The book has getd 3000+ stars, which is a wonderful accomplishment to me. Thanks to everyone. Hope it repartner helped you.

I also getd more code perestablishation from enthusiastic readers. For example, this GitHub page provided Python perestablishation of almost all examples in my book. On the one hand, I am very phired to see that. On the other hand, I am a little worried that my students in my offline class may use the code to do their homelabor:-). Overall, I am satisfied because it shows that the book and discleave out course are repartner encouraging to the readers; Otheralerted, they would not irritate to increase the code by themselves:-)

(Jun 2024) Minor modernize before printing

This is the fourth version of the book create. It is presumed to be the final one before the book is officipartner published. Specificpartner, when proofreading the book manuscript, I distinguished some very inpresentant publishs. Together with some telled by enthusiastic readers, they have been alterd in this version.

(Apr 2024) Code for the Grid-World Environment

We includeed the code for the grid-world environment in my book. Interested readers can increase and test their own algorithms in this environment. Both Python and MATLAB versions are provided.

Plmitigate remark that we do not provide the code of all the algorithms included in the book. That is because they are the homelabor for the students in offline teaching: the students necessitate to increase their own algorithms using the provided environment. Nevertheless, there are third-party perestablishations of some algorithms. Interested readers can verify the connects on the home page of the book.

I necessitate to thank my PhD students, Yize Mi and Jianan Li, who are also the Teaching Assistants of my offline teaching. They donated wonderfully to the code.

You are greet to provide any feedback about the code such as bugs if distinguished.

(Mar 2024) 2K stars

The book has getd 2K stars. I also getd many selectimistic evaluations of the book from many readers. Very phired that it can be encouraging.

(Mar 2024) Minor modernize

The third version of the create of the book is online now.

Compared to the second version, the third version is betterd in the sense that some inpresentant typos have been righted. Here, I would appreciate to thank the readers who sent me their feedback.

(Sep 2023) 1000+ stars

The book getd 1000+ stars! Thank everybody!

(Aug 2023) Major modernize – second version

The second version of the create of the book is online now!!

Compared to the first version, which was online one year ago, the second version has been betterd in various ways. For example, we replotted most of the figures, resystematic some satisfieds to create them evidgo in, righted some typos, and includeed Chapter 10, which was not included in the first version.

I put the first create of this book online in August 2022. Up to now, I have getd priceless feedback from many readers worldexpansive. I want to convey my gratitude to these readers.

(Nov 2022) Will be fusetly published

This book will be published fusetly by Springer Nature and Tsinghua University Press. It will probably be printed in the second half of 2023.

I have getd some comments and adviseions about this book from some readers. Thanks a lot, and I appreciate it. I am still accumulateing feedback and will probably alter the create in disjoinal months. Your feedback can create this book more encouraging for other readers!

(Oct 2022) Lecture remarks and vidoes

The lecture slides have been uploaded in the felderlyer “Lecture slides.”

The lecture videos (in Chinese) are online. Plmitigate verify our Bilibili channel https://space.bilibili.com/2044042934 or the Youtube channel https://www.youtube.com/channel/UCztGtS5YYiNv8x3pj9hLVgg/perestablishenumerates

(Aug 2022) First create

The first create of the book is online.