Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Online Course

Fundamentals of Reinforcement Learning

University of Alberta and Alberta Machine Intelligence Institute via Coursera

(24)
  • Provider Coursera
  • Cost Free Online Course (Audit)
  • Session In progress
  • Language English
  • Certificate Paid Certificate Available
  • Duration 4 weeks long
  • Learn more about MOOCs

Taken this course? Share your experience with other students. Write review

Overview

Reinforcement Learning is a subfield of Machine Learning, but is also a general purpose formalism for automated decision-making and AI. This course introduces you to statistical learning techniques where an agent explicitly takes actions and interacts with the world. Understanding the importance and challenges of learning agents that make decisions is of vital importance today, with more and more companies interested in interactive agents and intelligent decision-making.

This course introduces you to the fundamentals of Reinforcement Learning. When you finish this course, you will:
- Formalize problems as Markov Decision Processes
- Understand basic exploration methods and the exploration/exploitation tradeoff
- Understand value functions, as a general-purpose tool for optimal decision-making
- Know how to implement dynamic programming as an efficient solution approach to an industrial control problem

This course teaches you the key concepts of Reinforcement Learning, underlying classic and modern algorithms in RL. After completing this course, you will be able to start using RL for real problems, where you have or can specify the MDP.

This is the first course of the Reinforcement Learning Specialization.

Syllabus

Welcome to the Course!
-Welcome to: Fundamentals of Reinforcement Learning, the first course in a four-part specialization on Reinforcement Learning brought to you by the University of Alberta, Onlea, and Coursera. In this pre-course module, you'll be introduced to your instructors, get a flavour of what the course has in store for you, and be given an in-depth roadmap to help make your journey through this specialization as smooth as possible.

The K-Armed Bandit Problem
-For the first week of this course, you will learn how to understand the exploration-exploitation trade-off in sequential decision-making, implement incremental algorithms for estimating action-values, and compare the strengths and weaknesses to different algorithms for exploration. For this week’s graded assessment, you will implement and test an epsilon-greedy agent.

Markov Decision Processes
-When you’re presented with a problem in industry, the first and most important step is to translate that problem into a Markov Decision Process (MDP). The quality of your solution depends heavily on how well you do this translation. This week, you will learn the definition of MDPs, you will understand goal-directed behavior and how this can be obtained from maximizing scalar rewards, and you will also understand the difference between episodic and continuing tasks. For this week’s graded assessment, you will create three example tasks of your own that fit into the MDP framework.

Value Functions & Bellman Equations
-Once the problem is formulated as an MDP, finding the optimal policy is more efficient when using value functions. This week, you will learn the definition of policies and value functions, as well as Bellman equations, which is the key technology that all of our algorithms will use.

Dynamic Programming
-This week, you will learn how to compute value functions and optimal policies, assuming you have the MDP model. You will implement dynamic programming to compute value functions and optimal policies and understand the utility of dynamic programming for industrial applications and problems. Further, you will learn about Generalized Policy Iteration as a common template for constructing algorithms that maximize reward. For this week’s graded assessment, you will implement an efficient dynamic programming agent in a simulated industrial control problem.

Taught by

Martha White and Adam White

Help Center

Most commonly asked questions about Coursera

Reviews for Coursera's Fundamentals of Reinforcement Learning Based on 24 reviews

  • 5 stars 92%
  • 4 star 4%
  • 3 star 4%
  • 2 star 0%
  • 1 star 0%

Did you take this course? Share your experience with other students.

Write a review
  • 1
Anonymous
Anonymous completed this course.
This course has very good outline and appropriate level for RL beginners. The presentation and description in the lectures is simple but very accurate. I can totally follow it without reading the textbook. The workload is small. I finished the entire course in a week. The coding assignment is well organized and insightful as well. However, the quiz is sometimes confusing without enough details from the lecture. But I think it will be fine if you have more time than me and can read the materials from the textbook they told you. The math is harder for beginners than most other ML introductory course, which is unavoidable because that’s the most important part in reinforcement learning. Better to start with some background in probability and stochastic process.
Was this review helpful to you? Yes
Luiz C
Luiz completed this course, spending 2 hours a week on it and found the course difficulty to be easy.
Fantastic Course. That's the RL MOOC I have been waiting for so long. No surprise it is from Students of RL guru R. Sutton at Uni of Alberta. Very clearly and simply explained. Exercise and Test difficulty spot on. Wouldn't change a iota from this Course. Can't wait to do the rest of this RL specialization
Was this review helpful to you? Yes
Stewart A
Stewart completed this course, spending 5 hours a week on it and found the course difficulty to be medium.
This is a great course on Reinforcement Learning (RL) and I thoroughly recommend it. This is the first course in the four course Reinforcement Learning specialization from the Alberta Machine Intelligence Institute (AMII) at University of Alberta. The course introduces the key concepts and goals of RL...
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
First of all this course is based on an excellent book, "Reinforcement Learning, An Introduction - 2nd edition" by Sutton and Barto. The text is a clearly written with graphs and illustrations. I especially like the bibliographical and historical remarks to get context since I'm a contextual learner.

Ok, so I'm not a huge fan of multiple selection questions (the ones with the square boxes). But it is what it is.

The Jupyter notebooks for weeks 1 and 4 give you a feel of research quality results with just simple coding required. For an introductory course it's a good balance, but I suspect the programming workload to increase progressively in later courses.

My recommendation is to take this course even if machine learning isn't your speciality. You're sure to find plenty of applications for the tools you'll learn here.
Was this review helpful to you? Yes
Prateek J
Prateek completed this course, spending 20 hours a week on it and found the course difficulty to be medium.
This course comes straight from the capital of reinforcement learning and that is Alberta Machine Intelligence Institute which is co-headed by Rich Sutton himself. You can be sure of the fact that this course is free of all fat and nutritious. The course gives you the fundamental understanding of Markov Decision Processes and approximate Dynamic Programming required to build expert RL systems in course 2 and 3 of the specialization. I recommend everyone who has ever thought of breaking into RL come and take this course. It's hands-on with explanation of all key conceptual paradigms in Python and gives you that much needed head start you've been looking for so long after you heard about AlphaZero, AlphaGo and other recent advancements in AI.

Regards & Thanks

Prateek.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
I enjoyed taking this course and feel that it has expanded my tool-kit. The course was well constructed with reading assignments from the book followed by relatively short videos and assignments. I spent around 4-5 hours a week and most of the time was on the reading assignment. I enjoyed the programming assignments though it would have been nice if they had provided lesser scaffolding and asked us to write more code. The quizzes were ok and some of the questions were rather ambiguous and could have been improved with better wording. Overall I had a great learning experience and look forward to completing the series.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
The course is overall very good, and it actually introduces you to Reinforcement Learning from scratch. Lectures are very clear, quizzes are challenging and the course relies on a text book, provided when you enroll. The only weak point, but not a serious issue, is that most of the lectures do not add content to what is in the book. Since studying the book is in fact mandatory, they could have used the lectures to better explain some concepts, assuming people read the book. Sometimes they do, but not so often.
Was this review helpful to you? Yes
Matteo H
Matteo completed this course, spending 7 hours a week on it and found the course difficulty to be medium.
This course formed an exceptional introduction to RL as a field while only requiring a rudimentary background in Python. I used this course to grow my understanding of Python (which is a new programming language for me) as well as introduce some of the problems of Reinforcement Learning.

The "guest lecturers" in each module provided an interesting breadth of perspective on the material, while the primary instructors were clear, articulate, and engaging.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
Fundamentals of Reinforcement Learning is one of the best Online Courses I did on Coursera. I like that the course is based on a text book (Reinforcement Learning by Sutton), so you can really dig into the theory. Also the exercises are very helpful and ambitious which I like. I haven't found much advanced online courses which are so well explained like this one.
Was this review helpful to you? Yes
Andrei P
Andrei completed this course and found the course difficulty to be easy.
The course covers basics of reinforcement learning. It presents main ideas of the topic. The course instructors are very clear. Understanding of the course does not require neither advanced skills in mathematics nor in programming. However the presented concepts are useful, if you are beginner and know nothing about reinforcement learning
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
This is a great course which require a genuine commitment. The teachers have made a great effort to make you understand the Bellman equation in details! The quizzes and the coding exercises have an appropriate level of difficulty. You really have to take your pencil and your paper before answering.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
Really good course. It's really interesting and explains really well the basic concepts of the reinforcement learning. The mix between reading the text book and the videos gives the ability to understand very well, and the programming assignments let's you put to test the things you've learned.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
I really like the course. It's simple and easy to understand. I managed to finish the entire course in a day. The programming assignments are well thought out and are a good mix of easy and challenging, while also giving a good understanding of the underlying concepts.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
This course provided great value for me, the content and explanations are of good quality. Quizzes and programming exsersies are challenging enough to help you grasp nessesary concepts and get hands on experience. Look forward to the next course in the spesialisation.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
It gets the point across and the examples used are decent. But it's not a very engaging course. The assignments are not very difficult and walk you through the problem well. I found myself skipping through the lectures a bit, but learnt the basic ideas of RL fine.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
It is a reallui good course. It is basically an introduction course to RL but it has good reference (that you have to read) and video lectures which explain the reference book with some examples. The resources, such as notebooks, are well done and challenging enough.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
This course enforces one to become strong with the fundamentals of RL and implementing it in code just adds icing on the cake by giving confidence. I would recommend one to take this course and move ahead in this field.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
Best place to start reinforcement learning if you are new to it. Challenging assignments based concepts. Quizzes to test your fundamentals. Everything is organised.

Much more importantly good discussion forum.
Was this review helpful to you? Yes
Kim F
Kim completed this course, spending 3 hours a week on it and found the course difficulty to be medium.
I found the course very interesting. The videos very good an informative. I like the fact that the videos are describing theory (with real world examples) and not trying to teach Python while doing it.
Was this review helpful to you? Yes
Anonymous
Anonymous completed this course.
This course is from the number 1 origin of reinforcement learning.

I think this course is a basic course and passing it alone without the other courses in the Specialization is not enough.
Was this review helpful to you? Yes
  • 1

Class Central

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free

Never stop learning Never Stop Learning!

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free