Get started with custom lists to organize and share courses.

Sign up

Class Central is learner-supported. When you buy through links on our site, we may earn an affiliate commission.

Maximum Entropy Methods

Santa Fe Institute via Complexity Explorer

9 Reviews 1084 students interested
Found in Mathematics

Taken this course? Share your experience with other students. Write review

Overview

Sometimes the best description of a system is one that acknowledges our ignorance. Maximum entropy methods provide a coherent framework for doing this in a rigorous fashion, and their success has been felt in everything from physics to ecology to machine learning.

In this tutorial, Simon DeDeo introduces MaxEnt through a series of examples, taking students from the basics of the subject up to its applications in science and engineering. Basic comfort with the use of probabilities, and familiarity with exponentials and logarithms, is required. By the end of the course, students will have a tool for modeling complex systems, and a new set of concepts for thinking about what models are meant to do in the first place.

Note that Complexity Explorer tutorials are meant to introduce students to various important techniques and to provide illustrations of their application in complex systems.  A given tutorial is not meant to offer complete coverage of its topic or substitute for an entire course on that topic.   

This tutorial is designed for more advanced math students.  Math prerequisites for this course are an understanding of calculus, partial derivatives, shannon entropy; basic probability.

Read Simon DeDeo's Q&A about his Maximum Entropy tutorial here.

 

 

Syllabus

  1. Maximum Entropy Methods (Simon DeDeo)
  2. A Simple Example: Waiting for a Taxicab
  3. The Maximum Entropy Method
  4. MaxEnt Applied to the Taxicab Example, Part 1
  5. MaxEnt Applied to the Taxicab Example, Part 2
  6. Review of MaxEnt
  7. A Real-World Example: Modeling the Open Source Ecosystem, Part 1
  8. Modeling the Open Source Ecosystem, Part 2
  9. Modeling the Open Source Ecosystem, Part 3
  10. A Second Real-World Example: Modeling Sears-Roebuck Catalog Prices, Part 1
  11. Modeling Sears-Roebuck Catalog Prices, Part 2
  12. Conclusion
  13. Homework

 

Taught by

Simon DeDeo

Reviews for Complexity Explorer's Maximum Entropy Methods
4.9 Based on 9 reviews

  • 5 stars 89%
  • 4 star 11%
  • 3 star 0%
  • 2 star 0%
  • 1 star 0%

Did you take this course? Share your experience with other students.

Write a review
  • 1
Anonymous
5.0 3 months ago
Anonymous completed this course.
Really excellent course. Appreciate his approach from physics point of view.

These max entropy distributions reflect how little knowledge we have of the system, but are still able to glean valuable insights. Like the theory of gases, we can't possibly know the trajectories of each gas molecule, but we can talk of an average kinetic energy ...

Learned a lot from by the Open Source and Sears Roebuck examples.

Thought maybe this is a better way to introduce physicists to statistical mechanics by going through max entropy introduction.
Was this review helpful to you? Yes
Anonymous
4.0 a year ago
Anonymous completed this course.
Very interesting, I would have liked to have the scope/limitations of the method of Legrange multipliers explained a little more fully. Also, the possible appendix content about proving that the entropy function has a unique maximum, even subject to constraints, hasn't materialized or I haven't been able to find it.
Was this review helpful to you? Yes
Anonymous
5.0 6 months ago
Anonymous is taking this course right now.
Awesome course. Loved it. Brilliant teacher. Was just sad it was so short. More please - with more examples and more problems to do!

Would also love to see a course by this teacher on Lagrangian multipliers with more examples and further development of the geometrical intuitions he gives.
Was this review helpful to you? Yes
Ces L
5.0 a year ago
Ces completed this course, spending 2 hours a week on it and found the course difficulty to be easy.
The most interesting math course I have ever taken. I studied maximum entropy in my degree but what I have learnt in here is much more fundental. I adore the teacher of these lessons, is superclever . He must claim for the Nobel prize in something
Was this review helpful to you? Yes
Teresa H
5.0 a year ago
Teresa completed this course.
A great introduction to the key aspects of Maximum entropy. I could widen my knowledge with the help of more specialized books, thanks to have discovered Its potential through this tutorial.
Was this review helpful to you? Yes
Anonymous
5.0 a year ago
Anonymous completed this course.
Excellent introduction to subject. I am interested in understanding patterns in data from a flow cytometer, and this has given me insights, and inspired me to learn about a new subject.
Was this review helpful to you? Yes
Alejandro C
5.0 a year ago
by Alejandro completed this course, spending 3 hours a week on it and found the course difficulty to be medium.
Profesor DeDeo is an excellent professor, his explanations in this course are clear and profound. Course of Complexity Explore are extremely useful to develop a research area.
Was this review helpful to you? Yes
Rahul M
5.0 a year ago
by Rahul completed this course.
This really short tutorial should be a must take for anyone interested in modelling. For me it was a game changer in thinking about how to model any physical process.
Was this review helpful to you? Yes
Davide V
5.0 a year ago
Davide completed this course.
In my opinion, this course is very Interesting, easy to understand and to follow!

I have learned some mathematical issues that I didn't knew before. Thanks a lot!
Was this review helpful to you? Yes
  • 1

Class Central

Get personalized course recommendations, track subjects and courses with reminders, and more.

Sign up for free

Never stop learning Never Stop Learning!

Get personalized course recommendations, track subjects and courses with reminders, and more.