Entropy is a fundamental concept in data science because it shows up everywhere – from decision trees to similarity metrics to cutting-edge dimensionality reduction algorithms. It's also surprisingly simple, but often poorly explained. Traditionally, the equation is presented with the expectation that you'll memorize it without fully understanding what it means and where it comes from. This video takes a very different approach by showing you step-by-step where this simple equation comes from, so you can easily remember (and derive), understand, and explain it to your friends at parties.

For a complete index of all StatQuest videos, see:
https://statquest.org/video-index/

If you would like to support StatQuest, please consider…

Buy my book “The StatQuest Illustrated Guide to Machine Learning”:
PDF – https://statquest.gumroad.com/l/wvtmc
Paperback – https://www.amazon.com/dp/B09ZCKR4H6
Kindle eBook – https://www.amazon.com/dp/B09ZG79HXC

Patreon: https://www.patreon.com/statquest
…or…
YouTube membership: https://www.youtube.com/channel/UCtYLUTtgS3k1Fg4y5tAhLbw/join

…a cool StatQuest t-shirt or sweatshirt:
https://shop.spreadshirt.com/statquest-with-josh-starmer/

…buy one or two of my songs (or go all out and get a whole album!)
https://joshuastarmer.bandcamp.com/

… or simply donate to StatQuest!
https://www.paypal.me/statquest

Finally, if you want to stay up to date with my research and the creation of new StatQuests, follow me on Twitter:
https://twitter.com/joshuastarmer

0:00 Great song and great introduction
1:28 Introduction to the surprise
4:34 Equation for surprise
6:09 Calculate surprise for a series of events
9:35 Entropy defined for a coin
10:45 Entropy is the expected value of surprise
11:41 The entropy equation
13:01 Entropy in action!!!

#StatQuest #Entropy

Please take the opportunity to connect with your friends and family and share this video with them if you find it useful.