Other Reference Materials
As part of the HTM learning process, we wanted to give you a variety of reference materials to help you get up to speed quickly. We scoured the net to find the best links for mathematics, statistics, Python and more. We've also put together a wiki page on this very subject, so please suggest additional items of interest.
- Python
- Mathematics and Statistics
- Neuroscience and Theoretical Neuroscience
- Machine Learning
- Bayesian Networks
- Belief Propagation
- Hierarchical Temporal Memory
Python
Python Tutorial »
Guido van Rossum
Guido, the father of Python, has put together a tutorial that introduces the reader informally to the basic concepts and features of the Python language and system.
Python Reference Manual »
Guido van Rossum
Guido, the father of Python, created this reference manual describing the syntax and 'core semantics' of the language. It is terse, but attempts to be exact and complete.
Dive Into Python »
Mark Pilgrim
The Author has posted this book under the Gnu Free Documentation License. It is a book for experienced programmers, and covers everything from installation to detailed references on core functionality. Mark has made it available in a variety of formats, including HTML, PDF, Word, plain text, and more.
Non-Programmer’s Tutorial for Python »
Josh Cogliati, Wikibooks
The folks at Wikibooks have put together an introduction to the Python programming language for non-programmers. Although the Numenta technology is geared towards very experienced programmers, this tutorial is very good for any skill level. It is available in LaTeX, HTML, PDF and PostScript.
Mathematics and Statistics
Introduction to Probability and Statistics »
This course from MIT provides an elementary introduction to probability and statistics with applications, including basic probability models, combinatorics, probability distributions and more.
Statistics for Applications »
MIT OpenCourseWareThis undergraduate course from MIT includes a broad treatment of statistics, focusing on specific techniques used in science and the industry.
Theory of Probability »
MIT OpenCourseWareThis course from MIT covers the laws of large numbers and central limit theorems for sums of independent random variables, including Brownian motion and the elements of diffusion theory.
Statistical Learning Theory »
MIT OpenCourseWareThis course from MIT studies the generalization ability of a number of popular machine learning algorithms such as boosting, support vector machines and neural networks.
Mathematical Statistics »
MIT OpenCourseWareThis graduate level course form MIT covers decision theory, confidence intervals and hypothesis testing.
Statistical Methods in Brain and Cognition »
MIT OpenCourseWareThis course from MIT emphasizes statistics as a powerful tool for studying complex issues in behavioral and biological sciences, and includes inferential statistics, confidence intervals, regression and analysis of variance.
Information Theory, Inference and Learning Algorithms »
David MacKayThis book, available in nearly any format (including Google Books), unifies the teaching of information theory and inference in one textbook. This book includes a toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods and variational approximations.
Neuroscience and Theoretical Neuroscience
Introduction to Neuroscience »
MIT OpenCourseWareThis course begins with the study of nerve cells, and ultimately moves onto sensor systems such as hearing, vision and touch.
Neural Basis of Vision and Audition »
MIT OpenCourseWareThis undergraduate course from MIT covers speech and hearing, while focusing on physiological and anatomical studies of the mammalian nervous system.
Statistical Methods in Brain and Cognition »
MIT OpenCourseWareThis course from MIT emphasizes statistics as a powerful tool for studying complex issues in the brain, and covers descriptive statistics, probability, and inferential statistics.
Introduction to Computational Neuroscience »
MIT OpenCourseWareThis undergraduate course gives a mathematical introduction to neural coding and dynamics, including convolution, correlation, game theory, probability theory, and reinforcement learning.
Neuroscience for Kids »
Eric H. Chudler, Ph.D., University of WashingtonThe author has created a website for all students and teachers who would like to learn about the nervous system. This site helps you explore all about the brain, spinal cord, the nervous system, and more.
Machine Learning
Machine Learning Definition »
WikipediaVarious authors on Wikipedia have put together an article, and a journal, for Machine Learning, and more importantly, has a reference section at the bottom of the article.
CS229: Machine Learning »
Andrew Ng, Stanford UniversityThis course provides a broad introduction to machine learning and statistical pattern recognition, and was rated favorably among Numenta employees. Some of the topics it covers are supervised learning, unsupervised learning, learning theory and adaptive control.
Pattern Recognition and Machine Learning »
Christopher M. BishopThis textbook discusses many important algorithms and techniques used in the Machine Learning field. While it is aimed at advanced undergraduates or PhD students, it requires no previous knowledge of pattern recognition or machine learning. Additionally, Mr. Bishop has made a sample chapter available, which covers many of the necessary concepts.
An Introduction to Graphical Models »
Kevin P. MurphyIn this paper, Mr. Murphy discusses the concept of graphical models ("a marriage between probability theory and graph theory"), and fleshes out numerous topics including representing a joint probability distribution, inferring hidden states of a system, and eliminating parameters in learning.
MLpedia »
Machine Learning enthusiasts have created an editable Machine Learning resource along the lines of Wikipedia. Although there hasn’t been a lot of recent edits, there is a lot of information about Variational Bayes, Boosting, Markov chains, and more.
Bayesian Networks
Bayesian Network Definition »
WikipediaAuthors at Wikipedia have written a fairly complete definition of Bayesian networks, including structure learning and parameter learning. The article also has a list of software and external links from which you can learn more.
Probabilistic Reasoning in Intelligent Systems »
Judea PearlThis groundbreaking work by Judea Pearl introduces the reader to belief networks, covering Bayesian Networks, Markov Models, and Dempster-Shafer formulism.
A Brief Introduction to Graphical Models and Bayesian Networks »
Kevin MurphyMr. Murphy has written a tutorial on Bayesian Networks, and discusses Representation, Inference, Learning, Decision theory and Applications.
An Introduction to Bayesian Networks and their Contemporary Applications »
Daryle Niedermayer, I.S.P., PMP, B.Sc., B.A., M.Div.
Mr. Niedermayer has put together quite the site on Bayesian Networks, and includes a section on inference, as well as an in-depth discussion of the Bayes theorem and practical uses for Bayesian Networks.
Belief Propagation
Belief Propagation Definition »
WikipediaReferencing Judea Pearl, David MacKay and Brendan Frey, the writers at Wikipedia have put together a fairly complete definition of Belief Propagation. The article includes tree algorithms, algorithms for general graphs and complexity issues, and information about Generalized Belief Propagation.
Hierarchical Temporal Memory
While the papers presented here are older, the basic solutions we discuss haven't changed significantly. However, the way we discuss them (terminology) and represent them (graphically) have changed significantly. These documents may be helpful to some, but we encourage you to focus your attention on our newer algorithm-related documents, found in our Education section.
A Hierarchical Bayesian Model of Invariant Pattern Recognition in the Visual Cortex »
Dileep George & Jeff Hawkins
In this paper published in the proceedings of the International Joint Conference on Neural Networks, Dileep and Jeff give an in-depth look at how the visual cortex utilizes a hierarchical model of invariant visual pattern recognition. Written while both were at RNI, this is an early work, but many have found it extremely useful in understanding HTM.
Invariant Pattern Recognition Using Bayesian Inference on Hierarchical Sequences »
Dileep George
Dileep discusses the use of hierarchy in learning, inference and prediction, and discusses the simulation of a line drawing recognition system. Although this is an early work, it demonstrates the use of hierarchy quite nicely.
