• Home >
• For Developers >
• Numenta Blog

About the Zeta1 Learning Algorithms

Dear Developers,

As of this blog, over 3000 of you have downloaded NuPIC! Our forums have been very active with questions. From the nature of these questions, I conclude that several of you have read through our documentation and have started exciting explorations into different application domains.

I am writing this note to go into a little more depth on working with the algorithms, and to share our algorithm roadmap. The algorithms we released in NuPIC are a first step in implementing the full theory of HTMs. We believe that the current version has many strengths, as evidenced by the strong performance of our Pictures demonstration. However we also know there is an even longer list of improvements that are yet to be made. In this note I will describe some of the shortcomings of the Research Release and tie them to our roadmap. We are excited that you are working with us. Being aware of these limitations, I think, will help you minimize the number of wrong turns you may take.

1. Time based inference

Let us start with the obvious: the Research Release of NuPIC currently uses temporal information only during learning. It does not use time during inference (and therefore cannot do prediction). This release primarily addresses problems where temporal information is used during learning, but where inference is based on a single snapshot (such as vision). Using temporal information during learning is critical to learning invariant representations in HTMs. This concept is discussed in detail in our two whitepapers. Temporal inference, or Time Based Inference (TBI), is a feature that will be supported in a future release. While we believe that many applications can be implemented without TBI, other applications may need to wait.

2. Temporal learning is first-order

The 'order' of a temporal model refers to the number of prior time-steps that model takes into consideration for learning and representation. For example, a first-order temporal model uses the current time step and just one previous time step.

The temporal pooler in the NuPIC research release uses a first-order algorithm. The impact of this limitation is an inability to partition certain kinds of temporal structure. For example, assume that sequences 1 -> 2 -> 3 and 3 -> 2 -> 1 occur frequently at the input of a node. Ideally, we would like to have separate representations for these two different sequences. The temporal pooler in the research release views these sequences as the same.

This limitation affects problems that require higher-order temporal structure. Music is one example - B, D, F is a very different musical sequence than D, F, B. A side effect of not having a higher order temporal model is that the temporal groups do not share coincidences. In many problems, meaningful overlapping groups will be produced only when higher-order information is available.

If your problem requires higher-order information or groups with shared coincidences, you can use some simple tricks on the input representation to overcome this limitation. For example, suppose you have an input that rises and then falls, and suppose the solution requires the rising part and the falling part to be in different groups. In this case, one easy way to work around the limitation would be to augment the input information with the direction of the temporal derivative at that instant. That is, the input to a node at any time will be the value and the direction of the temporal derivative. This approach will create different coincidences for the rising part vs. the falling part, and the temporal pooling algorithm could then put them in different groups. This workaround is not a permanent fix, but it could alleviate the limitations of the first-order temporal model in some problems.

3. Nodes have abrupt, discrete boundaries.

Several of you already have observed that the nodes in NuPIC have discrete boundaries that abruptly stop the flow of information in space. These discrete boundaries create problems for us in domains like Pictures, or image recognition in general, where space varies in a continuous manner.

It is easy to see this problem in Pictures. In this example, the lower level nodes do a good job of forming invariant representations. However, some of that invariance is lost when multiple nodes are combined to form representations at a higher level. The coincidences learned at a higher level will be less invariant to objects that are not correctly aligned within the boundaries of the lower level nodes. This results in (1) coincidences that are not maximally invariant, and (2) too many coincidences. The discrete boundary problem could lead to issues in scaling to more complex tasks. We actively are working towards fixing this problem as we work towards a scaled-up vision system.

4. No sparse distributed representation within a node.

One fundamental assumption in a node is that only one temporal group and only one coincidence is active at one time. Although there can be uncertainty about which temporal group is active, the assumption behind the output distribution of a node is that only one temporal group is active.

The problem with not having a distributed representation is that learning is not shared efficiently. Suppose you have objects A, B and C. Assume C can be obtained as a superposition of A&B. (Note that this is not the same as composition. We do have compositionality in the research release). In this case, the invariant representation for C can be obtained as the superposition of the invariant representations of A&B. That is, the fact that the invariant representations for A&B are active is the invariant representation for C. Using this superposition property to represent C is much more efficient than having to learn a separate representation for C.

As with discrete boundaries, this issue is one of learning complexity and memory requirements. It is another limitation that manifests itself in larger problems, and again one we are actively researching.

5. Other Limitations

Lack of lateral constraints and not using feedback during learning are some of the other limitations of the existing algorithms. Having no lateral constraints can increase the number of false positives in a recognition task. Having no feedback during learning affects the types of pooling that can happen within a node. However, these limitations are not expected to significantly affect your progress while working on the research release.

Numenta's HTM Roadmap

In the short term, our next major internal milestone is building a real-world vision system for recognizing a set of objects in grey scale images. Clearly this system is a big step up in terms of complexity from our Pictures demo. I expect that it will require solutions to some or all of the problems that I listed above. Our strategy is to prioritize the problems in terms of their necessity to solve this real-world vision problem. Currently we have assigned equal importance to these problems, although that might change. Once the vision system is ready, we plan to release the full source code and data set, just as we did with the Pictures example.

Strategy for the Developer

In these early stages, as with any complex software package, it is important to understand the limitations and to work within them. Below, I list some general strategies you might adopt in your NuPIC explorations.

1) Use the research release to learn about HTMs and for experimentation.

2) Understand the strengths and limitations of the existing algorithms. Don't let the expectations about how brains might work cloud your understanding of the algorithms.

3) See whether you can map your problem to an existing example. Read our documentation on the kinds of applications suitable for HTMs.

4) Start with a simple version of your application and build up incrementally.

5) Modify the key algorithm parameters and understand their effects.

6) At each step, create a detailed hypothesis before doing experiments.

7) Use the forums to discuss your ideas with others.

Despite these limitations, I am confident that several of you will develop successful HTM applications with the research release of NuPIC. Thank you for being part of this endeavor.