About Time Based Inference
Time Based Inference (TBI) is a method that adds time to the inference paradigm, i.e. knowing what happened before enables better recognition. For example, it is hard to see a green snake hiding in a grassy field. As soon as the snake starts moving though, it becomes much easier to detect.
The examples on the left demonstrate the snake in the grass problem. In the static images you are not easily able to see the objects in all of the noise. However, once time is employed and the video begins to move you will see the objects within the noise.
The same is true of the movies on the right. Again, it is difficult to see the objects in the noise, except when the video begins to play. Even with the noise moving along with the objects, you can easily see the bus or the helicopter once the video plays.
Included with NuPIC 1.5
Release 1.5 of NuPIC includes an initial implementation of TBI that works at the first level of the hierarchy. As the graph below demonstrates, in experiments with Pictures, we consistently see a 10-30% improvement in classification accuracy in noisy domains. The improvement is seen with both static and dynamic noise.
TBI Accuracy
The movie below demonstrates the improved accuracy of TBI over flash inference in the Pictures example. The top half of the movie shows a trained Pictures network performing TBI inference and the bottom half shows the same network performing flash inference. In both cases, an image of a 'cat' is swept across a noisy background. As you can see, with TBI inference the 'cat' output (category output #2) gets more certain each time the cat image is stepped across the background. With flash inference on the other hand, the network is never able to reliably recognize the cat in the noise background (category #2 is never the most certain output).
Using TBI
TBI is enabled by simply setting the temporalPoolerAlgorithm parameter of the Zeta1Node to tbi. The other two choices for this parameter are sumProp and maxProp, which are both flash inference methods described in the Programmer's Guide. The temporalPoolerAlgorithm is a parameter which can be changed at inference time, so you can take any previously trained network and simply set this parameter value to tbi when performing inference. For example, you can first train the network with this parameter set to either maxProp or sumProp and then later run inference with it set to tbi.
Keep in mind that the current implementation of TBI only works well at the first level of the network. So the temporalPoolerAlgorithm at all levels other than the first level should always be set to either maxProp or sumProp.
Using TBI in the Images Application
The images application includes some sample experiments that use TBI on the Pictures data set. These experiments train up a network using the Pictures data set, then run both flash inference and TBI inference with different amounts of noise and report on the classification accuracy of each method.
To run a quick version (just two categories), run the following command which will train the network, then run both flash and TBI inference and report on the results:
python RunExperiment.py experiments/pictures/tbi/quickTo see the inference GUI and single-step through inference, you can run the following command which starts up the inference GUI using the network that was trained in the previous step:
python RunExperiment.py -r -g experiments/pictures/tbi/quick Click the "Step" button repeatedly to see a sequence of noisy pictures plus the network's inference.
TBI with the Images Example
We have a TBI example in the Images example application found in NuPIC 1.5. You can run this example by going to the Images directory (share/projects/images) and typing:
python RunExperiment.py pictures/tbi/normal
This example runs tests comparing the TBI and maxProp settings. For more information, you can see the set up in the params.py file for the TBI example.
