1 Getting Started With Numenta Node Plugins

This chapter gets you started with Numenta node plugins by discussing what custom nodes do and when you'd need them. The chapter gives an overview of the development process in C++ or Python, and discusses the interaction between Numenta tools, the Numenta Runtime engine (NRE), the plugin infrastructure, and your plugin.

Topics

Introduction
Developing a Plugin
Understanding the Node Plugin Architecture

Introduction

When you receive the NuPIC software, it includes a number of predefined nodes: sensors, effectors, and nodes that encapsulate the algorithm. Each of these nodes is a NuPIC plugin. Plugins are shared or dynamic libraries that interact with a main application (in this case NuPIC) to provide a certain functionality on demand.

Because all nodes included as part of the NuPIC platform are plugins, you can use those nodes as examples for node creation. NuPIC works with C++ and Python plugins:

VectorFileSensor, VectorFileEffector, Zeta1Node, and Zeta1TopNode are nodes that have been implemented as C++ plugins.
The ImageSensor node has been implemented as a Python plugin. Note that ImageSensor is fairly complex; your own custom node might well be more simple.

Why Custom Nodes?

In many situations, working with the existing nodes can give you good results. In fact, it often makes sense to preprocess your data to fit the existing VectorFileSensor instead of creating a custom node.

However, in some situations you might want to interact with the outside world in different ways than the existing sensors and effectors support. For example, your sensor might receive an image from a frame grabber. Your effector might send data to a GUI for display or send data to a robot arm. NuPIC places no limitation on what your custom sensor and effector can do, as long as they correspond to the existing APIs.

In some situations, you might wish to modify the existing algorithm in Zeta1Node or Zeta1TopNode, and a custom node is the appropriate way to do so.

What Custom Nodes Must Do

Each custom node must implement a fixed set of behaviors:

Initialization - During initialization, the Tools Framework creates the node and its inputs, outputs, and initial state.
Computation - During computation, the custom node processes the inputs and updates its outputs and state. If you're running in a multi-processing environment, the NRE distributes the output to nodes that are linked with the processing node.
Processing node commands - While initialization and computation are always performed by the NRE, node commands come from users. The Python API expects that your node has implemented an execute() command, the C++ API expects that your node has implemented execute(), getParameter(), and setParameter() commands.
Saving node state - Your custom node must be able to save its state when the tools issue a call to save an HTM Network file.

Usually, the save method must be able to save node state at any time, whether the node is trained, untrained, or in between.

Specifying a NodeSpec - The plugin must create a NodeSpec, which describes initial parameters, runtime parameters, and inputs and outputs. The Tools Framework uses the NodeSpec for validation when constructing the node. The NRE uses the NodeSpec when loading the HTM Network and when getting or setting parameters. A NodeSpec is required for both the C++ and the Python API.

Developing a Plugin

You can develop a plugin in either C++ or Python. The two APIs are separate; in contrast to the Tools Framework, the Python node plugin API is not implemented as bindings over the C++ API. The Python API uses a Python-specific object model and corresponds directly to derived Python classes.

Developing a Plugin in C++

The C++ node plugin API provides access to C/C++ libraries and APIs and is suitable for production applications. Most nodes included with NuPIC have been implemented as C++ plugins.

The recommended approach for developing a C++ plugin is to run the quickstart tool. The tool creates a template directory with files that include boilerplate functions for you. You can create your plugin using the resulting files. Comments are included.

This document explains how to run the tools and build your plugin, which files are generated, and which classes and methods you need to implement.

Developing a Plugin in Python

The Python node plug-in API supports rapid prototyping and debugging. It might be a good choice during development. You have access to Python standard libraries and external packages.

Because of performance and security considerations, Python might not be appropriate for production applications.

Developing a Python plugin is discussed in Using the Python Node Plugin API. A simplenode example that includes a sensor, effector, and learning node is available in the examples directory. The ImageSensor node can also serve as an example.

Understanding the Node Plugin Architecture

The plugin infrastructure allows the plugin that contains your custom nodes to interact with the Tools Framework and the NRE. This section explains how the components interact.

The C++ plugin infrastructure runs inside both the NRE and the Tools Framework. The main job of the infrastructure is to load and manage the node instances. What actually happens differs for the Tools Framework and the NRE.

Interaction Between Plugins and the Tools Framework

The tools and plugins interact during construction of the initial network and network file. When constructing a network that includes a node type, the tools load and read a nodeSpec structure for each node type. From the nodeSpec, the tools can validate and initialize the parameters, inputs, and outputs for each node type.

The tools framework uses the plugin manager to load all plugins. Each plugin registers its node types. When the tools are ready to save the HTM Network file, they call createInitialState() for each node, passing in the initial parameters specified in the node creation script.

Interaction Between Plugins and the NRE

At runtime, the plugins interact with the NRE as follows:

The description below uses the C++ method names, but the interaction is similar for Python plugins.

1. The session.loadNetwork() call loads a previously saved HTM Network file.

2. Each time the NRE sees a node, it goes to the plugin infrastructure and requests creation of that node.

3. The plugin infrastructure finds the NTA_createFunc() method for the requested node and calls it. The result of the call is an INode pointer that can be used by the NRE during creation of the HTM Network.

4. The NRE instantiates each node and creates the full network of nodes connected to each other through their inputs and outputs.

5. The NRE can now start running the network. The NRE's scheduler cycles through all enabled nodes and calls each node's compute() method in order. The exact order depends on the scheduler and the multi-CPU configuration (see "Using the NRE: Advanced Topics" in Advanced NuPIC Programming).

6. While the HTM Network is running or paused, the user (or Python script) can send execute commands to any node using the session API.

7. The network can be saved at any time by calling the session's saveRemoteNetwork() method. At this time, the NRE iterates through each node in turn and calls its saveState() method.