LCA CIFAR Tutorial Part4 Params Files

Introduction

Up to this point, we have been using the .lua file and the .params file generated from it as a black box, changing one or two lines between runs. It is helpful at this point to look in more detail at the structure of the .params file.

The params file defines the objects that make up the PetaVision run. These objects fall into the following categories:

1. The HyPerCol, a container for the other objects
2. layers
3. connections
4. probes

Let's look at the first .params file we generated, input/LCA_CIFAR-One-Image.params.

debugParsing = true;

The debugParsing directive controls whether the actions taken while parsing the params file are reported in the log file. Setting debugParsing to true can help in locating a syntax error in the params file. In a .lua script that uses PVModule.lua, set the lua variable debugParsing to either true or false; the resulting params file will begin with a line setting debugParsing to that value. The default in PVModule.lua is to set debugParsing to true.

The remainder of the .params file defines the HyPerCol, layers, connections, and probes. They are defined by parameter groups with the format

ObjectType "object_name" = {
    ParameterName1 = Value1;
    ParameterName2 = Value2;
    [...]
};

ObjectType defines the type of object to be created: a HyPerCol or a particular type of layer, connection, or probe, or the HyPerCol object. The HyPerCol object is the first object in the params file, and it manages the layers, connections, and probes.

object_name defines the name of the object being created. Two objects in a .params file cannot have the same name, even if they have different object types.

Inside the curly brackets is a list of several parameter definitions, that control the specific behavior of the object. Parameters come in three types: numerical, string, and array. Different parameters within the same group must have different names, even if they are different parameter types.

Numerical parameters may be integer, floating point, or the special values infinity, -infinity, true, or false.

String parameters may be a string enclosed in quotation marks ", or the special value NULL (without quotation marks).

Array parameters are a sequence of one or more numerical parameter values, separated by commas and enclosed as a whole in square brackets. If the array consists of only one value, the brackets are optional.

We now look at the various types of groups in the .params file. The first group must have the ObjectType "HyPerCol". The ordering of params within the group does not matter. If the .params file is generated from a .lua file, the ordering is determined by internals of the lua program that are not under user control.

1 HyPerCol

The column is the outermost wrapper of the simulation; all the layers are proportional to the column. The column sets up a bunch of key simulation details such as how long to run, where to save files, how frequently and where to checkpoint, and adaptive time-step parameters. Here are a few of the important ones:

HyPerCol Parameter	Description
stopTime	sets how long to run experiment; (start - stop)/dt = number of timesteps
dt	length of timestep; stopTime / dt is the number of timesteps to run
outputPath	sets directory path for experiment output
nx	x-dimensions of column; typically match to input image size
ny	y-dimensions of column; typically match to input image size
checkpointWriteDir	sets directory path for experiment checkpoints; usually output/Checkpoints

For more details on the HyPerCol please read the documentation: HyPerCol Parameters

2 HyPerLayer

The layers are where the neurons are contained and their dynamics described. You can set up layers that convolve inputs, have self-self interactions, or even just copy the layer properties or activities of one layer to another and more. All layers are subclassed from HyPerLayer and you can read about their individual properties by following some of the Doxygen documentation.

Some important parameters to notice are nxScale, nyScale and nf since they set up physical dimensions of the layer. phase and displayPeriod describe some of the temporal dynamics of the layer. Most layers have their own unique properties that you can explore further on your own. For now this is a good snapshot. The table below summarizes the types of layers we use and roles in this experiment:

Layer Class	"Name"	Description
ImageLayer	"Input"	loads images from inputPath
HyPerLayer	"InputError"	computes residual error between Input and LeakyIntegrator
HyPerLCALayer	"LeakyIntegrator"	codes a sparse representation of Input using LCA
HyPerLayer	"InputRecon"	reconstruction of input, feeds back into error

Let's look more closely at some more of the layer parameters: displayPeriod, writeStep, triggerFlag, and phase. ImageLayer has a parameter 'displayPeriod' that sets the number of timesteps an image is shown. We then typically set the writeStep and initialWriteTime to be some integer interval of displayPeriod, but this isn't necessary. For example if you want to see what the sparse reconstruction looks like while the same image is being shown to LeakyIntegrator, you can change the writeStep for "InputRecon" to 1 (just note that your output file will get very large very quickly so you may want change the stopTime to a smaller value if you want this sort of visualization).

writeStep determines how frequently OpenPV outputs to the .pvp file (this is the unique binary format used for OpenPV). Triggering can be used to specify when a layer or connection will perform updates. The default behavior is to update every time step. For example, the MomentumConn LeakyIntegratorToInputError has triggerLayerName = "Input". This means that the connection will only update its weights when Input presents a new image.

Normally, we would like to view the reconstruction after converging on a sparse approximation. This is where phase comes in. Phase determines the order of layers to update at a given timestep. In this tutorial, the phases are fairly straightforward, however there might be cases when a different behavior is desired.

Layer Class	"Name"	Phase
ImageLayer	"Input"	0
HyPerLayer	"InputError"	1
HyPerLCALayer	"LeakyIntegrator	2
HyPerLayer	"InputRecon"	3

For more details on the HyPerLayer parameters please read the documentation: HyPerLayer Parameters

3 HyPerConn

The connections connect neurons to other neurons in different layers. Similar to layers, connections are all subclassed from their base class HyPerConn. Connections can learn weights, transform, and scale data from one layer to another.

Connections in OpenPV are always described in terms of their pre and postLayerName, their channel code, and their patch size (or receptive field). We use a naming convention of [PreLayerName]To[PostLayerName] but it is not required if you explicitly define the pre and post layer.

The channelCode value determines if the connection is excitatory (0), inhibitory (1), or passive (-1). In a typical ANN layer, we subtract the input of the inhibitory channel from the excitatory channel. Passive does not actually deliver any data to the post layer, but is useful when you want a connection to only learn weights.

Patch size is determined by the nxp, nyp, and nfp parameters. Restrictions on how you can set these values are explained in detail in [Patch Size and Margin Width Requirements].

The following table summarizes the types of connections that are used and their roles in this experiment:

Connection Class	"Name"	Description
RescaleConn	"InputToError"	Copies and scales the input to the error layer
TransposeConn	"ErrorToLeakyIntegrator"	Transposes (flips pre/post) the LeakyIntegratorToError weights
MomentumConn	"LeakyIntegratorToError"	A learning connection that uses a momentum learning rate
CloneConn	"LeakyIntegratorToRecon"	Clones LeakyIntegratortoError weights to send to reconstruction
IdentConn	"ReconToError"	Copies the reconstruction activities to the error layers negative channel

For more details on the HyPerConn parameters please read the documentation: HyPerConn Parameters

4 Probes

Probes are used to calculate the energy of the network and adapt the timescale used for each tick. More information about this can be found on the AdaptiveTimeScaleProbe page.