Note

This documentation is for a development version. Click here for the latest stable release (v3.3.0).

# API reference¶

This section details the modules, classes, and functions available in NengoDL. It is divided into two sections. The first section describes the objects relevant to NengoDL users. More information on these objects can also be found in the User guide. The second section describes objects that only NengoDL developers need to worry about.

## Users¶

These objects are the main access points for the user-facing features of NengoDL.

### Simulator¶

The Simulator class is the access point for the main features of NengoDL, including running and training a model.

 nengo_dl.Simulator Simulate network using the nengo_dl backend. nengo_dl.Simulator.reset Resets the simulator to initial conditions. nengo_dl.Simulator.soft_reset Deprecated, use Simulator.reset instead. nengo_dl.Simulator.predict Generate output predictions for the input samples. nengo_dl.Simulator.predict_on_batch Generate output predictions for a single minibatch of input samples. nengo_dl.Simulator.compile Configure the model for training/evaluation. nengo_dl.Simulator.fit Trains the model on some dataset. nengo_dl.Simulator.evaluate Compute the loss and metric values for the network. nengo_dl.Simulator.step Run the simulation for one time step. nengo_dl.Simulator.run Run the simulation for the given length of time. nengo_dl.Simulator.run_steps Run the simulation for the given number of steps. nengo_dl.Simulator.train Deprecated, use Simulator.compile and Simulator.fit instead. nengo_dl.Simulator.loss Deprecated, use Simulator.compile and Simulator.evaluate instead. nengo_dl.Simulator.save_params Save network parameters to the given path. nengo_dl.Simulator.load_params Load network parameters from the given path. nengo_dl.Simulator.freeze_params Stores the live parameter values from the simulation back into a Nengo object definition. nengo_dl.Simulator.get_nengo_params Extract model parameters in a form that can be used to initialize Nengo objects in a different model. nengo_dl.Simulator.check_gradients Perform gradient checks for the network (used to verify that the analytic gradients are correct). nengo_dl.Simulator.trange Create a vector of simulation step times matching probed data. nengo_dl.Simulator.close Close the simulation, freeing resources. nengo_dl.Simulator.get_name Returns the standardized string name for input Nodes or output Probes.
 nengo_dl.simulator.SimulationData Data structure used to access simulation data from the model.
class nengo_dl.Simulator(network, dt=0.001, seed=None, model=None, device=None, unroll_simulation=1, minibatch_size=None, progress_bar=True)[source]

Simulate network using the nengo_dl backend.

Parameters
networknengo.Network

A network object to be built and then simulated.

dtfloat

Length of a simulator timestep, in seconds.

seedint

Seed for all stochastic operators used in this simulator.

modelModel

Pre-built model object (mainly used for debugging).

deviceNone or "/cpu:0" or "/gpu:[0-n]"

This specifies the computational device on which the simulation will run. The default is None, which means that operations will be assigned according to TensorFlow’s internal logic (generally speaking, this means that things will be assigned to the GPU if GPU support is available, otherwise everything will be assigned to the CPU). The device can be set manually by passing the TensorFlow device specification to this parameter. For example, setting device="/cpu:0" will force everything to run on the CPU. This may be worthwhile for small models, where the extra overhead of communicating with the GPU outweighs the actual computations. On systems with multiple GPUs, device="/gpu:0"/"/gpu:1"/etc. will select which one to use.

unroll_simulationint

This controls how many simulation iterations are executed each time through the outer simulation loop. That is, we could run 20 timesteps as

for i in range(20):
<run 1 step>


or

for i in range(5):
<run 1 step>
<run 1 step>
<run 1 step>
<run 1 step>


This is an optimization process known as “loop unrolling”, and unroll_simulation controls how many simulation steps are unrolled. The first example above would correspond to unroll_simulation=1, and the second would be unroll_simulation=4.

Unrolling the simulation will result in faster simulation speed, but increased build time and memory usage.

In general, unrolling the simulation will have no impact on the output of a simulation. The only case in which unrolling may have an impact is if the number of simulation steps is not evenly divisible by unroll_simulation. In that case extra simulation steps will be executed, which could change the internal state of the simulation and will affect any subsequent calls to sim.run. So it is recommended that the number of steps always be evenly divisible by unroll_simulation.

minibatch_sizeint

The number of simultaneous inputs that will be passed through the network. For example, a single call to Simulator.run will process minibatch_size input instances in parallel. Or when calling Simulator.predict/Simulator.fit with a batch of data, that data will be divided up into minibatch_size chunks.

progress_barbool

If True (default), display progress information when building a model. This will also be the default for the progress_bar argument within Simulator.run and Simulator.run_steps.

Attributes
dataSimulationData

Stores simulation data and parameter values (in particular, the recorded output from probes after calling Simulator.run can be accessed through sim.data[my_probe]).

modelnengo.builder.Model

Built Nengo model, containing the data that defines the network to be simulated.

keras_modeltf.keras.Model

Keras Model underlying the simulation (implements the inference/training loops).

tensor_graphtensor_graph.TensorGraph

Keras Layer implementing the Nengo simulation (built into keras_model).

reset(seed=None, include_trainable=True, include_probes=True, include_processes=True)

Resets the simulator to initial conditions.

Parameters
seedint

If not None, overwrite the default simulator seed with this value (note: this becomes the new default simulator seed).

include_trainablebool

If True (default), also reset any online or offline training that has been performed on simulator parameters (e.g., connection weights).

include_probesbool

If True (default), also clear probe data.

include_processes: bool

If True (default), also reset all nengo.Process objects in the model.

Notes

Changing the TensorFlow seed only affects ops created from then on; it has no impact on existing ops (either changing their seed or resetting their random state). So calling Simulator.reset will likely have no impact on any TensorFlow randomness (it will still affect numpy randomness, such as in a nengo.Process, as normal).

soft_reset(include_trainable=False, include_probes=False)

Deprecated, use Simulator.reset instead.

predict(x=None, n_steps=None, stateful=False, **kwargs)[source]

Generate output predictions for the input samples.

Computation is (optionally) done in batches.

This function implements the tf.keras.Model.predict API.

Parameters
x

Data for input Nodes in the model. This argument is optional; if it is not specified, then data will automatically be generated according to the inputs specified in the Node definitions (e.g., by calling the output function associated with that Node).

x can be specified as:

All inputs should have shape (batch_size, n_steps, node.size_out).

For example, if the model only has a single input Node, then x can simply be an ndarray of data for that Node.

with nengo.Network() as net:
a = nengo.Node([0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.predict(
x=np.ones((50, 10, 1)))


If the network has multiple inputs, then x can be specified as a dictionary mapping nengo.Node objects to arrays, e.g.

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node([0, 0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.predict(
x={
a: np.ones((50, 10, 1)),
b: np.ones((50, 10, 2))
}
)


If an input value is not specified for one of the Nodes in the model then data will be filled in automatically according to the Node definition.

For dynamic input types (e.g., tf.data pipelines or generators), NengoDL tries to avoid introspecting/altering the data before the simulation starts, as this may have unintended side-effects. So data must be specified via one of the standard Keras methods (arrays, list of arrays, or string name dictionary; using a dictionary of Node objects is not supported). In addition, data must be explicitly provided for all input nodes (it will not be automatically generated if data is not specified).

In addition, when using dynamic inputs, data must be provided for the special "n_steps" input. This specifies the number of timesteps that the simulation will run for. Technically this is just a single scalar value (e.g., 10). But Keras requires that all input data be batched, so that input value needs to be duplicated into an array with size (batch_size, 1) (where all entries have the same value, e.g. 10).

with nengo.Network() as net:
a = nengo.Node([0], label="a")
p = nengo.Probe(a, label="p")

with nengo_dl.Simulator(net) as sim:
dataset = tf.data.Dataset.from_tensor_slices(
{"a": tf.ones((50, 10, 1)),
"n_steps": tf.ones((50, 1), dtype=tf.int32) * 10}
).batch(sim.minibatch_size)

sim.predict(x=dataset)

n_stepsint

The number of simulation steps to be executed. This parameter is optional; if not specified, the number of simulation steps will be inferred from the input data. However, this parameter can be useful if you don’t want to specify input data (you just want to use the inputs defined by the Nengo Nodes), or if your model does not have any input Nodes (so there is no data to be passed in).

statefulbool

This parameter controls whether or not the saved internal stimulation state will be updated after a run completes. If stateful=False then the initial state of future runs will be unaffected by this run. With stateful=True, future runs will begin from the terminal state of this run.

For example,

# begins in state0, terminates in state1
sim.predict(..., stateful=False)
# begins in state0, terminates in state2
sim.predict(..., stateful=True)
# begins in state2, terminates in state3
sim.predict(..., stateful=False)
# begins in state2, terminates in state4
sim.predict(..., stateful=True)


Note that Simulator.reset can be used to reset the state to initial conditions at any point.

kwargs: dict

Will be passed on to tf.keras.Model.predict.

Returns
probe_valuesdict of {nengo.Probe: numpy.ndarray}

Output values from all the Probes in the network.

predict_on_batch(x=None, n_steps=None, stateful=False, **kwargs)[source]

Generate output predictions for a single minibatch of input samples.

Batch size is determined by sim.minibatch_size (i.e., inputs must have shape (sim.minibatch_size, n_steps, node.size_in).

This function implements the tf.keras.Model.predict_on_batch API.

Parameters
x

Data for input Nodes in the model. This argument is optional; if it is not specified, then data will automatically be generated according to the inputs specified in the Node definitions (e.g., by calling the output function associated with that Node).

x can be specified as:

All inputs should have shape (batch_size, n_steps, node.size_out).

For example, if the model only has a single input Node, then x can simply be an ndarray of data for that Node.

with nengo.Network() as net:
a = nengo.Node([0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.predict_on_batch(
x=np.ones((1, 10, 1)))


If the network has multiple inputs, then x can be specified as a dictionary mapping nengo.Node objects to arrays, e.g.

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node([0, 0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.predict_on_batch(
x={
a: np.ones((1, 10, 1)),
b: np.ones((1, 10, 2))
}
)


If an input value is not specified for one of the Nodes in the model then data will be filled in automatically according to the Node definition.

n_stepsint

The number of simulation steps to be executed. This parameter is optional; if not specified, the number of simulation steps will be inferred from the input data. However, this parameter can be useful if you don’t want to specify input data (you just want to use the inputs defined by the Nengo Nodes), or if your model does not have any input Nodes (so there is no data to be passed in).

statefulbool

This parameter controls whether or not the saved internal stimulation state will be updated after a run completes. If stateful=False then the initial state of future runs will be unaffected by this run. With stateful=True, future runs will begin from the terminal state of this run.

For example,

# begins in state0, terminates in state1
sim.predict_on_batch(..., stateful=False)
# begins in state0, terminates in state2
sim.predict_on_batch(..., stateful=True)
# begins in state2, terminates in state3
sim.predict_on_batch(..., stateful=False)
# begins in state2, terminates in state4
sim.predict_on_batch(..., stateful=True)


Note that Simulator.reset can be used to reset the state to initial conditions at any point.

kwargs: dict

Will be passed on to tf.keras.Model.predict_on_batch.

Returns
probe_valuesdict of {nengo.Probe: numpy.ndarray}

Output values from all the Probes in the network.

compile(*args, loss=None, metrics=None, loss_weights=None, **kwargs)

Configure the model for training/evaluation.

Parameters
args

Will be passed on to tf.keras.Model.compile.

loss

Loss functions define the error that will be minimized during training.

Losses can be specified as:

• A tf.losses.Loss instance.

• A string matching the name of one of the loss functions above.

• A function that accepts two arguments (y_true, y_pred) and returns a loss value (represented as a tf.Tensor).

• A list of some combination of the above, indicating different loss functions for each output Probe (ordered according to the order in which Probes were added to the model, which corresponds to the order found in Simulator.model.probes).

• A dictionary mapping Probe instances or names to loss functions.

The total loss minimized during training will be the sum over the loss computed on each Probe (possibly weighted by loss_weights).

For example,

with nengo.Network() as net:
node0 = nengo.Node([0])
node1 = nengo.Node([0])
probe0 = nengo.Probe(node0)
probe1 = nengo.Probe(node1)

with nengo_dl.Simulator(net) as sim:
sim.compile(loss={probe0: "mse", probe1: tf.losses.mae})


would compile probe0 to use mean squared error and probe1 to use mean absolute error.

metrics

Metrics are additional values (generally different kinds of losses) that will be computed during training for tracking purposes, but do not affect the result of the training.

They can be specified in all the same ways as loss above.

In addition, multiple metrics can be specified for each output Probe when using a list or dict, by providing multiple functions in a list (e.g., metrics={my_probe: ["mae", "mse"]}).

loss_weightslist or dict

Scalar weights that will be applied to the loss value computed for each output probe before summing them to compute the overall training loss. Can be a list (order corresponding to the order in loss) or a dict mapping Probe instances/names to weights.

kwargs

Will be passed on to tf.keras.Model.compile.

fit(x=None, y=None, n_steps=None, stateful=False, **kwargs)[source]

Trains the model on some dataset.

Note that if the model contains spiking neurons, during the execution of this function those neurons will be swapped for the equivalent non-spiking implementation (as opposed to, e.g., Simulator.evaluate, which will use the spiking implementation).

Optimizer and loss functions are defined separately in Simulator.compile.

This function implements the tf.keras.Model.fit API.

Parameters
x

Data for input Nodes in the model. This argument is optional; if it is not specified, then data will automatically be generated according to the inputs specified in the Node definitions (e.g., by calling the output function associated with that Node).

x can be specified as:

All inputs should have shape (batch_size, n_steps, node.size_out).

For example, if the model only has a single input Node, then x can simply be an ndarray of data for that Node.

with nengo.Network() as net:
a = nengo.Node([0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.compile(loss="mse")
sim.fit(
x=np.ones((50, 10, 1)), y=np.ones((50, 10, 1)))


If the network has multiple inputs, then x can be specified as a dictionary mapping nengo.Node objects to arrays, e.g.

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node([0, 0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.compile(loss="mse")
sim.fit(
x={
a: np.ones((50, 10, 1)),
b: np.ones((50, 10, 2))
},
y=np.ones((50, 10, 1))
)


If an input value is not specified for one of the Nodes in the model then data will be filled in automatically according to the Node definition.

For dynamic input types (e.g., tf.data pipelines or generators), NengoDL tries to avoid introspecting/altering the data before the simulation starts, as this may have unintended side-effects. So data must be specified via one of the standard Keras methods (arrays, list of arrays, or string name dictionary; using a dictionary of Node objects is not supported). In addition, data must be explicitly provided for all input nodes (it will not be automatically generated if data is not specified).

In addition, when using dynamic inputs, data must be provided for the special "n_steps" input. This specifies the number of timesteps that the simulation will run for. Technically this is just a single scalar value (e.g., 10). But Keras requires that all input data be batched, so that input value needs to be duplicated into an array with size (batch_size, 1) (where all entries have the same value, e.g. 10).

Also keep in mind that when using a dynamic input for x the y parameter is unused, and instead the generator should return (x, y) pairs.

with nengo.Network() as net:
a = nengo.Node([0], label="a")
p = nengo.Probe(a, label="p")

with nengo_dl.Simulator(net) as sim:
dataset = tf.data.Dataset.from_tensor_slices(
({"a": tf.ones((50, 10, 1)),
"n_steps": tf.ones((50, 1), dtype=tf.int32) * 10},
{"p": tf.ones((50, 10, 1))})
).batch(sim.minibatch_size)

sim.compile(loss="mse")
sim.fit(x=dataset)

y

Target values for Probes in the model. These can be specified in the same ways as the input values in x, except using Probes instead of Nodes. All targets should have shape (batch_size, n_steps, probe.size_in).

For example,

with nengo.Network() as net:
a = nengo.Node([0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.compile(loss="mse")
sim.fit(
x={a: np.zeros((50, 10, 1))}, y={p: np.zeros((50, 10, 1))})


Note that data is only specified for the probes used in the loss function (specified when calling Simulator.compile). For example, if we have two probes, but only one is used during training (the other is used for data collection during inference), we could set this up like:

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node([0])
p_a = nengo.Probe(a)
p_b = nengo.Probe(b)

with nengo_dl.Simulator(net) as sim:
# compiled loss function only depends on p_a
sim.compile(loss={p_a: "mse"})

# only specify data for p_a
sim.fit(
x={a: np.zeros((50, 10, 1))},  y={p_a: np.zeros((50, 10, 1))})


y is not used if x is a generator. Instead, the generator passed to x should yield (x, y) tuples, where y is in one of the formats described above.

n_stepsint

The number of simulation steps to be executed. This parameter is optional; if not specified, the number of simulation steps will be inferred from the input data. However, this parameter can be useful if you don’t want to specify input data (you just want to use the inputs defined by the Nengo Nodes), or if your model does not have any input Nodes (so there is no data to be passed in).

statefulbool

This parameter controls whether or not the saved internal stimulation state will be updated after a run completes. If stateful=False then the initial state of future runs will be unaffected by this run. With stateful=True, future runs will begin from the terminal state of this run.

For example,

# begins in state0, terminates in state1
sim.fit(..., stateful=False)
# begins in state0, terminates in state2
sim.fit(..., stateful=True)
# begins in state2, terminates in state3
sim.fit(..., stateful=False)
# begins in state2, terminates in state4
sim.fit(..., stateful=True)


Note that Simulator.reset can be used to reset the state to initial conditions at any point.

kwargs: dict

Will be passed on to tf.keras.Model.fit.

Returns
historytf.keras.callbacks.History

The history has two attributes: history.epoch is the list of epoch numbers, and history.history is a dictionary keyed by metric names (e.g., “loss”) containing a list of values of those metrics from each epoch.

evaluate(x=None, y=None, n_steps=None, stateful=False, **kwargs)[source]

Compute the loss and metric values for the network.

Loss functions and other metrics are defined separately in Simulator.compile.

This function implements the tf.keras.Model.evaluate API.

Parameters
x

Data for input Nodes in the model. This argument is optional; if it is not specified, then data will automatically be generated according to the inputs specified in the Node definitions (e.g., by calling the output function associated with that Node).

x can be specified as:

All inputs should have shape (batch_size, n_steps, node.size_out).

For example, if the model only has a single input Node, then x can simply be an ndarray of data for that Node.

with nengo.Network() as net:
a = nengo.Node([0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.compile(loss="mse")
sim.evaluate(
x=np.ones((50, 10, 1)), y=np.ones((50, 10, 1)))


If the network has multiple inputs, then x can be specified as a dictionary mapping nengo.Node objects to arrays, e.g.

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node([0, 0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.compile(loss="mse")
sim.evaluate(
x={
a: np.ones((50, 10, 1)),
b: np.ones((50, 10, 2))
},
y=np.ones((50, 10, 1))
)


If an input value is not specified for one of the Nodes in the model then data will be filled in automatically according to the Node definition.

For dynamic input types (e.g., tf.data pipelines or generators), NengoDL tries to avoid introspecting/altering the data before the simulation starts, as this may have unintended side-effects. So data must be specified via one of the standard Keras methods (arrays, list of arrays, or string name dictionary; using a dictionary of Node objects is not supported). In addition, data must be explicitly provided for all input nodes (it will not be automatically generated if data is not specified).

In addition, when using dynamic inputs, data must be provided for the special "n_steps" input. This specifies the number of timesteps that the simulation will run for. Technically this is just a single scalar value (e.g., 10). But Keras requires that all input data be batched, so that input value needs to be duplicated into an array with size (batch_size, 1) (where all entries have the same value, e.g. 10).

Also keep in mind that when using a dynamic input for x the y parameter is unused, and instead the generator should return (x, y) pairs.

with nengo.Network() as net:
a = nengo.Node([0], label="a")
p = nengo.Probe(a, label="p")

with nengo_dl.Simulator(net) as sim:
dataset = tf.data.Dataset.from_tensor_slices(
({"a": tf.ones((50, 10, 1)),
"n_steps": tf.ones((50, 1), dtype=tf.int32) * 10},
{"p": tf.ones((50, 10, 1))})
).batch(sim.minibatch_size)

sim.compile(loss="mse")
sim.evaluate(x=dataset)

y

Target values for Probes in the model. These can be specified in the same ways as the input values in x, except using Probes instead of Nodes. All targets should have shape (batch_size, n_steps, probe.size_in).

For example,

with nengo.Network() as net:
a = nengo.Node([0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.compile(loss="mse")
sim.evaluate(
x={a: np.zeros((50, 10, 1))}, y={p: np.zeros((50, 10, 1))})


Note that data is only specified for the probes used in the loss function (specified when calling Simulator.compile). For example, if we have two probes, but only one is used during training (the other is used for data collection during inference), we could set this up like:

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node([0])
p_a = nengo.Probe(a)
p_b = nengo.Probe(b)

with nengo_dl.Simulator(net) as sim:
# compiled loss function only depends on p_a
sim.compile(loss={p_a: "mse"})

# only specify data for p_a
sim.evaluate(
x={a: np.zeros((50, 10, 1))},  y={p_a: np.zeros((50, 10, 1))})


y is not used if x is a generator. Instead, the generator passed to x should yield (x, y) tuples, where y is in one of the formats described above.

n_stepsint

The number of simulation steps to be executed. This parameter is optional; if not specified, the number of simulation steps will be inferred from the input data. However, this parameter can be useful if you don’t want to specify input data (you just want to use the inputs defined by the Nengo Nodes), or if your model does not have any input Nodes (so there is no data to be passed in).

statefulbool

This parameter controls whether or not the saved internal stimulation state will be updated after a run completes. If stateful=False then the initial state of future runs will be unaffected by this run. With stateful=True, future runs will begin from the terminal state of this run.

For example,

# begins in state0, terminates in state1
sim.evaluate(..., stateful=False)
# begins in state0, terminates in state2
sim.evaluate(..., stateful=True)
# begins in state2, terminates in state3
sim.evaluate(..., stateful=False)
# begins in state2, terminates in state4
sim.evaluate(..., stateful=True)


Note that Simulator.reset can be used to reset the state to initial conditions at any point.

kwargs: dict

Will be passed on to tf.keras.Model.evaluate.

Returns
outputsdict of {str: numpy.ndarray}

Computed loss/metric values. The overall loss will be in outputs["loss"], and values for each Probe will be in outputs["probe_name_loss"] or outputs["probe_name_metric_name"].

step(**kwargs)[source]

Run the simulation for one time step.

Parameters
kwargsdict

Notes

Progress bar is disabled by default when running via this method.

run(time_in_seconds, **kwargs)[source]

Run the simulation for the given length of time.

Parameters
time_in_secondsfloat

Run the simulator for the given number of simulated seconds.

kwargsdict
run_steps(n_steps, data=None, progress_bar=None, stateful=True)[source]

Run the simulation for the given number of steps.

Parameters
n_stepsint

The number of simulation steps to be executed.

data :

Data for input Nodes in the model. This argument is optional; if it is not specified, then data will automatically be generated according to the inputs specified in the Node definitions (e.g., by calling the output function associated with that Node).

data can be specified as:

All inputs should have shape (batch_size, n_steps, node.size_out).

For example, if the model only has a single input Node, then data can simply be an ndarray of data for that Node.

with nengo.Network() as net:
a = nengo.Node([0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.run_steps(
10, data=np.ones((1, 10, 1)))


If the network has multiple inputs, then data can be specified as a dictionary mapping nengo.Node objects to arrays, e.g.

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node([0, 0])
p = nengo.Probe(a)

with nengo_dl.Simulator(net) as sim:
sim.run_steps(
10,
data={
a: np.ones((1, 10, 1)),
b: np.ones((1, 10, 2))
}
)


If an input value is not specified for one of the Nodes in the model then data will be filled in automatically according to the Node definition.

progress_barbool

If True, print information about the simulation status to standard output.

statefulbool

This parameter controls whether or not the saved internal stimulation state will be updated after a run completes. If stateful=False then the initial state of future runs will be unaffected by this run. With stateful=True, future runs will begin from the terminal state of this run.

For example,

# begins in state0, terminates in state1
sim.run_steps(..., stateful=False)
# begins in state0, terminates in state2
sim.run_steps(..., stateful=True)
# begins in state2, terminates in state3
sim.run_steps(..., stateful=False)
# begins in state2, terminates in state4
sim.run_steps(..., stateful=True)


Note that Simulator.reset can be used to reset the state to initial conditions at any point.

Notes

If unroll_simulation=x is specified, and n_steps > x, this will repeatedly execute x timesteps until the the number of steps executed is >= n_steps.

train(*args, **kwargs)[source]

Deprecated, use Simulator.compile and Simulator.fit instead.

loss(*args, **kwargs)[source]

Deprecated, use Simulator.compile and Simulator.evaluate instead.

save_params(path, include_state=False, include_non_trainable=None)

Save network parameters to the given path.

Parameters
pathstr

Filepath of parameter output file.

include_statebool

If True (default False) also save the internal simulation state.

Changed in version 3.2.0: Renamed from include_non_trainable to include_state.

Notes

This function is useful for saving/loading entire models; for saving/loading individual objects within a model, see get_nengo_params.

load_params(path, include_state=False, include_non_trainable=None)

Load network parameters from the given path.

Parameters
pathstr

Filepath of parameter input file.

include_statebool

If True (default False) also save the internal simulation state.

Changed in version 3.2.0: Renamed from include_non_trainable to include_state.

Notes

This function is useful for saving/loading entire models; for saving/loading individual objects within a model, see get_nengo_params.

freeze_params(objs)[source]

Stores the live parameter values from the simulation back into a Nengo object definition.

This can be helpful for reusing a NengoDL model inside a different Simulator. For example:

with nengo.Network() as net:
ens = nengo.Ensemble(10, 1)

with nengo_dl.Simulator(net) as sim:
# < run some optimization >
sim.freeze_params(net)

with nengo.Simulator(net) as sim2:
# run the network in the default Nengo simulator, with the
# trained parameters
sim2.run(1.0)

Parameters
obj(list of) NengoObject

The Nengo object(s) into which parameter values will be stored. Note that these objects must be members of the Network used to initialize the Simulator.

Notes

This modifies the source object in-place, and it may slightly modify the structure of that object. The goal is to have the object produce the same output as it would if run in the NengoDL simulator. It may not be possible to accurately freeze all possible object; if you run into errors in this process, try manually extracting the parameters you need in your model (from sim.data).

get_nengo_params(nengo_objs, as_dict=False)[source]

Extract model parameters in a form that can be used to initialize Nengo objects in a different model.

For example:

with nengo.Network() as net:
a = nengo.Ensemble(10, 1)
b = nengo.Ensemble(10, 1)
c = nengo.Connection(a, b)

with nengo_dl.Simulator(net) as sim:
# < do some optimization >
params = sim.get_nengo_params([a, b, c])

with nengo.Network() as new_net:
# < build some other network >

# now we want to insert two connected ensembles with
# the same parameters as our previous network:
d = nengo.Ensemble(10, 1, **params[0])
e = nengo.Ensemble(10, 1, **params[1])
f = nengo.Connection(d, e, **params[2])


Note that this function only returns trainable parameters (e.g. connection weights, biases, or encoders), or parameters that directly interact with those parameters (e.g. gains). Other arguments that are independent of the trainable parameters (e.g. Ensemble.neuron_type or Connection.synapse) should be specified manually (since they may change between models).

Parameters
nengo_objs(list of) Ensemble or Connection

A single object or list of objects for which we want to get the parameters.

as_dictbool

If True, return the values as a dictionary keyed by object label, instead of a list (the default). Note that in this case labels must be unique.

Returns
params(list or dict) of dicts

kwarg dicts corresponding to nengo_objs (passing these dicts as kwargs when creating new Nengo objects will result in a new object with the same parameters as the source object). A single kwarg dict if a single object was passed in, or a list (dict if as_dict=True) of kwargs corresponding to multiple input objects.

check_gradients(inputs=None, outputs=None, atol=1e-05, rtol=0.001)

Perform gradient checks for the network (used to verify that the analytic gradients are correct).

Raises a simulation error if the difference between analytic and numeric gradient is greater than atol + rtol * numeric_grad (elementwise).

Parameters
inputslist of numpy.ndarray

Input values for all the input Nodes in the model (ordered according to the order in which Nodes were added to the model). If None, will use all zeros.

outputslist of Probe

Compute gradients wrt this output (if None, computes wrt each output probe).

atolfloat

Absolute error tolerance.

rtolfloat

Relative (to numeric grad) error tolerance.

Notes

Calling this method will reset the internal simulation state.

trange(sample_every=None, dt=None)[source]

Create a vector of simulation step times matching probed data.

Note that the range does not start at 0 as one might expect, but at the first timestep (i.e., dt).

Parameters
sample_everyfloat (Default: None)

The sampling period of the probe to create a range for. If None, a time value for every dt will be produced.

close()[source]

Close the simulation, freeing resources.

Notes

The simulation cannot be restarted after it is closed.

get_name(obj)[source]

Returns the standardized string name for input Nodes or output Probes.

These are used when referring to inputs/outputs by string in Keras.

Parameters
obj

Input Node or output Probe

Returns
namestr

Name of the given object

property n_steps

The current simulation timestep.

property time

The current simulation time.

property seed

The simulation random seed.

class nengo_dl.simulator.SimulationData(sim, minibatched)[source]

Data structure used to access simulation data from the model.

The main use case for this is to access Probe data; for example, probe_data = sim.data[my_probe]. However, it is also used to access the parameters of objects in the model; for example, after the model has been optimized via Simulator.fit, the updated encoder values for an ensemble can be accessed via trained_encoders = sim.data[my_ens].encoders.

Parameters
simSimulator

The simulator from which data will be drawn

minibatchedbool

If False, discard the minibatch dimension on probe data

Notes

SimulationData shouldn’t be created/accessed directly by the user, but rather via sim.data (which is an instance of SimulationData).

__getitem__(obj)[source]

Return the data associated with obj.

Parameters
obj

Object whose simulation data is being accessed

Returns
data

Array containing probed data if obj is a Probe, otherwise the corresponding parameter object

get_params(*obj_attrs)[source]

Returns the current parameter values for the given objects.

Parameters
obj_attrslist of (NengoObject, str)

The Nengo object and attribute of that object for which we want to know the parameter values (each object-attribute pair specified as a tuple argument to the function).

Returns
paramslist of ndarray

Current values of the requested parameters

Notes

Parameter values should be accessed through sim.data[my_obj] (which will call this function if necessary), rather than directly through this function.

### TensorNodes¶

TensorNodes allow parts of a model to be defined using TensorFlow and smoothly integrated with the rest of a Nengo model.

See the documentation for more details.

 nengo_dl.TensorNode Inserts TensorFlow code into a Nengo model. nengo_dl.Layer A wrapper for constructing TensorNodes.
class nengo_dl.TensorNode(tensor_func, shape_in=Default, shape_out=Default, pass_time=Default, label=Default)[source]

Inserts TensorFlow code into a Nengo model.

Parameters
tensor_funccallable

A function that maps node inputs to outputs

shape_intuple of int

Shape of TensorNode input signal (not including batch dimension).

shape_outtuple of int

Shape of TensorNode output signal (not including batch dimension). If None, value will be inferred by calling tensor_func.

pass_timebool

If True, pass current simulation time to TensorNode function (in addition to the standard input).

labelstr (Default: None)

A name for the node, used for debugging and visualization

property output

Ensures that nothing tries to evaluate the output attribute (indicating that something is trying to simulate this as a regular nengo.Node rather than a TensorNode).

property size_in

Number of input elements (flattened).

property size_out

Number of output elements (flattened).

class nengo_dl.Layer(layer_func)[source]

A wrapper for constructing TensorNodes.

This is designed to mimic and integrate with the tf.keras.layers.Layer API, e.g.

with nengo.Network():
a = nengo.Ensemble(10, 1)
b = nengo_dl.Layer(tf.keras.layers.Dense(units=10))(a)
c = nengo_dl.Layer(lambda x: x + 1)(b)
d = nengo_dl.Layer(nengo.LIF())(c)

Parameters
layer_funccallable or tf.keras.Layer or NeuronType

A function or Keras Layer that takes the value from an input (represented as a tf.Tensor) and maps it to some output value, or a Nengo neuron type (which will be instantiated in a Nengo Ensemble and applied to the input).

__call__(input, transform=1, shape_in=None, synapse=None, return_conn=False, **layer_args)[source]

Apply the TensorNode layer to the given input object.

Parameters
inputNengoObject

Object providing input to the layer.

transformndarray

Transform matrix to apply on connection from input to this layer.

shape_intuple of int

If not None, reshape the input to the given shape.

synapsefloat or Synapse

Synapse to apply on connection from input to this layer.

return_connbool

If True, also return the connection linking this layer to input.

layer_argsdict

These arguments will be passed to TensorNode if layer_func is a callable or Keras Layer, or Ensemble if layer_func is a NeuronType.

Returns
obj

A TensorNode that implements the given layer function (if layer_func was a callable/Keras layer), or a Neuron object with the given neuron type, connected to input.

connConnection

If return_conn is True, also returns the connection object linking input and obj.

Notes

The input connection created for the new TensorNode will be marked as non-trainable by default.

__str__()[source]

Return str(self).

### Converter¶

nengo_dl.Converter can be used to automatically convert a Keras model to a native Nengo Network. This can be useful if, for example, you want to run a model in different Nengo Simulator backends (which will only support the core Nengo objects).

See the documentation for more details.

 nengo_dl.Converter Converts a Keras model to a Nengo network composed of native Nengo objects.
class nengo_dl.Converter(model, allow_fallback=True, inference_only=False, max_to_avg_pool=False, split_shared_weights=False, swap_activations=None, scale_firing_rates=None, synapse=None)[source]

Converts a Keras model to a Nengo network composed of native Nengo objects.

Parameters
modeltf.keras.Model

Keras model to be converted

allow_fallbackbool

If True, allow layers that cannot be converted to native Nengo objects to be added as a TensorNode instead. Note that if this occurs, the converted Nengo network will only be runnable in the NengoDL simulator.

inference_onlybool

Allow layers to be converted in such a way that inference behaviour will match the source model but training behaviour will not. If inference_only=False then some layers cannot be converted to native Nengo objects (but you can still use allow_fallback=True to use a TensorNode instead).

max_to_avg_poolbool

If True, convert max pooling layers to average pooling layers. Note that this will change the behaviour of the network, so parameters will probably need to be re-trained in NengoDL. If max_to_avg_pool=False then max pooling layers cannot be converted to native Nengo objects (but you can still use allow_fallback=True to use a TensorNode instead).

split_shared_weightsbool

In Keras, applying the same Layer object to different input layers will result in multiple instances of the given layer that share the same weights. This is not supported in Nengo. If split_shared_weights=True then those shared weights will be split into independent sets of weights. They will all be initialized to the same value, so the initial behaviour of the model will be unchanged, but if any further training is performed on the network then the weights in each of those instances may diverge.

swap_activationsdict

A dictionary mapping from TensorFlow activation functions or Nengo neuron types to TensorFlow activation functions or Nengo neuron types. This can be used to change all the activation types in a model to some other type. This is in addition to the default activation map (see LayerConverter). It can be keyed based on either TensorFlow or Nengo activation types, and will be applied both before and after the default activation map, in order to support whatever swap type is most useful for a given model. In particular, swap_activations can be useful for swapping rate neuron types to spiking neuron types, through e.g. {tf.nn.relu: nengo.SpikingRectifiedLinear()} or {nengo.RectifiedLinear(): nengo.SpikingRectifiedLinear()}. Or it can be used to swap activation types that don’t have a native Nengo implementation, e.g. {tf.keras.activatons.elu: tf.keras.activations.relu}.

scale_firing_rates: float or dict

Scales the inputs of neurons by x, and the outputs by 1/x. The idea is that this parameter can be used to increase the firing rates of spiking neurons (by scaling the input), without affecting the overall output (because the output spikes are being scaled back down). Note that this is only strictly true for neuron types with linear activation functions (e.g. ReLU). Nonlinear neuron types (e.g. LIF) will be skewed by this linear scaling on the input/output. scale_firing_rates can be specified as a float, which will be applied to all layers in the model, or as a dictionary mapping Keras model layers to a scale factor, allowing different scale factors to be applied to different layers.

synapsefloat or nengo.synapses.Synapse

Synaptic filter to be applied on the output of all neurons. This can be useful to smooth out the noise introduced by spiking neurons. Note, however, that increasing the synaptic filtering will make the network less responsive to rapid changes in the input, and you may need to present each input value for more timesteps in order to allow the network output to settle.

Attributes
modeltf.keras.Model

The input Keras model (if input was a Sequential model then this will be the equivalent Functional model).

netnengo.Network

The converted Nengo network.

inputsConverter.KerasTensorDict

Maps from Keras model inputs to input Nodes in the converted Nengo network. For example, my_node = Converter(my_model).inputs[my_model.input].

outputsConverter.KerasTensorDict

Maps from Keras model outputs to output Probes in the converted Nengo network. For example, my_probe = Converter(my_model).outputs[my_model.output].

layersConverter.KerasTensorDict

Maps from Keras model layers to the converted Nengo object. For example, my_neurons = Converter(my_model).layers[my_layer].

verify(training=False, inputs=None, atol=1e-08, rtol=1e-05)[source]

Verify that output of converted Nengo network matches the original Keras model.

Parameters
trainingbool

If True, check that optimizing the converted Nengo network produces the same results as optimizing the original Keras model.

inputslist of numpy.ndarray

Testing values for model inputs (if not specified, array of ones will be used).

atolfloat

Absolute tolerance for difference between Nengo and Keras outputs.

rtolfloat

Relative (to Nengo) tolerance for difference between nengo and Keras outputs.

Returns
successbool

True if output of Nengo network matches output of Keras model.

Raises
ValueError

If output of Nengo network does not match output of Keras model.

get_converter(layer)[source]

Get instantiated LayerConverter for the given Layer instance.

Note that this caches the results, so calling the function multiple times with the same Layer instance will return the same LayerConverter instance.

Parameters
layertf.keras.layers.Layer

The Keras Layer being converted.

Returns
converterLayerConverter

LayerConverter class for converting layer to Nengo objects.

classmethod register(keras_layer)[source]

A decorator for adding a class to the converter registry.

Parameters
keras_layertf.keras.layers.Layer

The Layer associated with the conversion function being registered.

class KerasTensorDict[source]

A dictionary-like object that has extra logic to handle Layer/Tensor keys.

class TrackedDict(dict)[source]

A dictionary-like object that keeps track of which keys have been accessed.

unused_keys()[source]

Returns any keys in the dictionary that have never been read.

### Configuration system¶

The configuration system is used to change NengoDL’s default behaviour in various ways.

See the documentation for more details.

 nengo_dl.configure_settings Pass settings to nengo_dl by setting them as parameters on the top-level Network config. nengo_dl.config.get_setting Returns config settings (created by configure_settings).
nengo_dl.configure_settings(**kwargs)[source]

Pass settings to nengo_dl by setting them as parameters on the top-level Network config.

The settings are passed as keyword arguments to configure_settings; e.g., to set trainable use configure_settings(trainable=True).

Parameters
trainablebool or None

Adds a parameter to Nengo Ensembles/Connections that controls whether or not they will be optimized by Simulator.fit. Passing None will use the default nengo_dl trainable settings, or True/False will override the default for all objects. In either case trainability can be further configured on a per-object basis (e.g. net.config[my_ensemble].trainable = True. See the documentation for more details.

plannergraph planning algorithm

Pass one of the graph planners to change the default planner.

sortersignal sorting algorithm

Pass one of the sort algorithms to change the default sorter.

simplifications: list of graph simplification functions

Pass a list of graph simplification functions to change the default simplifications applied. The default list of simplifications can be found in nengo_dl.graph_optimizer.default_simplifications.

inference_onlybool

Set to True if the network will only be run in inference mode (i.e., no calls to Simulator.fit). This may result in a small increase in the inference speed.

lif_smoothingfloat

If specified, use the smoothed SoftLIFRate neuron model, with the given smoothing parameter (sigma), to compute the gradient for LIF neurons (as opposed to using LIFRate).

dtypetf.DType

Set the floating point precision for simulation values.

keep_historybool

Adds a parameter to Nengo Probes that controls whether or not they will keep the history from all simulation timesteps or only the last simulation step. This can be further configured on a per-probe basis (e.g., net.config[my_probe].keep_history = False).

statefulbool

If True (default), the Simulator will be built to support stateful execution (where internal simulation state is preserved between simulator functions such as Simulator.predict). Otherwise all operations will be stateless. Note that this can also be configured individually through the stateful parameter on individual functions.

use_loopbool

If True (default), use a symbolic while loop to run the simulation. Otherwise, simulation iterations are explicitly built into the model, avoiding the while loop. This can improve performance, but the simulation can only run for exactly Simulator.unroll_simulation iterations.

learning_phasebool

The learning phase is used for models that have different behaviour in inference versus training mode (for example, spiking neurons swap their behaviour during training). Normally the learning phase is set automatically depending on what function is being called (e.g. sim.predict will run in inference mode and sim.fit will run in training mode). However, sometimes it can be useful to override this behaviour (for example if we want to check what the output of the model looks like during training, we might want to run sim.predict in training mode). Set learning_phase=True to always run the model in training mode (or False to always run in inference mode). Set learning_phase=None to use the default behaviour.

New in version 3.3.0.

nengo_dl.config.get_setting(model, setting, default=None, obj=None)[source]

Returns config settings (created by configure_settings).

Parameters
model

Built model or Network containing all the config settings.

settingstr

Name of the config option to return.

default

The default value to return if config option not set.

objNengoObject

The object on which config setting is stored (defaults to the top-level network).

Returns
config_val

Value of setting if it has been specified, else default.

### Neuron types¶

Additions to the neuron types included with Nengo.

 nengo_dl.SoftLIFRate LIF neuron with smoothing around the firing threshold. nengo_dl.LeakyReLU Rectified linear neuron with nonzero slope for values < 0. nengo_dl.SpikingLeakyReLU Spiking version of LeakyReLU.
class nengo_dl.SoftLIFRate(sigma=1.0, **lif_args)[source]

LIF neuron with smoothing around the firing threshold.

This is a rate version of the LIF neuron whose tuning curve has a continuous first derivative, due to the smoothing around the firing threshold. It can be used as a substitute for LIF neurons in deep networks during training, and then replaced with LIF neurons when running the network [1].

Parameters
sigmafloat

Amount of smoothing around the firing threshold. Larger values mean more smoothing.

tau_rcfloat

Membrane RC time constant, in seconds. Affects how quickly the membrane voltage decays to zero in the absence of input (larger = slower decay).

tau_reffloat

Absolute refractory period, in seconds. This is how long the membrane voltage is held at zero after a spike.

amplitudefloat

Scaling factor on the neuron output. Corresponds to the relative amplitude of the output spikes of the neuron.

Notes

References

1

Eric Hunsberger and Chris Eliasmith (2015): Spiking deep networks with LIF neurons. https://arxiv.org/abs/1510.08829.

rates(x, gain, bias)[source]

Estimates steady-state firing rate given gain and bias.

step(dt, J, output)[source]

Compute rates in Hz for input current (incl. bias)

step_math(dt, J, output)[source]

Backwards compatibility alias for self.step.

class nengo_dl.LeakyReLU(negative_slope=0.3, amplitude=1, **kwargs)[source]

Rectified linear neuron with nonzero slope for values < 0.

Parameters
negative_slopefloat

Scaling factor applied to values less than zero.

amplitudefloat

Scaling factor on the neuron output. Note that this will combine multiplicatively with negative_slope for values < 0.

step(dt, J, output)[source]

Implement the leaky relu nonlinearity.

step_math(dt, J, output)[source]

Backwards compatibility alias for self.step.

class nengo_dl.SpikingLeakyReLU(negative_slope=0.3, amplitude=1, **kwargs)[source]

Spiking version of LeakyReLU.

Note that this may output “negative spikes” (i.e. a spike with a sign of -1).

Parameters
negative_slopefloat

Scaling factor applied to values less than zero.

amplitudefloat

Scaling factor on the neuron output. Note that this will combine multiplicatively with negative_slope for values < 0.

rates(x, gain, bias)[source]

Use LeakyReLU to determine rates.

step(dt, J, output, voltage)[source]

Implement the spiking leaky relu nonlinearity.

step_math(dt, J, output, voltage)[source]

Backwards compatibility alias for self.step.

### Distributions¶

Additions to the distributions included with Nengo. These distributions are usually used to initialize weight matrices, e.g. nengo.Connection(a.neurons, b.neurons, transform=nengo_dl.dists.Glorot()).

 nengo_dl.dists.TruncatedNormal Normal distribution where any values more than some distance from the mean are resampled. nengo_dl.dists.VarianceScaling Variance scaling distribution for weight initialization (analogous to tf.initializers.VarianceScaling). nengo_dl.dists.Glorot Weight initialization method from [1] (also known as Xavier initialization). nengo_dl.dists.He Weight initialization method from [1].
class nengo_dl.dists.TruncatedNormal(mean=0, stddev=1, limit=None)[source]

Normal distribution where any values more than some distance from the mean are resampled.

Parameters
meanfloat

Mean of the normal distribution.

stddevfloat

Standard deviation of the normal distribution.

limitfloat

Resample any values more than this distance from the mean. If None, then limit will be set to 2 standard deviations.

sample(n, d=None, rng=None)[source]

Samples the distribution.

Parameters
nint

Number samples to take.

dint or None

The number of dimensions to return. If this is an int, the return value will be of shape (n, d). If None, the return value will be of shape (n,).

rngRandomState

Random number generator state (if None, will use the default numpy random number generator).

Returns
samples(n,) or (n, d) array_like

Samples as a 1d or 2d array depending on d. The second dimension enumerates the dimensions of the process.

class nengo_dl.dists.VarianceScaling(scale=1, mode='fan_avg', distribution='uniform')[source]

Variance scaling distribution for weight initialization (analogous to tf.initializers.VarianceScaling).

Parameters
scalefloat

Overall scale on values.

mode“fan_in” or “fan_out” or “fan_avg”

Whether to scale based on input or output dimensionality, or average of the two.

distribution: “uniform” or “normal”

Whether to use a uniform or truncated normal distribution for weights.

sample(n, d=None, rng=None)[source]

Samples the distribution.

Parameters
nint

Number samples to take.

dint or None

The number of dimensions to return. If this is an int, the return value will be of shape (n, d). If None, the return value will be of shape (n,).

rngRandomState

Random number generator state (if None, will use the default numpy random number generator).

Returns
samples(n,) or (n, d) array_like

Samples as a 1d or 2d array depending on d. The second dimension enumerates the dimensions of the process.

class nengo_dl.dists.Glorot(scale=1, distribution='uniform')[source]

Weight initialization method from [1] (also known as Xavier initialization).

Parameters
scalefloat

Scale on weight distribution. For rectified linear units this should be sqrt(2), otherwise usually 1.

distribution: “uniform” or “normal”

Whether to use a uniform or normal distribution for weights

References

1(1,2)

Xavier Glorot and Yoshua Bengio (2010): Understanding the difficulty of training deep feedforward neural networks. International conference on artificial intelligence and statistics. http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf.

class nengo_dl.dists.He(scale=1, distribution='normal')[source]

Weight initialization method from [1].

Parameters
scalefloat

Scale on weight distribution. For rectified linear units this should be sqrt(2), otherwise usually 1.

distribution: “uniform” or “normal”

Whether to use a uniform or normal distribution for weights

References

1(1,2)

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. (2015): Delving deep into rectifiers: Surpassing human-level performance on ImageNet classification. https://arxiv.org/abs/1502.01852.

### Loss functions¶

Some common loss functions (for use with the loss argument in Simulator.compile).

 nengo_dl.losses.nan_mse Compute Mean Squared Error between given outputs and targets. nengo_dl.losses.Regularize An objective function to apply regularization penalties.
nengo_dl.losses.nan_mse(y_true, y_pred)[source]

Compute Mean Squared Error between given outputs and targets.

If any values in y_true are nan, that will be treated as zero error for those elements.

Parameters
y_truetf.Tensor

Target values for a Probe in a network.

y_predtf.Tensor

Output values from a Probe in a network.

Returns
msetf.Tensor

Tensor representing the mean squared error.

class nengo_dl.losses.Regularize(order=2, axis=None)[source]

An objective function to apply regularization penalties.

This is designed to be applied to a probed signal, e.g.

with nengo.Network() as net:
a = nengo.Node([0])
b = nengo.Node(size_in=1)
c = nengo.Connection(a, b, transform=1)
p = nengo.Probe(c, "weights")
...

# this part is optional, but we may often want to only keep the data from
# the most recent timestep when probing in this way, to save memory
nengo_dl.configure_settings(keep_history=True)
net.config[p].keep_history = False

with nengo_dl.Simulator(net) as sim:
sim.compile(loss={p: nengo_dl.losses.Regularize()})

Parameters
orderint or str

Order of the regularization norm (e.g. 1 for L1 norm, 2 for L2 norm). See https://www.tensorflow.org/api_docs/python/tf/norm for a full description of the possible values for this parameter.

axisint or None

The axis of the probed signal along which to compute norm. If None (the default), the signal is flattened and the norm is computed across the resulting vector. Note that these are only the axes with respect to the output on a single timestep (i.e. batch/time dimensions are not included).

Notes

The mean will be computed across all the non-axis dimensions after computing the norm (including batch/time) in order to compute the overall objective value.

call(y_true, y_pred)[source]

Invoke the loss function.

Parameters
y_truetf.Tensor

Ignored

y_predtf.Tensor

The value to be regularized

Returns
outputtf.Tensor

Scalar regularization loss value.

### Callbacks¶

Objects to be used with the Keras callback functionality.

The short answer is that these can be passed to, e.g., Simulator.fit like

sim.fit(..., callbacks=[nengo_dl.callbacks.NengoSummaries(...)]

 nengo_dl.callbacks.NengoSummaries Logs the values of Nengo object parameters, to be displayed in TensorBoard. nengo_dl.callbacks.TensorBoard A version of the Keras TensorBoard callback that also profiles inference. nengo_dl.callbacks.IsolateState Isolate the internal state of the simulation from any other stateful operations.
class nengo_dl.callbacks.NengoSummaries(log_dir, sim, objects)[source]

Logs the values of Nengo object parameters, to be displayed in TensorBoard.

See https://www.tensorflow.org/tensorboard/get_started for general instructions on using TensorBoard.

Parameters
log_dirstr

Directory where log file will be written.

simSimulator

Simulator object which will be used to look up parameter values.

objects

The object whose parameter values we want to record (passing an Ensemble will log its encoders, Neurons will log biases, and Connection will log connection weights/decoders).

on_epoch_end(epoch, logs=None)[source]

Log parameter values at the end of each epoch.

on_train_end(logs=None)[source]

Close summary writer at end of training.

class nengo_dl.callbacks.TensorBoard(log_dir='logs', histogram_freq=0, write_graph=True, write_images=False, update_freq='epoch', profile_batch=2, embeddings_freq=0, embeddings_metadata=None, **kwargs)[source]

A version of the Keras TensorBoard callback that also profiles inference.

on_predict_batch_end(*args, **kwargs)[source]

Redirect to training function.

on_predict_begin(*args, **kwargs)[source]

Redirect to training function.

on_predict_end(*args, **kwargs)[source]

Redirect to training function.

class nengo_dl.callbacks.IsolateState(sim)[source]

Isolate the internal state of the simulation from any other stateful operations.

This will cause every batch to begin from the same initial state (the state of the simulation whenever this callback is created). And when this operation completes, the simulation state will be returned to that initial state.

Parameters
simSimulator

The Simulator containing the state we want to control.

reset()[source]

Resets the simulation state to the saved state.

on_train_batch_end(batch, logs=None)[source]

Reset state at the end of each batch.

on_predict_batch_end(batch, logs=None)[source]

Reset state at the end of each batch.

on_test_batch_end(batch, logs=None)[source]

Reset state at the end of each batch.

## Developers¶

These objects are only relevant to people interested in modifying the implementation of NengoDL (e.g., adding a new neuron type).

### Builder¶

The builder manages the mapping between (groups of) Nengo operators and the builder objects that know how to translate those operators into a TensorFlow graph.

 nengo_dl.builder.Builder Manages the operator build classes known to the nengo_dl build process. nengo_dl.builder.BuildConfig Stores configuration parameters that may be relevant to parts of the build process. nengo_dl.builder.OpBuilder Base class for build classes, which implement the logic for building a group of Nengo Operators into TensorFlow. nengo_dl.builder.NengoBuilder Copy of the default Nengo builder. nengo_dl.builder.NengoModel Copy of the default Nengo model.
class nengo_dl.builder.Builder(plan)[source]

Manages the operator build classes known to the nengo_dl build process.

Parameters
planlist of tuple of Operator

The groups of operators that will be built

build_pre(signals, config, progress=None)[source]

Setup step for build classes, in which they compute any of the values that are constant across simulation timesteps.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

Configuration parameters for the build process

progressutils.ProgressBar

Progress bar for ops in plan

build_step(signals, progress=None)[source]

Build the computations implementing a single simulator timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

progressutils.ProgressBar

Progress bar for ops in plan

Returns
side_effectslist of tf.Tensor

Outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used.

build_post(signals, progress=None)[source]

Calls post build functions for all ops in plan.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

progressutils.ProgressBar

Progress bar for ops in plan

name_scope(ops)[source]

Returns a new TensorFlow name scope for the given ops.

classmethod register(nengo_op)[source]

A decorator for adding a class to the build function registry.

Parameters
nengo_opOperator

The operator associated with the build function being decorated.

class nengo_dl.builder.BuildConfig[source]

Stores configuration parameters that may be relevant to parts of the build process.

Parameters
inference_onlybool

If True the network should be constructed in “inference only” mode (omitting any support for training operations).

lif_smoothingfloat

Smoothing parameter for LIF gradient approximation.

cpu_onlybool

True if TensorFlow is only running on the CPU (because that was specified by the user or because GPU support is not available).

rngRandomState

Seeded random number generator.

trainingtf.Tensor (bool)

True if building in training mode, False for inference mode.

Create new instance of BuildConfig(inference_only, lif_smoothing, cpu_only, rng, training)

class nengo_dl.builder.OpBuilder(ops)[source]

Base class for build classes, which implement the logic for building a group of Nengo Operators into TensorFlow.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

build_post(signals)[source]

This function will be called after the graph has been built and each time the Simulator is reset.

Note that this function may be called multiple times per session, so it should do any required operations in-place.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.builder.NengoBuilder[source]

Copy of the default Nengo builder.

This class is here so that we can register new build functions for NengoDL without affecting the default Nengo build process.

classmethod build(model, obj, *args, **kwargs)[source]

Build obj into model.

This method looks up the appropriate build function for obj and calls it with the model and other arguments provided.

In addition to the parameters listed below, further positional and keyword arguments will be passed unchanged into the build function.

Parameters
modelModel

The Model instance in which to store build artifacts.

objobject

The object to build into the model.

class nengo_dl.builder.NengoModel(*args, fail_fast=True, **kwargs)[source]

Copy of the default Nengo model.

This allows us to override certain model behaviours.

Parameters
fail_fastbool

If True, try to call op.make_step when ops are added to the model. Note that NengoDL doesn’t actually use make_step, so errors in that function are not necessarily errors in NengoDL (which is why we want to disable that check). But it might still be useful when debugging new op/build functions, which is why we leave the option.

add_op(op)[source]

Add an operator to the model.

Parameters
opOperator

Operator being added to the model.

Notes

This is a copy of the parent nengo.builder.Model.add_op, with the addition of the if self.fail_fast condition.

### Operator builders¶

These objects are used to convert Nengo operators into TensorFlow graph elements.

#### Basic operators¶

Build classes for basic Nengo operators.

 nengo_dl.op_builders.ResetInc A version of Reset that increments the target value rather than overwriting. nengo_dl.op_builders.ElementwiseSet A version of ElementwiseInc that overwrites the target rather than incrementing. nengo_dl.op_builders.DotSet A version of DotInc that overwrites the target rather than incrementing. nengo_dl.op_builders.SparseDotSet A version of SparseDotInc that overwrites the target rather than incrementing. nengo_dl.op_builders.ResetBuilder Build a group of Reset operators. nengo_dl.op_builders.CopyBuilder Build a group of Copy operators. nengo_dl.op_builders.ElementwiseIncBuilder Build a group of ElementwiseInc operators. nengo_dl.op_builders.sparse_matmul Matrix multiplication between sparse matrix A and dense matrix X nengo_dl.op_builders.DotIncBuilder Build a group of DotInc operators. nengo_dl.op_builders.SimPyFuncBuilder Build a group of SimPyFunc operators. nengo_dl.op_builders.SparseDotIncBuilder Build a group of SparseDotInc operators. nengo_dl.op_builders.TimeUpdateBuilder Build a group of TimeUpdate operators.
class nengo_dl.op_builders.ResetInc(*args, **kwargs)[source]

A version of Reset that increments the target value rather than overwriting.

property dst

dst is stored in incs rather than sets.

class nengo_dl.op_builders.ElementwiseSet(*args, **kwargs)[source]

A version of ElementwiseInc that overwrites the target rather than incrementing.

property Y

Y is stored in sets rather than incs.

class nengo_dl.op_builders.DotSet(*args, **kwargs)[source]

A version of DotInc that overwrites the target rather than incrementing.

property Y

Y is stored in sets rather than incs.

class nengo_dl.op_builders.SparseDotSet(*args, **kwargs)[source]

A version of SparseDotInc that overwrites the target rather than incrementing.

property Y

Y is stored in sets rather than incs.

class nengo_dl.op_builders.ResetBuilder(ops)[source]

Build a group of Reset operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.op_builders.CopyBuilder(ops)[source]

Build a group of Copy operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.op_builders.ElementwiseIncBuilder(ops)[source]

Build a group of ElementwiseInc operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

nengo_dl.op_builders.sparse_matmul(A_indices, A_data, A_shape, X, transpose_x=False)[source]

Matrix multiplication between sparse matrix A and dense matrix X

Parameters
A_indicestf.Tensor

(N, 2) array of [row,col] non-zero entries

A_datatf.Tensor

(N,) array of data in the nonzero entries specified in A_indices

A_shapetuple of int

Shape of full A matrix

Xtf.Tensor

Dense matrix being multiplied by A

transpose_xbool

Transpose X before multiply

Returns
dottf.Tensor

Result of matrix multiplication between A and X

class nengo_dl.op_builders.DotIncBuilder(ops)[source]

Build a group of DotInc operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.op_builders.SimPyFuncBuilder(ops)[source]

Build a group of SimPyFunc operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.op_builders.SparseDotIncBuilder(ops)[source]

Build a group of SparseDotInc operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.op_builders.TimeUpdateBuilder(ops)[source]

Build a group of TimeUpdate operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

#### Neuron types¶

Build classes for Nengo neuron operators.

 nengo_dl.neuron_builders.GenericNeuronBuilder Builds all neuron types for which there is no custom TensorFlow implementation. nengo_dl.neuron_builders.TFNeuronBuilder Base class for NeuronType builders with a TF implementation. nengo_dl.neuron_builders.RectifiedLinearBuilder Build a group of RectifiedLinear neuron operators. nengo_dl.neuron_builders.SpikingRectifiedLinearBuilder Build a group of SpikingRectifiedLinear neuron operators. nengo_dl.neuron_builders.SigmoidBuilder Build a group of Sigmoid neuron operators. nengo_dl.neuron_builders.TanhBuilder Build a group of Tanh neuron operators. nengo_dl.neuron_builders.LIFRateBuilder Build a group of LIFRate neuron operators. nengo_dl.neuron_builders.SoftLIFRateBuilder Build a group of SoftLIFRate neuron operators. nengo_dl.neuron_builders.LIFBuilder Build a group of LIF neuron operators. nengo_dl.neuron_builders.RegularSpikingBuilder Build a group of RegularSpiking neuron operators. nengo_dl.neuron_builders.StochasticSpikingBuilder Build a group of StochasticSpiking neuron operators. nengo_dl.neuron_builders.PoissonSpikingBuilder Build a group of PoissonSpiking neuron operators. nengo_dl.neuron_builders.SimNeuronsBuilder Builds a group of SimNeurons operators.
class nengo_dl.neuron_builders.GenericNeuronBuilder(ops)[source]

Builds all neuron types for which there is no custom TensorFlow implementation.

Notes

These will be executed as native Python functions, requiring execution to move in and out of TensorFlow. This can significantly slow down the simulation, so any performance-critical neuron models should consider adding a custom TensorFlow implementation for their neuron type instead.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

class nengo_dl.neuron_builders.TFNeuronBuilder(ops)[source]

Base class for NeuronType builders with a TF implementation.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

step(J, dt, **state)[source]

Implements the logic for a single inference step.

training_step(J, dt, **state)[source]

Implements the logic for a single training step.

Note: subclasses only need to implement this if spiking=True. It is used to specify an alternate (differentiable) implementation of the neuron model to be used during training.

build_step(signals, **step_kwargs)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

class nengo_dl.neuron_builders.RectifiedLinearBuilder(ops)[source]

Build a group of RectifiedLinear neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

step(J, dt)[source]

Implements the logic for a single inference step.

class nengo_dl.neuron_builders.SpikingRectifiedLinearBuilder(ops)[source]

Build a group of SpikingRectifiedLinear neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

step(J, dt, voltage)[source]

Implements the logic for a single inference step.

training_step(J, dt, **state)[source]

Implements the logic for a single training step.

Note: subclasses only need to implement this if spiking=True. It is used to specify an alternate (differentiable) implementation of the neuron model to be used during training.

class nengo_dl.neuron_builders.SigmoidBuilder(ops)[source]

Build a group of Sigmoid neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

step(J, dt)[source]

Implements the logic for a single inference step.

class nengo_dl.neuron_builders.TanhBuilder(ops)[source]

Build a group of Tanh neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

step(J, dt)[source]

Implements the logic for a single inference step.

class nengo_dl.neuron_builders.LIFRateBuilder(ops)[source]

Build a group of LIFRate neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

step(J, dt)[source]

Implements the logic for a single inference step.

class nengo_dl.neuron_builders.SoftLIFRateBuilder(ops)[source]

Build a group of SoftLIFRate neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

step(J, dt)[source]

Implements the logic for a single inference step.

class nengo_dl.neuron_builders.LIFBuilder(ops)[source]

Build a group of LIF neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

step(J, dt, voltage, refractory_time)[source]

Implements the logic for a single inference step.

training_step(J, dt, **state)[source]

Implements the logic for a single training step.

Note: subclasses only need to implement this if spiking=True. It is used to specify an alternate (differentiable) implementation of the neuron model to be used during training.

class nengo_dl.neuron_builders.RegularSpikingBuilder(ops)[source]

Build a group of RegularSpiking neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

step(J, dt, voltage)[source]

Implements the logic for a single inference step.

training_step(J, dt, **state)[source]

Implements the logic for a single training step.

Note: subclasses only need to implement this if spiking=True. It is used to specify an alternate (differentiable) implementation of the neuron model to be used during training.

class nengo_dl.neuron_builders.StochasticSpikingBuilder(ops)[source]

Build a group of StochasticSpiking neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

step(J, dt)[source]

Implements the logic for a single inference step.

training_step(J, dt)[source]

Implements the logic for a single training step.

Note: subclasses only need to implement this if spiking=True. It is used to specify an alternate (differentiable) implementation of the neuron model to be used during training.

class nengo_dl.neuron_builders.PoissonSpikingBuilder(ops)[source]

Build a group of PoissonSpiking neuron operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

step(J, dt)[source]

Implements the logic for a single inference step.

training_step(J, dt)[source]

Implements the logic for a single training step.

Note: subclasses only need to implement this if spiking=True. It is used to specify an alternate (differentiable) implementation of the neuron model to be used during training.

class nengo_dl.neuron_builders.SimNeuronsBuilder(ops)[source]

Builds a group of SimNeurons operators.

Calls the appropriate sub-build class for the different neuron types.

Attributes
TF_NEURON_IMPL

Mapping from neuron types to custom build classes (neurons without a custom builder will use the generic builder).

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

#### Learning rules¶

Build classes for Nengo learning rule operators.

 nengo_dl.learning_rule_builders.SimBCMBuilder Build a group of SimBCM operators. nengo_dl.learning_rule_builders.SimOjaBuilder Build a group of SimOja operators. nengo_dl.learning_rule_builders.SimVojaBuilder Build a group of SimVoja operators. nengo_dl.learning_rule_builders.build_pes Builds a nengo.PES object into a Nengo model. nengo_dl.learning_rule_builders.SimPESBuilder Build a group of SimPES operators.
class nengo_dl.learning_rule_builders.SimBCMBuilder(ops)[source]

Build a group of SimBCM operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.learning_rule_builders.SimOjaBuilder(ops)[source]

Build a group of SimOja operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

class nengo_dl.learning_rule_builders.SimVojaBuilder(ops)[source]

Build a group of SimVoja operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

nengo_dl.learning_rule_builders.build_pes(model, pes, rule)[source]

Builds a nengo.PES object into a Nengo model.

Overrides the standard Nengo PES builder in order to avoid slicing on axes > 0 (not currently supported in NengoDL).

Parameters
modelModel

The model to build into.

pesPES

Learning rule type to build.

ruleLearningRule

The learning rule object corresponding to the neuron type.

Notes

Does not modify model.params[] and can therefore be called more than once with the same nengo.PES instance.

class nengo_dl.learning_rule_builders.SimPESBuilder(ops)[source]

Build a group of SimPES operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

#### Processes¶

Build classes for Nengo process operators.

 nengo_dl.process_builders.GenericProcessBuilder Builds all process types for which there is no custom TensorFlow implementation. nengo_dl.process_builders.LowpassBuilder Build a group of Lowpass synapse operators. nengo_dl.process_builders.LinearFilterBuilder Build a group of LinearFilter synapse operators. nengo_dl.process_builders.SimProcessBuilder Builds a group of SimProcess operators.
class nengo_dl.process_builders.GenericProcessBuilder(ops)[source]

Builds all process types for which there is no custom TensorFlow implementation.

Notes

These will be executed as native Python functions, requiring execution to move in and out of TensorFlow. This can significantly slow down the simulation, so any performance-critical processes should consider adding a custom TensorFlow implementation for their type instead.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

build_post(signals)[source]

This function will be called after the graph has been built and each time the Simulator is reset.

Note that this function may be called multiple times per session, so it should do any required operations in-place.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

class nengo_dl.process_builders.LowpassBuilder(ops)[source]

Build a group of Lowpass synapse operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

class nengo_dl.process_builders.LinearFilterBuilder(ops)[source]

Build a group of LinearFilter synapse operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

class nengo_dl.process_builders.SimProcessBuilder(ops)[source]

Builds a group of SimProcess operators.

Calls the appropriate sub-build class for the different process types.

Attributes
TF_PROCESS_IMPLdict of {Process: builder.OpBuilder}

Mapping from process types to custom build classes (processes without a custom builder will use the generic builder).

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

build_post(signals)[source]

This function will be called after the graph has been built and each time the Simulator is reset.

Note that this function may be called multiple times per session, so it should do any required operations in-place.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

#### Transforms¶

Build classes for Nengo transform operators.

 nengo_dl.transform_builders.ConvSet A version of ConvInc that overwrites the target rather than incrementing. nengo_dl.transform_builders.ConvIncBuilder Build a group of nengo.builder.transforms.ConvInc operators.
class nengo_dl.transform_builders.ConvSet(*args, **kwargs)[source]

A version of ConvInc that overwrites the target rather than incrementing.

property Y

Y is stored in sets rather than incs.

class nengo_dl.transform_builders.ConvIncBuilder(ops)[source]

Build a group of nengo.builder.transforms.ConvInc operators.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

static mergeable(x, y)[source]

Compute the mergeability of two operators of this builder’s type.

Parameters
xnengo.builder.Operator

The operator being tested

ynengo.builder.Operator

The operator being merged into (this is representative of a group of operators that have already been merged)

Returns
mergeablebool

True if x and y can be merged into a single built op, else False.

#### TensorNodes¶

To build TensorNode objects we need to define a new Nengo operator (tensor_node.SimTensorNode), a build function that adds that operator into a Nengo graph (tensor_node.build_tensor_node), and a NengoDL build class that maps that new Nengo operator to TensorFlow operations (tensor_node.SimTensorNodeBuilder).

 nengo_dl.tensor_node.SimTensorNode Operator for TensorNodes (constructed by build_tensor_node). nengo_dl.tensor_node.build_tensor_node This is the Nengo build function, so that Nengo knows what to do with TensorNodes. nengo_dl.tensor_node.SimTensorNodeBuilder Builds a SimTensorNode operator into a NengoDL model.
class nengo_dl.tensor_node.SimTensorNode(func, time, input, output, shape_in, tag=None)[source]

Operator for TensorNodes (constructed by build_tensor_node).

Parameters
funccallable

The TensorNode function (tensor_func).

timeSignal or None

Signal representing the current simulation time (or None if pass_time is False).

inputSignal or None

Input Signal for the TensorNode (or None if no inputs).

outputSignal

Output Signal for the TensorNode.

shape_intuple of int or None

Shape of input to TensorNode (if None, will leave the shape of input signal unchanged).

tagstr

A label associated with the operator, for debugging

Notes

1. sets [output]

2. incs []

3. reads [time] (if pass_time=True) + [input] (if input is not None)

4. updates []

nengo_dl.tensor_node.build_tensor_node(model, node)[source]

This is the Nengo build function, so that Nengo knows what to do with TensorNodes.

class nengo_dl.tensor_node.SimTensorNodeBuilder(ops)[source]

Builds a SimTensorNode operator into a NengoDL model.

Initialize internal OpBuilder implementation.

Note: this should not be building any model operations, this is purely for internal setup of the OpBuilder itself.

Parameters
opslist of Operator

The operator group to build into the model

build_pre(signals, config)[source]

This function should set up any computations that are fixed for this op (i.e., things that do not need to be recomputed each timestep).

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

configBuildConfig

General repository for config information builders might want (conglomerated into this object so that we can add/remove config data without having to change the function signature all the time).

build_step(signals)[source]

This function builds whatever computations need to be executed in each simulation timestep.

Parameters
signalssignals.SignalDict

Mapping from Signal to tf.Tensor (updated by operations)

Returns
side_effectslist of tf.Tensor

If not None, the returned tensors correspond to outputs with possible side-effects, i.e. computations that need to be executed in the TensorFlow graph even if their output doesn’t appear to be used

### Graph construction¶

Manages the data and build processes associated with implementing a Nengo simulation in TensorFlow.

 nengo_dl.tensor_graph.TensorGraph Implement the Nengo simulation as a Keras Layer.
class nengo_dl.tensor_graph.TensorGraph(model, dt, unroll_simulation, minibatch_size, device, progress, seed)[source]

Implement the Nengo simulation as a Keras Layer.

Parameters
modelModel

Pre-built Nengo model describing the network to be simulated.

dtfloat

Length of a simulator timestep, in seconds.

unroll_simulationint

Unroll simulation loop by explicitly building unroll_simulation iterations into the computation graph.

minibatch_sizeint

The number of simultaneous inputs that will be passed through the network.

deviceNone or "/cpu:0" or "/gpu:[0-n]"

Device on which to execute computations (if None then uses the default device as determined by TensorFlow).

progressutils.ProgressBar

Progress bar for optimization stage.

seedint

Seed for random number generation.

build_inputs()[source]

Generates a set of Input layers that can be used as inputs to a TensorGraph layer.

Returns
n_stepstf.keras.layers.Input

Input layer for specifying the number of simulation timesteps.

inputsdict of {nengo.Node: tf.keras.layers.Input}

Input layers for each of the Nodes in the network.

build(input_shape=None)[source]

Create any Variables used in the model.

Parameters
input_shapelist of tuple of int

Shapes of all the inputs to this layer.

call(inputs, training=None, progress=None, stateful=False)[source]

Constructs the graph elements to simulate the model.

Parameters
inputslist of tf.Tensor

Input layers/tensors for the network (must match the structure defined in build_inputs).

trainingbool

Whether the network is being run in training or inference mode. If None, uses the symbolic Keras learning phase variable.

progressutils.ProgressBar

Progress bar for construction stage.

statefulbool

Whether or not to build the model to support preserving the internal state between executions.

Returns
probe_arrayslist of tf.Tensor

Tensors representing the output of all the Probes in the network (order corresponding to self.model.probes, which is the order the Probes were instantiated).

build_post()[source]

Executes post-build processes for operators (after the graph has been constructed and whenever Simulator is reset).

get_tensor(sig)[source]

Returns a Tensor corresponding to the given Signal.

Parameters
sigSignal

A signal in the Nengo model.

Returns
tensortf.Tensor

Tensor containing the value of the given Signal.

mark_signals()[source]

Mark all the signals in self.model according to whether they represent trainable parameters of the model (parameters that can be optimized by deep learning methods).

Trainable parameters include connection weights, ensemble encoders, and neuron biases. Unless one of those signals is targeted by a Nengo learning rule (otherwise the learning rule update conflicts with the deep learning optimization).

Users can manually specify whether signals are trainable or not using the config system (e.g., net.config[nengo.Ensemble].trainable = False).

The trainable attribute will be set to one of three values:

• True: Signal is trainable

• False: Signal could be trainable, but has been set to non-trainable (e.g., because the user manually configured that object not to be trainable).

• None: Signal is never trainable (e.g., simulator state)

create_signals(sigs)[source]

Groups signal data together into larger arrays, and represent each individual signal as a slice into that array.

Parameters
sigslist of Signal

Base signals arranged into the order in which they should reside in memory (e.g., output from graph_optimizer.order_signals)

### Signals¶

Represents and manages the internal simulation signals.

 nengo_dl.signals.TensorSignal Represents a tensor as an indexed view into a base array. nengo_dl.signals.SignalDict Handles the mapping from Signal to tf.Tensor.
class nengo_dl.signals.TensorSignal(slices, key, dtype, shape, minibatch_size, label='TensorSignal')[source]

Represents a tensor as an indexed view into a base array.

Parameters
slicestuple of tuple of int

Start/stop indices of slices along the first axis of the base array, corresponding to the data for this signal.

keyobject

Key mapping to the base array that contains the data for this signal.

dtypestr

dtype of the values represented by this signal.

shapetuple of int

View shape of this signal (may differ from shape of base array).

minibatch_sizeint

If not None then this signal contains a minibatch dimension with the given size.

labelstr

Name for this signal, used to make debugging easier.

reset()[source]

Reset cached Tensors.

property slices

The slices containing the data for this signal in the base array.

property ndim

The rank of this signal.

__getitem__(indices)[source]

Create a new TensorSignal representing a subset (slice or advanced indexing) of the indices of this TensorSignal.

Parameters
indicesslice or list of int

The desired subset of the indices in this TensorSignal

Returns
sigsignals.TensorSignal

A new TensorSignal representing the subset of this TensorSignal

reshape(shape)[source]

Create a new TensorSignal representing a reshaped view of the same data in this TensorSignal (size of data must remain unchanged).

Parameters
shapetuple of int

New shape for the signal (one dimension can be -1 to indicate an inferred dimension size, as in numpy)

Returns
sigsignals.TensorSignal

New TensorSignal representing the same data as this signal but with the given shape

property tf_shape

A tf.Tensor representing the shape of this signal.

property tf_indices

A tf.Tensor representing the indices of this signal.

property tf_indices_nd

A tf.Tensor representing the indices of this signal for use with e.g. scatter_nd.

property tf_slice

A tuple of tf.Tensors representing the (start, stop, stride) slice within the base array containing the data for this signal.

This can be used as a more efficient representation of TensorSignal.tf_indices.

property full_shape

Shape of the signal including the minibatch dimension.

property minibatched

Whether or not this TensorSignal contains a minibatch dimension.

class nengo_dl.signals.SignalDict(dtype, minibatch_size)[source]

Handles the mapping from Signal to tf.Tensor.

Takes care of gather/scatter logic to read/write signals within the base arrays.

Parameters
dtypestr

Floating point precision used in signals (e.g. “float32”)

minibatch_sizeint

Number of items in each minibatch

reset()[source]

Reset build-specific data structures.

These are data structures that are filled out during the TensorGraph build process (and therefore need to be re-initialized if we build the model again), as opposed to data that is constant for a given Nengo model.

scatter(dst, val, mode='update')[source]

Updates the base data corresponding to dst.

Parameters
dstTensorSignal

Signal indicating the data to be modified in base array

valtf.Tensor

Update data (same shape as dst, i.e. a dense array <= the size of the base array)

mode“update” or “inc”

Overwrite/add the data at dst with val

gather(src, force_copy=False)[source]

Fetches the data corresponding to src from the base array.

Parameters
srcTensorSignal

Signal indicating the data to be read from base array

force_copybool

If True, always perform a gather, not a slice (this forces a copy). Note that setting force_copy=False does not guarantee that a copy won’t be performed.

Returns
gatheredtf.Tensor

Tensor object corresponding to a dense subset of data from the base array

combine(sigs, label='Combine')[source]

Combines several TensorSignals into one by concatenating along the first axis.

Parameters
sigs

Signals to be combined

labelstr

Name for combined signal (to help with debugging)

Returns
sigTensorSignal

New TensorSignal representing the concatenation of the data in sigs

get_tensor_signal(slices, key, dtype, shape, minibatched, signal=None, label='TensorSignal')[source]

Creates a new TensorSignal with the given properties.

This should be used rather than instantiating a new TensorSignal directly, as it handles some extra book-keeping.

Parameters
slicestuple of tuple of int

Start/stop indices of slices along the first axis of the base array, corresponding to the data for this signal.

keyobject

Key mapping to the base array that contains the data for this signal

dtypestr

dtype of the values represented by this signal

shapetuple of int

View shape of this signal (may differ from shape of base array)

minibatchedbool

Whether or not this signal contains a minibatch dimension

signalSignal

If not None, associate the new TensorSignal with the given Signal in the sig_map

labelstr

Name for this signal, used to make debugging easier

Returns
sigTensorSignal

A new TensorSignal with the given properties

op_constant(ops, op_sizes, attr, dtype, shape=1, - 1)[source]

Creates a tensor representing the constant parameters of an op group.

Parameters
opslist of object

The operators for some merged group of ops

op_sizeslist of int

The number of constant elements in each op

attrstr

The attribute of the op that describes the constant parameter

dtypestr

Numeric type of the parameter

shapetuple of int

Shape for returned constant (this will be ignored in the scalar case). The default adds an empty dimension for broadcasting along the batch axis.

Returns
constanttf.Tensor

Tensor containing the values of attr for the given ops. This will be a scalar if all the ops have the same parameter value, or an array giving the parameter value for each element in each op.

### Graph optimization¶

These functions are used to restructure the Nengo operator graph so that it can be simulated more efficiently when converted into a TensorFlow graph.

 nengo_dl.graph_optimizer.mergeable Check if the given op can be merged with the candidate group. nengo_dl.graph_optimizer.greedy_planner Create merged execution plan through greedy selection. nengo_dl.graph_optimizer.tree_planner Create merged execution plan through exhaustive tree search. nengo_dl.graph_optimizer.transitive_planner Create merged execution plan through transitive closure construction. nengo_dl.graph_optimizer.transitive_closure_recurse Computes the transitive closure for the given graph, restricted to the operators with the given builder type. nengo_dl.graph_optimizer.noop_planner Orders operators into a valid execution order, but does not perform any merging. nengo_dl.graph_optimizer.order_signals Orders signals and operators to try to structure reads/writes in contiguous blocks. nengo_dl.graph_optimizer.hamming_sort Reorder signals using heuristics to try to place signals that are accessed by the same operators into adjacent positions (giving priority to larger blocks). nengo_dl.graph_optimizer.sort_ops_by_signals Rearrange operators to match the order of signals. nengo_dl.graph_optimizer.sort_signals_by_ops Attempts to rearrange sigs so that it is in the same order as operator signals, without changing the overall block order. nengo_dl.graph_optimizer.noop_order_signals A version of graph_optimizer.order_signals that doesn’t do any reordering, for debugging. nengo_dl.graph_optimizer.remove_unmodified_resets Remove any Reset operators that are targeting a signal that is never modified. nengo_dl.graph_optimizer.remove_zero_incs Remove any operators where we know the input (and therefore output) is zero. nengo_dl.graph_optimizer.remove_reset_incs Replace y=Reset(0) + x with y=x. nengo_dl.graph_optimizer.remove_constant_copies Change Copies with constant input to Resets. nengo_dl.graph_optimizer.remove_identity_muls Change y=x*1 ops to y=x Copy ops. nengo_dl.graph_optimizer.signal_io_dicts Organizes operators into dictionaries according to the signals they set/inc/read/update. nengo_dl.graph_optimizer.display_signal_blocks Creates a visual depiction of the signals blocks read by each operator group.
nengo_dl.graph_optimizer.mergeable(op, chosen_ops)[source]

Check if the given op can be merged with the candidate group.

Parameters
opOperator

The operator to be merged

chosen_opslist of Operator

The operator group to be merged in to

Returns
mergeablebool

True if op can be merged into chosen_ops, else False

nengo_dl.graph_optimizer.greedy_planner(operators)[source]

Create merged execution plan through greedy selection.

Always selects the largest available group of operators as the next to be executed.

Parameters
operatorslist of Operator

All the nengo operators in a model (unordered)

Returns
planlist of tuple of Operator

Operators combined into mergeable groups and in execution order

Notes

Originally based on nengo_ocl greedy planner

nengo_dl.graph_optimizer.tree_planner(op_list, max_depth=3)[source]

Create merged execution plan through exhaustive tree search.

The max_depth parameter scales the planner between full tree search and greedy search. max_depth==1 is equivalent to greedy_planner, and max_depth==len(op_list) is full tree search (guaranteed to find the optimal plan, but likely very slow).

Parameters
op_listlist of Operator

All the nengo operators in a model (unordered)

max_depthint

The planner will search this many steps ahead before selecting which group to schedule next

Returns
planlist of tuple of Operator

Operators combined into mergeable groups and in execution order

nengo_dl.graph_optimizer.transitive_planner(op_list)[source]

Create merged execution plan through transitive closure construction.

This is something like a middle ground between greedy_planner and tree_planner; it can improve simulation time over the greedy planner, but comes with potentially significant build time increases.

Parameters
op_listlist of Operator

All the nengo operators in a model (unordered)

Returns
planlist of tuple of Operator

Operators combined into mergeable groups and in execution order

nengo_dl.graph_optimizer.transitive_closure_recurse(dg, ops, trans, builder_type, builder_types, cache)[source]

Computes the transitive closure for the given graph, restricted to the operators with the given builder type.

Parameters
dgdict of {int: set of int}

Dependency graph where dg[a] = {b, c} indicates that operators b and c are dependent on a

opslist of int

The operators for which we want to compute the transitive closure

transdict of {int: set of int}

The transitive closure for the graph (will be filled in-place)

builder_typetype

One of the nengo_dl build classes (e.g., CopyBuilder), specifying the type of operators to include in the transitive closure

builder_typeslist of type

The build class for each operator

cachedict of {frozenset of int: set of int}

Stores base sets which trans will reference (to reduce memory usage, since many elements in trans will have the same value)

Notes

This function uses ints to refer to operators, where the int indicates the index of the operator in the overall op list (this is done to save memory). See transitive_planner.

nengo_dl.graph_optimizer.noop_planner(operators)[source]

Orders operators into a valid execution order, but does not perform any merging.

Parameters
operatorslist of Operator

All the nengo operators in a model (unordered)

Returns
planlist of tuple of Operator

Operators in execution order

nengo_dl.graph_optimizer.order_signals(plan, n_passes=10)[source]

Orders signals and operators to try to structure reads/writes in contiguous blocks.

Parameters
planlist of tuple of Operator

Operator execution plan (e.g., output from greedy_planner)

n_passesint

Number of repeated passes through the operator reordering stage

Returns
signalslist of Signal

Signals organized into the order in which we want them arranged in memory

planlist of tuple of Operator

Input plan with operators reordered within groups to align with order of signals

nengo_dl.graph_optimizer.hamming_sort(blocks)[source]

Reorder signals using heuristics to try to place signals that are accessed by the same operators into adjacent positions (giving priority to larger blocks).

Parameters
blocksdict of {Signal: frozenset of int}

Dictionary indicating which io blocks each signal is a part of

Returns
sort_idxsdict of {Signal: int}

Indices indicating where each signal should be in the sorted list

nengo_dl.graph_optimizer.sort_ops_by_signals(sorted_io, sigs, sig_idxs, new_plan, blocks, op_sigs)[source]

Rearrange operators to match the order of signals.

Note: the same operators can be associated with multiple IO blocks if they have multiple inputs, so rearranging the operators according to one of those blocks could mess up the order with respect to the other IO block. We iterate through the IO blocks in increasing size so that the largest blocks win out.

Parameters
sorted_iolist of tuple of (Operator, int)

The operators that form each IO block, sorted by increasing size of the block. In the case that a group of operators participate in multiple IO blocks, the integer distinguishes which one of those blocks this block is associated with.

sigslist of Signal

Signals that have been arranged into a given order by other parts of the algorithm

sig_idxsdict of {Signal: int}

Sorted indices of signals

new_plandict of {tuple of Operator: tuple of Operator}

Mapping from original operator group to the sorted operators

blocksdict of {Signal: frozenset of int}

Indicates which IO blocks each signal participates in

op_sigsdict of {Operator: list of Signal}

The signals accessed by each operator

Returns
new_plandict of {tuple of Operator: tuple of Operator}

Mapping from original operator group to the sorted operators

sig_idxsdict of {Signal: int}

Signal indices, possibly updated to match new op order

nengo_dl.graph_optimizer.sort_signals_by_ops(sorted_io, sigs, sig_idxs, new_plan, blocks, op_sigs)[source]

Attempts to rearrange sigs so that it is in the same order as operator signals, without changing the overall block order.

Parameters
sorted_iolist of tuple of (Operator, int)

The operators that form each io block, sorted by increasing size of the io block. In the case that a group of operators participate in multiple io blocks, the integer distinguishes which one of those blocks this block is associated with.

sigslist of Signal

Signals to be sorted

sig_idxsdict of {Signal: int}

Sorted indices of signals

new_plandict of {tuple of Operator: tuple of Operator}

Mapping from original operator group to the sorted operators

blocksdict of {Signal: frozenset of int}

Indicates which io blocks each signal participates in

op_sigsdict of {Operator: list of Signal}

The signals accessed by each operator

Returns
sig_idxsdict of {Signal: int}

Sorted indices of signals

nengo_dl.graph_optimizer.noop_order_signals(plan, **_)[source]

A version of graph_optimizer.order_signals that doesn’t do any reordering, for debugging.

nengo_dl.graph_optimizer.remove_unmodified_resets(operators)[source]

Remove any Reset operators that are targeting a signal that is never modified.

If a signal is reset, but never inced/updated after that, we can just set the default signal value to the reset value and remove the reset. Note: this wouldn’t normally happen, but it can happen if we removed some of the incs (e.g. in remove_zero_incs).

Parameters
operatorslist of Operator

Operators in the model

Returns
new_operatorslist of Operator

Modified list of operators

nengo_dl.graph_optimizer.remove_zero_incs(operators)[source]

Remove any operators where we know the input (and therefore output) is zero.

If the input to a DotInc/ElementwiseInc/Copy/ConvInc is zero then we know that the output of the op will be zero, so we can just get rid of it.

Parameters
operatorslist of Operator

Operators in the model

Returns
new_operatorslist of Operator

Modified list of operators

nengo_dl.graph_optimizer.remove_reset_incs(operators)[source]

Replace y=Reset(0) + x with y=x.

If a signal is Reset and Inc’d, we can change that to a Set that combines the two ops (note: any other incs of that signal can proceed as normal)

Parameters
operatorslist of Operator

operators in the model

Returns
new_operatorslist of Operator

modified list of operators

Notes

In practice, this modification can hurt more than it helps. Inc operators are cheaper to compute the gradient for, and changing Incs to Incs and Sets splits up the Inc merge groups. It tends to provide the most value for models consisting of long linear chains of objects.

nengo_dl.graph_optimizer.remove_constant_copies(operators)[source]

Change Copies with constant input to Resets.

If a Copy has no dependencies, or just one Reset() dependency, then we can change it to an op that just directly sets the output signal to the Copy input value.

Parameters
operatorslist of Operator

Operators in the model

Returns
new_operatorslist of Operator

Modified list of operators

nengo_dl.graph_optimizer.remove_identity_muls(operators)[source]

Change y=x*1 ops to y=x Copy ops.

If one of the inputs to a DotInc/ElementwiseInc is 1 then we can skip the multiplication and change it to a Copy op.

Parameters
operatorslist of Operator

Operators in the model

Returns
new_operatorslist of Operator

Modified list of operators

nengo_dl.graph_optimizer.signal_io_dicts(operators)[source]

Organizes operators into dictionaries according to the signals they set/inc/read/update.

Parameters
operatorslist of Operator

Operators in the model

Returns
setsdict of {Signal: list of Operator}

A dictionary indicating all the Operators that set each signal.

incsdict of {Signal: list of Operator}

A dictionary indicating all the Operators that inc each signal.

readsdict of {Signal: list of Operator}

A dictionary indicating all the Operators that read each signal.

updatesdict of {Signal: list of Operator}

A dictionary indicating all the Operators that update each signal.

nengo_dl.graph_optimizer.display_signal_blocks(operators, all_signals)[source]

Creates a visual depiction of the signals blocks read by each operator group.

Parameters
operatorslist of tuple of Operator

Operator execution plan

all_signalslist of Signal

Base signals arranged into some order

Returns
signal_blocksstr

A string where each row corresponds to one operator group, and the non-blank characters in the line indicate that the operator group reads/writes that signal (with a number used to distinguish the different signal blocks within the operator group).

### Layer converters¶

Tools for automatically converting a Keras model to a Nengo network.

 nengo_dl.converter.LayerConverter Base class for converter classes, which contain the logic for mapping some Keras layer type to Nengo objects. nengo_dl.converter.ConvertModel Convert tf.keras.Model to Nengo objects. nengo_dl.converter.ConvertSequential Convert tf.keras.Sequential to Nengo objects. nengo_dl.converter.ConvertFallback Convert layers which do not have a native Nengo equivalent into a TensorNode. nengo_dl.converter.ConvertAvgPool Base class for converting average pooling layers to Nengo objects. nengo_dl.converter.ConvertActivation Convert tf.keras.layers.Activation to Nengo objects. nengo_dl.converter.ConvertAdd Convert tf.keras.layers.Add to Nengo objects. nengo_dl.converter.ConvertAverage Convert tf.keras.layers.Average to Nengo objects. nengo_dl.converter.ConvertAvgPool1D Convert tf.keras.layers.AvgPool1D to Nengo objects. nengo_dl.converter.ConvertAvgPool2D Convert tf.keras.layers.AvgPool2D to Nengo objects. nengo_dl.converter.ConvertAvgPool3D Convert tf.keras.layers.AvgPool3D to Nengo objects. nengo_dl.converter.ConvertBatchNormalization Convert tf.keras.layers.BatchNormalization to Nengo objects. nengo_dl.converter.ConvertConcatenate Convert tf.keras.layers.Concatenate to Nengo objects. nengo_dl.converter.ConvertConv Base class for converting convolutional layers to Nengo objects. nengo_dl.converter.ConvertConv1D Convert tf.keras.layers.Conv1D to Nengo objects. nengo_dl.converter.ConvertConv2D Convert tf.keras.layers.Conv2D to Nengo objects. nengo_dl.converter.ConvertConv3D Convert tf.keras.layers.Conv3D to Nengo objects. nengo_dl.converter.ConvertDense Convert tf.keras.layers.Dense to Nengo objects. nengo_dl.converter.ConvertFlatten Convert tf.keras.layers.Flatten to Nengo objects. nengo_dl.converter.ConvertInput Convert tf.keras.layers.InputLayer to Nengo objects. nengo_dl.converter.ConvertReLU Convert tf.keras.layers.ReLU to Nengo objects. nengo_dl.converter.ConvertLeakyReLU Convert tf.keras.layers.LeakyReLU to Nengo objects. nengo_dl.converter.ConvertReshape Convert tf.keras.layers.Reshape to Nengo objects. nengo_dl.converter.ConvertUpSampling Base class for converting upsampling layers to Nengo objects. nengo_dl.converter.ConvertUpSampling1D Convert tf.keras.layers.UpSampling1D to Nengo objects. nengo_dl.converter.ConvertUpSampling2D Convert tf.keras.layers.UpSampling2D to Nengo objects. nengo_dl.converter.ConvertUpSampling3D Convert tf.keras.layers.UpSampling3D to Nengo objects. nengo_dl.converter.ConvertZeroPadding Base class for converting zero-padding layers to Nengo objects. nengo_dl.converter.ConvertZeroPadding1D Convert tf.keras.layers.ZeroPadding1D to Nengo objects. nengo_dl.converter.ConvertZeroPadding2D Convert tf.keras.layers.ZeroPadding2D to Nengo objects. nengo_dl.converter.ConvertZeroPadding3D Convert tf.keras.layers.ZeroPadding3D to Nengo objects.
class nengo_dl.converter.LayerConverter(layer, converter)[source]

Base class for converter classes, which contain the logic for mapping some Keras layer type to Nengo objects.

Subclasses must implement the LayerConverter.convert method. They may optionally extend LayerConverter.convertible if this layer type requires custom logic for whether or not a layer can be converted to Nengo objects.

Subclasses should override the unsupported_args class parameter if there are certain non-default Layer attributes that are not supported by the converter. This is a list of names for attributes that must have the default value for the layer to be convertible. The default is assumed to be None, or a tuple of ("attribute_name", default_value) can be specified. If there are parameters that are supported in inference mode but not in training mode, they should be added to the unsupported_training_args parameter.

Subclasses should override the has_weights class parameter if the layer type being converted contains internal weights (this affects how the converter will handle duplicate layers).

Parameters
layertf.keras.layers.Layer

The Layer object being converted.

converterConverter

The parent Converter class running the conversion process.

add_nengo_obj(node_id, biases=None, activation=None)[source]

Builds a Nengo object for the given Node of this layer.

Parameters
node_idint

The index of the Keras Node currently being built on this layer.

biasesnumpy.ndarray or None

If not None, add trainable biases with the given value.

activationcallable or None

The TensorFlow activation function to be used (None will be interpreted as linear activation).

Returns
obj

The Nengo object whose output corresponds to the output of the given Keras Node.

add_connection(node_id, obj, input_idx=0, trainable=False, **kwargs)[source]

Adds a Connection from one of the inputs of the Node being built to the Nengo object.

Parameters
node_idint

The index of the Keras Node currently being built on this layer.

objNengoObject

The Nengo object implementing this Node.

input_idxint

Which of the inputs we want to add a Connection for (in the case of layers that have multiple inputs).

trainablebool

Whether or not the weights associated with the created Connection should be trainable.

kwargsdict

Will be passed on to nengo.Connection.

Returns
connnengo.Connection

The constructed Connection object.

get_input_obj(node_id, tensor_idx=0)[source]

Returns the Nengo object corresponding to the given input of this layer.

Parameters
node_idint

The index of the Keras Node currently being built on this layer.

tensor_idxint

The index of the input we want to look up (for layers with multiple inputs).

Returns
obj

The Nengo object whose output corresponds to the given input of this layer.

input_shape(node_id, include_batch=False)[source]

Returns the input shape of the given node.

Parameters
node_idint

The node whose shape we want to look up.

include_batchbool

Whether or not the returned shape should include the batch dimension.

Returns
shape(list of) tuple of int

A single tuple shape if the node has one input, or a list of shapes if the node as multiple inputs.

output_shape(node_id, include_batch=False)[source]

Returns the output shape of the given node.

Parameters
node_idint

The node whose shape we want to look up.

include_batchbool

Whether or not the returned shape should include the batch dimension.

Returns
shape(list of) tuple of int

A single tuple shape if the node has one output, or a list of shapes if the node as multiple outputs.

static set_trainable(obj, trainable)[source]

Set trainable config attribute of object.

Parameters
objNengoObject

The object to be assigned.

trainable: bool

Trainability value of obj.

classmethod convertible(layer, converter)[source]

Check whether the given Keras layer is convertible to native Nengo objects.

Parameters
layertf.keras.layers.Layer

The Keras Layer we want to convert.

converterConverter

The Converter object running the conversion process.

Returns
convertiblebool

True if the layer can be converted to native Nengo objects, else False.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertModel(layer, converter)[source]

Convert tf.keras.Model to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertSequential(seq_model, converter)[source]

Convert tf.keras.Sequential to Nengo objects.

class nengo_dl.converter.ConvertFallback(*args, **kwargs)[source]

Convert layers which do not have a native Nengo equivalent into a TensorNode.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertAvgPool(layer, converter)[source]

Base class for converting average pooling layers to Nengo objects.

convert(node_id, dimensions)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

classmethod convertible(layer, converter)[source]

Check whether the given Keras layer is convertible to native Nengo objects.

Parameters
layertf.keras.layers.Layer

The Keras Layer we want to convert.

converterConverter

The Converter object running the conversion process.

Returns
convertiblebool

True if the layer can be converted to native Nengo objects, else False.

class nengo_dl.converter.ConvertActivation(layer, converter)[source]

Convert tf.keras.layers.Activation to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertAdd(layer, converter)[source]

Convert tf.keras.layers.Add to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertAverage(layer, converter)[source]

Convert tf.keras.layers.Average to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertAvgPool1D(layer, converter)[source]

Convert tf.keras.layers.AvgPool1D to Nengo objects.

Also works for tf.keras.layers.GlobalAvgPool1D, and tf.keras.layers.MaxPool1D/GlobalMaxPool1D (if max_to_avg_pool=True).

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertAvgPool2D(layer, converter)[source]

Convert tf.keras.layers.AvgPool2D to Nengo objects.

Also works for tf.keras.layers.GlobalAvgPool2D, and tf.keras.layers.MaxPool2D/GlobalMaxPool2D (if max_to_avg_pool=True).

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertAvgPool3D(layer, converter)[source]

Convert tf.keras.layers.AvgPool3D to Nengo objects.

Also works for tf.keras.layers.GlobalAvgPool3D, and tf.keras.layers.MaxPool3D/GlobalMaxPool3D (if max_to_avg_pool=True).

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertBatchNormalization(layer, converter)[source]

Convert tf.keras.layers.BatchNormalization to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

classmethod convertible(layer, converter)[source]

Check whether the given Keras layer is convertible to native Nengo objects.

Parameters
layertf.keras.layers.Layer

The Keras Layer we want to convert.

converterConverter

The Converter object running the conversion process.

Returns
convertiblebool

True if the layer can be converted to native Nengo objects, else False.

class nengo_dl.converter.ConvertConcatenate(layer, converter)[source]

Convert tf.keras.layers.Concatenate to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

classmethod convertible(layer, converter)[source]

Check whether the given Keras layer is convertible to native Nengo objects.

Parameters
layertf.keras.layers.Layer

The Keras Layer we want to convert.

converterConverter

The Converter object running the conversion process.

Returns
convertiblebool

True if the layer can be converted to native Nengo objects, else False.

class nengo_dl.converter.ConvertConv(layer, converter)[source]

Base class for converting convolutional layers to Nengo objects.

convert(node_id, dimensions)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertConv1D(layer, converter)[source]

Convert tf.keras.layers.Conv1D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertConv2D(layer, converter)[source]

Convert tf.keras.layers.Conv2D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertConv3D(layer, converter)[source]

Convert tf.keras.layers.Conv3D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertDense(layer, converter)[source]

Convert tf.keras.layers.Dense to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertFlatten(layer, converter)[source]

Convert tf.keras.layers.Flatten to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertInput(layer, converter)[source]

Convert tf.keras.layers.InputLayer to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertReLU(layer, converter)[source]

Convert tf.keras.layers.ReLU to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertLeakyReLU(layer, converter)[source]

Convert tf.keras.layers.LeakyReLU to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertReshape(layer, converter)[source]

Convert tf.keras.layers.Reshape to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertUpSampling(layer, converter)[source]

Base class for converting upsampling layers to Nengo objects.

convert(node_id, dimensions)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertUpSampling1D(layer, converter)[source]

Convert tf.keras.layers.UpSampling1D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertUpSampling2D(layer, converter)[source]

Convert tf.keras.layers.UpSampling2D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertUpSampling3D(layer, converter)[source]

Convert tf.keras.layers.UpSampling3D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertZeroPadding(layer, converter)[source]

Base class for converting zero-padding layers to Nengo objects.

convert(node_id, dimensions)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertZeroPadding1D(layer, converter)[source]

Convert tf.keras.layers.ZeroPadding1D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertZeroPadding2D(layer, converter)[source]

Convert tf.keras.layers.ZeroPadding2D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

class nengo_dl.converter.ConvertZeroPadding3D(layer, converter)[source]

Convert tf.keras.layers.ZeroPadding3D to Nengo objects.

convert(node_id)[source]

Convert the given node of this layer to Nengo objects

Parameters
node_idint

The index of the inbound node to be converted.

Returns
outputNengoObject

Nengo object whose output corresponds to the output of the Keras layer node.

### Utilities¶

Utility objects used throughout the code base.

 nengo_dl.utils.sanitize_name Remove illegal TensorFlow name characters from string. nengo_dl.utils.function_name Get the name of the callable object func. nengo_dl.utils.align_func Decorator that ensures the output of func is a valid ndarray with the given dtype. nengo_dl.utils.ProgressBar Handles progress bar display for some tracked process. nengo_dl.utils.SubProgressBar A progress bar representing a sub-task within an overall progress bar. nengo_dl.utils.NullProgressBar A progress bar that does nothing.
nengo_dl.utils.sanitize_name(name)[source]

Remove illegal TensorFlow name characters from string.

Valid TensorFlow name characters are [A-Za-z0-9_.\-/]

Parameters
namestr

Name to be sanitized

Returns
sanitizedstr

Sanitized name

nengo_dl.utils.function_name(func, sanitize=True)[source]

Get the name of the callable object func.

Parameters
funccallable

Callable object (e.g., function, callable class)

sanitizebool

If True, remove any illegal TensorFlow name characters from name

Returns
namestr

Name of func (optionally sanitized)

nengo_dl.utils.align_func(output_dtype)[source]

Decorator that ensures the output of func is a valid ndarray with the given dtype.

Parameters
output_dtypestr or tf.DType or dtype

Desired dtype of function output(s)

Raises
nengo.exceptions.SimulationError

If the function returns None or a non-finite value.

class nengo_dl.utils.ProgressBar(present='', past=None, max_value=1, **kwargs)[source]

Handles progress bar display for some tracked process.

Parameters
presentstr

Description of process in present (e.g., “Simulating”)

paststr

Description of process in past (e.g., “Simulation”)

max_valueint or None

The maximum number of steps in the tracked process (or None if the maximum number of steps is unknown)

Notes

Initializes a progress bar with sane defaults

start(**kwargs)[source]

Start tracking process, initialize display.

finish(**kwargs)[source]

Stop tracking process, finish display.

step()[source]

Advance the progress bar one step.

sub(msg=None, **kwargs)[source]

Creates a new progress bar for tracking a sub-process.

Parameters
msgstr

Description of sub-process

property max_steps

Alias for max_value to allow this to work with Nengo progress bar interface.

class nengo_dl.utils.SubProgressBar(present='', past=None, max_value=1, **kwargs)[source]

A progress bar representing a sub-task within an overall progress bar.

Initializes a progress bar with sane defaults

finish(**kwargs)[source]

Finishing a sub-progress bar doesn’t start a new line.

class nengo_dl.utils.NullProgressBar(present='', past=None, max_value=1, **kwargs)[source]

A progress bar that does nothing.

Used to replace ProgressBar when we want to disable output.

Initializes a progress bar with sane defaults

sub(*args, **kwargs)[source]

Noop for creating a sub-progress bar.

step(**kwargs)[source]

Noop for incrementing the progress bar.

### Benchmarks¶

Benchmark networks and utilities for evaluating NengoDL’s performance.

 nengo_dl.benchmarks.cconv Circular convolution (EnsembleArray) benchmark. nengo_dl.benchmarks.integrator Single integrator ensemble benchmark. nengo_dl.benchmarks.pes PES learning rule benchmark. nengo_dl.benchmarks.basal_ganglia Basal ganglia network benchmark. nengo_dl.benchmarks.mnist A network designed to stress-test tensor layers (based on mnist net). nengo_dl.benchmarks.spaun Builds the Spaun network. nengo_dl.benchmarks.random_network A randomly interconnected network of ensembles. nengo_dl.benchmarks.lmu A network containing a single Legendre Memory Unit cell and dense readout. nengo_dl.benchmarks.run_profile Run profiler on a benchmark network.
nengo_dl.benchmarks.cconv(dimensions, neurons_per_d, neuron_type)[source]

Circular convolution (EnsembleArray) benchmark.

Parameters
dimensionsint

Number of dimensions for vector values

neurons_per_dint

Number of neurons to use per vector dimension

neuron_typeNeuronType

Simulation neuron type

Returns
netnengo.Network

benchmark network

nengo_dl.benchmarks.integrator(dimensions, neurons_per_d, neuron_type)[source]

Single integrator ensemble benchmark.

Parameters
dimensionsint

Number of dimensions for vector values

neurons_per_dint

Number of neurons to use per vector dimension

neuron_typeNeuronType

Simulation neuron type

Returns
netnengo.Network

benchmark network

nengo_dl.benchmarks.pes(dimensions, neurons_per_d, neuron_type)[source]

PES learning rule benchmark.

Parameters
dimensionsint

Number of dimensions for vector values

neurons_per_dint

Number of neurons to use per vector dimension

neuron_typeNeuronType

Simulation neuron type

Returns
netnengo.Network

benchmark network

nengo_dl.benchmarks.basal_ganglia(dimensions, neurons_per_d, neuron_type)[source]

Basal ganglia network benchmark.

Parameters
dimensionsint

Number of dimensions for vector values

neurons_per_dint

Number of neurons to use per vector dimension

neuron_typeNeuronType

Simulation neuron type

Returns
netnengo.Network

benchmark network

nengo_dl.benchmarks.mnist(use_tensor_layer=True)[source]

A network designed to stress-test tensor layers (based on mnist net).

Parameters
use_tensor_layerbool

If True, use individual tensor_layers to build the network, as opposed to a single TensorNode containing all layers.

Returns
netnengo.Network

benchmark network

nengo_dl.benchmarks.spaun(dimensions)[source]

Builds the Spaun network.

See [1] for more details.

Parameters
dimensionsint

Number of dimensions for vector values

Returns
netnengo.Network

benchmark network

Notes

This network needs to be installed via

pip install git+https://github.com/drasmuss/spaun2.0.git


References

1

Chris Eliasmith, Terrence C. Stewart, Xuan Choo, Trevor Bekolay, Travis DeWolf, Yichuan Tang, and Daniel Rasmussen (2012). A large-scale model of the functioning brain. Science, 338:1202-1205.

nengo_dl.benchmarks.random_network(dimensions, neurons_per_d, neuron_type, n_ensembles, connections_per_ensemble, seed=0)[source]

A randomly interconnected network of ensembles.

Parameters
dimensionsint

Number of dimensions for vector values

neurons_per_dint

Number of neurons to use per vector dimension

neuron_typeNeuronType

Simulation neuron type

n_ensemblesint

Number of ensembles in the network

connections_per_ensembleint

Outgoing connections from each ensemble

Returns
netnengo.Network

benchmark network

nengo_dl.benchmarks.lmu(theta, input_d, native_nengo=False, dtype='float32')[source]

A network containing a single Legendre Memory Unit cell and dense readout.

See [1] for more details.

Parameters
thetaint

Time window parameter for LMU.

input_dint

Dimensionality of input signal.

native_nengobool

If True, build the LMU out of Nengo objects. Otherwise, build the LMU directly in TensorFlow, and use a TensorNode to wrap the whole cell.

dtypestr

Float dtype to use for internal parameters of LMU when native_nengo=False (native_nengo=True will use the dtype of the Simulator).

Returns
netnengo.Network

Benchmark network

References

1

Aaron R. Voelker, Ivana Kajić, and Chris Eliasmith. Legendre memory units: continuous-time representation in recurrent neural networks. In Advances in Neural Information Processing Systems. 2019. https://papers.nips.cc/paper/9689-legendre-memory-units-continuous-time-representation-in-recurrent-neural-networks.

nengo_dl.benchmarks.run_profile(net, train=False, n_steps=150, do_profile=True, reps=1, dtype='float32', **kwargs)[source]

Run profiler on a benchmark network.

Parameters
netNetwork

The nengo Network to be profiled.

trainbool

If True, profile the Simulator.fit function. Otherwise, profile the Simulator.run function.

n_stepsint

The number of timesteps to run the simulation.

do_profilebool

Whether or not to run profiling

repsint

Repeat the run this many times (only profile data from the last run will be kept).

dtypestr

Simulation dtype (e.g. “float32”)

Returns
exec_timefloat

Time (in seconds) taken to run the benchmark, taking the minimum over reps.

Notes

kwargs will be passed on to Simulator

#### Interface¶

The benchmark module also includes a command-line interface for building and running the benchmarks:

##### benchmarks¶

Command-line interface for benchmarks.

benchmarks [OPTIONS] COMMAND1 [ARGS]... [COMMAND2 [ARGS]...]...

###### build¶

Builds one of the benchmark networks

benchmarks build [OPTIONS]


Options

--benchmark <benchmark>

Name of benchmark network

--dimensions <dimensions>

Number of dimensions

--neurons_per_d <neurons_per_d>

Neurons per dimension

--neuron_type <neuron_type>

Nengo neuron model

--kwarg <kwarg>

Arbitrary kwarg to pass to benchmark network (key=value)

###### profile¶

Runs profiling on a network (call after ‘build’)

benchmarks profile [OPTIONS]


Options

--train, --no-train

Whether to profile training (as opposed to running) the network

--n_steps <n_steps>

Number of steps for which to run the simulation

--batch_size <batch_size>

Number of inputs to the model

--device <device>

TensorFlow device on which to run the simulation

--unroll <unroll>

Number of steps for which to unroll the simulation

--time-only

Only count total time, rather than profiling internals