Model for Experiments¶

Available Experiments

Experiment Models¶

Experiment models make explicit the different steps needed to perform a measurement. The Base Experiment class defines some methods that are transversal to all experiments (such as loading a configuration file) but individual experiment models can overwrite this methods to develop custom solutions.

Moreover, PyNTA introduces the PUB/SUB pattern in order to exchange information between different parts of the program in a flexible and efficient way. You can find more information on publisher and subscriber.

copyright: Aquiles Carattino <aquiles@uetke.com>
license: GPLv3, see LICENSE for more details

base_experiment.py¶

Base class for the experiments. BaseExperiment defines the common patterns that every experiment should have. Importantly, it starts an independent process called publisher, that will be responsible for broadcasting messages that are appended to a queue. The messages rely on the pyZMQ library and should be tested further in order to assess their limitations. The general pattern is that of the PUB/SUB, with one publisher and several subscribers.

The messages should include a topic and data. For this, the elements in the queue should be dictionaries with two keywords: data and topic. data['data'] will be serialized through the use of cPickle, and is handled automatically by pyZQM through the use of send_pyobj. The subscribers should be aware of this and use either unpickle or recv_pyobj.

In order to stop the publisher process, the string 'stop' should be placed in data['data']. The message will be broadcast and can be used to stop other processes, such as subscribers.

Todo

Check whether the serialization of objects with cPickle may be a bottleneck for performance.

copyright: Aquiles Carattino <aquiles@uetke.com>
license: GPLv3, see LICENSE for more details

class pynta.model.experiment.base_experiment.BaseExperiment(filename=None)[source]¶

Bases: object

Base class to define experiments. Should keep track of the basic methods needed regardless of the experiment to be performed. For instance, a way to start and a way to finalize a measurement.

property alive_threads¶

clear_threads()[source]¶: Keep only the threads that are alive.

connect(method, topic, *args, **kwargs)[source]¶

Async method that connects the running publisher to the given method on a specific topic.

Parameters

method – method that will be connected on a given topic
topic (str) – the topic that will be used by the subscriber to discriminate what information to collect.
args – extra arguments will be passed to the subscriber, which in turn will pass them to the function
kwargs – extra keyword arguments will be passed to the subscriber, which in turn will pass them to the function

property connections¶

finalize()[source]¶: Needs to be overridden by child classes.

property list_alive_threads¶

load_configuration(filename)[source]¶

Loads the configuration file in YAML format.

Parameters: filename (str) – full path to where the configuration file is located.
Raises: FileNotFoundError – if the file does not exist.

property num_threads¶

set_up()[source]¶: Needs to be overridden by child classes.

stop_publisher()[source]¶: Puts the proper data to the queue in order to stop the running publisher process

stop_subscribers()[source]¶: Puts the proper data into every alive subscriber in order to stop it.

update_config(**kwargs)[source]¶

Publisher¶

Publishers are responsible for broadcasting the message over the ZMQ PUB/SUB architecture. The publisher runs continuously on a separated process and grabs elements from a queue, which in turn are sent through a socket to any other processes listening.

Todo

In the current implementation, data is serialized for being added to a Queue, then deserialized by the publisher and serialized again to be sent. These three steps could be simplify into one if, for example, one assumes that objects where pickled. There is also a possibility of assuming numpy arrays and using a zero-copy strategy.

copyright: Aquiles Carattino <aquiles@uetke.com>
license: GPLv3, see LICENSE for more details

class pynta.model.experiment.publisher.Publisher(port=None)[source]¶

Bases: object

Publisher class in which the queue for publishing messages is defined and also a separated process is started. It is important to have a new process, since the serialization/deserialization of messages from the QUEUE may be a bottleneck for performance.

empty_queue()[source]¶: If the publisher stops before broadcasting all the messages, the Queue may still be using some memory. This method is simply getting all the elements in order to free memory. Can be useful for garbage collection or better control of the downstream program.

join(timeout=0)[source]¶

property port¶

publish(topic, data)[source]¶

Adapts the data to make it faster to broadcast

Parameters

topic (str) – Topic in which to publish the data
data – Data to be published

Returns

None

start()[source]¶: Start a new process that will be responsible for broadcasting the messages.

Todo

Find a way to start the publisher on a different port if the one specified is in use.

stop()[source]¶

pynta.model.experiment.publisher.publisher(queue, event, port)[source]¶

Simple method that starts a publisher on the port 5555.

Parameters

queue (multiprocessing.Queue) – Queue of messages to be broadcasted
event (multiprocessing.Event) – Event to stop the publisher
port (int) – port in which to broadcast data

Todo

The publisher’s port should be determined in a configuration file.

Deprecated since version 0.1.0.

Subscriber¶

Example script on how to run separate processes to process the data coming from a publisher like the one on publisher.py. The first process just grabs the frame and puts it in a Queue. The Queue is then used by another process in order to analyse, process, save, etc. It has to be noted that on UNIX systems, getting from a queue with Queue.get() is particularly slow, much slower than serializing a numpy array with cPickle.

pynta.model.experiment.subscriber.subscribe(port, topic)[source]¶

pynta.model.experiment.subscriber.subscriber(func, topic, event, *args, **kwargs)[source]¶

Experiment Configuration¶

Class which holds some parameters that need to be used throughout the lifetime of a program. Keeping them in a separate class gives great flexibility, because it allows to overwrite values at run time.

Todo

Changes to config at runtime will have no effect on other processes. Find a way in which config can broadcast itself to all the instances of the class

copyright: Aquiles Carattino <aquiles@uetke.com>
license: GPLv3, see LICENSE for more details

class pynta.model.experiment.config.Config[source]¶: Bases: object