ansar-create
The ansar-create library uses multi-threading and multi-processing to solve difficult software challenges such as concurrency, interruption and cancellation. It wraps those platform facilities in a standard runtime model, giving developers the ability to express that type of software in a clear and consistent manner.
This type of software is often referred to as asynchronous, event-driven or reactive software. It acknowledges the fundamental fact that significant events can occur at any time, and that software must be able to respond to those events in a reliable and timely way.
Features
Based on a standard model for complex software operations (SDL).
Uniform management of threads, processes and state machines
Built-in runtime facilities such as timers and logging.
Persistent application configuration.
Process orchestration.
Development automation.
ansar-create uses ansar-encode to pass application data between processes and for persistence of configuration information. Refer to the associated documentation for detailed information on the encoding and decoding of complex application data.
A Quick Taste
Solving multi-threading and multi-processing requirements is a good way to illustrate the use of ansar-create. To start a thread;
import sys
import ansar.create as ar
def analyzer(self, datafile):
return ar.Ack()
ar.bind(analyzer)
def run_analyzer(datafile):
with ar.OpenChannel() as channel:
a = channel.create(analyzer, datafile)
m = channel.select(ar.Completed)
return m.value
if __name__ == '__main__':
datafile = sys.argv[1]
a = run_analyzer(datafile)
assert isinstance(a, ar.Ack)
Note
Source files appearing in this section can be downloaded from here.
The repo Makefile contains the setup needed for this guide. Related background information can be
found here.
This code runs the analyzer function in its own dedicated thread. With ansar, if you want to run
something in a different thread then code it up as a standard function, with parameters and return
statements. Register it with bind() and you are good to go.
The application calls the run_analyzer function in the normal synchronous manner. An OpenChannel
context is used to automate the construction and cleanup of a channel and that object is used to initiate the
thread. Eventually the analysis finishes and this is detected by select(). The argument passed
to run_analyzer is forwarded to the analyzer thread and the object returned by that thread
(i.e. Ack) is available as m.value.
This code illustrates the integration of multi-threading into a standard Python application, without involving the developer in any of the technical details associated with multi-threading. There is the passing of arbitrary data to an instance of a thread and the safe return of data produced by that thread.
As an implementation of the analysis this use of multi-threading is somewhat redundant -
the run_analyzer function waits for the results of the thread immediately after starting
it. The value of multi-threading is better realized when there is a need for multiple concurrent
analyzers. Consider this simple change;
def run_analyzer(datafile1, datafile2):
with ar.OpenChannel() as channel:
a1 = channel.create(analyzer, datafile1)
a2 = channel.create(analyzer, datafile2)
m1 = channel.select(ar.Completed)
m2 = channel.select(ar.Completed)
if channel.return_address == a1:
m1, m2 = m2, m1
return m1.value, m2.value
if __name__ == '__main__':
datafile1 = sys.argv[1]
datafile2 = sys.argv[2]
a1, a2 = run_analyzer(datafile1, datafile2)
assert isinstance(a1, ar.Ack)
assert isinstance(a2, ar.Ack)
Note
There is no guarantee that analyzer 1 will complete before analyzer 2. This issue is resolved by the conditional flip of the 2 results variables.
The run_analyzer function now accepts 2 data files and returns the pair of results,
as soon as they both become available. As a last quick introduction to ansar-create, consider
the changes needed to run those concurrent analyzers as separate processes;
def run_analyzer(datafile1, datafile2):
with ar.OpenChannel() as channel:
a1 = channel.create(ar.Process, 'analyzer', input=datafile1)
a2 = channel.create(ar.Process, 'analyzer', input=datafile2)
m1 = channel.select(ar.Completed)
m2 = channel.select(ar.Completed)
if channel.return_address == a1:
m1, m2 = m2, m1
return m1.value, m2.value
The changes are minimal. Rather than creating instances of analyzer functions there is now
the creation of Process objects. These objects operate as the internal proxies
for external processes, each one creating a new platform process, sending an encoding of the argument
over an input pipe and expecting the encoding of a result on the output pipe. That result is then
returned in the Completed object, in exactly the same fashion as the analyzer
function did previously.
Moving the analyzer function into an executable is trivial;
import ansar.create as ar
def analyzer(self, settings, datafile):
return ar.Ack()
ar.bind(analyzer)
if __name__ == '__main__':
ar.create_object(analyzer, factory_input='default-datafile-name')
All the hard work is done by create_object(). It processes the command-line, decodes
the input pipe, constructs the analyzer object, passes the datafile and encodes the
results.
Note
The calling convention between create_object() and analyzer() requires the
settings parameter to be defined. In this particular scenario it is a placeholder
and not used. Further reading on the construction of async executables can be
found here.
For the proper operation of Process objects the supplied sub-process name is
expected to be an executable file, somewhere on the current $PATH. A standard technique for resolving
this issue can be found in the repo Makefile, mentioned at the top of this document.
Documentation
Links to further reading can be found below. These include a tutorial for asynchronous programming, how to quickly create a standard async executable, details on the logging facility, a sample multi-processing project, and more.
Author
The ansar-create library was developed by Scott Woods <scott.18.ansar@gmail.com>. The library has evolved under the influence of several large projects and more recently has moved from C++ to Python.
License
The ansar-create library is released under the MIT License.