Download 1.x PDF - Read the Docs

Transcript
IPython Documentation, Release 1.2.1: An Afternoon Hack
does from the outside, but have rich internal logic to determine destinations, as well as handle dependency
graphs Their sockets are always ROUTER on both sides.
The Python task schedulers have an additional message type, which informs the Hub of the destination of a
task as soon as that destination is known.
Message type: task_destination:
content = {
'msg_id' : 'abcd-1234-...', # the msg's uuid
'engine_id' : '1234-abcd-...', # the destination engine's zmq.IDENTITY
}
apply()
In terms of message classes, the MUX scheduler and Task scheduler relay the exact same message types.
Their only difference lies in how the destination is selected.
The Namespace model suggests that execution be able to use the model:
ns.apply(f, *args, **kwargs)
which takes f, a function in the user’s namespace, and executes f(*args, **kwargs) on a remote
engine, returning the result (or, for non-blocking, information facilitating later retrieval of the result). This
model, unlike the execute message which just uses a code string, must be able to send arbitrary (pickleable)
Python objects. And ideally, copy as little data as we can. The buffers property of a Message was introduced
for this purpose.
Utility method build_apply_message() in IPython.kernel.zmq.serialize wraps a function signature and builds a sendable buffer format for minimal data copying (exactly zero copies of numpy
array data or buffers or large strings).
Message type: apply_request:
metadata = {
'after' : ['msg_id',...], # list of msg_ids or output of Dependency.as_dict()
'follow' : ['msg_id',...], # list of msg_ids or output of Dependency.as_dict()
}
content = {}
buffers = ['...'] # at least 3 in length
# as built by build_apply_message(f,args,kwargs)
after/follow represent task dependencies. ‘after’ corresponds to a time dependency. The request will not
arrive at an engine until the ‘after’ dependency tasks have completed. ‘follow’ corresponds to a location
dependency. The task will be submitted to the same engine as these msg_ids (see Dependency docs for
details).
Message type: apply_reply:
content = {
'status' : 'ok' # 'ok' or 'error'
# other error info here, as in other messages
}
406
Chapter 7. IPython developer’s guide