Skip to content
This repository has been archived by the owner on Jul 16, 2024. It is now read-only.

GSoC 2015 ZeroMQ

Jump to bottom
Krzysztof Nowak edited this page May 6, 2015 · 3 revisions

#Asynchronous communication layer (ØMQ) for the island model

PaGMO provides powerful parallelization built around the generalized island-model paradigm ("optimization in an archipelago of islands"). The optimization algorithms are automatically parallelized (asynchronously, coarse-grained approach) thus making efficient use of todays multi/many-core architectures. Islands may run in complete isolation in parallel but once candidate solutions start to migrate between islands the optimization process is speed up. The current implementation of the island model in PaGMO is mainly based on the two classes island_base and archipelago. The island_base class is a polymorphic class that takes care of starting computational tasks. Currently three tpyes of islands are available. A POSIX type of island (spawning a thread), a python type of island (spawning a process), and an MPI island.

This project will make use of a modern approach, namely ØMQ, an asynchronous communication layer that it very "light-weight" to define a new type of island (the ØMQ island) able to communicate to similar islands (anywhere in the network) and exchange information (regardless of their belonging or not to an archipelago object). This would allow an easy access to grid-computing architectures where different machines could be running isolated ØMQ island (on the same "topic") or have them embedded in some local archipelago and still have communication (migration) happening via the network.

Implementation Details:

We target a completely broker-less, peer-to-peer, asynchronous message passing solution for migration of solutions between islands.

Computer 1:

isl = island_zeromq(algo, prob, 20, topic="cool_stuff")
isl.evolve(100)

... in the meantime, somewhere else ....

Computer 2:

isl = island_zeromq(algo, prob, 20, topic="cool_stuff")
isl.evolve(100)

In this setup the two islands would be exchanging individuals as they are subscribed to and talk on the same topic (and only in case the exchange makes sense, i.e., same problem dimensions). The island_zeromq could also, on one or both computers, be embedded into a local archipelago containing other (non-ØMQ) islands. Here is an example of a more complex setup, connecting three local archipelagos (I, J, K) with three ØMQ connections (dashed edges):

References:

Mentors:

Clone this wiki locally