Event Transfer (ET)

The Event Transport (ET) system provides the user with an efficient method for moving bulk data between processes. ET was not designed as a messaging system, for that you need cMsg. The ET system can be visualized as data containers moving around a circular railway track. In this metaphor a producer of data requests an empty container, fills it with data, tags it with metadata describing the contents and places it at the start of the track. Stations along the track are assigned algorithms for testing the metadata and selecting containers of interest. The user has the option of making a station blocking or non-blocking. A container stopped at a blocking station holds up all of the other containers behind it on the track. A data consumer attached to the station processes the data in the container and has the option of either putting the container back on the track, where it proceeds to the next station, or returning the container to the station at the start of the track, effectively discarding the data. In the simple example diagram, a data monitoring consumer attaches to a non-blocking station which samples data of interest. A data recorder attaches to a blocking station and records the content of every container to disk. The container is then returned to the start of the track.

Using these simple concepts complicated data pathways can be constructed. The ET package supports remote consumers connecting to stations over the network allowing load sharing between multiple machines. The ET package is supported in C and Java on Linux and MacOS.