Hi Folks,

First of all a special thanks to everyone who attended attend our last workshop session on 24^th September which was focused around the complementary topics of Source Discovery, Publisher Events and Local Data Sharing. Once again I am happy to report it was a very productive session and we covered a lot of ground!

Glenn and I have aggregated some notes from the session which are listed below - any feedback is welcome.

Source Discovery

Implementation Notes:

·         Client applications should be notified when new sources become available. There is also a requirement for client applications to be notified of state changes of sources. At the very minimum “UP” and “DOWN” states but existing OpenMAMA status should be reused where appropriate. We also need to include in this a clear definition of each state e.g. what “STALE” actually means.

·         Other metadata for sources is a requirement. This data should be flexible and consist of a well-known set of key-value pairs. Existing OpenMAMA mechanisms such as messages, field caches or properties could be reused to cache the meta data. Also, the concept of an initial message, with the full set of metadata fields, and then delta messages after that, should be used and closely mirror how regular subscriptions work in OpenMAMA. 

Proposed standard metadata for sources includes:

-          Name

-          State

-          Capability list

-          Transport

-          Data Dictionary

-          Symbology

·         Sources should be able to be queried on a per bridge and transport basis. Specifically, the topology shall be defined as:

Bridge -> Transport -> Source -> Group -> Publisher -> Topic

Note that the concept of Group isn’t currently defined within OpenMAMA, but should provide a useful and logical way of grouping symbols together.

·         An OpenMAMA API implementing the above should be defined, with options of the implementation being an “OpenMAMA native” source discovery solution, or by another market data platform. A flexible solution to this would use OpenMAMA messages as a way of passing source information back and forth between the client application and the middleware bridge implementations.

·         The existing MamaSourceManager object can be extended to store the metadata, provide callbacks and implement the rest of the functionality above. Note that MamaSourceManager will store all sources available to an application and not be tied to one particular bridge. This would be optional to use, and would intercept messages and cache data in the messages from the source discovery implementation.

·         For an OpenMAMA native implementation of source discovery, it should be optional to have a central lookup point on the platform. Also, if implemented using standard OpenMAMA pub/sub semantics, then multi-tier solutions, i.e. Ones with some sort of OpenMAMA proxy, should also work.

·         The basic mechanisms should be based on OpenMAMA messages, with a listener in place to cache the messages and their fields for later query via a new OpenMAMA source manager method. This listener can also propagate the event through to the calling application if it has registered interest in receiving such updates.

·         Transport callbacks may be used to advise the MAMA Source Manager when a source or group of sources have gone offline

Related Notes:

·         Source discovery is particularly useful for screen based application.

·         Topic discovery is out of scope for this implementation.

·         There is an existing SR Labs commercial component called Worldview which provides a limited ability to query sources. Because of dependencies it is not possible to open source this at the moment, however the underlying protocol to serialise and deserialise sources, which is all based on OpenMAMA, could be open sourced and used as a basis for the above.

Local Data Sharing

Implementation Notes:

·         One of the major differences between this and existing forms of OpenMAMA publishing, is the concept of fan in, or multiple active publishers on the same topic.  This currently will not work with OpenMAMA because sequence numbers must be unique and contiguous per topic (per transport). Therefore, the major requirement/change is that there is an option, on a per source basis, that sequence numbers should be considered on a per topic + publisher basis.  The mechanism to provide this variation in data quality validation could leverage the OpenMAMA plugin framework discussed in the previous workshop.

·         Another difference from existing publishing is the optional requirement for acknowledged delivery of messages. There is the possibility that the ACK would be delivered via a bridge or using OpenMAMA native constructs such as an inbox, but the callbacks implemented for Publisher Events should be used to notify the user.

·         Currently when OpenMAMA makes a non-basic subscription it will time out after a configurable number of initial image requests and delays. For local data sharing there will need to be an option to wait indefinitely. 

·         A gateway or proxy server to aggregate published messages should not be a requirement, however it may be used within certain deployments to provide a layer enabling auditing, caching, authentication etc.

·         Again, this should be defined and implemented in a top down manner, with an OpenMAMA API implementing the above features, with pluggable implementations.

Related Notes:

·         Suggestions made to call contributions a “Message / Notice Board” rather than “contributions”

·         Other implementations send such contributed back up through the subscription. As an implementation like this would be excessively middleware dependant, OpenMAMA should not adopt this approach.

·         Roundtrip subscription to contributed data is a simple and foolproof way to ensure that the contributed data has been propagated.

Publisher Events

Implementation Notes

·         General agreement of the design but a desire to get it distributed to the OpenMAMA community in the for of RFC, with a period of 2 weeks for feedback.

·         A bridge interface change is required, so this would be dependent on the new bridge loading mechanism discussed in the first workshop.

·         Guaranteed messaging agreed as out of scope

Tangents and Wish Lists

·         Question asked around lag to be expected for getting OpenMAMA changes into Enterprise MAMA 6. Confirmed it should take around 1-2 months from the cut taken depending on successful testing.

·         There was a query to see if anything could be done to make joining the Linux Foundation easier for large financial institutions.

·         The same mechanism used for source metadata detection could be used to propagate market events such as trading statuses which are currently sent on a per topic basis.

·         A mechanism closely related to source discovery is topic discovery on a per source basis. OpenMAMA currently has a feature called symbol lists which satisfies some of this requirement but there are limitations with it including lack of multiple announce topics. Ideally a new mechanism should be provided which allows a sub set of symbols from a particular source to be queried i.e. symbol groups, with symbols being added, updated or deleted dynamically. There may also be a requirement for this list to be ordered in some way. The logic for this is similar to how order books are handled in OpenMAMDA.

Cheers,

Frank & Glenn