Date   

Re: OpenMAMA Publisher Events RFC

Frank Quinn <f.quinn@...>
 

Hi Folks,

 

Note that as per the notes from the workshops in September, the normal RFC window for response is typically 2 weeks, which we are well past at this point.

 

In saying that, as this is the first RFC since that decision was made, we’ll leave this open until the end of the week (6th November), at which point we will consider this RFC approved.

 

Cheers,

Frank

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Glenn McClements
Sent: 28 October 2015 17:51
To: Alpert, Reed <reed.alpert@...>; openmama-dev@...
Subject: Re: [Openmama-dev] OpenMAMA Publisher Events RFC

 

Folks, please spend some time reviewing this if you get a chance. We’ve gone though quite a few iterations of this so this is the final sign off.

 

Cheers,

Glenn 

 

Glenn McClements, SVP Engineering 

Adelaide Exchange, 24-26 Adelaide Street,  Belfast, UK, BT2 8GD

www.srtechlabs.com

 

SR.LABS Proven High Speed Electronic Trading Solutions

 

From: <openmama-dev-bounces@...> on behalf of "Alpert, Reed via Openmama-dev" <openmama-dev@...>
Reply-To: "Alpert, Reed" <reed.alpert@...>
Date: Friday, 9 October 2015 16:26
To: "openmama-dev@..." <openmama-dev@...>
Cc: CIB Market Data Services Engineers <CIB_Market_Data_Services_Engineers@...>
Subject: [Openmama-dev] OpenMAMA Publisher Events RFC

 

Hi,

 

Attached is an RFC for OpenMAMA Publisher Events.

 

Please send any comments/suggestions/etc.

 

Publisher events provides a way for bridges to provide events back to the application.

An example is permission_denied for a publish, but not limited to that.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: OpenMAMA Publisher Events RFC

Glenn McClements <g.mcclements@...>
 

Folks, please spend some time reviewing this if you get a chance. We’ve gone though quite a few iterations of this so this is the final sign off.

Cheers,
Glenn 


Glenn McClements, SVP Engineering 

Adelaide Exchange, 24-26 Adelaide Street,  Belfast, UK, BT2 8GD

www.srtechlabs.com

 

SR.LABS Proven High Speed Electronic Trading Solutions


From: <openmama-dev-bounces@...> on behalf of "Alpert, Reed via Openmama-dev" <openmama-dev@...>
Reply-To: "Alpert, Reed" <reed.alpert@...>
Date: Friday, 9 October 2015 16:26
To: "openmama-dev@..." <openmama-dev@...>
Cc: CIB Market Data Services Engineers <CIB_Market_Data_Services_Engineers@...>
Subject: [Openmama-dev] OpenMAMA Publisher Events RFC

Hi,

 

Attached is an RFC for OpenMAMA Publisher Events.

 

Please send any comments/suggestions/etc.

 

Publisher events provides a way for bridges to provide events back to the application.

An example is permission_denied for a publish, but not limited to that.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: [Openmama-users] Problem bringing fields from bloomberg professional terminal

Tom Doust
 

Hi Nestor

 

Do you have mappings for these fields in the field map file?

 

Feel free to mail me a copy of the file directly and I will take a look

 

 

Best regards

 

Tom

 

 

TOM DOUST | Head of Consultancy                                                                                                         


TICK42

P: +44 (0) 1628 477444 | M: +44 (0) 7710 479924 | E: tom.doust@... | skype:  tom.doust |  http://www.tick42.com

 

 

From: openmama-users-bounces@... [mailto:openmama-users-bounces@...] On Behalf Of Macrux
Sent: 26 October 2015 15:26
To: openmama-dev@...; openmama-users@...
Subject: [Openmama-users] Problem bringing fields from bloomberg professional terminal

 

Hi there,

I have a problem bringing some fields with the tick42blp bridge, I'm trying to get the data for the price levels (wBestBidPrice1, wBestBidPrice2,..., wBestBidPrice10) as well as the sizes for those levels and also for the ask side, but when the mamalisten example receives the MamaMsg, it doesn't has those fields, even if they are specified as the documentation shows:

mamalistenc -m tick42blp -tport blp_tport -S BlpMktData -s "AAPL US Equity,[BEST_BID1, BEST_BID2,BEST_BID3,BEST_ASK1,BEST_ASK2,BEST_ASK3]"

And so on...

I have the fieldmap.csv file in WOMBAT_PATH, and I'm using Blomberg's professional terminal, in fact, I can bring these fields using the example included in Blomberg Open API, but not through the bridge which brings other fields like wBidPrice, wAskPrice, wBidSize, wAskSize, wEventTime, among others, but not information about levels.

Has anyone had this  issue?

Thanks in advance,

Nestor.


Problem bringing fields from bloomberg professional terminal

macrux
 

Hi there,

I have a problem bringing some fields with the tick42blp bridge, I'm trying to get the data for the price levels (wBestBidPrice1, wBestBidPrice2,..., wBestBidPrice10) as well as the sizes for those levels and also for the ask side, but when the mamalisten example receives the MamaMsg, it doesn't has those fields, even if they are specified as the documentation shows:

mamalistenc -m tick42blp -tport blp_tport -S BlpMktData -s "AAPL US Equity,[BEST_BID1, BEST_BID2,BEST_BID3,BEST_ASK1,BEST_ASK2,BEST_ASK3]"

And so on...

I have the fieldmap.csv file in WOMBAT_PATH, and I'm using Blomberg's professional terminal, in fact, I can bring these fields using the example included in Blomberg Open API, but not through the bridge which brings other fields like wBidPrice, wAskPrice, wBidSize, wAskSize, wEventTime, among others, but not information about levels.

Has anyone had this  issue?

Thanks in advance,

Nestor.


Notes from Workshop Session 24-Sep-2015

Frank Quinn <f.quinn@...>
 

Hi Folks,

 

First of all a special thanks to everyone who attended attend our last workshop session on 24th September which was focused around the complementary topics of Source Discovery, Publisher Events and Local Data Sharing. Once again I am happy to report it was a very productive session and we covered a lot of ground!

 

Glenn and I have aggregated some notes from the session which are listed below - any feedback is welcome.

 

Source Discovery

 

Implementation Notes:

 

·         Client applications should be notified when new sources become available. There is also a requirement for client applications to be notified of state changes of sources. At the very minimum “UP” and “DOWN” states but existing OpenMAMA status should be reused where appropriate. We also need to include in this a clear definition of each state e.g. what “STALE” actually means.

 

·         Other metadata for sources is a requirement. This data should be flexible and consist of a well-known set of key-value pairs. Existing OpenMAMA mechanisms such as messages, field caches or properties could be reused to cache the meta data. Also, the concept of an initial message, with the full set of metadata fields, and then delta messages after that, should be used and closely mirror how regular subscriptions work in OpenMAMA. 

 

Proposed standard metadata for sources includes:

 

-          Name

-          State

-          Capability list

-          Transport

-          Data Dictionary

-          Symbology

 

·         Sources should be able to be queried on a per bridge and transport basis. Specifically, the topology shall be defined as:

 

Bridge -> Transport -> Source -> Group -> Publisher -> Topic

 

Note that the concept of Group isn’t currently defined within OpenMAMA, but should provide a useful and logical way of grouping symbols together.

 

·         An OpenMAMA API implementing the above should be defined, with options of the implementation being an “OpenMAMA native” source discovery solution, or by another market data platform. A flexible solution to this would use OpenMAMA messages as a way of passing source information back and forth between the client application and the middleware bridge implementations.

 

·         The existing MamaSourceManager object can be extended to store the metadata, provide callbacks and implement the rest of the functionality above. Note that MamaSourceManager will store all sources available to an application and not be tied to one particular bridge. This would be optional to use, and would intercept messages and cache data in the messages from the source discovery implementation.

 

·         For an OpenMAMA native implementation of source discovery, it should be optional to have a central lookup point on the platform. Also, if implemented using standard OpenMAMA pub/sub semantics, then multi-tier solutions, i.e. Ones with some sort of OpenMAMA proxy, should also work.

 

·         The basic mechanisms should be based on OpenMAMA messages, with a listener in place to cache the messages and their fields for later query via a new OpenMAMA source manager method. This listener can also propagate the event through to the calling application if it has registered interest in receiving such updates.

 

·         Transport callbacks may be used to advise the MAMA Source Manager when a source or group of sources have gone offline

 

 

Related Notes:

 

·         Source discovery is particularly useful for screen based application.

 

·         Topic discovery is out of scope for this implementation.

 

·         There is an existing SR Labs commercial component called Worldview which provides a limited ability to query sources. Because of dependencies it is not possible to open source this at the moment, however the underlying protocol to serialise and deserialise sources, which is all based on OpenMAMA, could be open sourced and used as a basis for the above.

 

 

Local Data Sharing

 

Implementation Notes:

 

·         One of the major differences between this and existing forms of OpenMAMA publishing, is the concept of fan in, or multiple active publishers on the same topic.  This currently will not work with OpenMAMA because sequence numbers must be unique and contiguous per topic (per transport). Therefore, the major requirement/change is that there is an option, on a per source basis, that sequence numbers should be considered on a per topic + publisher basis.  The mechanism to provide this variation in data quality validation could leverage the OpenMAMA plugin framework discussed in the previous workshop.

 

·         Another difference from existing publishing is the optional requirement for acknowledged delivery of messages. There is the possibility that the ACK would be delivered via a bridge or using OpenMAMA native constructs such as an inbox, but the callbacks implemented for Publisher Events should be used to notify the user.

 

·         Currently when OpenMAMA makes a non-basic subscription it will time out after a configurable number of initial image requests and delays. For local data sharing there will need to be an option to wait indefinitely. 

 

·         A gateway or proxy server to aggregate published messages should not be a requirement, however it may be used within certain deployments to provide a layer enabling auditing, caching, authentication etc.

 

·         Again, this should be defined and implemented in a top down manner, with an OpenMAMA API implementing the above features, with pluggable implementations.

 

Related Notes:

 

·         Suggestions made to call contributions a “Message / Notice Board” rather than “contributions”

 

·         Other implementations send such contributed back up through the subscription. As an implementation like this would be excessively middleware dependant, OpenMAMA should not adopt this approach.

 

·         Roundtrip subscription to contributed data is a simple and foolproof way to ensure that the contributed data has been propagated.

 

 

Publisher Events

 

Implementation Notes

 

·         General agreement of the design but a desire to get it distributed to the OpenMAMA community in the for of RFC, with a period of 2 weeks for feedback.

 

·         A bridge interface change is required, so this would be dependent on the new bridge loading mechanism discussed in the first workshop.

 

·         Guaranteed messaging agreed as out of scope

 

 

Tangents and Wish Lists

 

·         Question asked around lag to be expected for getting OpenMAMA changes into Enterprise MAMA 6. Confirmed it should take around 1-2 months from the cut taken depending on successful testing.

 

·         There was a query to see if anything could be done to make joining the Linux Foundation easier for large financial institutions.

 

·         The same mechanism used for source metadata detection could be used to propagate market events such as trading statuses which are currently sent on a per topic basis.

 

·         A mechanism closely related to source discovery is topic discovery on a per source basis. OpenMAMA currently has a feature called symbol lists which satisfies some of this requirement but there are limitations with it including lack of multiple announce topics. Ideally a new mechanism should be provided which allows a sub set of symbols from a particular source to be queried i.e. symbol groups, with symbols being added, updated or deleted dynamically. There may also be a requirement for this list to be ordered in some way. The logic for this is similar to how order books are handled in OpenMAMDA.

 

 

Cheers,

Frank & Glenn

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: Inconsistencies/bugs with OpenMama

Sridharan, Venkatesan <Venkatesan.Sridharan@...>
 

Thanks Frank. Have raised a github issue for each point.

-----Original Message-----
From: Frank Quinn [mailto:f.quinn@...]
Sent: Monday, October 12, 2015 3:22 PM
To: Sridharan, Venkatesan <Venkatesan.Sridharan@...>; openmama-dev@...
Subject: RE: Inconsistencies/bugs with OpenMama

Hi Venkatesan,

Glad you're getting something from the project and appreciate your efforts around this!

At first look, I agree in principle with a lot of the below, but I feel like it could get unruly trying to cover everything in a single thread. With that in mind, could you raise a github issue for each of the below so we can track each point in detail independently and make sure nothing slips through the net?

Also note there was also a series of patches submitted in the past to cover datetime / price vectors which may address at least some of your concerns, though we were holding off on implementing any bridge-related changes until dynamic bridge loading is completed:

http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001002.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001003.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001004.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001005.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001006.html

Cheers,
Frank

-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Sridharan, Venkatesan
Sent: 12 October 2015 01:59
To: openmama-dev@...
Subject: [Openmama-dev] Inconsistencies/bugs with OpenMama

Hi Folks,

This is my first post here. First off, thanks for your efforts on OpenMama, the code is clean, well commented and the documentation is great too!

We've been using OpenMama for a few months now and have written bridges for TibRv and Qpid (using qpid c++ API). I am talking to my company about opensourcing them. I came across a number of seeming errors/inconsistencies and wanted to get the community's take. I can provide a patch to fix the issues if we agree that they indeed are bugs and that they do need to be fixed.

* xxxxBridge_getDefaultPayloadId (char ***name, char **id) The function retrieves the payload id which are const and hence params must be const qualified -> (const char ***name, const char **id).

* xxxPayload_getMsg(const msgPayload msg, const char* name, mama_fid_t fid, msgPayload* result) We are passing in a const msg and extracting its field. so, last arg should be const msgPayload*

* loadbridge uses property file before open() During creation of the default queue, properties subsystem is accessed and this always prints an error/warning if we use a custom properties file.

* bridgeMamaPublisherSendReplyToInbox => request is void* instead of mamaMsg typedef mama_status (*bridgeMamaPublisher_sendReplyToInbox)(publisherBridge publisher,
=>void * request, <=
mamaMsg reply);
This is needless as the request is a mamaMsg


* throw in dtor!
MamaTransport::~MamaTransport (void)
{
if (mPimpl)
{
if ( mamaInternal_getCatchCallbackExceptions() && mPimpl->mCallback)
{
delete mPimpl->mCallback;
}
if (mPimpl->mDeleteCTransport)
{
=>mamaTry (mamaTransport_destroy (mTransport)); <=
}
delete mPimpl;
mPimpl = NULL;
}
}

* Bridges are expected to allocate memory for some functions (toString, getReplyHandle etc) but there is no hook to free the memory. Since each bridge is allowed to use custom memory allocation, there should be a hook from mamaMsg_freeString back to the bridge. Also, currently mamaMsg_freeString is a no-op

* mamaMsg_copy does not check for self copy

* Some fields are not initialized in INITIALIZE_BRIDGE () macro. Eg, mamaBridgeImpl::mInternalDispatcher is not set to NULL on bridge creation. Got a crash on this one.

* As per comments and documentation, mamaQueue_timedDispatch is supposed to keep dispatching until the given timeout has elapsed. But, the current bridge implementations use wombatQueue_timedDispatch which dispatches only one message. So, EITHER wombatQueue_timedDispatch should be changed to keep dispatching till timeout OR another helper function should be provided to do this (as this functionality is used across bridges)

* msgPayload_unSerialize prototype incorrect (takes in void** instead of void*) typedef mama_status
(*msgPayload_unSerialize) (const msgPayload msg,
=>const void** buffer, <=
mama_size_t bufferLength);
This is an input buffer and the only function using this is mamaMsg_setNewBuffer which calls the above function and passes a void* buffer as a void**

* Incorrect test in MamaMsg::setNewBuffer ():
bool MamaMsg::setNewBuffer (
void* buffer,
mama_size_t size)
{
int result = 0;
// should be MAMA_STATUS_OK == ...
=> if (MAMA_STATUS_OK != (mamaTry (mamaMsg_setNewBuffer ( <=
mMsg, buffer, size))))
{
return result;
}
return result != 0;
}

* These functions are not defined, I encountered this when trying to build python, perl interfaces with swig. A default implementation returning MAMA_STATUS_NOT_IMPLEMENTED or nullptr should be added mamaMsg_updateVectorTime, mamaMsg_updateVectorPrice, mamaDispatcher_getQueue, mama_forceLogVaWithPrefix, mamaMsgField_getVectorBool, mama_logPolicyToString, mama_tryStringToLogPolicy, mamaMsgIterator_end

* Inconsistent prototypes :
mama_status
mamaMsg_updateVectorPrice (
mamaMsg msg,
const char* fname,
mama_fid_t fid,
//should be const mamaPrice priceList[]
=> const mamaPrice* priceList[], =<
mama_size_t numElements);

msgPayload_getPrice, msgPayload_getDateTime should take pointer to result as last parameter
Eg:
typedef mama_status
(*msgPayload_getDateTime) (const msgPayload msg,
const char* name,
mama_fid_t fid,
//should be mamaDateTime * result
=> mamaDateTime result); <=

* There are some pesky macros defined in port.h (close, sleep etc) which causes problems when using other libraries (eg boost). We should be defining them as inline functions instead of using macros


Cheers,
Venkatesan




Confidentiality Note: This e-mail and any attachments are confidential and may be protected by legal privilege. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this e-mail or any attachment is prohibited. If you have received this e-mail in error, please notify us immediately by returning it to the sender and delete this copy from your system. Thank you for your cooperation.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC



Confidentiality Note: This e-mail and any attachments are confidential and may
be protected by legal privilege. If you are not the intended recipient, be aware
that any disclosure, copying, distribution or use of this e-mail or any
attachment is prohibited. If you have received this e-mail in error, please
notify us immediately by returning it to the sender and delete this copy from
your system. Thank you for your cooperation.


Re: Inconsistencies/bugs with OpenMama

Frank Quinn <f.quinn@...>
 

Hi Venkatesan,

Glad you're getting something from the project and appreciate your efforts around this!

At first look, I agree in principle with a lot of the below, but I feel like it could get unruly trying to cover everything in a single thread. With that in mind, could you raise a github issue for each of the below so we can track each point in detail independently and make sure nothing slips through the net?

Also note there was also a series of patches submitted in the past to cover datetime / price vectors which may address at least some of your concerns, though we were holding off on implementing any bridge-related changes until dynamic bridge loading is completed:

http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001002.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001003.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001004.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001005.html
http://lists.celinuxforum.org/pipermail/openmama-dev/2014-January/001006.html

Cheers,
Frank

-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Sridharan, Venkatesan
Sent: 12 October 2015 01:59
To: openmama-dev@...
Subject: [Openmama-dev] Inconsistencies/bugs with OpenMama

Hi Folks,

This is my first post here. First off, thanks for your efforts on OpenMama, the code is clean, well commented and the documentation is great too!

We've been using OpenMama for a few months now and have written bridges for TibRv and Qpid (using qpid c++ API). I am talking to my company about opensourcing them. I came across a number of seeming errors/inconsistencies and wanted to get the community's take. I can provide a patch to fix the issues if we agree that they indeed are bugs and that they do need to be fixed.

* xxxxBridge_getDefaultPayloadId (char ***name, char **id) The function retrieves the payload id which are const and hence params must be const qualified -> (const char ***name, const char **id).

* xxxPayload_getMsg(const msgPayload msg, const char* name, mama_fid_t fid, msgPayload* result) We are passing in a const msg and extracting its field. so, last arg should be const msgPayload*

* loadbridge uses property file before open() During creation of the default queue, properties subsystem is accessed and this always prints an error/warning if we use a custom properties file.

* bridgeMamaPublisherSendReplyToInbox => request is void* instead of mamaMsg typedef mama_status (*bridgeMamaPublisher_sendReplyToInbox)(publisherBridge publisher,
=>void * request, <=
mamaMsg reply);
This is needless as the request is a mamaMsg


* throw in dtor!
MamaTransport::~MamaTransport (void)
{
if (mPimpl)
{
if ( mamaInternal_getCatchCallbackExceptions() && mPimpl->mCallback)
{
delete mPimpl->mCallback;
}
if (mPimpl->mDeleteCTransport)
{
=>mamaTry (mamaTransport_destroy (mTransport)); <=
}
delete mPimpl;
mPimpl = NULL;
}
}

* Bridges are expected to allocate memory for some functions (toString, getReplyHandle etc) but there is no hook to free the memory. Since each bridge is allowed to use custom memory allocation, there should be a hook from mamaMsg_freeString back to the bridge. Also, currently mamaMsg_freeString is a no-op

* mamaMsg_copy does not check for self copy

* Some fields are not initialized in INITIALIZE_BRIDGE () macro. Eg, mamaBridgeImpl::mInternalDispatcher is not set to NULL on bridge creation. Got a crash on this one.

* As per comments and documentation, mamaQueue_timedDispatch is supposed to keep dispatching until the given timeout has elapsed. But, the current bridge implementations use wombatQueue_timedDispatch which dispatches only one message. So, EITHER wombatQueue_timedDispatch should be changed to keep dispatching till timeout OR another helper function should be provided to do this (as this functionality is used across bridges)

* msgPayload_unSerialize prototype incorrect (takes in void** instead of void*) typedef mama_status
(*msgPayload_unSerialize) (const msgPayload msg,
=>const void** buffer, <=
mama_size_t bufferLength);
This is an input buffer and the only function using this is mamaMsg_setNewBuffer which calls the above function and passes a void* buffer as a void**

* Incorrect test in MamaMsg::setNewBuffer ():
bool MamaMsg::setNewBuffer (
void* buffer,
mama_size_t size)
{
int result = 0;
// should be MAMA_STATUS_OK == ...
=> if (MAMA_STATUS_OK != (mamaTry (mamaMsg_setNewBuffer ( <=
mMsg, buffer, size))))
{
return result;
}
return result != 0;
}

* These functions are not defined, I encountered this when trying to build python, perl interfaces with swig. A default implementation returning MAMA_STATUS_NOT_IMPLEMENTED or nullptr should be added mamaMsg_updateVectorTime, mamaMsg_updateVectorPrice, mamaDispatcher_getQueue, mama_forceLogVaWithPrefix, mamaMsgField_getVectorBool, mama_logPolicyToString, mama_tryStringToLogPolicy, mamaMsgIterator_end

* Inconsistent prototypes :
mama_status
mamaMsg_updateVectorPrice (
mamaMsg msg,
const char* fname,
mama_fid_t fid,
//should be const mamaPrice priceList[]
=> const mamaPrice* priceList[], =<
mama_size_t numElements);

msgPayload_getPrice, msgPayload_getDateTime should take pointer to result as last parameter
Eg:
typedef mama_status
(*msgPayload_getDateTime) (const msgPayload msg,
const char* name,
mama_fid_t fid,
//should be mamaDateTime * result
=> mamaDateTime result); <=

* There are some pesky macros defined in port.h (close, sleep etc) which causes problems when using other libraries (eg boost). We should be defining them as inline functions instead of using macros


Cheers,
Venkatesan




Confidentiality Note: This e-mail and any attachments are confidential and may be protected by legal privilege. If you are not the intended recipient, be aware that any disclosure, copying, distribution or use of this e-mail or any attachment is prohibited. If you have received this e-mail in error, please notify us immediately by returning it to the sender and delete this copy from your system. Thank you for your cooperation.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Inconsistencies/bugs with OpenMama

Sridharan, Venkatesan <Venkatesan.Sridharan@...>
 

Hi Folks,

This is my first post here. First off, thanks for your efforts on OpenMama, the code is clean, well commented and the documentation is great too!

We've been using OpenMama for a few months now and have written bridges for TibRv and Qpid (using qpid c++ API). I am talking to my company about opensourcing them. I came across a number of seeming errors/inconsistencies and wanted to get the community's take. I can provide a patch to fix the issues if we agree that they indeed are bugs and that they do need to be fixed.

* xxxxBridge_getDefaultPayloadId (char ***name, char **id)
The function retrieves the payload id which are const and hence params must be const qualified -> (const char ***name, const char **id).

* xxxPayload_getMsg(const msgPayload msg, const char* name, mama_fid_t fid, msgPayload* result)
We are passing in a const msg and extracting its field. so, last arg should be const msgPayload*

* loadbridge uses property file before open()
During creation of the default queue, properties subsystem is accessed and this always prints an error/warning if we use a custom properties file.

* bridgeMamaPublisherSendReplyToInbox => request is void* instead of mamaMsg
typedef mama_status
(*bridgeMamaPublisher_sendReplyToInbox)(publisherBridge publisher,
=>void * request, <=
mamaMsg reply);
This is needless as the request is a mamaMsg


* throw in dtor!
MamaTransport::~MamaTransport (void)
{
if (mPimpl)
{
if ( mamaInternal_getCatchCallbackExceptions() && mPimpl->mCallback)
{
delete mPimpl->mCallback;
}
if (mPimpl->mDeleteCTransport)
{
=>mamaTry (mamaTransport_destroy (mTransport)); <=
}
delete mPimpl;
mPimpl = NULL;
}
}

* Bridges are expected to allocate memory for some functions (toString, getReplyHandle etc) but there is no hook to free the memory. Since each bridge is allowed to use custom memory allocation, there should be a hook from mamaMsg_freeString back to the bridge. Also, currently mamaMsg_freeString is a no-op

* mamaMsg_copy does not check for self copy

* Some fields are not initialized in INITIALIZE_BRIDGE () macro. Eg, mamaBridgeImpl::mInternalDispatcher is not set to NULL on bridge creation. Got a crash on this one.

* As per comments and documentation, mamaQueue_timedDispatch is supposed to keep dispatching until the given timeout has elapsed. But, the current bridge implementations use
wombatQueue_timedDispatch which dispatches only one message. So, EITHER wombatQueue_timedDispatch should be changed to keep dispatching till timeout OR another helper function
should be provided to do this (as this functionality is used across bridges)

* msgPayload_unSerialize prototype incorrect (takes in void** instead of void*)
typedef mama_status
(*msgPayload_unSerialize) (const msgPayload msg,
=>const void** buffer, <=
mama_size_t bufferLength);
This is an input buffer and the only function using this is mamaMsg_setNewBuffer which calls the above function and passes a void* buffer as a void**

* Incorrect test in MamaMsg::setNewBuffer ():
bool MamaMsg::setNewBuffer (
void* buffer,
mama_size_t size)
{
int result = 0;
// should be MAMA_STATUS_OK == ...
=> if (MAMA_STATUS_OK != (mamaTry (mamaMsg_setNewBuffer ( <=
mMsg, buffer, size))))
{
return result;
}
return result != 0;
}

* These functions are not defined, I encountered this when trying to build python, perl interfaces with swig. A default implementation returning MAMA_STATUS_NOT_IMPLEMENTED or nullptr should be added
mamaMsg_updateVectorTime, mamaMsg_updateVectorPrice, mamaDispatcher_getQueue, mama_forceLogVaWithPrefix, mamaMsgField_getVectorBool, mama_logPolicyToString, mama_tryStringToLogPolicy, mamaMsgIterator_end

* Inconsistent prototypes :
mama_status
mamaMsg_updateVectorPrice (
mamaMsg msg,
const char* fname,
mama_fid_t fid,
//should be const mamaPrice priceList[]
=> const mamaPrice* priceList[], =<
mama_size_t numElements);

msgPayload_getPrice, msgPayload_getDateTime should take pointer to result as last parameter
Eg:
typedef mama_status
(*msgPayload_getDateTime) (const msgPayload msg,
const char* name,
mama_fid_t fid,
//should be mamaDateTime * result
=> mamaDateTime result); <=

* There are some pesky macros defined in port.h (close, sleep etc) which causes problems when using other libraries (eg boost). We should be defining them as inline functions instead of using macros


Cheers,
Venkatesan




Confidentiality Note: This e-mail and any attachments are confidential and may
be protected by legal privilege. If you are not the intended recipient, be aware
that any disclosure, copying, distribution or use of this e-mail or any
attachment is prohibited. If you have received this e-mail in error, please
notify us immediately by returning it to the sender and delete this copy from
your system. Thank you for your cooperation.


Re: Proposed end of support for Microsoft Visual Studio 2005 (VC8)

Alpert, Reed <reed.alpert@...>
 

HI,

 

We don’t use VS2005, but do build with 2012 and soon 2013.

We also have some interest in 2008, but haven’t gone down that road yet.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Frank Quinn
Sent: Saturday, October 10, 2015 7:45 AM
To: openmama-dev
Subject: [Openmama-dev] Proposed end of support for Microsoft Visual Studio 2005 (VC8)

 

Hi Folks,

 

Does anyone have any objections to ending support for Visual Studio 2005 for OpenMAMA? There are several reasons for wanting to do this:

 

1. Visual Studio 2005 is 10 years old

2. It is out of mainstream support from Microsoft

3. We are limited to Visual Studio 2005 Solution file syntax to maintain backwards compatibility

4. We suspect none of our users or developers actually using it

 

Cheers,

Frank

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Proposed end of support for Microsoft Visual Studio 2005 (VC8)

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

Does anyone have any objections to ending support for Visual Studio 2005 for OpenMAMA? There are several reasons for wanting to do this:

1. Visual Studio 2005 is 10 years old
2. It is out of mainstream support from Microsoft
3. We are limited to Visual Studio 2005 Solution file syntax to maintain backwards compatibility
4. We suspect none of our users or developers actually using it

Cheers,
Frank


OpenMAMA Publisher Events RFC

Alpert, Reed <reed.alpert@...>
 

Hi,

 

Attached is an RFC for OpenMAMA Publisher Events.

 

Please send any comments/suggestions/etc.

 

Publisher events provides a way for bridges to provide events back to the application.

An example is permission_denied for a publish, but not limited to that.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Re: [Openmama-users] Capturec and tick42blp bridge

macrux
 

Hi,

I recorded just a few seconds of the AAPL US Equity from Bloomberg yesterday (I sent a bigger capture in my first message). The playback is attached in the file 1325551007.playback. The log of the capturerec is attached in the file capturerec-aapl.log, and for the capturereplayc two logs are attached: one in the file capturereplayc-aapl-ubuntu.log and the other in the file capturereplayc-aapl-windows.log. It was done using the openmama 2.3.1 release in windows using the bridge tick42blp to connect to the bloomberg terminal. I used the same parameters mentioned before:

capturec -m tick42blp -MS BlpMktData:blp_tport:symbols.sym

Where symbols.sym is a file containing only one symbol: "AAPL US Equity"

It starts to recording a file. After a while I stop the process with CTLR+C. Then I try to use the capturereplayc to play the file:

capturereplayc -S TEST -m qpid -tport pub -dictionary data/dictionaries/data.dict -f 1325551007.playback

The fact is that capturereplayc cannot even start when I try to reproduce the file in ubuntu (in this machine I have the openmama 2.3.3 release), it happens a "segmentation fault" as you can see in the ubuntu log. If I try to reproduce it in a windows machine (with windows 10 and openmama 2.3.1), then the capturereplayc starts listening and when I start a mamalistenc, it stops abruptly, but the verbose log of capturereplayc doesn't say anything. The same happens if I do it in the machine where I have the bloomberg terminal (with windows 7 machine and openmama 2.3.1), so I'm stuck in this problem.


Thanks in advance,

Nestor.


On 2 October 2015 at 02:45, Damian Maguire <d.maguire@...> wrote:
Hey Nestor, 

Do you have a verbose log file from capturereplay up to the point at which it stops? And does the process simply hang, or is it crashing? I’ve never tried capturereplay with the Bloomberg bridge, though Tom and co will likely have a better idea of the issues there, but if it’s a bug in capturereplay we should be able to get a better idea of where from the logs.

Cheers, 

D

Damian Maguire
Senior Sales Engineer

SR.LABS Proven High Speed Electronic Trading Solutions

Adelaide Exchange | 24-26 Adelaide Street | Belfast | UK |BT2 8GD

d.maguire@...  

+44 7835 844770



From: <openmama-users-bounces@...> on behalf of Macrux
Date: Thursday, October 1, 2015 at 10:23 PM
To: "openmama-dev@...", "openmama-users@..."
Subject: [Openmama-users] Capturec and tick42blp bridge

Hi folks,

As part of the work I'm doing with bloomberg and openmama, I'm trying to record the data coming from bloomberg through the bridge, I'm using this instruction:

capturec -m tick42blp -MS BlpMktData:blp_tport:symbols.sym


Where symbols.sym is a file containing only one symbol: "AAPL US Equity"

It starts to recording a file. After a while I stop the process with CTLR+C. Then I try to use the capturereplayc to play the file:

capturereplayc -S TEST -m qpid -tport pub -dictionary bbdict.txt -f 1221191001.playback

But the capturereplayc stops abruptly. The same happens if I try to use the dictionary provided in the openmama release.

So my question is, how could I record the data coming from bloomberg and then play them with the capturereplayc tool.

Thanks for your help. I appreciate your patience, I'm really new with openama.

Best regards,



PD: I attached the playback file recorded today just in case someone wants to test it.




The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: Open MAMA DateTime object

Alpert, Reed <reed.alpert@...>
 

Hi,

 

Some notes on the datetime 2038 & 2106 issues.

 

·         Today

o   OM holds seconds in unsigned 4 bytes (as part of 8 byte struct).

§  4,294,967,295 secs, ~136 years.

o   This is the same for 32- and 64-bit builds.

o   Max stored date is Jan 28, 2106.

§  32 bits.

§  4,294,967,295 seconds.

§  49,710.26961805556 days.

§  136 years.

·         Why

o   Some bonds have issue dates before 1970.

§  Perpetual bonds issued a long time ago.

§  Don’t think we can fix this since Linux/C standard is 1970.

o   Some bonds mature after 2106.

§  This is fixed with expanding the seconds storage.

 

·         However

o   Windows has fixed the 2038 issue (VS8 or later).

o   Linux it is still there:

§  On 32-bit build time_t is 4 bytes.

§  On 64-bit build time_t is 8 bytes.

§  Calling gmtime_r gives: (used in utcTm, which is called ~7 times in datetime.c):

·         2038 for 32-bit.

·         2106 for 64-bit.

 

·         Possible solution:

o   There are 4 bits available in the data struct, but they are not contiguous to existing seconds.

o   Adding 4 more bits goes to the year 4147.

§  97,982,352,000 seconds from 1970.

o   Need to redefine the masks so the newly-used 4 bits are contiguous.

o   Need full rebuild: OM, bridges, apps.

o   Update methods to use u64 for seconds.

o   Potential break of existing apps.

§  But it may just be a cast to u32 or change to use u64.

§  Or truncate will occur depending upon the compiler; this causes the date to stop at the same as now.

o   Upsides:

§  Keeps datetime 4-bytes for better performance.

o   Downsides:

§  All bits in struct are used, no room for any other expansion.

§  Big-bang, the bridge, OM, and apps all need to rebuild at once.

·         But that is mostly common with a major release.

§  Capture data will not work if stored as binary.

 

Thanks,

 

Reed.

 

________________________________________

Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

From: Frank Quinn [mailto:f.quinn@...]

Sent: Tuesday, September 08, 2015 9:25 AM

To: openmama-dev@...

Cc: Glenn McClements; Alpert, Reed; Phelan, Nigel

Subject: RE: Open MAMA DateTime object

 

Hi Folks,

Please see thread below - interesting point raised about date time and the Year 2038 problem. I’ve weighed in with my thoughts, further opinions welcome…

Cheers,

Frank

From: Frank Quinn

Sent: 08 September 2015 14:10

To: 'Phelan, Nigel' <nigel.phelan@...>

Cc: Glenn McClements <g.mcclements@...>; Alpert, Reed <reed.alpert@...>

Subject: RE: Open MAMA DateTime object

 

Hi Nigel,

 

In short, yes – your understanding aligns with mine. Using a uint32 rather than an int32 has doubled our date range, but sooner or later it looks like we will need to face this.

 

It’s interesting because when we discussed this last week, I had the uint64 bitmap in my head and I had a pretty good idea that there were a few bits going spare that we could use if we needed to (even if they weren’t sequential), but I had overlooked the function prototypes themselves – yes some of those would need changed or new implementations added to use 64 bit values.

There is a nibble going spare to store some additional data, and we should be able to use a bit in the hints for letting MAMA know which date format to use for encode / decode… so yes at a glance I think we can extend in a backwards compatible way with some creative bit shifting.

 

I’m in two minds about whether it’s best in OpenMAMA 6 or OpenMAMA 7 – I’m sure Glenn will weigh in with an opinion here, but the only way I can think of that we can add it to OpenMAMA 6 is if we create new MAMA function calls and don’t modify any existing ones (e.g. mamaDateTime_setEpochTimeEx), and then deprecate that method when OpenMAMA 7 is released. If I had to choose I would prefer to just go big bang with OpenMAMA 7 rather than create an interim method in OpenMAMA 6 which is destined to be retired in the next major release anyway, but it will all depend on how urgent a requirement this is for Year 2106+.

 

Cheers,

Frank

 

From: Phelan, Nigel [mailto:nigel.phelan@...]

Sent: 07 September 2015 14:17

To: Frank Quinn <f.quinn@...>

Cc: Glenn McClements <g.mcclements@...>; Alpert, Reed <reed.alpert@...>

Subject: Open MAMA DateTime object

 

Hi Frank – following on from our discussion last week I spoke to Reed in our New York team, and he thought:

•         The range of dates you can store in a MamaDateTime is from 1970 to 2106 (further out than I thought, but still a problem for us).

•         The underlying representation has 4 bytes of seconds and 20 bits of microseconds; he thought the type could be extended in a backward compatible way, but that new methods might be needed to handle an extended range as some of them currently return a uint_32 seconds value.

How does that fit with your observations?  Do you think this is something that could be addressed in Open MAMA 6, or are we going to have to defer it to Open MAMA 7?

 

Nigel

 

________________________________________

Nigel Phelan | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 25 Bank Street, London E14 5JP| T: +44-(0)20-7134-9558 | nigel.phelan@...

 

This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email

 

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Re: [Openmama-users] Capturec and tick42blp bridge

Damian Maguire <d.maguire@...>
 

Hey Nestor, 

Do you have a verbose log file from capturereplay up to the point at which it stops? And does the process simply hang, or is it crashing? I’ve never tried capturereplay with the Bloomberg bridge, though Tom and co will likely have a better idea of the issues there, but if it’s a bug in capturereplay we should be able to get a better idea of where from the logs.

Cheers, 

D

Damian Maguire
Senior Sales Engineer

SR.LABS Proven High Speed Electronic Trading Solutions

Adelaide Exchange | 24-26 Adelaide Street | Belfast | UK |BT2 8GD

d.maguire@...  

+44 7835 844770



From: <openmama-users-bounces@...> on behalf of Macrux
Date: Thursday, October 1, 2015 at 10:23 PM
To: "openmama-dev@...", "openmama-users@..."
Subject: [Openmama-users] Capturec and tick42blp bridge

Hi folks,

As part of the work I'm doing with bloomberg and openmama, I'm trying to record the data coming from bloomberg through the bridge, I'm using this instruction:

capturec -m tick42blp -MS BlpMktData:blp_tport:symbols.sym


Where symbols.sym is a file containing only one symbol: "AAPL US Equity"

It starts to recording a file. After a while I stop the process with CTLR+C. Then I try to use the capturereplayc to play the file:

capturereplayc -S TEST -m qpid -tport pub -dictionary bbdict.txt -f 1221191001.playback

But the capturereplayc stops abruptly. The same happens if I try to use the dictionary provided in the openmama release.

So my question is, how could I record the data coming from bloomberg and then play them with the capturereplayc tool.

Thanks for your help. I appreciate your patience, I'm really new with openama.

Best regards,



PD: I attached the playback file recorded today just in case someone wants to test it.




The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Capturec and tick42blp bridge

macrux
 

Hi folks,

As part of the work I'm doing with bloomberg and openmama, I'm trying to record the data coming from bloomberg through the bridge, I'm using this instruction:

capturec -m tick42blp -MS BlpMktData:blp_tport:symbols.sym


Where symbols.sym is a file containing only one symbol: "AAPL US Equity"

It starts to recording a file. After a while I stop the process with CTLR+C. Then I try to use the capturereplayc to play the file:

capturereplayc -S TEST -m qpid -tport pub -dictionary bbdict.txt -f 1221191001.playback

But the capturereplayc stops abruptly. The same happens if I try to use the dictionary provided in the openmama release.

So my question is, how could I record the data coming from bloomberg and then play them with the capturereplayc tool.

Thanks for your help. I appreciate your patience, I'm really new with openama.

Best regards,



PD: I attached the playback file recorded today just in case someone wants to test it.




Re: [Openmama-users] Build Mamda Book from Mama Subscription

Tom Doust
 

Hi Nestor, Glenn

 

Order book handling is outside the scope of the current version of our BLP bridge.

 

In our TREP bridge we take OMM MarketByPrice and MarketByOrder messages and build mamda book messages from them. Conceptually, it’s quite simple because the OMM and the Mamda models are almost identical. In practice, it’s a bit messy because in order to key the price levels correctly you have to maintain some state between updates.

 

I think generating the messages in the bridge is the right way to go.

 

From what I recall of the Open Bloomberg  API there are several, source dependent representations, some of which are similar to the TR OMM model.

 

The TREP bridge code described above is in our  open source release in git hub if you want to take a look. A lot of the code we used to build the mama book messages is based on code doing the same thing in the mamda book publisher example that ships with OpenMAMA.


Apologies that this doesn’t really give any easy answers but hopefully will point you in the right direction.

 

Best regards

 

Tom

 

From: openmama-users-bounces@... [mailto:openmama-users-bounces@...] On Behalf Of Glenn McClements
Sent: 01 October 2015 08:23
To: Macrux <kmacrux@...>; openmama-dev@...; openmama-users@...
Subject: Re: [Openmama-users] Build Mamda Book from Mama Subscription

 

Hi Nestor,

Perhaps one of the tick42blp bridge developers could help out, but one thing that may help is a orderbook message spec for OpenMAMDA. This isn't currently published on the OpenMAMA website but I see when we can get it out.

 

Regards,

Glenn 

 

Glenn McClements, SVP Engineering 

Adelaide Exchange | 24-26 Adelaide Street | Belfast | UK | BT2 8GD

www.srtechlabs.com

 

SR.LABS Proven High Speed Electronic Trading Solutions

 


From: openmama-users-bounces@... <openmama-users-bounces@...> on behalf of Macrux <kmacrux@...>
Sent: 30 September 2015 22:39
To: openmama-dev@...; openmama-users@...
Subject: [Openmama-users] Build Mamda Book from Mama Subscription

 

Hi there,

I'm wondering if there is any example in which data received through OpenMAMA is transformed into a Mamda Order book. For example, I have a subcription to Bloomberg through the tick42blp bridge but I need to transform the data received into a Mamda Order Book because I have a client (other program) which works with OpenMamda order books, that's the point.


Thanks in advance for any help you could give about that case.

Regards,

Nestor.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: [Openmama-users] Build Mamda Book from Mama Subscription

Glenn McClements <g.mcclements@...>
 

Hi Nestor,

Perhaps one of the tick42blp bridge developers could help out, but one thing that may help is a orderbook message spec for OpenMAMDA. This isn't currently published on the OpenMAMA website but I see when we can get it out.


Regards,

Glenn 


Glenn McClements, SVP Engineering 

Adelaide Exchange | 24-26 Adelaide Street | Belfast | UK | BT2 8GD

www.srtechlabs.com

 

SR.LABS Proven High Speed Electronic Trading Solutions




From: openmama-users-bounces@... <openmama-users-bounces@...> on behalf of Macrux <kmacrux@...>
Sent: 30 September 2015 22:39
To: openmama-dev@...; openmama-users@...
Subject: [Openmama-users] Build Mamda Book from Mama Subscription
 
Hi there,


I'm wondering if there is any example in which data received through OpenMAMA is transformed into a Mamda Order book. For example, I have a subcription to Bloomberg through the tick42blp bridge but I need to transform the data received into a Mamda Order Book because I have a client (other program) which works with OpenMamda order books, that's the point.


Thanks in advance for any help you could give about that case.
Regards,


Nestor.

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Build Mamda Book from Mama Subscription

macrux
 

Hi there,


I'm wondering if there is any example in which data received through OpenMAMA is transformed into a Mamda Order book. For example, I have a subcription to Bloomberg through the tick42blp bridge but I need to transform the data received into a Mamda Order Book because I have a client (other program) which works with OpenMamda order books, that's the point.


Thanks in advance for any help you could give about that case.
Regards,


Nestor.


Problem with the OpenMAMA .Net layer

Tom Doust
 

Hi

 

We came across a subtle (and interesting) issue in the .Net implementation. I’m going to describe the problem and how we worked around but I want to open it up for discussion before proposing any fixes. I’d also note that although we came across it in one particular context it is possible there may be similar cases elsewhere.

 

The Open MAMA .Net layer declares the getVectorAsString() method like this:

 

            [DllImport(Mama.DllName, CallingConvention = CallingConvention.Cdecl)]

                     public static extern int mamaMsgField_getVectorString (IntPtr nativeHandle,

                           ref IntPtr result,

                           ref uint size);

 

and the other vector-methods are similar. The functions that they are calling are declared in the C layer like this:

 

            MAMAExpDLL

            extern mama_status

            mamaMsgField_getVectorString (

                const mamaMsgField   msgField,

                const char***        result,

                mama_size_t*         size);

 

A problem arises in 64-bit Windows because sizeof(uint) != sizeof(mama_size_t) and this causes heap corruption, when the P/Invoke layer tries to marshal the values of result and size back to the C# code. The C code in the bridge writes size as a 64-bit value (because sizeof(mama_size_t) == 8), but the P/Invoke layer has allocated only 32-bits to hold it (because sizeof(uint) == 4, even in 64-bit Windows).

 

 

Our temporary workaround is as follows:

 

As it happens, the P/Invoke layer seems to assign the addresses of result and size next to each other in the heap, so we are able to avoid the problem by assigning to size first (overwriting the lower 32-bits of result) and then assigning  the value of result afterwards. This is a horrible solution and is not acceptable for the longer term, though, as we would be relying on undocumented behaviour of the P/invoke marshaller, and any changes to the bridge implementation of the field-handling code may break it again.

 

We think a reasonable solution is to declare the size argument in the  .Net getVectorAsString  as ‘ref IntPtr’ rather than ‘ref uint’  which should guarantee the correct byte length on both 32 bit and 64 bit systems, but we are very interested to hear what other people think about this both in terms of the solution and our understanding of what the p/Invoke implementation is doing. Our analysis is based on inference from stepping the disassembled code J

 

Best regards

 

Tom Doust

 

 

TOM DOUST | Head of Consultancy                                                                                                         


TICK42

P: +44 (0) 1628 477444 | M: +44 (0) 7710 479924 | E: tom.doust@... | skype:  tom.doust |  http://www.tick42.com

 

 


Re: [PATCH 2/2] [MAMADOTNET] Add getName() to MamaTransport

Matt Mulhern
 

Hi,

This is a quick notification that this has been pushed to the OpenMAMA::next branch.

Thanks,
Matt Mulhern

On 21/05/2015 17:29, Gary Molloy wrote:

Testing Strategy

--------------------

 

Modify the MamaPublisherCS.cs example application and add the following after you have created your publisher:

 

            publisher.create(transport, outboundTopic);

 

            MamaTransport aTransport = new MamaTransport();

            aTransport = publisher.getTransport();

            string tportName = aTransport.getName();

            Console.WriteLine("TEST:: getTransport name {0}.", tportName);

 

You can expect to see the following output:

 

> MamaPublisher.exe -m wmw -tport tcp_sub

Created inbound subscription.
TEST:: getTransport name tcp_sub. <--------------------------- test output
Publishing message to MAMA_TOPIC.
Publishing message to MAMA_TOPIC.
Publishing message to MAMA_TOPIC.
Publishing message to MAMA_TOPIC.

 

 

======================================================================================================================

 

From 3abc196ba5a09beecbe000463c015c086c86fbea Mon Sep 17 00:00:00 2001

From: Gary Molloy <g.molloy@...>

Date: Mon, 18 May 2015 13:55:43 +0100

Subject: [PATCH 2/2] [MAMADOTNET] Add getName() to MamaTransport

 

Function already exists for C / C++ / JNI

 

Signed-off-by: Gary Molloy <g.molloy@...>

---

mama/dotnet/src/cs/MamaTransport.cs | 15 +++++++++++++++

1 file changed, 15 insertions(+)

 

diff --git a/mama/dotnet/src/cs/MamaTransport.cs b/mama/dotnet/src/cs/MamaTransport.cs

index fee71cb..11a8f71 100644

--- a/mama/dotnet/src/cs/MamaTransport.cs

+++ b/mama/dotnet/src/cs/MamaTransport.cs

@@ -260,6 +260,18 @@ namespace Wombat

                                  GC.KeepAlive(callback);

       }        

 

+      /// <summary>

+      /// Get the name of the transport.

+      /// </summary>

+      public string getName()

+      {

+                             // Get the transport name from the native layer

+                               IntPtr ret = IntPtr.Zero;

+                               CheckResultCode(NativeMethods.mamaTransport_getName(nativeHandle, ref ret));

+

+                               // Convert to an ANSI string

+                               return Marshal.PtrToStringAnsi(ret);

+      }

 

                               #region Implementation details

 

@@ -430,6 +442,9 @@ namespace Wombat

             [DllImport(Mama.DllName, CallingConvention = CallingConvention.Cdecl)]

             public static extern int mamaTransport_getQuality(IntPtr transport,

                                                               ref MamaQuality qual);

+            [DllImport(Mama.DllName, CallingConvention = CallingConvention.Cdecl)]

+            public static extern int mamaTransport_getName(IntPtr transport,

+                                                           ref IntPtr ret);

                                }

 

                               // state

--

2.1.0

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 



_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

681 - 700 of 2311