Date   

Re: OpenMAMA-2.4.0-rc1 Now Available

Alpert, Reed <reed.alpert@...>
 

Hi,

 

Submitted one pull request for Sconscript.win to create entitlementlibraries.c.

 

With that the build works fine via scons on Windows 7 32b/64b, RHEL5 32b/64b, RHEL6 32b/64b.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB Market Data Services Engineers | CIB_Market_Data_Services_Engineers@...

 

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Frank Quinn
Sent: Thursday, February 18, 2016 6:37 AM
To: openmama-dev
Subject: [Openmama-dev] OpenMAMA-2.4.0-rc1 Now Available

 

Hi Folks,

 

We are pleased to announce the first release candidate for OpenMAMA 2.4.0 is now available:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-2.4.0-rc1

 

As many of you will be aware, we have been working on closing off the work required for a new OpenMAMA release for several weeks now and it is one of the biggest we have ever assembled - both in terms of features assembled and sheer quantity of new and modified lines of code.

 

With that in mind, I would appreciate everyone’s help in trying out the latest release candidate and helping weed out any bugs or unexpected behavior which may have crept in, or may be specific to your environment / bridges. At a high level, the main new functionality is in the following areas:

 

·         Qpid proton broker support

·         Several qpid proton bridge bug fixes including fixing support for using the qpid payload on non-qpid transports and vice versa

·         CentOS / RHEL 7 support

·         Timer fixes

·         Several changes to work with recent OSX

·         Publisher Events

·         Dynamic Entitlements (phase 1 - mainly focused on defining existing interface points)

·         Dynamic Bridge loading (the ability to load any middleware bridge, even if there is no reference to it in OpenMAMA code, and adding flexibility to the bride interface).

·         Ability to specify a default payload via configuration

 

With that in mind, we recommend that any testing you plan on doing pays particular attention to those areas.

 

NB for bridge developers, see updated wiki page detailing the changes required in the bridge which have changed slightly since our last notification: https://github.com/OpenMAMA/OpenMAMA/wiki/Dynamic-Bridge-Loading

 

As well as new functionality, we have also been undergoing several major operational changes since the last OpenMAMA Release:

 

·         Migrated Wiki, Issue tracking and code review all in Github out in the open (see http://github.com/OpenMAMA/OpenMAMA)

·         Continuous integration now includes Microsoft Windows builds (see http://ci.openmama.org)

·         All compiler warnings have been removed from our CentOS 6.x builds

·         All supported unit tests should now pass on Avis

 

Another focal point of this release is a general bug scrub of all outstanding issues, leading to over 100 issues of various sizes being resolved since we moved to Github less than 6 months ago.

 

Thank you all in advance for your help in testing - if you spot any issues, please follow our guidelines for raising an issue, or even better, follow our guidelines for raising a patch. Good hunting.

 

Cheers,

Frank

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


OpenMAMA-2.4.0-rc1 Now Available

Frank Quinn <fquinn.ni@...>
 

Hi Folks,


We are pleased to announce the first release candidate for OpenMAMA 2.4.0 is now available:


https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-2.4.0-rc1


As many of you will be aware, we have been working on closing off the work required for a new OpenMAMA release for several weeks now and it is one of the biggest we have ever assembled - both in terms of features assembled and sheer quantity of new and modified lines of code.


With that in mind, I would appreciate everyone’s help in trying out the latest release candidate and helping weed out any bugs or unexpected behavior which may have crept in, or may be specific to your environment / bridges. At a high level, the main new functionality is in the following areas:


  • Qpid proton broker support

  • Several qpid proton bridge bug fixes including fixing support for using the qpid payload on non-qpid transports and vice versa

  • CentOS / RHEL 7 support

  • Timer fixes

  • Several changes to work with recent OSX

  • Publisher Events

  • Dynamic Entitlements (phase 1 - mainly focused on defining existing interface points)

  • Dynamic Bridge loading (the ability to load any middleware bridge, even if there is no reference to it in OpenMAMA code, and adding flexibility to the bride interface).

  • Ability to specify a default payload via configuration


With that in mind, we recommend that any testing you plan on doing pays particular attention to those areas.


NB for bridge developers, see updated wiki page detailing the changes required in the bridge which have changed slightly since our last notification: https://github.com/OpenMAMA/OpenMAMA/wiki/Dynamic-Bridge-Loading


As well as new functionality, we have also been undergoing several major operational changes since the last OpenMAMA Release:


  • Migrated Wiki, Issue tracking and code review all in Github out in the open (see http://github.com/OpenMAMA/OpenMAMA)

  • Continuous integration now includes Microsoft Windows builds (see http://ci.openmama.org)

  • All compiler warnings have been removed from our CentOS 6.x builds

  • All supported unit tests should now pass on Avis


Another focal point of this release is a general bug scrub of all outstanding issues, leading to over 100 issues of various sizes being resolved since we moved to Github less than 6 months ago.


Thank you all in advance for your help in testing - if you spot any issues, please follow our guidelines for raising an issue, or even better, follow our guidelines for raising a patch. Good hunting.


Cheers,

Frank


Re: OpenMAMA 2.4.0

Dmitri Fedorov <dfedorov.solace@...>
 

Great, thank you very much Frank, we're looking forward to it.

Regards
Dmitri

On Mon, Feb 8, 2016 at 11:08 AM, Frank Quinn <f.quinn@...> wrote:

Hi Dmitri,

 

Yeah most of the work is done at this point so with any luck we’ll be taking the RC cut at the end of this week or the beginning of next week.

 

Cheers,

Frank

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Dmitri Fedorov
Sent: 08 February 2016 16:04
To: openmama-dev@...
Subject: Re: [Openmama-dev] OpenMAMA 2.4.0

 

Thank you Frank,

 

No, there is nothing, I'm just trying to understand when we are here at Solace Systems should expect this release.

 

I suppose it will take a few days to complete the remaining issues, not as few weeks, right?

 

Regards

Dmitri Fedorov

 

On Mon, Feb 8, 2016 at 10:59 AM, Frank Quinn <f.quinn@...> wrote:

Hi Dmitri,

 

Yeah that’s all that is to be completed before we take the 2.4.0 cut, why is there something that you need squeezed in?

 

Cheers,

Frank

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Dmitri Fedorov
Sent: 08 February 2016 15:23
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA 2.4.0

 

Hi all,

 

I'm with Solace R&D for OpenMAMA.

 

Am I right to assume that this list of open issues https://github.com/OpenMAMA/OpenMAMA/milestones/OpenMAMA-2.4.0 is all what is remained before 2.4.0 is declared, please?

 

Thank you

Dmitri Fedorov

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: OpenMAMA 2.4.0

Frank Quinn <f.quinn@...>
 

Hi Dmitri,

 

Yeah most of the work is done at this point so with any luck we’ll be taking the RC cut at the end of this week or the beginning of next week.

 

Cheers,

Frank

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Dmitri Fedorov
Sent: 08 February 2016 16:04
To: openmama-dev@...
Subject: Re: [Openmama-dev] OpenMAMA 2.4.0

 

Thank you Frank,

 

No, there is nothing, I'm just trying to understand when we are here at Solace Systems should expect this release.

 

I suppose it will take a few days to complete the remaining issues, not as few weeks, right?

 

Regards

Dmitri Fedorov

 

On Mon, Feb 8, 2016 at 10:59 AM, Frank Quinn <f.quinn@...> wrote:

Hi Dmitri,

 

Yeah that’s all that is to be completed before we take the 2.4.0 cut, why is there something that you need squeezed in?

 

Cheers,

Frank

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Dmitri Fedorov
Sent: 08 February 2016 15:23
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA 2.4.0

 

Hi all,

 

I'm with Solace R&D for OpenMAMA.

 

Am I right to assume that this list of open issues https://github.com/OpenMAMA/OpenMAMA/milestones/OpenMAMA-2.4.0 is all what is remained before 2.4.0 is declared, please?

 

Thank you

Dmitri Fedorov

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: OpenMAMA 2.4.0

Dmitri Fedorov <dfedorov.solace@...>
 

Thank you Frank,

No, there is nothing, I'm just trying to understand when we are here at Solace Systems should expect this release.

I suppose it will take a few days to complete the remaining issues, not as few weeks, right?

Regards
Dmitri Fedorov

On Mon, Feb 8, 2016 at 10:59 AM, Frank Quinn <f.quinn@...> wrote:

Hi Dmitri,

 

Yeah that’s all that is to be completed before we take the 2.4.0 cut, why is there something that you need squeezed in?

 

Cheers,

Frank

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Dmitri Fedorov
Sent: 08 February 2016 15:23
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA 2.4.0

 

Hi all,

 

I'm with Solace R&D for OpenMAMA.

 

Am I right to assume that this list of open issues https://github.com/OpenMAMA/OpenMAMA/milestones/OpenMAMA-2.4.0 is all what is remained before 2.4.0 is declared, please?

 

Thank you

Dmitri Fedorov

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: OpenMAMA 2.4.0

Frank Quinn <f.quinn@...>
 

Hi Dmitri,

 

Yeah that’s all that is to be completed before we take the 2.4.0 cut, why is there something that you need squeezed in?

 

Cheers,

Frank

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Dmitri Fedorov
Sent: 08 February 2016 15:23
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA 2.4.0

 

Hi all,

 

I'm with Solace R&D for OpenMAMA.

 

Am I right to assume that this list of open issues https://github.com/OpenMAMA/OpenMAMA/milestones/OpenMAMA-2.4.0 is all what is remained before 2.4.0 is declared, please?

 

Thank you

Dmitri Fedorov

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


OpenMAMA 2.4.0

Dmitri Fedorov <dfedorov.solace@...>
 

Hi all,

I'm with Solace R&D for OpenMAMA.

Am I right to assume that this list of open issues https://github.com/OpenMAMA/OpenMAMA/milestones/OpenMAMA-2.4.0 is all what is remained before 2.4.0 is declared, please?

Thank you
Dmitri Fedorov


Re: Discussion Point: Dynamic Bridge Loading and Default payloads

Alpert, Reed <reed.alpert@...>
 

For us it is mostly a support issue, in that when working with a bridge vendor it is easier if we are also using their payload.

If we are mixing bridges and payloads a vendor can quite reasonably ask “Do you still get that issue if you are using our payload?”

I did accidentally mix bridges and payloads early on, and it worked just fine, but still ….

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB Market Data Services Engineers | CIB_Market_Data_Services_Engineers@...

 

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Frank Quinn
Sent: Thursday, February 04, 2016 5:02 AM
To: openmama-dev@...
Subject: [Openmama-dev] Discussion Point: Dynamic Bridge Loading and Default payloads

 

Hi Folks,

As brought up on a recent thread from Ben:

    2)My only observation, which is somewhat related, is that we kept the
    concept of a default payload. I think this break the whole purpose of
    OpenMama as that means mw bridge must come with their default
    payload, which is loaded by default, irrespective to the fact you are
    going to use it or not. I think this is a defect, and was hoping that we
    will take the occasion of dynamic bridge loading to get this changed.
    Although it might be a big change as it involves probably deprecating
    MamaMsg_create in favour of _createForPayload....

    I'm very keen to get feedback from the community, and particularly
    our user community about this.


I have written both enterprise applications based on OpenMAMA as well as internal OpenMAMA code, and as a user, I think mamaMsg_create() without any payload provided is very handy. I think there are a couple of reasons for this:

  1. mamaMsg_create is one of those functions which you can call from anywhere in your application. It doesn't depend on which bridge you happen to be in during a callback for or anything like that so it can literally come from anywhere. Part of its traditional appeal is specifically that the application developer doesn't need to consider which payloads are in play, and if he / she does care, there is now a separate _createForPayload method which will provide this level of control.
  2. mamaMsg_create usually does exactly what the user will want in almost every scenario. If there is only one middleware involved as it's a one sided API (i.e. a MAMA client or a MAMA publisher), the first payload recommended by the bridge is typically what the user wants to load. It's the same if there is a mid-tier (MAMA pub + sub in one MAMA instance) application where the middlewares are the same north and south.
  3. Deprecating mamaMsg_create will (I expect) have a reasonably large impact on users as a whole. In fact mamaMsg_createforPayload is actually a new method for users moving from MAMA 5 to OpenMAMA, so migrating customers will never have seen it before.

OK so that's why I think mamaMsg_create() is a nice thing to have wearing my "user hat". Here is where I think it's broken and potential solutions wearing my "bridge developer helmet":

  1. It makes the default payload library to be the first payload it finds among all payloads. This is OK in most scenarios, but I think there is a notable exception where there is a multi-bridge app present and the preferred payload for bridge 1 is not even supported by bridge 2. That means any payloads created in this application instance cannot be published by bridge 2 without additional transcoding.
    Potential Solution: Wait until all middleware bridges are loaded before selecting a default payload based on which payload bridges exist across all bridges.
  2. It has no way to be aware of optimal payload selections for each middleware bridge. An example would, again, be in a multi-bridge environment where the algorithm from #1 (let's say) is used to decide a payload compromise, but in the context where a mamaMsg_create is being called, if you always know that it will only ever be create for use with one middleware bridge, you will want to select whichever payload is preferred for that bridge.
    Potential Solution: Have an application-facing way to provide the user with the preferred payload for a given bridge object and then the user can call _createforPayload based on that value.
  3. For middleware bridges that want to support all possible payload bridges that support serialization and deserialization and possibly don't even have their own payload implementation at all (e.g. ZMQ), they may not wish to select a preference for which payload bridge to use at all.
    Potential Solution: Provide a mama.properties parameter which may be used to find supplementary, user-provided compatible bridges.

So I agree with you in that mamaMsg_create is a little broken, but I think they're all fixable without deprecating a pretty handy convenience function for most users.

Cheers,
Frank

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Re: Dynamic Bridge Loading in 2.4.0 and Bridge Backwards Compatibility

Frank Quinn <f.quinn@...>
 

Hi Folks,

On the technical committee call today there was unanimous agreement that this is the right course of action for the 2.4.0 release. However, if anyone else from the community has any objections to this, please let us know and we'll take any feedback on board.

If there are no significant objections, we will consider the issue closed on February 8th, and start actively removing the relevant backwards compatibility components in the code.

Cheers,
Frank

On 03/02/16 18:17, Frank Quinn wrote:

Hi Folks,

 

Dynamic Bridge Loading is coming in 2.4.0 and it brings with it a level of flexibility which is new to OpenMAMA’s internal bridges. It effectively allows:

 

1.       OpenMAMA to have the concept of mandatory bridge functions (thereby failing to load incompatible bridges where these bridge functions are not present).

2.       Creation of new and optional bridge functions will no longer break the bridge interface itself.

3.       Reordering of methods in each bridge will no longer break the bridge.

4.       Third parties may create OpenMAMA bridges without having to reserve enum values with the OpenMAMA project.

 

Dynamic bridge loading itself was designed with backwards compatibility in mind, but the more we spoke with users in the community, the more we understood that backwards compatibility in the bridge (as opposed to within the OpenMAMA API itself) is nowhere near as important as avoiding run time crashes due to incompatible bridges being loaded. Note that the upcoming Publisher Events framework in its current form actually depends on this functionality as it involves updating of bridge functions.

 

With that in mind, we would like to propose that the internal bridge interface (again, not to be confused with the OpenMAMA API which will remain unchanged) for OpenMAMA 2.4.x is not backwards compatible with 2.3.x.

 

The impact of this would be that each bridge vendor will need to make a few small changes to work with OpenMAMA 2.4.x as documented over at https://github.com/OpenMAMA/OpenMAMA/wiki/Dynamic-Bridge-Loading. Failure to conform with the new changes will result in OpenMAMA gracefully failing to load the bridge in question at runtime.

 

Taking this stance now will:

 

1.       Reduce the amount of code which third party bridge developers need to maintain.

2.       Ensure that we do not get hit with compatibility issues due to third party bridges not populating the bridge in a way which OpenMAMA expects in future.

3.       Allow us to retire a bunch of code rather than have a large conditional revolving around “if this is not a recent bridge”.

4.       Make it much easier for bridge developers to remain compatible with future OpenMAMA bridge changes going forward.

 

We will be discussing on a technical committee call tomorrow whether or not it’s OK to break backwards compatibility. Depending on the outcome of that, we may be sending around a note to this list offering an opportunity for other members of the community to object before pulling the trigger on this.

 

Cheers,

Frank



The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Discussion Point: Dynamic Bridge Loading and Default payloads

Frank Quinn <f.quinn@...>
 

Hi Folks,

As brought up on a recent thread from Ben:

    2)My only observation, which is somewhat related, is that we kept the
    concept of a default payload. I think this break the whole purpose of
    OpenMama as that means mw bridge must come with their default
    payload, which is loaded by default, irrespective to the fact you are
    going to use it or not. I think this is a defect, and was hoping that we
    will take the occasion of dynamic bridge loading to get this changed.
    Although it might be a big change as it involves probably deprecating
    MamaMsg_create in favour of _createForPayload....

    I'm very keen to get feedback from the community, and particularly
    our user community about this.


I have written both enterprise applications based on OpenMAMA as well as internal OpenMAMA code, and as a user, I think mamaMsg_create() without any payload provided is very handy. I think there are a couple of reasons for this:

  1. mamaMsg_create is one of those functions which you can call from anywhere in your application. It doesn't depend on which bridge you happen to be in during a callback for or anything like that so it can literally come from anywhere. Part of its traditional appeal is specifically that the application developer doesn't need to consider which payloads are in play, and if he / she does care, there is now a separate _createForPayload method which will provide this level of control.
  2. mamaMsg_create usually does exactly what the user will want in almost every scenario. If there is only one middleware involved as it's a one sided API (i.e. a MAMA client or a MAMA publisher), the first payload recommended by the bridge is typically what the user wants to load. It's the same if there is a mid-tier (MAMA pub + sub in one MAMA instance) application where the middlewares are the same north and south.
  3. Deprecating mamaMsg_create will (I expect) have a reasonably large impact on users as a whole. In fact mamaMsg_createforPayload is actually a new method for users moving from MAMA 5 to OpenMAMA, so migrating customers will never have seen it before.
OK so that's why I think mamaMsg_create() is a nice thing to have wearing my "user hat". Here is where I think it's broken and potential solutions wearing my "bridge developer helmet":

  1. It makes the default payload library to be the first payload it finds among all payloads. This is OK in most scenarios, but I think there is a notable exception where there is a multi-bridge app present and the preferred payload for bridge 1 is not even supported by bridge 2. That means any payloads created in this application instance cannot be published by bridge 2 without additional transcoding.
    Potential Solution: Wait until all middleware bridges are loaded before selecting a default payload based on which payload bridges exist across all bridges.
  2. It has no way to be aware of optimal payload selections for each middleware bridge. An example would, again, be in a multi-bridge environment where the algorithm from #1 (let's say) is used to decide a payload compromise, but in the context where a mamaMsg_create is being called, if you always know that it will only ever be create for use with one middleware bridge, you will want to select whichever payload is preferred for that bridge.
    Potential Solution: Have an application-facing way to provide the user with the preferred payload for a given bridge object and then the user can call _createforPayload based on that value.
  3. For middleware bridges that want to support all possible payload bridges that support serialization and deserialization and possibly don't even have their own payload implementation at all (e.g. ZMQ), they may not wish to select a preference for which payload bridge to use at all.
    Potential Solution: Provide a mama.properties parameter which may be used to find supplementary, user-provided compatible bridges.

So I agree with you in that mamaMsg_create is a little broken, but I think they're all fixable without deprecating a pretty handy convenience function for most users.

Cheers,
Frank

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: Dynamic Bridge Loading in 2.4.0 and Bridge Backwards Compatibility

Frank Quinn <f.quinn@...>
 

Thanks Ben - glad to see you're happy to take the plunge!

On point #2, I was actually talking to Damian about this very point yesterday and I think I agree with the sentiment of what you're saying, but it's a _little_ off topic as that is related to a customer-facing API change which is a little hairier because it may involve recompiles and possibly code changes on the client side... in saying that, I think there are ways we can alleviate your concerns without needing to change the customer facing interface, so I'll start up a new thread to discuss that.

Cheers,
Frank

On 04/02/16 08:25, Benjamin Taieb wrote:

Hi Frank,

I won't make it to today's call, but wanted to give some feedback :

 

1)I think it is perfectly OK to break backwards compatibility given:

·         We are all aware that the API need some architectural improvement, that will yield further benefit down the line. In this case, we would break compatibility to allow introducing change in the future without breaking backward compatibility (bonus point here if you understood what I meant).

·         Those non backwards compatibles changes are limited in scope (here just bridges), and here only mw vendors/open source bridge are concerned. The user community doesn't change their application code.

 

2)My only observation, which is somewhat related, is that we kept the concept of a default payload. I think this break the whole purpose of OpenMama as that means mw bridge must come with their default payload, which is loaded by default, irrespective to the fact you are going to use it or not. I think this is a defect, and was hoping that we will take the occasion of dynamic bridge loading to get this changed. Although it might be a big change as it involves probably deprecating MamaMsg_create in favour of _createForPayload....

I'm very keen to get feedback from the community, and particularly our user community about this.

 

Cheers,

Ben.

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Frank Quinn
Sent: 03 February 2016 18:18
To: openmama-dev@...
Subject: [Openmama-dev] Dynamic Bridge Loading in 2.4.0 and Bridge Backwards Compatibility

 

Hi Folks,

 

Dynamic Bridge Loading is coming in 2.4.0 and it brings with it a level of flexibility which is new to OpenMAMA’s internal bridges. It effectively allows:

 

1.       OpenMAMA to have the concept of mandatory bridge functions (thereby failing to load incompatible bridges where these bridge functions are not present).

2.       Creation of new and optional bridge functions will no longer break the bridge interface itself.

3.       Reordering of methods in each bridge will no longer break the bridge.

4.       Third parties may create OpenMAMA bridges without having to reserve enum values with the OpenMAMA project.

 

Dynamic bridge loading itself was designed with backwards compatibility in mind, but the more we spoke with users in the community, the more we understood that backwards compatibility in the bridge (as opposed to within the OpenMAMA API itself) is nowhere near as important as avoiding run time crashes due to incompatible bridges being loaded. Note that the upcoming Publisher Events framework in its current form actually depends on this functionality as it involves updating of bridge functions.

 

With that in mind, we would like to propose that the internal bridge interface (again, not to be confused with the OpenMAMA API which will remain unchanged) for OpenMAMA 2.4.x is not backwards compatible with 2.3.x.

 

The impact of this would be that each bridge vendor will need to make a few small changes to work with OpenMAMA 2.4.x as documented over at https://github.com/OpenMAMA/OpenMAMA/wiki/Dynamic-Bridge-Loading. Failure to conform with the new changes will result in OpenMAMA gracefully failing to load the bridge in question at runtime.

 

Taking this stance now will:

 

1.       Reduce the amount of code which third party bridge developers need to maintain.

2.       Ensure that we do not get hit with compatibility issues due to third party bridges not populating the bridge in a way which OpenMAMA expects in future.

3.       Allow us to retire a bunch of code rather than have a large conditional revolving around “if this is not a recent bridge”.

4.       Make it much easier for bridge developers to remain compatible with future OpenMAMA bridge changes going forward.

 

We will be discussing on a technical committee call tomorrow whether or not it’s OK to break backwards compatibility. Depending on the outcome of that, we may be sending around a note to this list offering an opportunity for other members of the community to object before pulling the trigger on this.

 

Cheers,

Frank


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC



The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: Dynamic Bridge Loading in 2.4.0 and Bridge Backwards Compatibility

Benjamin Taieb
 

Hi Frank,

I won't make it to today's call, but wanted to give some feedback :

 

1)I think it is perfectly OK to break backwards compatibility given:

·         We are all aware that the API need some architectural improvement, that will yield further benefit down the line. In this case, we would break compatibility to allow introducing change in the future without breaking backward compatibility (bonus point here if you understood what I meant).

·         Those non backwards compatibles changes are limited in scope (here just bridges), and here only mw vendors/open source bridge are concerned. The user community doesn't change their application code.

 

2)My only observation, which is somewhat related, is that we kept the concept of a default payload. I think this break the whole purpose of OpenMama as that means mw bridge must come with their default payload, which is loaded by default, irrespective to the fact you are going to use it or not. I think this is a defect, and was hoping that we will take the occasion of dynamic bridge loading to get this changed. Although it might be a big change as it involves probably deprecating MamaMsg_create in favour of _createForPayload....

I'm very keen to get feedback from the community, and particularly our user community about this.

 

Cheers,

Ben.

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Frank Quinn
Sent: 03 February 2016 18:18
To: openmama-dev@...
Subject: [Openmama-dev] Dynamic Bridge Loading in 2.4.0 and Bridge Backwards Compatibility

 

Hi Folks,

 

Dynamic Bridge Loading is coming in 2.4.0 and it brings with it a level of flexibility which is new to OpenMAMA’s internal bridges. It effectively allows:

 

1.       OpenMAMA to have the concept of mandatory bridge functions (thereby failing to load incompatible bridges where these bridge functions are not present).

2.       Creation of new and optional bridge functions will no longer break the bridge interface itself.

3.       Reordering of methods in each bridge will no longer break the bridge.

4.       Third parties may create OpenMAMA bridges without having to reserve enum values with the OpenMAMA project.

 

Dynamic bridge loading itself was designed with backwards compatibility in mind, but the more we spoke with users in the community, the more we understood that backwards compatibility in the bridge (as opposed to within the OpenMAMA API itself) is nowhere near as important as avoiding run time crashes due to incompatible bridges being loaded. Note that the upcoming Publisher Events framework in its current form actually depends on this functionality as it involves updating of bridge functions.

 

With that in mind, we would like to propose that the internal bridge interface (again, not to be confused with the OpenMAMA API which will remain unchanged) for OpenMAMA 2.4.x is not backwards compatible with 2.3.x.

 

The impact of this would be that each bridge vendor will need to make a few small changes to work with OpenMAMA 2.4.x as documented over at https://github.com/OpenMAMA/OpenMAMA/wiki/Dynamic-Bridge-Loading. Failure to conform with the new changes will result in OpenMAMA gracefully failing to load the bridge in question at runtime.

 

Taking this stance now will:

 

1.       Reduce the amount of code which third party bridge developers need to maintain.

2.       Ensure that we do not get hit with compatibility issues due to third party bridges not populating the bridge in a way which OpenMAMA expects in future.

3.       Allow us to retire a bunch of code rather than have a large conditional revolving around “if this is not a recent bridge”.

4.       Make it much easier for bridge developers to remain compatible with future OpenMAMA bridge changes going forward.

 

We will be discussing on a technical committee call tomorrow whether or not it’s OK to break backwards compatibility. Depending on the outcome of that, we may be sending around a note to this list offering an opportunity for other members of the community to object before pulling the trigger on this.

 

Cheers,

Frank


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Dynamic Bridge Loading in 2.4.0 and Bridge Backwards Compatibility

Frank Quinn <f.quinn@...>
 

Hi Folks,

 

Dynamic Bridge Loading is coming in 2.4.0 and it brings with it a level of flexibility which is new to OpenMAMA’s internal bridges. It effectively allows:

 

1.       OpenMAMA to have the concept of mandatory bridge functions (thereby failing to load incompatible bridges where these bridge functions are not present).

2.       Creation of new and optional bridge functions will no longer break the bridge interface itself.

3.       Reordering of methods in each bridge will no longer break the bridge.

4.       Third parties may create OpenMAMA bridges without having to reserve enum values with the OpenMAMA project.

 

Dynamic bridge loading itself was designed with backwards compatibility in mind, but the more we spoke with users in the community, the more we understood that backwards compatibility in the bridge (as opposed to within the OpenMAMA API itself) is nowhere near as important as avoiding run time crashes due to incompatible bridges being loaded. Note that the upcoming Publisher Events framework in its current form actually depends on this functionality as it involves updating of bridge functions.

 

With that in mind, we would like to propose that the internal bridge interface (again, not to be confused with the OpenMAMA API which will remain unchanged) for OpenMAMA 2.4.x is not backwards compatible with 2.3.x.

 

The impact of this would be that each bridge vendor will need to make a few small changes to work with OpenMAMA 2.4.x as documented over at https://github.com/OpenMAMA/OpenMAMA/wiki/Dynamic-Bridge-Loading. Failure to conform with the new changes will result in OpenMAMA gracefully failing to load the bridge in question at runtime.

 

Taking this stance now will:

 

1.       Reduce the amount of code which third party bridge developers need to maintain.

2.       Ensure that we do not get hit with compatibility issues due to third party bridges not populating the bridge in a way which OpenMAMA expects in future.

3.       Allow us to retire a bunch of code rather than have a large conditional revolving around “if this is not a recent bridge”.

4.       Make it much easier for bridge developers to remain compatible with future OpenMAMA bridge changes going forward.

 

We will be discussing on a technical committee call tomorrow whether or not it’s OK to break backwards compatibility. Depending on the outcome of that, we may be sending around a note to this list offering an opportunity for other members of the community to object before pulling the trigger on this.

 

Cheers,

Frank


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: Openmama Documentation for subscription onDestroy

cmorgan
 

I’ve raised an issue to cover this on github.

 

Here is a link to the issue:

https://github.com/OpenMAMA/OpenMAMA/issues/114

 

Chris Morgan

 

 

From: Alpert, Reed [mailto:reed.alpert@...]
Sent: February-01-16 10:48 AM
To: Duane Pauls <Duane.Pauls@...>; Christopher Morgan <Christopher.Morgan@...>; openmama-dev@...
Subject: RE: Openmama Documentation for subscription onDestroy

 

I think the app needs to call sub->destroy(), then wait for the onDestroy(), and then ‘delete sub’.

 

onDestroy() is a completion event – once you get it you won’t hear from that subscription again (no callbacks of any kind).

 

In order for an app to call C++ or Java ‘delete sub’ w/o sub->destroy the destructor needs to completely disconnect the app from the impl (which as Duane & Chris point out it does not).

In that case the inline onDestroy() does have value in that the app knows it can safely destroy any of its data structures.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Duane Pauls
Sent: Thursday, January 28, 2016 1:39 PM
To: Christopher Morgan; openmama-dev@...
Subject: Re: [Openmama-dev] Openmama Documentation for subscription onDestroy

 

I’ll add a bit of clarification to the problem with the CPP interface if the middleware bridge queues an event to call the onDestroy callback.

 

When an application destructs a MamaSubscription object, the application onDestroy is called inline, and the MamaSubscription object is freed, but the corresponding MamaSubscriptionImpl object is not freed.  So far so good – it is the MamaSubscriptionImpl object pointer that is the closure to the C-layer and this is still valid, unfreed memory.  However, at this point the MamaSubscriptionImpl object’s mCallback pointer still points to the application’s MamaSubscriptionCallback object.  The application may have freed this memory since MamaSubscriptionCallback::onDestroy() has been called.

 

If we look at MamdaSubscription as an example, it in fact does not implement MamaSubscriptionCallback::onDestroy at all.  It will always destroy the callback (MamdaSubscription::mImpl) immediately after destroying the underlying MamaSubscription.  This happens to occur after onDestroy is called since it is called inline with MamaSubscription::~MamaSubscription.  But nevertheless, it is freed immediately after onDestroy is called inline with the call to MamaSubscription::~MamaSubscription().

 

Now, sometime later, the C-layer onDestroy is called.  This calls the static MamaSubscription::onSubscriptionDestroy which calls MamaSubscriptionImpl::InvokeDestroy().  This checks if MamaSubscriptionImpl::mCallback is NULL (it is not – in the case of a MamdaSubscription it is still pointing to a freed MamdaSubscriptionImpl object) and tries to invoke the onDestroy() method.  If the vtable pointer is no longer valid, a core here is quite likely.

 

Even if a core does not occur, the application would see onDestroy() twice – once inline with when the destroy was started and a second time when the lower layer destroy was completed.  It seems the design intent here is to decouple the MamaSubscriptionImpl from everything above it when the MamaSubscription is destroyed.  To do this, it seems the right thing to do in the C++ layer would be to set MamaSubscriptionImpl::mCallback to NULL when setting MamaSubscriptionImpl::mFreed to true (or alternatively include mFreed in the tests that check mCallback == NULL).

 

Should a bug be raised for this issue?  Or should a middleware bridge in fact NOT enqueue the onDestroy callback as stated in the OpenMAMA Bridge Documentation Wiki and instead invoke onDestroy inline?  If the intent is to always invoke the callback inline, it seems the benefit of having a callback in the first place is diminished.

 

Cheers,

Duane

 

From: Christopher Morgan
Sent: Thursday, January 28, 2016 10:30 AM
To:
openmama-dev@...
Subject: Openmama Documentation for subscription onDestroy

 

Hi,

 

I’m with solace dev for the openmama middle bridge and I have question about mamaSubscription onDestroy with the CPP openmama. On the openmama Developer Wiki under Bridge Documentation for myMiddlewareBridgeMamaSubscription_destroy () it states “…the destroy method adds an event to the queue to invokes the destroy callback, which notifies the client application of the successful destroy of the subscription.” Our Bridge currently does this but in the CPP API for openmama the MamaSubscription seems to do an inline onDestroy as a part of the destructor. This does not work well with the enqueuerd onDestroy callback (potential for causing segmentation faults). I’ve also compared to the Qpid bridge which seems to do an inline onDestroy callback as a part their MiddlewareBridgeMamaSubscription_destroy(). Should the onDestroy callback be inline the MiddlewareBridgeMamaSubscription_destroy() like the Qpid to work with the CPP API? Or Is there a problem with the CPP API?

 

Chris Morgan

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Re: Openmama Documentation for subscription onDestroy

Alpert, Reed <reed.alpert@...>
 

I think the app needs to call sub->destroy(), then wait for the onDestroy(), and then ‘delete sub’.

 

onDestroy() is a completion event – once you get it you won’t hear from that subscription again (no callbacks of any kind).

 

In order for an app to call C++ or Java ‘delete sub’ w/o sub->destroy the destructor needs to completely disconnect the app from the impl (which as Duane & Chris point out it does not).

In that case the inline onDestroy() does have value in that the app knows it can safely destroy any of its data structures.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Duane Pauls
Sent: Thursday, January 28, 2016 1:39 PM
To: Christopher Morgan; openmama-dev@...
Subject: Re: [Openmama-dev] Openmama Documentation for subscription onDestroy

 

I’ll add a bit of clarification to the problem with the CPP interface if the middleware bridge queues an event to call the onDestroy callback.

 

When an application destructs a MamaSubscription object, the application onDestroy is called inline, and the MamaSubscription object is freed, but the corresponding MamaSubscriptionImpl object is not freed.  So far so good – it is the MamaSubscriptionImpl object pointer that is the closure to the C-layer and this is still valid, unfreed memory.  However, at this point the MamaSubscriptionImpl object’s mCallback pointer still points to the application’s MamaSubscriptionCallback object.  The application may have freed this memory since MamaSubscriptionCallback::onDestroy() has been called.

 

If we look at MamdaSubscription as an example, it in fact does not implement MamaSubscriptionCallback::onDestroy at all.  It will always destroy the callback (MamdaSubscription::mImpl) immediately after destroying the underlying MamaSubscription.  This happens to occur after onDestroy is called since it is called inline with MamaSubscription::~MamaSubscription.  But nevertheless, it is freed immediately after onDestroy is called inline with the call to MamaSubscription::~MamaSubscription().

 

Now, sometime later, the C-layer onDestroy is called.  This calls the static MamaSubscription::onSubscriptionDestroy which calls MamaSubscriptionImpl::InvokeDestroy().  This checks if MamaSubscriptionImpl::mCallback is NULL (it is not – in the case of a MamdaSubscription it is still pointing to a freed MamdaSubscriptionImpl object) and tries to invoke the onDestroy() method.  If the vtable pointer is no longer valid, a core here is quite likely.

 

Even if a core does not occur, the application would see onDestroy() twice – once inline with when the destroy was started and a second time when the lower layer destroy was completed.  It seems the design intent here is to decouple the MamaSubscriptionImpl from everything above it when the MamaSubscription is destroyed.  To do this, it seems the right thing to do in the C++ layer would be to set MamaSubscriptionImpl::mCallback to NULL when setting MamaSubscriptionImpl::mFreed to true (or alternatively include mFreed in the tests that check mCallback == NULL).

 

Should a bug be raised for this issue?  Or should a middleware bridge in fact NOT enqueue the onDestroy callback as stated in the OpenMAMA Bridge Documentation Wiki and instead invoke onDestroy inline?  If the intent is to always invoke the callback inline, it seems the benefit of having a callback in the first place is diminished.

 

Cheers,

Duane

 

From: Christopher Morgan
Sent: Thursday, January 28, 2016 10:30 AM
To:
openmama-dev@...
Subject: Openmama Documentation for subscription onDestroy

 

Hi,

 

I’m with solace dev for the openmama middle bridge and I have question about mamaSubscription onDestroy with the CPP openmama. On the openmama Developer Wiki under Bridge Documentation for myMiddlewareBridgeMamaSubscription_destroy () it states “…the destroy method adds an event to the queue to invokes the destroy callback, which notifies the client application of the successful destroy of the subscription.” Our Bridge currently does this but in the CPP API for openmama the MamaSubscription seems to do an inline onDestroy as a part of the destructor. This does not work well with the enqueuerd onDestroy callback (potential for causing segmentation faults). I’ve also compared to the Qpid bridge which seems to do an inline onDestroy callback as a part their MiddlewareBridgeMamaSubscription_destroy(). Should the onDestroy callback be inline the MiddlewareBridgeMamaSubscription_destroy() like the Qpid to work with the CPP API? Or Is there a problem with the CPP API?

 

Chris Morgan

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Re: Openmama Documentation for subscription onDestroy

Duane Pauls <Duane.Pauls@...>
 

I’ll add a bit of clarification to the problem with the CPP interface if the middleware bridge queues an event to call the onDestroy callback.

 

When an application destructs a MamaSubscription object, the application onDestroy is called inline, and the MamaSubscription object is freed, but the corresponding MamaSubscriptionImpl object is not freed.  So far so good – it is the MamaSubscriptionImpl object pointer that is the closure to the C-layer and this is still valid, unfreed memory.  However, at this point the MamaSubscriptionImpl object’s mCallback pointer still points to the application’s MamaSubscriptionCallback object.  The application may have freed this memory since MamaSubscriptionCallback::onDestroy() has been called.

 

If we look at MamdaSubscription as an example, it in fact does not implement MamaSubscriptionCallback::onDestroy at all.  It will always destroy the callback (MamdaSubscription::mImpl) immediately after destroying the underlying MamaSubscription.  This happens to occur after onDestroy is called since it is called inline with MamaSubscription::~MamaSubscription.  But nevertheless, it is freed immediately after onDestroy is called inline with the call to MamaSubscription::~MamaSubscription().

 

Now, sometime later, the C-layer onDestroy is called.  This calls the static MamaSubscription::onSubscriptionDestroy which calls MamaSubscriptionImpl::InvokeDestroy().  This checks if MamaSubscriptionImpl::mCallback is NULL (it is not – in the case of a MamdaSubscription it is still pointing to a freed MamdaSubscriptionImpl object) and tries to invoke the onDestroy() method.  If the vtable pointer is no longer valid, a core here is quite likely.

 

Even if a core does not occur, the application would see onDestroy() twice – once inline with when the destroy was started and a second time when the lower layer destroy was completed.  It seems the design intent here is to decouple the MamaSubscriptionImpl from everything above it when the MamaSubscription is destroyed.  To do this, it seems the right thing to do in the C++ layer would be to set MamaSubscriptionImpl::mCallback to NULL when setting MamaSubscriptionImpl::mFreed to true (or alternatively include mFreed in the tests that check mCallback == NULL).

 

Should a bug be raised for this issue?  Or should a middleware bridge in fact NOT enqueue the onDestroy callback as stated in the OpenMAMA Bridge Documentation Wiki and instead invoke onDestroy inline?  If the intent is to always invoke the callback inline, it seems the benefit of having a callback in the first place is diminished.

 

Cheers,

Duane

 

From: Christopher Morgan
Sent: Thursday, January 28, 2016 10:30 AM
To: openmama-dev@...
Subject: Openmama Documentation for subscription onDestroy

 

Hi,

 

I’m with solace dev for the openmama middle bridge and I have question about mamaSubscription onDestroy with the CPP openmama. On the openmama Developer Wiki under Bridge Documentation for myMiddlewareBridgeMamaSubscription_destroy () it states “…the destroy method adds an event to the queue to invokes the destroy callback, which notifies the client application of the successful destroy of the subscription.” Our Bridge currently does this but in the CPP API for openmama the MamaSubscription seems to do an inline onDestroy as a part of the destructor. This does not work well with the enqueuerd onDestroy callback (potential for causing segmentation faults). I’ve also compared to the Qpid bridge which seems to do an inline onDestroy callback as a part their MiddlewareBridgeMamaSubscription_destroy(). Should the onDestroy callback be inline the MiddlewareBridgeMamaSubscription_destroy() like the Qpid to work with the CPP API? Or Is there a problem with the CPP API?

 

Chris Morgan


Openmama Documentation for subscription onDestroy

cmorgan
 

Hi,

 

I’m with solace dev for the openmama middle bridge and I have question about mamaSubscription onDestroy with the CPP openmama. On the openmama Developer Wiki under Bridge Documentation for myMiddlewareBridgeMamaSubscription_destroy () it states “…the destroy method adds an event to the queue to invokes the destroy callback, which notifies the client application of the successful destroy of the subscription.” Our Bridge currently does this but in the CPP API for openmama the MamaSubscription seems to do an inline onDestroy as a part of the destructor. This does not work well with the enqueuerd onDestroy callback (potential for causing segmentation faults). I’ve also compared to the Qpid bridge which seems to do an inline onDestroy callback as a part their MiddlewareBridgeMamaSubscription_destroy(). Should the onDestroy callback be inline the MiddlewareBridgeMamaSubscription_destroy() like the Qpid to work with the CPP API? Or Is there a problem with the CPP API?

 

Chris Morgan


FW: [Openmama-ci] Code change(s) just landed on origin/next (Fixed)

Frank Quinn <f.quinn@...>
 

Hi Folks,

 

As discussed in the technical committee meeting, an example of the notifications I was considering subscribing openmama-dev to is listed below:

 

They should only get triggered when a change lands on either ‘next’ or ‘master’ branches.


Currently these notifications are only sent to members of the openmama-ci mailing list, but if we can collapse the two without adding too much noise, then the openmama-ci mailing list can be removed.

 

Cheers,

Frank

 

From: openmama-ci-bounces@... [mailto:openmama-ci-bounces@...] On Behalf Of jenkins@...
Sent: 12 January 2016 15:15
To: openmama-ci@...
Subject: [Openmama-ci] Code change(s) just landed on origin/next (Fixed)

 

Some changes have just been added to the origin/next branch!

 
[Matt Mulhern] UNITTEST: disabling invalid msg test.
        mama/c_cpp/src/gunittest/c/mamamsg/msggeneraltests.cpp
 
 

Results for OpenMAMA Next Branch (Qpid Proton) CI run with latest changes:

 

  • CI Project Name: OpenMAMA Next Branch (Qpid Proton)
  • Build Number: #8
  • Build Status: Fixed
  • Total Amount of Tests: 1762
  • Passed: 1762
  • Failed: 0
  • Skipped / Disabled: 0

You may also check CI console output to view the full results.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Re: Feature freeze for OpenMAMA 2.4.0

Frank Quinn <f.quinn@...>
 

Hi Folks,

 

After a few extra days grace, I’m now declaring the feature set for OpenMAMA 2.4.0 frozen. Work will be completed on the outstanding issues to be merged into next, then the 2.4.0 code cut will be taken.

 

Also note that Dynamic Bridge Loading has just landed into the next branch (and Phil just fixed OSX to work with it too which we’re about to merge in).

 

Cheers,

Frank

 

From: Frank Quinn
Sent: 06 January 2016 13:11
To: 'openmama-dev@...' <openmama-dev@...>
Subject: Feature freeze for OpenMAMA 2.4.0

 

Hi Folks,

 

Happy new year!

 

Now, let's round up the features / bug fixes that are going to make it into the next OpenMAMA release (2.4.0).

 

Are there any existing issues that are not listed below that anyone would like to ensure is included in the next release? If so, please reply to this email on-list with the github ticket number and we'll see if it's feasible to squeeze them in.

 

Features in progress for being included in next release:

 

·         Publisher Events (Under Development)

·         Dynamic Entitlements for OEA abstraction (Under Development)

·         Dynamic Bridge Loading  (Code Review)

·         MSVC 2015 Support (Code Review)

 

Summary of existing changes already completed for next release (full list available at https://github.com/OpenMAMA/OpenMAMA/compare/master...next):

 

·         Qpid proton broker support

·         Several qpid proton bridge bug fixes

·         CentOS / RHEL 7 support

·         Timer fixes

·         New Unit tests

·         Several changes to work with recent OSX

·         Avis fixes to provide a clean CI run (See the clouds over at http://ci.openmama.org)

 

I will aggregate and appraise responses until Wednesday 13th January, at which point I will send another email advising when the 2.4.0 cut will be taken.

 

Cheers,

Frank


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Feature freeze for OpenMAMA 2.4.0

Frank Quinn <f.quinn@...>
 

Hi Folks,

 

Happy new year!

 

Now, let's round up the features / bug fixes that are going to make it into the next OpenMAMA release (2.4.0).

 

Are there any existing issues that are not listed below that anyone would like to ensure is included in the next release? If so, please reply to this email on-list with the github ticket number and we'll see if it's feasible to squeeze them in.

 

Features in progress for being included in next release:

 

·         Publisher Events (Under Development)

·         Dynamic Entitlements for OEA abstraction (Under Development)

·         Dynamic Bridge Loading  (Code Review)

·         MSVC 2015 Support (Code Review)

 

Summary of existing changes already completed for next release (full list available at https://github.com/OpenMAMA/OpenMAMA/compare/master...next):

 

·         Qpid proton broker support

·         Several qpid proton bridge bug fixes

·         CentOS / RHEL 7 support

·         Timer fixes

·         New Unit tests

·         Several changes to work with recent OSX

·         Avis fixes to provide a clean CI run (See the clouds over at http://ci.openmama.org)

 

I will aggregate and appraise responses until Wednesday 13th January, at which point I will send another email advising when the 2.4.0 cut will be taken.

 

Cheers,

Frank


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC

661 - 680 of 2314