Date   

Re: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Tom Doust
 

Hi Yury, Konstantin

Are you using the current github version of the bridge code? We looked at and fixed some of the issues around locking the subscription destroy some time back.

It would be good to know if we have missed something.

Best regards

Tom

-----Original Message-----
From: openmama-users-bounces@... [mailto:openmama-users-bounces@...] On Behalf Of Konstantin Baydarov
Sent: 20 June 2017 11:32
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi, guys.

I'm working on the issue with Yury. I spotted the deadlock possibility during debugging tick42rmds bridge crash on unsubscribe, will be interested knowing the solution as well.

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 12:38 PM
To: Frank Quinn <fquinn@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Cc: Konstantin Baydarov <konstantin.baydarov@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Frank,

Let me answer your question in random order :) 2. It looks like a designed behavior of RMDS bridge - callbacks are invoked in a thread servicing transport events from a server. One thread per mamaTransport is created.
1. Therefore race conditions are possible. Our case is: mute is called for a subscription, then mamaSubscription_cleanup frees self->mInitialRequest but concurrent mamaSubscription_processMsg call tries to access self->mInitialRequest because the message it processes is initial message from a server.
3. Taking in account p.2 we cannot process mute request asynchronously as MAMA starts freeing subscription resources immediately after mute is called. Do you think it is possible for MAMA not to invoke bridgeMamaSubscriptionMute() under mamaSubscription locks? Thus the contract for this function would be:
- This call should be synchronous and no events should be processed after it returned (like before)
- It should be reentrant and synchronize it's operations by itself (quite sensible requirement)

-----Original Message-----
From: Frank Quinn [mailto:fquinn@...]
Sent: Tuesday, June 20, 2017 10:05 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury,

Thanks for the detailed query, I have a few outstanding questions and suggestions on this one:

1. I would question whether or not mamaSubscription_processMsg should crash for a muted subscription. Muting is a state which exists to attempt to stop new events coming in. However if you're in the dispatcher thread and you just received an object, it's too late this time - mute should only be invoked prior to prevent the *next* read. So perhaps (knowing nothing about the RMDS bridge) the straightforward solution would be simply to do the checks which may cause muting after the callback?
2. Should the thread which processes events from RMDS server invoke the callback method directly (inline)? Is it on the same thread as is assigned to the MAMA Subscription object? It should be to match the application's expected concurrency behaviour.
3. Rather than muting immediately, you could consider creating a muting callback event which gets enqueued onto your subscription thread. That way the mute event will always be synchronous with the subscription thread and you don't need to worry about locking any associated resources. Locking with subscription objects (particularly MAMA core objects) is hairy stuff and you should avoid it where possible to avoid race conditions.

Cheers,
Frank



-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Yury Batrakov
Sent: 19 June 2017 17:42
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi guys,

I've faced the following issue when using OpenMAMA and tick42rmds bridge.
The bridge internally creates a thread to process events from RMDS server, once a message is received that thread invokes mamaSubscription_processMsg. While the message is processed user may want to destroy the subscription (obviously in other thread). To avoid corruption of mamaSubscription object, mamaSubscription_destroy() function calls bridge->mute for the bridge to stop calling mamaSubscription_processMsg() and only then deallocates mamaSubscription. The problem with this approach is the following: here's the pseudo code for RMDS dispatching thread


if(muted) {
// Do not dispatch
return;
}

// Do some other checks <-- mute() may be invoked here
mamaSubscription_processMsg() // processMsg for muted subscription, may crash


The solution for that is to change RMDS bridge to block in bridge->mute() call until mamaSubscription_processMsg() returns but there's another problem: mamaSubscription_processMsg and mamaSubscription_deactivate may deadlock on wombatThrottle. Consider the following scenario:
1. RMDS bridge thread invokes mamaSubscription_processMsg() for message of type initial
2. User thread invokes mamaSubscription_destroy() which acquires wombat throttle lock:
if (impl->mTransport)
throttle = mamaTransportImpl_getThrottle(impl->mTransport,
MAMA_THROTTLE_DEFAULT);

if(NULL != throttle)
{
wombatThrottle_lock(throttle);
}
3. Then mamaSubscription_destroy calls mamaSubscription_deactivate_internal which calls our new version of bridge->mute() which waits for RMDS bridge thread to finish message processing
if (impl->mSubscBridge)
{
impl->mBridgeImpl->bridgeMamaSubscriptionMute (impl->mSubscBridge);
}
4. RMDS bridge handles initial message and tries to acquire the same throttle:
#5 0x00007ffff78d4f32 in wombatThrottle_lock (throttle=0x6298e0) at mama/c_cpp/src/c/throttle.c:441
#6 0x00007ffff78a34e2 in imageRequest_stopWaitForResponse (request=0x14d1a20) at mama/c_cpp/src/c/imagerequest.c:774
#7 0x00007ffff78cbe06 in mamaSubscription_stopWaitForResponse (subscription=0xe36280, ctx=0xe364a0) at mama/c_cpp/src/c/subscription.c:1262
#8 0x00007ffff78a38fe in processPointToPointMessage (callback=0x1527e50, msg=0x642460, msgType=6, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:169
#9 0x00007ffff78a3f9c in listenerMsgCallback_processMsg (callback=0x1527e50, msg=0x642460, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:480
#10 0x00007ffff78cd825 in mamaSubscription_processMsg (subscription=0xe36280, msg=0x642460) at mama/c_cpp/src/c/subscription.c:2259

What do you think is the best way to avoid this?


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-users mailing list
Openmama-users@...
https://lists.openmama.org/mailman/listinfo/openmama-users


Re: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Konstantin Baydarov
 

Classification: Public

Hi, guys.

I'm working on the issue with Yury. I spotted the deadlock possibility during debugging tick42rmds bridge crash on unsubscribe, will be interested knowing the solution as well.

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 12:38 PM
To: Frank Quinn <fquinn@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Cc: Konstantin Baydarov <konstantin.baydarov@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Frank,

Let me answer your question in random order :)
2. It looks like a designed behavior of RMDS bridge - callbacks are invoked in a thread servicing transport events from a server. One thread per mamaTransport is created.
1. Therefore race conditions are possible. Our case is: mute is called for a subscription, then mamaSubscription_cleanup frees self->mInitialRequest but concurrent mamaSubscription_processMsg call tries to access self->mInitialRequest because the message it processes is initial message from a server.
3. Taking in account p.2 we cannot process mute request asynchronously as MAMA starts freeing subscription resources immediately after mute is called. Do you think it is possible for MAMA not to invoke bridgeMamaSubscriptionMute() under mamaSubscription locks? Thus the contract for this function would be:
- This call should be synchronous and no events should be processed after it returned (like before)
- It should be reentrant and synchronize it's operations by itself (quite sensible requirement)

-----Original Message-----
From: Frank Quinn [mailto:fquinn@...]
Sent: Tuesday, June 20, 2017 10:05 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury,

Thanks for the detailed query, I have a few outstanding questions and suggestions on this one:

1. I would question whether or not mamaSubscription_processMsg should crash for a muted subscription. Muting is a state which exists to attempt to stop new events coming in. However if you're in the dispatcher thread and you just received an object, it's too late this time - mute should only be invoked prior to prevent the *next* read. So perhaps (knowing nothing about the RMDS bridge) the straightforward solution would be simply to do the checks which may cause muting after the callback?
2. Should the thread which processes events from RMDS server invoke the callback method directly (inline)? Is it on the same thread as is assigned to the MAMA Subscription object? It should be to match the application's expected concurrency behaviour.
3. Rather than muting immediately, you could consider creating a muting callback event which gets enqueued onto your subscription thread. That way the mute event will always be synchronous with the subscription thread and you don't need to worry about locking any associated resources. Locking with subscription objects (particularly MAMA core objects) is hairy stuff and you should avoid it where possible to avoid race conditions.

Cheers,
Frank



-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Yury Batrakov
Sent: 19 June 2017 17:42
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi guys,

I've faced the following issue when using OpenMAMA and tick42rmds bridge.
The bridge internally creates a thread to process events from RMDS server, once a message is received that thread invokes mamaSubscription_processMsg. While the message is processed user may want to destroy the subscription (obviously in other thread). To avoid corruption of mamaSubscription object, mamaSubscription_destroy() function calls bridge->mute for the bridge to stop calling mamaSubscription_processMsg() and only then deallocates mamaSubscription. The problem with this approach is the following: here's the pseudo code for RMDS dispatching thread


if(muted) {
// Do not dispatch
return;
}

// Do some other checks <-- mute() may be invoked here
mamaSubscription_processMsg() // processMsg for muted subscription, may crash


The solution for that is to change RMDS bridge to block in bridge->mute() call until mamaSubscription_processMsg() returns but there's another problem: mamaSubscription_processMsg and mamaSubscription_deactivate may deadlock on wombatThrottle. Consider the following scenario:
1. RMDS bridge thread invokes mamaSubscription_processMsg() for message of type initial
2. User thread invokes mamaSubscription_destroy() which acquires wombat throttle lock:
if (impl->mTransport)
throttle = mamaTransportImpl_getThrottle(impl->mTransport,
MAMA_THROTTLE_DEFAULT);

if(NULL != throttle)
{
wombatThrottle_lock(throttle);
}
3. Then mamaSubscription_destroy calls mamaSubscription_deactivate_internal which calls our new version of bridge->mute() which waits for RMDS bridge thread to finish message processing
if (impl->mSubscBridge)
{
impl->mBridgeImpl->bridgeMamaSubscriptionMute (impl->mSubscBridge);
}
4. RMDS bridge handles initial message and tries to acquire the same throttle:
#5 0x00007ffff78d4f32 in wombatThrottle_lock (throttle=0x6298e0) at mama/c_cpp/src/c/throttle.c:441
#6 0x00007ffff78a34e2 in imageRequest_stopWaitForResponse (request=0x14d1a20) at mama/c_cpp/src/c/imagerequest.c:774
#7 0x00007ffff78cbe06 in mamaSubscription_stopWaitForResponse (subscription=0xe36280, ctx=0xe364a0) at mama/c_cpp/src/c/subscription.c:1262
#8 0x00007ffff78a38fe in processPointToPointMessage (callback=0x1527e50, msg=0x642460, msgType=6, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:169
#9 0x00007ffff78a3f9c in listenerMsgCallback_processMsg (callback=0x1527e50, msg=0x642460, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:480
#10 0x00007ffff78cd825 in mamaSubscription_processMsg (subscription=0xe36280, msg=0x642460) at mama/c_cpp/src/c/subscription.c:2259

What do you think is the best way to avoid this?


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


OpenMAMA 6.2.1 Released

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

We are pleased to announce the final release of OpenMAMA 6.2.1 is now available:


This release exists mainly to address several issues coming out of the recent MAMA Datetime changes:

  • New explicit mamaDateTime_[gs]etEpochTimeExt methods to allow bridges and applications to directly set the underlying timestamp value regardless of whether or not time_t on the target system has sufficient resolution
  • Fixed issue with extended datetime representation on 32-bit systems (Windows and Linux)
  • Fixed issue with multiple subscribers for the same topic on qpid
  • Fixed crash in conflated order book processing
  • New explicit public accessors for datetime precision and hints
  • Release distributions will now include dependent libraries inside the target package and related license information is included (including the new Apache APR dependency)

NB: This release includes the removal of the legacy _USE_32BIT_TIME_T compile time macro for 32 bit windows. Please ensure that third party application and bridges are not compiled using this macro to avoid potential corruption of data.

For a complete list of all 17 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/7?closed=1

A special thanks to all developers, contributors and testers who helped is getting this out door.

Cheers,
Frank


OpenMAMA-6.2.1-rc2 Now Available

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

The second release candidate for OpenMAMA 6.2.1 has just been cut and contains a few fixes put in since RC1 - see https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.1-rc2.

Once again, we appreciate the continued assistance from the community. Assuming no major issues are found, we'll aim to release the GA version on 15th June.

If you find any issues, please see here for details on how to report it to us:


Cheers,
Frank


Last call for OpenMAMA 6.2.1 RC2 bugs

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

I will soon be preparing a release for the second release candidate for OpenMAMA 6.2.1.

Note that there are a fair few changes and bug fixes to make extended datetime work on both 32 and 64 bit platforms:

1. New explicit mamaDateTime_[gs]etFromEpochExt methods to allow bridges to explicitly set the underlying value regardless of whether or not time_t on the target system has sufficient resolution
2. New explicit getters and setters for datetime precision and hints
3. Release distributions will now include dependent libraries inside the target package and related license information will be included (changes about to land).

If anyone has any further issues they would like to report please do so before the end of the day, otherwise we'll be cutting RC2 tomorrow morning.

I'll send a full notification after the cut is made.

Cheers,
Frank


OpenMAMA-6.2.1-rc1 Now Available

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

We are pleased to announce the first release candidate for openMAMA 6.2.1 is now available:


This is a bugfix release to address a few main issues:
  • Fixed issue with extended datetime representation on 32-bit systems (Windows and Linux)
  • Fixed issue with multiple subscribers for the same topic on qpid
  • Fixed crash in conflated order book processing
For a complete list of all 9 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/7?closed=1

Thank you all in advance for your help in testing - if you spot any issues, please follow our guidelines for raising an issue, or even better, follow our guidelines for raising a patch.

As this release is fairly small bugfix release, we're not expecting a large volume of bugs to fall out, so I will be acting on a reasonably brief test window of 1 week commencing today. If no issues have been found, we can aim to release on 29th May. If critical issues are found, we will continue to go through weekly release candidates until have a stable release ready.

Cheers,
Frank


Re: [Openmama-dev] OpenMAMA Forum Selection

Tom Doust
 

I’ve used proboards before in a different context – it works fine but as Frank says its very old-school.

My only other comment is that I think there is a lot of value in getting quite tightly integrated with github

 

Tom

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Damian Maguire
Sent: 20 April 2017 08:34
To: fquinn.ni@...; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-dev] [Openmama-users] OpenMAMA Forum Selection

 

Speaking from experience of the different tools, Disqus seems to be a solid option. Not a strong opinion though, if other people have feedback. 

 

Thanks, 

 

Damian

 

On Wed, Apr 19, 2017 at 8:30 AM Frank Quinn <fquinn.ni@...> wrote:

Hi Folks,

 

After suggestion from a steering committee action, I have been looking at various options available with respect to providing OpenMAMA forum functionality. The forum would operate alongside the existing mailing lists and should be accessible and searchable for anyone who wants to avail of its knowledge.

  • Disqus (recommended): This is a freely hosted half-forum, half-comment-provider. It’s really designed to have follow ups on documentation pages but can also be browsed like a forum. You can log in with Disqus, facebook, twitter or google accounts. No maintenance is required and it already has integration support with our new technical documentation framework. I like this solution because I think it would operate nicely alongside what we have rather forking discussion arenas.

Others considered:

  • Proboards: I actually stumbled across since the initial research and it's a proper forum with topics etc and is freely hosted. Ticks all the boxes if you want more of an old-school forum, and is searchable without an account and unlimited in its access.
  • Discourse: Feels like it would overlap with github issues and wiki. Nice though as you can sign in with a github account. However would require us to host it unless you pay $100 pm (unless you go offbrand).
  • Flarum: Nice open source tool but is in beta phase and requires hosting to be set up which ultimately adds overhead to maintaining the project.

Let me know if you have any strong opinions either way folks. For me, it's really a two horse race between Disqus and Proboards but I'm open to suggestions (or even others that aren't listed here). Think about which one you'd be most likely to use.

 

Cheers,

Frank

_______________________________________________
Openmama-users mailing list
Openmama-users@...
https://lists.openmama.org/mailman/listinfo/openmama-users


Re: OpenMAMA Forum Selection

Damian Maguire
 

Speaking from experience of the different tools, Disqus seems to be a solid option. Not a strong opinion though, if other people have feedback. 

Thanks, 

Damian

On Wed, Apr 19, 2017 at 8:30 AM Frank Quinn <fquinn.ni@...> wrote:
Hi Folks,

After suggestion from a steering committee action, I have been looking at various options available with respect to providing OpenMAMA forum functionality. The forum would operate alongside the existing mailing lists and should be accessible and searchable for anyone who wants to avail of its knowledge.
  • Disqus (recommended): This is a freely hosted half-forum, half-comment-provider. It’s really designed to have follow ups on documentation pages but can also be browsed like a forum. You can log in with Disqus, facebook, twitter or google accounts. No maintenance is required and it already has integration support with our new technical documentation framework. I like this solution because I think it would operate nicely alongside what we have rather forking discussion arenas.
Others considered:
  • Proboards: I actually stumbled across since the initial research and it's a proper forum with topics etc and is freely hosted. Ticks all the boxes if you want more of an old-school forum, and is searchable without an account and unlimited in its access.
  • Discourse: Feels like it would overlap with github issues and wiki. Nice though as you can sign in with a github account. However would require us to host it unless you pay $100 pm (unless you go offbrand).
  • Flarum: Nice open source tool but is in beta phase and requires hosting to be set up which ultimately adds overhead to maintaining the project.
Let me know if you have any strong opinions either way folks. For me, it's really a two horse race between Disqus and Proboards but I'm open to suggestions (or even others that aren't listed here). Think about which one you'd be most likely to use.

Cheers,
Frank
_______________________________________________
Openmama-users mailing list
Openmama-users@...
https://lists.openmama.org/mailman/listinfo/openmama-users


OpenMAMA Forum Selection

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

After suggestion from a steering committee action, I have been looking at various options available with respect to providing OpenMAMA forum functionality. The forum would operate alongside the existing mailing lists and should be accessible and searchable for anyone who wants to avail of its knowledge.
  • Disqus (recommended): This is a freely hosted half-forum, half-comment-provider. It’s really designed to have follow ups on documentation pages but can also be browsed like a forum. You can log in with Disqus, facebook, twitter or google accounts. No maintenance is required and it already has integration support with our new technical documentation framework. I like this solution because I think it would operate nicely alongside what we have rather forking discussion arenas.
Others considered:
  • Proboards: I actually stumbled across since the initial research and it's a proper forum with topics etc and is freely hosted. Ticks all the boxes if you want more of an old-school forum, and is searchable without an account and unlimited in its access.
  • Discourse: Feels like it would overlap with github issues and wiki. Nice though as you can sign in with a github account. However would require us to host it unless you pay $100 pm (unless you go offbrand).
  • Flarum: Nice open source tool but is in beta phase and requires hosting to be set up which ultimately adds overhead to maintaining the project.
Let me know if you have any strong opinions either way folks. For me, it's really a two horse race between Disqus and Proboards but I'm open to suggestions (or even others that aren't listed here). Think about which one you'd be most likely to use.

Cheers,
Frank


Re: OpenMAMA 6.2.0 Released

Frank Quinn <fquinn.ni@...>
 

Apologies folks, for some reason this email seems to render badly in outlook.

If this effects you, check out the official announcement instead: http://www.openmama.org/news/openmama-620-released


On Thu, Mar 30, 2017 at 2:30 PM, Frank Quinn <fquinn.ni@...> wrote:

Hi Folks,

We are pleased to announce the final release of OpenMAMA 6.2.0 is now available:

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.0-release

This release bumps the version of OpenMAMA up to 6.2.0 due to recent MAMA Payload changes which could potentially break the bridge for payload bridges which treat mamaDateTime as a U64 pointer and attempt direct access. See the RFC on the change for for more details (particularly payload bridge developers as changes may be required).

Key changes and bugfixes included in this release:

  • New extended MamaDateTime changes to support date ranges beyond the 2038 problem
  • Added thread pinning for process thread affinity
  • Performance improvements made for mamaDqPublisher_send
  • Fixed transient access violation issues in windows when using wtimegm
  • New MAMA methods for looking up middleware and payload bridges
  • Performance improvements to mamaproducerc_v2
  • Added new document generating script for updating openmama.github.io
  • Added new helper method for finding a file in a delimited path (e.g. PATH or WOMBAT_PATH)
  • Added new helper method for loading in a MAMA format symbol file in a standard way

For a complete list of all 24 issues and pull requests included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/6?closed=1

As well as new functionality, we have also continued with a few devops changes since the last OpenMAMA Release:

  • Main github landing page now includes Travis CI buttons
  • Nightly RPM builds are now available and accessible via the github landing page
  • The ci-run python script is now used across all platforms simplifying the CI process

A special thanks to all developers, contributors and testers who helped is getting this out door.

Cheers,
Frank



OpenMAMA 6.2.0 Released

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

We are pleased to announce the final release of OpenMAMA 6.2.0 is now available:

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.0-release

This release bumps the version of OpenMAMA up to 6.2.0 due to recent MAMA Payload changes which could potentially break the bridge for payload bridges which treat mamaDateTime as a U64 pointer and attempt direct access. See the RFC on the change for for more details (particularly payload bridge developers as changes may be required).

Key changes and bugfixes included in this release:

  • New extended MamaDateTime changes to support date ranges beyond the 2038 problem
  • Added thread pinning for process thread affinity
  • Performance improvements made for mamaDqPublisher_send
  • Fixed transient access violation issues in windows when using wtimegm
  • New MAMA methods for looking up middleware and payload bridges
  • Performance improvements to mamaproducerc_v2
  • Added new document generating script for updating openmama.github.io
  • Added new helper method for finding a file in a delimited path (e.g. PATH or WOMBAT_PATH)
  • Added new helper method for loading in a MAMA format symbol file in a standard way

For a complete list of all 24 issues and pull requests included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/6?closed=1

As well as new functionality, we have also continued with a few devops changes since the last OpenMAMA Release:

  • Main github landing page now includes Travis CI buttons
  • Nightly RPM builds are now available and accessible via the github landing page
  • The ci-run python script is now used across all platforms simplifying the CI process

A special thanks to all developers, contributors and testers who helped is getting this out door.

Cheers,
Frank


OpenMAMA-6.2.0-rc1 Now Available

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

We are pleased to announce the first release candidate for OpenMAMA 6.2.0 is now available including binary releases:

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.0-rc1

This release bumps the version of OpenMAMA up to 6.2.0 due to recent MAMA Payload changes which could potentially break the bridge for payload bridges which treat mamaDateTime as a U64 pointer and attempt direct access. See the RFC on the change for for more details.

Payload bridge developers in particular are encouraged to rigorously test their bridges with this release.

Key changes and bugfixes included in this release:

  • New extended MamaDateTime changes to support date ranges beyond the 2038 problem
  • Added thread pinning for process thread affinity
  • Performance improvements made for mamaDqPublisher_send
  • Fixed transient access violation issues in windows when using wtimegm
  • New MAMA methods for looking up middleware and payload bridges
  • Performance improvements to mamaproducerc_v2
  • Added new document generating script for updating openmama.github.io
  • Added new helper method for finding a file in a delimited path (e.g. PATH or WOMBAT_PATH)
  • Added new helper method for loading in a MAMA format symbol file in a standard way

For a complete list of all 22 issues and pull requests included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/6?closed=1

Cheers,
Frank


New Technical Documentation Resource for OpenMAMA

Frank Quinn <fquinn.ni@...>
 

Hi folks,

As we have discussed before, documentation is something that we are all keen to make improvements upon. With that in mind, I am pleased to announce the launch of our new technical documentation website https://openmama.github.io.

There's not much there at the moment that doesn't already exist, but going forward, it should be considered the new home for:

- Reference documentation
- Application Developer Guides
- Bridge Developer Guides
- Plugin Developer guides
- Tutorials
- Example Code walkthroughs
- RFCs
- Any other technical documentation (e.g. contents of current wiki)

In the coming months, we'll be aggregating all existing technical documentation to this website and building out additional content as we go. We'll also be updating links etc.

You are all encouraged to get involved - anyone can pull down the git repository behind this site (https://github.com/OpenMAMA/openmama.github.io) and run it locally using instructions provided in the Contribute section or make minor changes (e.g. to RFCs) using the Edit functionality to participate.

If you have any problems, drop in on our Gitter channel and let us know.

Cheers,
Frank


OpenMAMA 6.1.0 Released

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

We are pleased to announce the final release of OpenMAMA 6.1.0 is now available:

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.1.0

This mostly a maintenance / bugfix release, but it also jumps the version number from 2.4.1 to 6.1.0. I appreciate it's a big jump but it should make the following things clear to the community and anyone watching the project:

  • OpenMAMA is the equivalent to Vela's "MAMA 6" (Vela have also agreed to move to 6.1.x to align with OpenMAMA).
  • Users of both Vela's Enterprise release of MAMA and OpenMAMA can easily see what equivalent versions will be.
  • Users upgrading from Vela's older MAMA 5 can have more sane #ifdefs in their code if they're trying to support both versions during migration.
Note that bridge has not been changed as part of this release so all bridges and plugins which worked with 2.4.x will also work with 6.1.x.

At a high level, the main new functionality is in the following areas:
  • Removed all known valgrind reported memory leaks from our CI test bed and API
  • Avis now removed (see the mailing list entry)
  • Added Payload and Middleware unit tests on Visual Studio projects
  • Added ability to provide a separate timeout for recaps
  • Fixed issue where book recaps were being ignored duing FT takeover
  • Fixed issue where wildcard subscription OnMsg callback is called with NULL instead of topic
  • Wired up mamamsg vector price and vector datetime field types
  • Qpid support to stop publishing to departed subscribers added
  • Fixed core on startup where no entitlements were defined
  • Fixed race condition deadlock in mamaDispatcher_destroy

For a complete list of all 69 issues and pull requests included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/5?closed=1

As well as new functionality, we have also continued with a few devops changes since the last OpenMAMA Release:
  • New python script in place to do jenkins CI builds
  • Qpid proton build script now modified to include the qpid proton DLL on windows scons builds
  • Modified release generating script to allow binary drops for RC releases
  • Added Fedora 24 RPM and removed Fedora 21 RPM
  • Github landing page has gotten a bit of a facelift including CI status
A special thanks to all developers, contributors and testers who helped is getting this out door.

Cheers,
Frank


Solace OpenMAMA Publish/Subscribe tutorial is available

Dmitri Fedorov
 

Hi all,

Solace Systems publishes a series of OpenMAMA tutorials, helping developers to adapt OpenMAMA and get a better understanding of proper configuration of the Solace Middleware Bridge for OpenMAMA.

The posts are meant to complement existing OpenMAMA samples and documentation. 

Publish/Subscribe tutorial has been posted:


If you have any feedback about these posts and/or wishes for the following posts topics, please don't hesitate to email it to me by replying to this email.

Regards,
Dmitri Fedorov
Software Architect
Solace Systems, Inc.
Ottawa, ON Canada

Solace Systems accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Solace Systems.


Re: [Openmama-dev] OpenMAMA 6.1.0 En Route

Frank Quinn <fquinn.ni@...>
 

Hi Dmitri,

Yes this is the release that we were going to call 2.4.2 and ideally release at the end of next month.

Cheers,
Frank


On Fri, 29 Jul 2016, 20:19 Dmitri Fedorov, <dfedorov.solace@...> wrote:
Hi Frank,

This is a cut from the "next" branch, right? The one that was going to be 2.4.2 and planned for the end of summer?

Regards,
Dmitri Fedorov
Software Architect
Solace Systems, Inc.
Ottawa, ON Canada

Solace Systems accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Solace Systems.

On 29 July 2016 at 14:54, Frank Quinn <fquinn.ni@...> wrote:
Hi Folks,

In the interest of simplifying version compatibility, OpenMAMA's next release version is going to be 6.1.0. I appreciate it's a big jump but it should make the following things clear to the community and anyone watching the project:
  1. OpenMAMA is the equivalent to Vela's "MAMA 6" (Vela have also agreed to move to 6.1.x to align with OpenMAMA).
  2. Users of both Vela's Enterprise release of MAMA and OpenMAMA can easily see what equivalent versions will be.
  3. Users upgrading from MAMA 5 can have more sane #ifdefs in their code if they're trying to support both versions during migration.
I plan on taking the cut within the next week, so if anyone has any issues that they would like to be included or patches you would like to be included in the next release, please reply to this thread with details.

Cheers,
Frank

_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: [Openmama-dev] OpenMAMA 6.1.0 En Route

Dmitri Fedorov
 

Hi Frank,

This is a cut from the "next" branch, right? The one that was going to be 2.4.2 and planned for the end of summer?

Regards,
Dmitri Fedorov
Software Architect
Solace Systems, Inc.
Ottawa, ON Canada

Solace Systems accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Solace Systems.

On 29 July 2016 at 14:54, Frank Quinn <fquinn.ni@...> wrote:
Hi Folks,

In the interest of simplifying version compatibility, OpenMAMA's next release version is going to be 6.1.0. I appreciate it's a big jump but it should make the following things clear to the community and anyone watching the project:
  1. OpenMAMA is the equivalent to Vela's "MAMA 6" (Vela have also agreed to move to 6.1.x to align with OpenMAMA).
  2. Users of both Vela's Enterprise release of MAMA and OpenMAMA can easily see what equivalent versions will be.
  3. Users upgrading from MAMA 5 can have more sane #ifdefs in their code if they're trying to support both versions during migration.
I plan on taking the cut within the next week, so if anyone has any issues that they would like to be included or patches you would like to be included in the next release, please reply to this thread with details.

Cheers,
Frank

_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev



OpenMAMA 6.1.0 En Route

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

In the interest of simplifying version compatibility, OpenMAMA's next release version is going to be 6.1.0. I appreciate it's a big jump but it should make the following things clear to the community and anyone watching the project:
  1. OpenMAMA is the equivalent to Vela's "MAMA 6" (Vela have also agreed to move to 6.1.x to align with OpenMAMA).
  2. Users of both Vela's Enterprise release of MAMA and OpenMAMA can easily see what equivalent versions will be.
  3. Users upgrading from MAMA 5 can have more sane #ifdefs in their code if they're trying to support both versions during migration.
I plan on taking the cut within the next week, so if anyone has any issues that they would like to be included or patches you would like to be included in the next release, please reply to this thread with details.

Cheers,
Frank


Solace OpenMAMA tutorials available

Dmitri Fedorov
 

Hi all,

Solace Systems has started publishing a series of OpenMAMA tutorials, helping developers to adapt OpenMAMA and get a better understanding of proper configuration of the Solace Middleware Bridge for OpenMAMA.

The posts are meant to complement existing OpenMAMA samples and documentation. 

This is the first two posts:

If you have any feedback about these posts and/or wishes for the following posts topics, please don't hesitate to email it to me by replying to this email.

Regards,
Dmitri Fedorov
Software Architect
Solace Systems, Inc.
Ottawa, ON Canada

Solace Systems accepts no liability for the content of this email, or for the consequences of any actions taken on the basis of the information provided, unless that information is subsequently confirmed in writing. Any views or opinions presented in this email are solely those of the author and do not necessarily represent those of Solace Systems.


Re: [Openmama-dev] OpenMAMA End of Support for Avis

Frank Quinn <fquinn.ni@...>
 

Cheers Damian,

With that in mind, I just created: https://github.com/OpenMAMA/OpenMAMA-avis including a README explaining the situation.

As I have heard no objections, we'll be be removing avis from the core code and updating the documentation and website etc. over the next week or so.

Cheers,
Frank

On Mon, Jul 25, 2016 at 10:19 AM, Damian Maguire <dmaguire@...> wrote:

For what it's worth, I'm fully supportive of this move, as was everyone I spoke to about it. I think Avis is a distraction for core OpenMAMA that we don't need - the CI work, build system changes, core code changes, unit tests etc all have additional costs when taking Avis into account, and I think it's actually very confusing for end users of the API. 


I do wonder if it would be worth extracting the code to a separate repo anyway, with a deprecation notice - at least it remains visible and available to other users who are interested (and might be an opportunity to firm up the process/framework for non-core bridges). 


Cheers, 


D



DAMIAN MAGUIRE 

Senior Sales Engineer 

 

M. +44 783 584 4770 

 

Adelaide Exchange Building2nd Floor, 24-26 Adelaide StreetBelfast, BT2 8GD  

velatradingtech.com | @vela_tt


1467620971571_PastedImage


From: openmama-dev-bounces@... <openmama-dev-bounces@...> on behalf of Frank Quinn <fquinn.ni@...>
Sent: 22 July 2016 09:35:41
To: openmama-dev; openmama-users@...
Subject: Re: [Openmama-dev] OpenMAMA End of Support for Avis
 
Hi Folks,

Last call on this - if I hear no word back, it's getting taken out on Monday.

Cheers,
Frank

On Sat, Jul 16, 2016 at 7:18 PM, Frank Quinn <fquinn.ni@...> wrote:
Hi Folks,

I'd like too propose that we stop maintaining the avis bridge for OpenMAMA and that the release which just went out (2.4.1) is the last to support avis, which may then be removed from the code base.

This has already been agreed by the Steering Committee, but I want to put it out there to the users and dev mailing lists to see if there are any further objections, or if anyone would like to step up and take ownership of the bridge (e.g. split out an external OpenMAMA-avis project).

There are several reasons for wanting to retire support for avis:

1. To the best of my knowledge, nobody is using it (we never hear of it on the mailing list any more).
2. Avis itself hasn't been updated in 7 years
3. It's a resource drain on any new functionality we may add for zero return.
4. Even today, doesn't pass all unit tests because the payload doesn't support all OpenMAMA functionality.

It's all about focus - I want to make sure that we spend time focusing on what people are actively using.

If anyone has any sentimental attachment to the avis bridge, please speak up now.

We are currently scheduling the removal of avis from the codebase on 25th July.

Cheers,
Frank


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC