Date   
Use of multiple bridges

Sergey Emantayev
 

I'm exploring an option for my MAMDA application to receive market data from multiple bridges. However, the Mamda initialisation API like MamdaCommonFields::setDictionary is confusing me because it is the static function accepting a single dictionary. Different bridges may have different dictionaries and it seems to work with only a single bridge. Is there any solution or workaround?

Best Regards,
Sergey Emantayev

OpenMAMA 6.2.3 Released

Frank Quinn <fquinn@...>
 

Hi Folks,

 

We are pleased to announce the final release of OpenMAMA 6.2.3 is now available:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.3-release

 

This is a hotfix release to address two key issues which were discovered as part of the recent 6.2.2 release:

 

* Restore mamaSubscription RecoverGaps functions accidentally removed in the last release

* Restore missing wombat portability headers in 6.2.2 Release

 

For a complete list of the issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/9?closed=1

 

Cheers,

Frank

 

 

Frank Quinn

Cascadium

T: +44 (0) 28 8678 8015

E: fquinn@...

W: http://cascadium.io

 

Re: OpenMAMA 6.2.2 Released

Damian Maguire
 

Thanks for this Frank, some fantastic work gone into this one. Great to have the release available in so many of the standard repos now, and with Cmake support getting bedded in it should be a lot easier for people to build from scratch as well (big thanks to Victor for all the effort on that one).


Huge amount of other work in there as well, so thanks to all the contributors along the way.


Thanks,


Damian


DAMIAN MAGUIRE 

Senior Sales Engineer

Vela

 

O. +44 289 568 0298  

M. +44 783 584 4770 

dmaguire@... 

 

Adelaide Exchange Building2nd Floor, 24-26 Adelaide StreetBelfast, BT2 8GD  

tradevela.com | @tradevela


1518792739120_PastedImage




From: Openmama-dev@... <Openmama-dev@...> on behalf of Frank Quinn <fquinn@...>
Sent: 02 July 2018 21:11
To: openmama-dev@...; openmama-users@...
Subject: [Openmama-dev] OpenMAMA 6.2.2 Released
 

Hi Folks,

 

We are pleased to announce the final release of OpenMAMA 6.2.2 is now available:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.2-release

 

Note that for the first time, a OpenMAMA generally available release is now available via Maven Central, Microsoft’s vcpkg and yum repositories (via Cloudsmith).

 

Documentation will be coming in the following weeks with more details including how to use our new experimental cmake build system!

 

This is a maintenance release which fixes several outstanding bugs and introduces some new functionality.

 

Key features include:

 

  • Introduction of pluggable DQ strategies (Market Data Subscription recovery mechanisms). See https://openmama.github.io/openmama_rfc_dq_pluggability.html
  • Added new methods mamaMsg_toJsonString and mamaMsg_toNormalizedString
  • OpenMAMA source structure moved to maven and build system moved to gradle
  • Cmake support now available (experimental). Note it will replace scons in the next release and supports Windows, Linux and OSX
  • OpenMAMA added to Microsoft vcpkg for easy nuget packaging and building from source
  • OpenMAMA Integration headers now available to allow developers to build plugins and bridges without access to the source code. See https://openmama.github.io/openmama_bridge.html#openmama-integration-headers.
  • Added implementation for mamaPrice_setFromString
  • MamaPrice can now support decimal point precision up to 16 places
  • Implement MamaFieldCache in JNI enhancement
  • Support for setting mamaDateTime with pre-1970 dates on Unix platforms
  • Removal of Visual Studio compiler warnings
  • Added Appveyor integration for CI
  • Add support for autoloading payload bridges from config

 

For a complete list of all 55 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/8?closed=1

 

A special thanks to all developers, contributors and testers who helped is getting this out door.

 

Cheers,

Frank

 

 

Frank Quinn

Cascadium

T: +44 (0) 28 8678 8015

E: fquinn@...

W: http://cascadium.io

 


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Systems LLC

OpenMAMA 6.2.2 Released

Frank Quinn <fquinn@...>
 

Hi Folks,

 

We are pleased to announce the final release of OpenMAMA 6.2.2 is now available:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.2-release

 

Note that for the first time, a OpenMAMA generally available release is now available via Maven Central, Microsoft’s vcpkg and yum repositories (via Cloudsmith).

 

Documentation will be coming in the following weeks with more details including how to use our new experimental cmake build system!

 

This is a maintenance release which fixes several outstanding bugs and introduces some new functionality.

 

Key features include:

 

  • Introduction of pluggable DQ strategies (Market Data Subscription recovery mechanisms). See https://openmama.github.io/openmama_rfc_dq_pluggability.html
  • Added new methods mamaMsg_toJsonString and mamaMsg_toNormalizedString
  • OpenMAMA source structure moved to maven and build system moved to gradle
  • Cmake support now available (experimental). Note it will replace scons in the next release and supports Windows, Linux and OSX
  • OpenMAMA added to Microsoft vcpkg for easy nuget packaging and building from source
  • OpenMAMA Integration headers now available to allow developers to build plugins and bridges without access to the source code. See https://openmama.github.io/openmama_bridge.html#openmama-integration-headers.
  • Added implementation for mamaPrice_setFromString
  • MamaPrice can now support decimal point precision up to 16 places
  • Implement MamaFieldCache in JNI enhancement
  • Support for setting mamaDateTime with pre-1970 dates on Unix platforms
  • Removal of Visual Studio compiler warnings
  • Added Appveyor integration for CI
  • Add support for autoloading payload bridges from config

 

For a complete list of all 55 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/8?closed=1

 

A special thanks to all developers, contributors and testers who helped is getting this out door.

 

Cheers,

Frank

 

 

Frank Quinn

Cascadium

T: +44 (0) 28 8678 8015

E: fquinn@...

W: http://cascadium.io

 

OpenMAMA-6.2.2-rc2 Now Available

Frank Quinn <fquinn@...>
 

Hi Folks,

 

We were planning on releasing today but a pull request landed yesterday containing some bugfixes for the recently added plugin code which I have deemed as necessary for this release, so I have now cut OpenMAMA 6.2.2-rc2 which can be found here:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.2-rc2

 

Considering this is bugfix only, the release candidate window will be one week, making the target release date (all being well) 28th June.

 

I encourage all application and bridge developers to rigorously test this new release with their software especially with respect to the market data subscription life cycle.

 

If there are any further incoming changes please advise me asap to see if the above dates need to be revised.

 

Cheers,

Frank

 

Frank Quinn, Cascadium | +44 (0) 28 8678 8015 | http://cascadium.io

 

From: Frank Quinn
Sent: 13 June 2018 20:36
To: 'openmama-dev@...' <openmama-dev@...>; 'openmama-users@...' <openmama-users@...>
Subject: RE: OpenMAMA-6.2.2-rc1 Now Available

 

Hi Folks,

 

Hope testing is going well!

 

We’ve just submitted a change to correct a few unit test compiler warnings and memory leaks recently introduced but nothing which impacts core code so there’s no current reason to extend the RC window.

 

Just a reminder that we’re going into the final week of testing here so if anyone has spotted anything unusual please speak up now if you want a fix to make it into this release!

 

Cheers,

Frank

 

Frank Quinn, Cascadium | +44 (0) 28 8678 8015 | http://cascadium.io

 

From: Frank Quinn
Sent: 29 May 2018 21:07
To: 'openmama-dev@...' <openmama-dev@...>; 'openmama-users@...' <openmama-users@...>
Subject: OpenMAMA-6.2.2-rc1 Now Available

 

Hi Folks,

 

We are pleased to announce the first release candidate for OpenMAMA 6.2.2 is now available:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.2-rc1

 

This is a maintenance release which fixes several outstanding bugs and introduces some new functionality.

 

Key features include:

 

  • Introduction of pluggable DQ strategies (Market Data Subscription recovery mechanisms). See https://openmama.github.io/openmama_rfc_dq_pluggability.html
  • Added new methods mamaMsg_toJsonString and mamaMsg_toNormalizedString
  • OpenMAMA source structure moved to maven and build system moved to gradle
  • Cmake support now available (experimental). Note it will replace scons in the next release and supports Windows, Linux and OSX
  • OpenMAMA added to Microsoft vcpkg for easy nuget packaging and building from source
  • OpenMAMA Integration headers now available to allow developers to build plugins and bridges without access to the source code. See https://openmama.github.io/openmama_bridge.html#openmama-integration-headers.
  • Added implementation for mamaPrice_setFromString
  • MamaPrice can now support decimal point precision up to 16 places
  • Implement MamaFieldCache in JNI enhancement
  • Support for setting mamaDateTime with pre-1970 dates on Unix platforms
  • Removal of Visual Studio compiler warnings
  • Added Appveyor integration for CI
  • Add support for autoloading payload bridges from config

 

For a complete list of all 54 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/8?closed=1

 

Thank you all in advance for your help in testing - if you spot any issues, please follow our guidelines for raising an issue, or even better, follow our guidelines for raising a patch.

 

Since the release is significant, the testing period will be 3 weeks from today making the target release date 20th June and everyone is invited to try it out - binary releases are available at the link above.

 

If critical issues are found and not resolved before this date, we will continue to go through weekly release candidates until have a stable release ready.

 

Cheers,

Frank

 

Frank Quinn

Cascadium

T: +44 (0) 28 8678 8015

E: fquinn@...

W: http://cascadium.io

 

Re: OpenMAMA-6.2.2-rc1 Now Available

Frank Quinn <fquinn@...>
 

Hi Folks,

 

Hope testing is going well!

 

We’ve just submitted a change to correct a few unit test compiler warnings and memory leaks recently introduced but nothing which impacts core code so there’s no current reason to extend the RC window.

 

Just a reminder that we’re going into the final week of testing here so if anyone has spotted anything unusual please speak up now if you want a fix to make it into this release!

 

Cheers,

Frank

 

Frank Quinn, Cascadium | +44 (0) 28 8678 8015 | http://cascadium.io

 

From: Frank Quinn
Sent: 29 May 2018 21:07
To: 'openmama-dev@...' <openmama-dev@...>; 'openmama-users@...' <openmama-users@...>
Subject: OpenMAMA-6.2.2-rc1 Now Available

 

Hi Folks,

 

We are pleased to announce the first release candidate for OpenMAMA 6.2.2 is now available:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.2-rc1

 

This is a maintenance release which fixes several outstanding bugs and introduces some new functionality.

 

Key features include:

 

  • Introduction of pluggable DQ strategies (Market Data Subscription recovery mechanisms). See https://openmama.github.io/openmama_rfc_dq_pluggability.html
  • Added new methods mamaMsg_toJsonString and mamaMsg_toNormalizedString
  • OpenMAMA source structure moved to maven and build system moved to gradle
  • Cmake support now available (experimental). Note it will replace scons in the next release and supports Windows, Linux and OSX
  • OpenMAMA added to Microsoft vcpkg for easy nuget packaging and building from source
  • OpenMAMA Integration headers now available to allow developers to build plugins and bridges without access to the source code. See https://openmama.github.io/openmama_bridge.html#openmama-integration-headers.
  • Added implementation for mamaPrice_setFromString
  • MamaPrice can now support decimal point precision up to 16 places
  • Implement MamaFieldCache in JNI enhancement
  • Support for setting mamaDateTime with pre-1970 dates on Unix platforms
  • Removal of Visual Studio compiler warnings
  • Added Appveyor integration for CI
  • Add support for autoloading payload bridges from config

 

For a complete list of all 54 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/8?closed=1

 

Thank you all in advance for your help in testing - if you spot any issues, please follow our guidelines for raising an issue, or even better, follow our guidelines for raising a patch.

 

Since the release is significant, the testing period will be 3 weeks from today making the target release date 20th June and everyone is invited to try it out - binary releases are available at the link above.

 

If critical issues are found and not resolved before this date, we will continue to go through weekly release candidates until have a stable release ready.

 

Cheers,

Frank

 

Frank Quinn

Cascadium

T: +44 (0) 28 8678 8015

E: fquinn@...

W: http://cascadium.io

 

OpenMAMA-6.2.2-rc1 Now Available

Frank Quinn <fquinn@...>
 

Hi Folks,

 

We are pleased to announce the first release candidate for OpenMAMA 6.2.2 is now available:

 

https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.2-rc1

 

This is a maintenance release which fixes several outstanding bugs and introduces some new functionality.

 

Key features include:

 

  • Introduction of pluggable DQ strategies (Market Data Subscription recovery mechanisms). See https://openmama.github.io/openmama_rfc_dq_pluggability.html
  • Added new methods mamaMsg_toJsonString and mamaMsg_toNormalizedString
  • OpenMAMA source structure moved to maven and build system moved to gradle
  • Cmake support now available (experimental). Note it will replace scons in the next release and supports Windows, Linux and OSX
  • OpenMAMA added to Microsoft vcpkg for easy nuget packaging and building from source
  • OpenMAMA Integration headers now available to allow developers to build plugins and bridges without access to the source code. See https://openmama.github.io/openmama_bridge.html#openmama-integration-headers.
  • Added implementation for mamaPrice_setFromString
  • MamaPrice can now support decimal point precision up to 16 places
  • Implement MamaFieldCache in JNI enhancement
  • Support for setting mamaDateTime with pre-1970 dates on Unix platforms
  • Removal of Visual Studio compiler warnings
  • Added Appveyor integration for CI
  • Add support for autoloading payload bridges from config

 

For a complete list of all 54 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/8?closed=1

 

Thank you all in advance for your help in testing - if you spot any issues, please follow our guidelines for raising an issue, or even better, follow our guidelines for raising a patch.

 

Since the release is significant, the testing period will be 3 weeks from today making the target release date 20th June and everyone is invited to try it out - binary releases are available at the link above.

 

If critical issues are found and not resolved before this date, we will continue to go through weekly release candidates until have a stable release ready.

 

Cheers,

Frank

 

Frank Quinn

Cascadium

T: +44 (0) 28 8678 8015

E: fquinn@...

W: http://cascadium.io

 

Re: [Openmama-dev] Let's test cmake support

Frank Quinn <fquinn@...>
 

Hi folks,

Further to this, we have now wired up with C#, install rules and unit tests on Linux, Windows and even native OSX (with some recent changes).

This is a huge step because it effectively outsources compiler support which we effectively had to manage ourselves with our previous scons infrastructure as well as fight with python environments.

With that in mind I have now raised https://github.com/OpenMAMA/OpenMAMA/pull/361 which will hopefully make it into next soon!

After the next release goes out (which I propose is soon), we can look at making cmake the default for CI and the release following that one.

Cheers,
Frank

On 27 Apr 2018 10:05, Victor Maleyev <imnotmindlin@...> wrote:
Hi guys,

Me and Frank made some efforts to support CMake build system: it builds MAMA on Linux and Windows. Unfortunately it is not in trunk yet but I desperately need any feedback on how it works to make it stable and ready for release. Just clone the repo from here: https://github.com/fquinner/OpenMAMA/tree/feature-cmake-support and try build it like this:
mkdir build
cmake ..
make

Make sure that flex, Apache portable runtime and gradle are installed.

Feel free to mail me if issues are found.



[Openmama-dev] Let's test cmake support

Victor Maleyev
 

Added openmama-users

-------- Пересылаемое сообщение--------
27.04.2018, 12:05, "Victor Maleyev" <imnotmindlin@...>:

Hi guys,

Me and Frank made some efforts to support CMake build system: it builds MAMA on Linux and Windows. Unfortunately it is not in trunk yet but I desperately need any feedback on how it works to make it stable and ready for release. Just clone the repo from here: https://github.com/fquinner/OpenMAMA/tree/feature-cmake-supp… and try build it like this:
mkdir build
cmake ..
make

Make sure that flex, Apache portable runtime and gradle are installed.

Feel free to mail me if issues are found.




-------- Завершение пересылаемого сообщения --------



Re: [Openmama-dev] Did MAMAIgnoreDeprecatedOpen ever work on Linux? [I]

Frank Quinn <fquinn.ni@...>
 

Hi Yury,

Yes that method is deprecated because OpenMAMA prefers dynamic loading methods now (so that payloads don't need to be registered in OpenMAMA's core code to load or depend on magic characters).

Have you looked at mamaMsg_createForPayloadBridge? It should do the same thing, but accepts a payload bridge rather than a char identifier and is not deprecated.

Damian also provided some helper functions included 6.2.1 to help look up bridges by name which might be helpful for lookup and cache, including mama_getPayloadBridge.

Cheers,
Frank

On Tue, Oct 3, 2017 at 10:07 AM, Yury Batrakov <yury.batrakov@...> wrote:

Classification: For internal use only

Hey Damian,

 

OK, thank you for clarification. The reason why I thought that deprecation warnings should be deleted automatically (after defining MAMA_DLL and BRIDGE preprocessor variables) is the following declaration of mamaMsg_createForPayload

MAMAIgnoreDeprecatedOpen

MAMAExpDeprecatedDLL(

        "mamaMsg_createForPayload has been deprecated, use dynamic loading instead!")

extern mama_status

mamaMsg_createForPayload (mamaMsg* msg, const char id);

MAMAIgnoreDeprecatedClose

 

The background of this question is: I’m writing a middleware bridge which always has to create messages for specific payload - it’s a problem because mamaMsg_createForPayload is deprecated and mamaMsg_create creates messages for “default” payload - the payload initialized by a first bridge created by OpenMAMA. This will be a problem when two or more bridges are loaded to a process.

 

 

From: Damian Maguire [mailto:dmaguire@...]
Sent: Tuesday, October 03, 2017 10:27 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-users <openmama-users@lists.openmama.org>; openmama-dev <openmama-dev@....org>
Subject: Re: Did MAMAIgnoreDeprecatedOpen ever work on Linux?

 

Hey Yury,

 

I'm not sure the behaviour you're describing is an issue - the purpose of the pragma's is to allow you to use 'deprecated' features of the API without generating warnings, rather than disable the deprecation completely. For example, when we implemented dynamic loading, we deprecated the 'mamaPayloadType' here. However, in order to support legacy clients, we continue to use the type within the MAMA codebase. We generally aim to be 'warning free' in the code, and understand the risks associated with using the type, so where we make use of mamaPayloadType we mark them with MamaIgnoreDeprecatedOpen and MamaIgnoreDeprecatedClose pragma's. This disables the deprecation specific warnings at those sites - for example, here.

 

Using your code, the equivalent would be:

 

int __attribute__((deprecated)) b() {

   return 0;

}

 

int main() {

        printf("GCC %d %d\n" , __GNUC__, __GNUC_MINOR__);

MamaIgnoreDeprecatedOpen

        b();

MamaIgnoreDeprecatedClose

        return 0;

}

 

Which shouldn't warn about the use of b();.

 

Let me know if that makes sense, or if you have further questions around this.

 

Thanks,

 

Damian

 

 

DAMIAN MAGUIRE 

Senior Sales Engineer 

 

 

Adelaide Exchange Building, 2nd Floor, 24-26 Adelaide Street, Belfast, BT2 8GD  

velatradingtech.com | @vela_tt

 

 


From: openmama-dev-bounces@lists.openmama.org <openmama-dev-bounces@lists.openmama.org> on behalf of Yury Batrakov <yury.batrakov@...>
Sent: 02 October 2017 17:16
To: openmama-users; openmama-dev
Subject: [Openmama-dev] Did MAMAIgnoreDeprecatedOpen ever work on Linux?

 

Classification: Public

Hi team,

 

Is MAMAIgnoreDeprecatedOpen supposed to work for Linux with gcc > 4.6?

See the following example:

 

// Next 3 lines were copied from wombat/…/linux/port.h

_Pragma ("GCC diagnostic push")

_Pragma ("GCC diagnostic ignored \"-Wdeprecated\"")

_Pragma ("GCC diagnostic ignored \"-Wdeprecated-declarations\"")

 

int __attribute__((deprecated)) b() {

   return 0;

}

 

_Pragma ("GCC diagnostic pop")

 

int main() {

        printf("GCC %d %d\n" , __GNUC__, __GNUC_MINOR__);

        b();

        return 0;

}

 

When compiling with 4.8 it shows the following warnings anyway:

/opt/gcc/gcc-4.8.1/bin/g++ -Wall -Wextra 123.c

123.c: In function ‘int main()’:

123.c:15:9: warning: ‘int b()’ is deprecated (declared at 123.c:7) [-Wdeprecated-declarations]

         b();

         ^

123.c:15:11: warning: ‘int b()’ is deprecated (declared at 123.c:7) [-Wdeprecated-declarations]

         b();

           ^

 

But if we place those pragmas around invocation of b() (not around the definition) all warnings go away



---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC



---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

_______________________________________________
Openmama-dev mailing list
Openmama-dev@....org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: [Openmama-dev] Questions about MamaStartCallback

Frank Quinn <fquinn.ni@...>
 

Hi Yury,

I did have this proposed change merged, but have since discovered that it causes a deadlock in our unit test on windows.

On closer inspection, it looks like the problem is caused by the fact that making mama_stop blocking will effectively deadlock any application which attempts to call mama_stop from a MAMA event coming off the default queue (like our unit test application). Since I expect this is a common enough pattern, we'll have to revert this change.

Note that the argument to startBackground is misleadingly a stop callback - i.e. it will be called when mama_start *unblocks*. With this in mind, you should be able to get your MyMamaConnection destructor to wait on a semaphore raised by onStartComplete method (use mama_startBackgroundEx rather than mama_startBackground to provide a closure) to raise a semaphore to simulate the equivalent behaviour and prevent the crash.

Cheers,
Frank

On Mon, Aug 14, 2017 at 1:21 PM, Yury Batrakov <yury.batrakov@...> wrote:
Classification: Public

Hi guys,

As everybody knows there is possibility to start MAMA bridge in background thread in C++ API: function Mama::startBackground (mamaBridge bridgeImpl, MamaStartCallback* cb)
There are a couple of problems with this function though:
    1. We can't ignore "MAMA startup finished" event setting  the value for the second argument to nullptr - this can be fixed as a simple nullptr check in void MAMACALLTYPE stopCb (mama_status status, mamaBridge, void* closure) in mamacpp.cpp
    2. There is race condition between thread calling Mama::stop and internal MAMA thread spawned  by startBackground function. This race may lead to program crash.

Let me describe #2 - consider the following scenario:

We have a simple class MyMamaConnection

class MyMamaStartCallback : public MamaStartCallback {
    void onStartComplete (Wombat::MamaStatus status) override {
           // .... Some bridge specific callback code
    }
};

class MyMamaConnection {
    mamaBridge m_bridge;
    MyMamaStartCallback m_callback;

    MyMamaConnection() {
        Mama::startBackground (m_bridge, &m_callback)
    }

    ~MyMamaConnection() {
        Mama::stop(m_bridge);
    }
};

The crash occur in destructor because Mama::stop doesn't guarantee that no threads are still using m_callback pointer after this function exited. In our case MyMamaConnection's destructor destroys m_callback object while it is still in use by the thread created by Mama::startBackground.

Here are more details, file mama.c:

static void* mamaStartThread (void* closure)
{
...
    rval = mama_start (cb->mBridgeImpl); // Imagine user thread Thread_1 calling ~MyMamaConnection() from our example.

    // Current thread may be suspended by OS at this point while Thread_1 continues its work and deletes the object pointed by closure
...

    if (cb->mStopCallbackEx)

        // cb->mClosure points to destroyed object.

        cb->mStopCallbackEx (rval, cb->mBridgeImpl, cb->mClosure);
...
}

One of the possible solutions is to move this code from mama_close() to mama_stop():
                        // Get name of start background thread and destroy it
                        const char* threadName = wombatThread_getThreadName(middlewareLib->bridge->mStartBackgroundThread);
                        wombatThread_destroy (threadName);

Let me know if it's possible to have this fixed in the next release.


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@....org
https://lists.openmama.org/mailman/listinfo/openmama-dev

Re: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Konstantin Baydarov
 

Classification: Public

Hi, Tom.

As Frank suggested, I switched our app from destroy () to destroyEx() (MamaSubscription), but I still get a segmentation fault. I think the reason of segfault is that while RMDSBridgeSubscription::OnMessage() calls mamaSubscription_processMsg() in the tick42rmds thread - the subscription is destroyed in the mamaQueue_dispatch() by the mama_start() thread. The problem is that tick42rmds processing thread, calling RMDSBridgeSubscription::OnMessage(), is not synchronized with mama_start() thread, so mama_start() thread can destroy mama subscription at any time including in the moment of running mamaSubscription_processMsg().
Here are stacks of both threads that operates on the same subscription concurrently without any synchronization:
1) mama_start() thread that destroys subscription:
#0 mamaSubscription_destroy (subscription=0x0) at mama/c_cpp/src/c/subscription.c:2929
#1 0x00007ffff51f259a in tick42rmdsBridgeMamaInbox_destroy () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#2 0x00007ffff78b836b in mamaInbox_destroy (inbox=0x1065f10) at mama/c_cpp/src/c/inbox.c:249
#3 0x00007ffff78a2aec in imageRequest_destroy (request=0x104a300) at mama/c_cpp/src/c/imagerequest.c:425
#4 0x00007ffff78a1425 in dqContext_cleanup (ctx=0x7fffec3add10) at mama/c_cpp/src/c/dqstrategy.c:621
#5 0x00007ffff78cc4d3 in mamaSubscription_cleanup (subscription=0x7fffec3adae0) at mama/c_cpp/src/c/subscription.c:1510
#6 0x00007ffff78ce5df in mamaSubscription_destroy (subscription=0x7fffec3adae0) at mama/c_cpp/src/c/subscription.c:2948
#7 0x00007ffff78cc7b6 in mamaSubscription_DestroyThroughQueueCB (Queue=0x61e340, closure=0x7fffec3adae0) at mama/c_cpp/src/c/subscription.c:1604
#8 0x00007ffff78e634f in wombatQueue_dispatchInt (queue=0x61e430, data=0x0, closure=0x0, isTimed=1 '\001', timout=500) at common/c_cpp/src/c/queue.c:319
#9 0x00007ffff78e63d2 in wombatQueue_timedDispatch (queue=0x61e430, data=0x0, closure=0x0, timeout=500) at common/c_cpp/src/c/queue.c:335
#10 0x00007ffff51fa6d9 in tick42rmdsBridgeMamaQueue_dispatch () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#11 0x00007ffff78c3a8b in mamaQueue_dispatch (queue=0x61e340) at mama/c_cpp/src/c/queue.c:819
#12 0x00007ffff51f20a8 in tick42rmdsBridge_start () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#13 0x00007ffff78a8810 in mama_start (bridgeImpl=0x61d660) at mama/c_cpp/src/c/mama.c:1591
#14 0x00007ffff7b8ee53 in Wombat::Mama::start (bridgeImpl=0x61d660) at mama/c_cpp/src/cpp/mamacpp.cpp:198
#15 0x000000000040c53a in MamaListen::start (this=0x7fffffffd690) at mamalistencpp_mt.cpp:1072
#16 0x000000000040ebaf in main (argc=9, argv=0x7fffffffd948) at mamalistencpp_mt.cpp:1915

2) tick42rmds thread - moment of Segfault:
Program received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7ffff38ff700 (LWP 25476)]
0x00007ffff78a3508 in imageRequest_stopWaitForResponse (request=0x0) at mama/c_cpp/src/c/imagerequest.c:772
772 mamaSubscription_getTransport ( impl->mSubscription, &tport );
(gdb) bt
#0 0x00007ffff78a3508 in imageRequest_stopWaitForResponse (request=0x0) at mama/c_cpp/src/c/imagerequest.c:772
#1 0x00007ffff78cbe53 in mamaSubscription_stopWaitForResponse (subscription=0x7fffe0587200, ctx=0x7fffe0587420) at mama/c_cpp/src/c/subscription.c:1264
#2 0x00007ffff78a393e in processPointToPointMessage (callback=0x7fffe0587a00, msg=0x63eda0, msgType=6, ctx=0x7fffe0587420) at mama/c_cpp/src/c/listenermsgcallback.c:171
#3 0x00007ffff78a3fdc in listenerMsgCallback_processMsg (callback=0x7fffe0587a00, msg=0x63eda0, ctx=0x7fffe0587420) at mama/c_cpp/src/c/listenermsgcallback.c:482
#4 0x00007ffff78cd9a7 in _mamaSubscription_processMsg (subscription=0x7fffe0587200, msg=0x63eda0) at mama/c_cpp/src/c/subscription.c:2322
#5 0x00007ffff78cd7ff in mamaSubscription_processMsg (subscription=0x7fffe0587200, msg=0x63eda0) at mama/c_cpp/src/c/subscription.c:2272
#6 0x00007ffff51fae33 in RMDSBridgeSubscription::OnMessage(mamaMsgImpl_*, mamaMsgType) () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#7 0x00007ffff5257632 in UPASubscription::NotifyListenersRefreshMessage(mamaMsgImpl_*, boost::shared_ptr<RMDSBridgeSubscription>, bool) ()
from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#8 0x00007ffff525bad2 in UPASubscription::InternalProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#9 0x00007ffff52611d5 in UPASubscription::ProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#10 0x00007ffff522a268 in UPAConsumer::ProcessResponse(RsslChannel*, RwfBuffer*) () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#11 0x00007ffff522a850 in UPAConsumer::ReadFromChannel(RsslChannel*) () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#12 0x00007ffff522bb1d in UPAConsumer::Run() () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#13 0x00007ffff5211b1c in RMDSSubscriber::threadFunc(void*) () from /export/home/dnauat/komodo/UAT/UK/kb/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#14 0x00007ffff6733806 in start_thread () from /lib64/libpthread.so.0
#15 0x00007ffff5af59bd in clone () from /lib64/libc.so.6
#16 0x0000000000000000 in ?? ()

BR,
Konstantin Baydarov

-----Original Message-----
From: Tom Doust [mailto:tom.doust@...]
Sent: Tuesday, June 20, 2017 8:40 PM
To: Konstantin Baydarov <konstantin.baydarov@...>; Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-dev] [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Yes we call back on the bridge’s thread. We don’t queue the message, we leave the client code to do that if it wants to. This is by design.


On 20/06/2017, 18:18, "Konstantin Baydarov" <konstantin.baydarov@...> wrote:

Classification: Public

Hi, Tom.

I noticed, that comparing to qpid bridge(that comes with openmama sources), tick42rmds calls mamaSubscription_processMsg() method from separate thread and not from mamaQueue_dispatch(), wondering if it's correct. Probably it's one of the reasons of the issue that we facing?
Qpid bridge call stack:
#0 mamaSubscription_processMsg (subscription=0x76e150, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:2226
#1 0x00007ffff7b4c580 in imageRequestImpl_onInitialMessage (msg=0x7aebb0, closure=0x772710) at mama/c_cpp/src/c/imagerequest.c:225
#2 0x00007ffff648bede in qpidBridgeMamaInboxImpl_onMsg (subscription=0x772900, msg=0x7aebb0, closure=0x7727c0, itemClosure=0x0) at mama/c_cpp/src/c/bridge/qpid/inbox.c:298
#3 0x00007ffff7b76ab4 in mamaSubscription_forwardMsg (subscription=0x772900, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:1422
#4 0x00007ffff7b781ef in mamaSubscription_processMsg (subscription=0x772900, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:2315
#5 0x00007ffff6490818 in qpidBridgeMamaTransportImpl_queueCallback (queue=0x60eb50, closure=0x61df80) at mama/c_cpp/src/c/bridge/qpid/transport.c:1083
#6 0x00007ffff7b90a1f in wombatQueue_dispatchInt (queue=0x60ecb0, data=0x0, closure=0x0, isTimed=1 '\001', timout=500) at common/c_cpp/src/c/queue.c:319
#7 0x00007ffff7b90aa2 in wombatQueue_timedDispatch (queue=0x60ecb0, data=0x0, closure=0x0, timeout=500) at common/c_cpp/src/c/queue.c:335
#8 0x00007ffff648e720 in qpidBridgeMamaQueue_dispatch (queue=0x60ec40) at mama/c_cpp/src/c/bridge/qpid/queue.c:265
#9 0x00007ffff7b6e1de in mamaQueue_dispatch (queue=0x60eb50) at mama/c_cpp/src/c/queue.c:824
#10 0x00007ffff648a8c3 in qpidBridge_start (defaultEventQueue=0x60eb50) at mama/c_cpp/src/c/bridge/qpid/bridge.c:196
#11 0x00007ffff7b52976 in mama_start (bridgeImpl=0x60e750) at mama/c_cpp/src/c/mama.c:1659
#12 0x0000000000403e61 in buildDataDictionary () at mama/c_cpp/src/examples/c/mamalistenc.c:647
#13 0x000000000040366f in main (argc=9, argv=0x7fffffffd728) at mama/c_cpp/src/examples/c/mamalistenc.c:335

tick42rmds bridge call stack:
#0 0x0000000000000000 in ?? ()
#1 0x00007ffff78cc259 in mamaSubscription_forwardMsg (subscription=0x7fffe9757c50, msg=0x641d60) at mama/c_cpp/src/c/subscription.c:1426
#2 0x00007ffff78a38ec in processPointToPointMessage (callback=0x7fffe9768fb0, msg=0x641d60, msgType=6, ctx=0x7fffe939b500) at mama/c_cpp/src/c/listenermsgcallback.c:174
#3 0x00007ffff78a3f37 in listenerMsgCallback_processMsg (callback=0x7fffe9768fb0, msg=0x641d60, ctx=0x7fffe939b500) at mama/c_cpp/src/c/listenermsgcallback.c:471
#4 0x00007ffff78cd7e5 in mamaSubscription_processMsg (subscription=0x7fffe939b2e0, msg=0x641d60) at mama/c_cpp/src/c/subscription.c:2260
#5 0x00007ffff51fdaf3 in RMDSBridgeSubscription::OnMessage(mamaMsgImpl_*, mamaMsgType) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#6 0x00007ffff5258492 in UPASubscription::NotifyListenersRefreshMessage(mamaMsgImpl_*, boost::shared_ptr<RMDSBridgeSubscription>, bool) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#7 0x00007ffff5259032 in UPASubscription::InternalProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#8 0x00007ffff525e785 in UPASubscription::ProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#9 0x00007ffff522ccc8 in UPAConsumer::ProcessResponse(RsslChannel*, RwfBuffer*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#10 0x00007ffff522d2b0 in UPAConsumer::ReadFromChannel(RsslChannel*) () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#11 0x00007ffff522e62d in UPAConsumer::Run() () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#12 0x00007ffff52146cc in RMDSSubscriber::threadFunc(void*) () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#13 0x00007ffff6733806 in start_thread () from /lib64/libpthread.so.0
#14 0x00007ffff5af59bd in clone () from /lib64/libc.so.6
#15 0x0000000000000000 in ?? ()

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 8:12 PM
To: Tom Doust <tom.doust@...>; Konstantin Baydarov <konstantin.baydarov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: [Openmama-dev] [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Tom,

We are using version 1.3.
As I see from latest github code the problem still exists. See RMDSBridgeSubscription::OnMessage method:

if (isShutdown_ || ((0 != source_) && source_->IsPausedUpdates()))
{
return;
}

// ... Shutdown() may be called here
// And then MAMA can start destroying subscription_'s fields try
{
status = mamaSubscription_processMsg(subscription_, msg); // This function will be examining subscription_'s fields being destroyed
}


-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Tom Doust
Sent: Tuesday, June 20, 2017 4:09 PM
To: Konstantin Baydarov <konstantin.baydarov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-dev] [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury, Konstantin

Are you using the current github version of the bridge code? We looked at and fixed some of the issues around locking the subscription destroy some time back.

It would be good to know if we have missed something.

Best regards

Tom

-----Original Message-----
From: openmama-users-bounces@... [mailto:openmama-users-bounces@...] On Behalf Of Konstantin Baydarov
Sent: 20 June 2017 11:32
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi, guys.

I'm working on the issue with Yury. I spotted the deadlock possibility during debugging tick42rmds bridge crash on unsubscribe, will be interested knowing the solution as well.

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 12:38 PM
To: Frank Quinn <fquinn@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Cc: Konstantin Baydarov <konstantin.baydarov@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Frank,

Let me answer your question in random order :) 2. It looks like a designed behavior of RMDS bridge - callbacks are invoked in a thread servicing transport events from a server. One thread per mamaTransport is created.
1. Therefore race conditions are possible. Our case is: mute is called for a subscription, then mamaSubscription_cleanup frees self->mInitialRequest but concurrent mamaSubscription_processMsg call tries to access self->mInitialRequest because the message it processes is initial message from a server.
3. Taking in account p.2 we cannot process mute request asynchronously as MAMA starts freeing subscription resources immediately after mute is called. Do you think it is possible for MAMA not to invoke bridgeMamaSubscriptionMute() under mamaSubscription locks? Thus the contract for this function would be:
- This call should be synchronous and no events should be processed after it returned (like before)
- It should be reentrant and synchronize it's operations by itself (quite sensible requirement)

-----Original Message-----
From: Frank Quinn [mailto:fquinn@...]
Sent: Tuesday, June 20, 2017 10:05 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury,

Thanks for the detailed query, I have a few outstanding questions and suggestions on this one:

1. I would question whether or not mamaSubscription_processMsg should crash for a muted subscription. Muting is a state which exists to attempt to stop new events coming in. However if you're in the dispatcher thread and you just received an object, it's too late this time - mute should only be invoked prior to prevent the *next* read. So perhaps (knowing nothing about the RMDS bridge) the straightforward solution would be simply to do the checks which may cause muting after the callback?
2. Should the thread which processes events from RMDS server invoke the callback method directly (inline)? Is it on the same thread as is assigned to the MAMA Subscription object? It should be to match the application's expected concurrency behaviour.
3. Rather than muting immediately, you could consider creating a muting callback event which gets enqueued onto your subscription thread. That way the mute event will always be synchronous with the subscription thread and you don't need to worry about locking any associated resources. Locking with subscription objects (particularly MAMA core objects) is hairy stuff and you should avoid it where possible to avoid race conditions.

Cheers,
Frank



-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Yury Batrakov
Sent: 19 June 2017 17:42
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi guys,

I've faced the following issue when using OpenMAMA and tick42rmds bridge.
The bridge internally creates a thread to process events from RMDS server, once a message is received that thread invokes mamaSubscription_processMsg. While the message is processed user may want to destroy the subscription (obviously in other thread). To avoid corruption of mamaSubscription object, mamaSubscription_destroy() function calls bridge->mute for the bridge to stop calling mamaSubscription_processMsg() and only then deallocates mamaSubscription. The problem with this approach is the following: here's the pseudo code for RMDS dispatching thread


if(muted) {
// Do not dispatch
return;
}

// Do some other checks <-- mute() may be invoked here
mamaSubscription_processMsg() // processMsg for muted subscription, may crash


The solution for that is to change RMDS bridge to block in bridge->mute() call until mamaSubscription_processMsg() returns but there's another problem: mamaSubscription_processMsg and mamaSubscription_deactivate may deadlock on wombatThrottle. Consider the following scenario:
1. RMDS bridge thread invokes mamaSubscription_processMsg() for message of type initial
2. User thread invokes mamaSubscription_destroy() which acquires wombat throttle lock:
if (impl->mTransport)
throttle = mamaTransportImpl_getThrottle(impl->mTransport,
MAMA_THROTTLE_DEFAULT);

if(NULL != throttle)
{
wombatThrottle_lock(throttle);
}
3. Then mamaSubscription_destroy calls mamaSubscription_deactivate_internal which calls our new version of bridge->mute() which waits for RMDS bridge thread to finish message processing
if (impl->mSubscBridge)
{
impl->mBridgeImpl->bridgeMamaSubscriptionMute (impl->mSubscBridge);
}
4. RMDS bridge handles initial message and tries to acquire the same throttle:
#5 0x00007ffff78d4f32 in wombatThrottle_lock (throttle=0x6298e0) at mama/c_cpp/src/c/throttle.c:441
#6 0x00007ffff78a34e2 in imageRequest_stopWaitForResponse (request=0x14d1a20) at mama/c_cpp/src/c/imagerequest.c:774
#7 0x00007ffff78cbe06 in mamaSubscription_stopWaitForResponse (subscription=0xe36280, ctx=0xe364a0) at mama/c_cpp/src/c/subscription.c:1262
#8 0x00007ffff78a38fe in processPointToPointMessage (callback=0x1527e50, msg=0x642460, msgType=6, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:169
#9 0x00007ffff78a3f9c in listenerMsgCallback_processMsg (callback=0x1527e50, msg=0x642460, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:480
#10 0x00007ffff78cd825 in mamaSubscription_processMsg (subscription=0xe36280, msg=0x642460) at mama/c_cpp/src/c/subscription.c:2259

What do you think is the best way to avoid this?


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-users mailing list
Openmama-users@...
https://lists.openmama.org/mailman/listinfo/openmama-users
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.



---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

Re: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Tom Doust
 

Yes we call back on the bridge’s thread. We don’t queue the message, we leave the client code to do that if it wants to. This is by design.

On 20/06/2017, 18:18, "Konstantin Baydarov" <konstantin.baydarov@...> wrote:

Classification: Public

Hi, Tom.

I noticed, that comparing to qpid bridge(that comes with openmama sources), tick42rmds calls mamaSubscription_processMsg() method from separate thread and not from mamaQueue_dispatch(), wondering if it's correct. Probably it's one of the reasons of the issue that we facing?
Qpid bridge call stack:
#0 mamaSubscription_processMsg (subscription=0x76e150, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:2226
#1 0x00007ffff7b4c580 in imageRequestImpl_onInitialMessage (msg=0x7aebb0, closure=0x772710) at mama/c_cpp/src/c/imagerequest.c:225
#2 0x00007ffff648bede in qpidBridgeMamaInboxImpl_onMsg (subscription=0x772900, msg=0x7aebb0, closure=0x7727c0, itemClosure=0x0) at mama/c_cpp/src/c/bridge/qpid/inbox.c:298
#3 0x00007ffff7b76ab4 in mamaSubscription_forwardMsg (subscription=0x772900, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:1422
#4 0x00007ffff7b781ef in mamaSubscription_processMsg (subscription=0x772900, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:2315
#5 0x00007ffff6490818 in qpidBridgeMamaTransportImpl_queueCallback (queue=0x60eb50, closure=0x61df80) at mama/c_cpp/src/c/bridge/qpid/transport.c:1083
#6 0x00007ffff7b90a1f in wombatQueue_dispatchInt (queue=0x60ecb0, data=0x0, closure=0x0, isTimed=1 '\001', timout=500) at common/c_cpp/src/c/queue.c:319
#7 0x00007ffff7b90aa2 in wombatQueue_timedDispatch (queue=0x60ecb0, data=0x0, closure=0x0, timeout=500) at common/c_cpp/src/c/queue.c:335
#8 0x00007ffff648e720 in qpidBridgeMamaQueue_dispatch (queue=0x60ec40) at mama/c_cpp/src/c/bridge/qpid/queue.c:265
#9 0x00007ffff7b6e1de in mamaQueue_dispatch (queue=0x60eb50) at mama/c_cpp/src/c/queue.c:824
#10 0x00007ffff648a8c3 in qpidBridge_start (defaultEventQueue=0x60eb50) at mama/c_cpp/src/c/bridge/qpid/bridge.c:196
#11 0x00007ffff7b52976 in mama_start (bridgeImpl=0x60e750) at mama/c_cpp/src/c/mama.c:1659
#12 0x0000000000403e61 in buildDataDictionary () at mama/c_cpp/src/examples/c/mamalistenc.c:647
#13 0x000000000040366f in main (argc=9, argv=0x7fffffffd728) at mama/c_cpp/src/examples/c/mamalistenc.c:335

tick42rmds bridge call stack:
#0 0x0000000000000000 in ?? ()
#1 0x00007ffff78cc259 in mamaSubscription_forwardMsg (subscription=0x7fffe9757c50, msg=0x641d60) at mama/c_cpp/src/c/subscription.c:1426
#2 0x00007ffff78a38ec in processPointToPointMessage (callback=0x7fffe9768fb0, msg=0x641d60, msgType=6, ctx=0x7fffe939b500) at mama/c_cpp/src/c/listenermsgcallback.c:174
#3 0x00007ffff78a3f37 in listenerMsgCallback_processMsg (callback=0x7fffe9768fb0, msg=0x641d60, ctx=0x7fffe939b500) at mama/c_cpp/src/c/listenermsgcallback.c:471
#4 0x00007ffff78cd7e5 in mamaSubscription_processMsg (subscription=0x7fffe939b2e0, msg=0x641d60) at mama/c_cpp/src/c/subscription.c:2260
#5 0x00007ffff51fdaf3 in RMDSBridgeSubscription::OnMessage(mamaMsgImpl_*, mamaMsgType) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#6 0x00007ffff5258492 in UPASubscription::NotifyListenersRefreshMessage(mamaMsgImpl_*, boost::shared_ptr<RMDSBridgeSubscription>, bool) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#7 0x00007ffff5259032 in UPASubscription::InternalProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#8 0x00007ffff525e785 in UPASubscription::ProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#9 0x00007ffff522ccc8 in UPAConsumer::ProcessResponse(RsslChannel*, RwfBuffer*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#10 0x00007ffff522d2b0 in UPAConsumer::ReadFromChannel(RsslChannel*) () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#11 0x00007ffff522e62d in UPAConsumer::Run() () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#12 0x00007ffff52146cc in RMDSSubscriber::threadFunc(void*) () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#13 0x00007ffff6733806 in start_thread () from /lib64/libpthread.so.0
#14 0x00007ffff5af59bd in clone () from /lib64/libc.so.6
#15 0x0000000000000000 in ?? ()

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 8:12 PM
To: Tom Doust <tom.doust@...>; Konstantin Baydarov <konstantin.baydarov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: [Openmama-dev] [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Tom,

We are using version 1.3.
As I see from latest github code the problem still exists. See RMDSBridgeSubscription::OnMessage method:

if (isShutdown_ || ((0 != source_) && source_->IsPausedUpdates()))
{
return;
}

// ... Shutdown() may be called here
// And then MAMA can start destroying subscription_'s fields try
{
status = mamaSubscription_processMsg(subscription_, msg); // This function will be examining subscription_'s fields being destroyed
}


-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Tom Doust
Sent: Tuesday, June 20, 2017 4:09 PM
To: Konstantin Baydarov <konstantin.baydarov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-dev] [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury, Konstantin

Are you using the current github version of the bridge code? We looked at and fixed some of the issues around locking the subscription destroy some time back.

It would be good to know if we have missed something.

Best regards

Tom

-----Original Message-----
From: openmama-users-bounces@... [mailto:openmama-users-bounces@...] On Behalf Of Konstantin Baydarov
Sent: 20 June 2017 11:32
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi, guys.

I'm working on the issue with Yury. I spotted the deadlock possibility during debugging tick42rmds bridge crash on unsubscribe, will be interested knowing the solution as well.

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 12:38 PM
To: Frank Quinn <fquinn@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Cc: Konstantin Baydarov <konstantin.baydarov@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Frank,

Let me answer your question in random order :) 2. It looks like a designed behavior of RMDS bridge - callbacks are invoked in a thread servicing transport events from a server. One thread per mamaTransport is created.
1. Therefore race conditions are possible. Our case is: mute is called for a subscription, then mamaSubscription_cleanup frees self->mInitialRequest but concurrent mamaSubscription_processMsg call tries to access self->mInitialRequest because the message it processes is initial message from a server.
3. Taking in account p.2 we cannot process mute request asynchronously as MAMA starts freeing subscription resources immediately after mute is called. Do you think it is possible for MAMA not to invoke bridgeMamaSubscriptionMute() under mamaSubscription locks? Thus the contract for this function would be:
- This call should be synchronous and no events should be processed after it returned (like before)
- It should be reentrant and synchronize it's operations by itself (quite sensible requirement)

-----Original Message-----
From: Frank Quinn [mailto:fquinn@...]
Sent: Tuesday, June 20, 2017 10:05 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury,

Thanks for the detailed query, I have a few outstanding questions and suggestions on this one:

1. I would question whether or not mamaSubscription_processMsg should crash for a muted subscription. Muting is a state which exists to attempt to stop new events coming in. However if you're in the dispatcher thread and you just received an object, it's too late this time - mute should only be invoked prior to prevent the *next* read. So perhaps (knowing nothing about the RMDS bridge) the straightforward solution would be simply to do the checks which may cause muting after the callback?
2. Should the thread which processes events from RMDS server invoke the callback method directly (inline)? Is it on the same thread as is assigned to the MAMA Subscription object? It should be to match the application's expected concurrency behaviour.
3. Rather than muting immediately, you could consider creating a muting callback event which gets enqueued onto your subscription thread. That way the mute event will always be synchronous with the subscription thread and you don't need to worry about locking any associated resources. Locking with subscription objects (particularly MAMA core objects) is hairy stuff and you should avoid it where possible to avoid race conditions.

Cheers,
Frank



-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Yury Batrakov
Sent: 19 June 2017 17:42
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi guys,

I've faced the following issue when using OpenMAMA and tick42rmds bridge.
The bridge internally creates a thread to process events from RMDS server, once a message is received that thread invokes mamaSubscription_processMsg. While the message is processed user may want to destroy the subscription (obviously in other thread). To avoid corruption of mamaSubscription object, mamaSubscription_destroy() function calls bridge->mute for the bridge to stop calling mamaSubscription_processMsg() and only then deallocates mamaSubscription. The problem with this approach is the following: here's the pseudo code for RMDS dispatching thread


if(muted) {
// Do not dispatch
return;
}

// Do some other checks <-- mute() may be invoked here
mamaSubscription_processMsg() // processMsg for muted subscription, may crash


The solution for that is to change RMDS bridge to block in bridge->mute() call until mamaSubscription_processMsg() returns but there's another problem: mamaSubscription_processMsg and mamaSubscription_deactivate may deadlock on wombatThrottle. Consider the following scenario:
1. RMDS bridge thread invokes mamaSubscription_processMsg() for message of type initial
2. User thread invokes mamaSubscription_destroy() which acquires wombat throttle lock:
if (impl->mTransport)
throttle = mamaTransportImpl_getThrottle(impl->mTransport,
MAMA_THROTTLE_DEFAULT);

if(NULL != throttle)
{
wombatThrottle_lock(throttle);
}
3. Then mamaSubscription_destroy calls mamaSubscription_deactivate_internal which calls our new version of bridge->mute() which waits for RMDS bridge thread to finish message processing
if (impl->mSubscBridge)
{
impl->mBridgeImpl->bridgeMamaSubscriptionMute (impl->mSubscBridge);
}
4. RMDS bridge handles initial message and tries to acquire the same throttle:
#5 0x00007ffff78d4f32 in wombatThrottle_lock (throttle=0x6298e0) at mama/c_cpp/src/c/throttle.c:441
#6 0x00007ffff78a34e2 in imageRequest_stopWaitForResponse (request=0x14d1a20) at mama/c_cpp/src/c/imagerequest.c:774
#7 0x00007ffff78cbe06 in mamaSubscription_stopWaitForResponse (subscription=0xe36280, ctx=0xe364a0) at mama/c_cpp/src/c/subscription.c:1262
#8 0x00007ffff78a38fe in processPointToPointMessage (callback=0x1527e50, msg=0x642460, msgType=6, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:169
#9 0x00007ffff78a3f9c in listenerMsgCallback_processMsg (callback=0x1527e50, msg=0x642460, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:480
#10 0x00007ffff78cd825 in mamaSubscription_processMsg (subscription=0xe36280, msg=0x642460) at mama/c_cpp/src/c/subscription.c:2259

What do you think is the best way to avoid this?


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-users mailing list
Openmama-users@...
https://lists.openmama.org/mailman/listinfo/openmama-users
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

Re: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Konstantin Baydarov
 

Classification: Public

Hi, Tom.

I noticed, that comparing to qpid bridge(that comes with openmama sources), tick42rmds calls mamaSubscription_processMsg() method from separate thread and not from mamaQueue_dispatch(), wondering if it's correct. Probably it's one of the reasons of the issue that we facing?
Qpid bridge call stack:
#0 mamaSubscription_processMsg (subscription=0x76e150, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:2226
#1 0x00007ffff7b4c580 in imageRequestImpl_onInitialMessage (msg=0x7aebb0, closure=0x772710) at mama/c_cpp/src/c/imagerequest.c:225
#2 0x00007ffff648bede in qpidBridgeMamaInboxImpl_onMsg (subscription=0x772900, msg=0x7aebb0, closure=0x7727c0, itemClosure=0x0) at mama/c_cpp/src/c/bridge/qpid/inbox.c:298
#3 0x00007ffff7b76ab4 in mamaSubscription_forwardMsg (subscription=0x772900, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:1422
#4 0x00007ffff7b781ef in mamaSubscription_processMsg (subscription=0x772900, msg=0x7aebb0) at mama/c_cpp/src/c/subscription.c:2315
#5 0x00007ffff6490818 in qpidBridgeMamaTransportImpl_queueCallback (queue=0x60eb50, closure=0x61df80) at mama/c_cpp/src/c/bridge/qpid/transport.c:1083
#6 0x00007ffff7b90a1f in wombatQueue_dispatchInt (queue=0x60ecb0, data=0x0, closure=0x0, isTimed=1 '\001', timout=500) at common/c_cpp/src/c/queue.c:319
#7 0x00007ffff7b90aa2 in wombatQueue_timedDispatch (queue=0x60ecb0, data=0x0, closure=0x0, timeout=500) at common/c_cpp/src/c/queue.c:335
#8 0x00007ffff648e720 in qpidBridgeMamaQueue_dispatch (queue=0x60ec40) at mama/c_cpp/src/c/bridge/qpid/queue.c:265
#9 0x00007ffff7b6e1de in mamaQueue_dispatch (queue=0x60eb50) at mama/c_cpp/src/c/queue.c:824
#10 0x00007ffff648a8c3 in qpidBridge_start (defaultEventQueue=0x60eb50) at mama/c_cpp/src/c/bridge/qpid/bridge.c:196
#11 0x00007ffff7b52976 in mama_start (bridgeImpl=0x60e750) at mama/c_cpp/src/c/mama.c:1659
#12 0x0000000000403e61 in buildDataDictionary () at mama/c_cpp/src/examples/c/mamalistenc.c:647
#13 0x000000000040366f in main (argc=9, argv=0x7fffffffd728) at mama/c_cpp/src/examples/c/mamalistenc.c:335

tick42rmds bridge call stack:
#0 0x0000000000000000 in ?? ()
#1 0x00007ffff78cc259 in mamaSubscription_forwardMsg (subscription=0x7fffe9757c50, msg=0x641d60) at mama/c_cpp/src/c/subscription.c:1426
#2 0x00007ffff78a38ec in processPointToPointMessage (callback=0x7fffe9768fb0, msg=0x641d60, msgType=6, ctx=0x7fffe939b500) at mama/c_cpp/src/c/listenermsgcallback.c:174
#3 0x00007ffff78a3f37 in listenerMsgCallback_processMsg (callback=0x7fffe9768fb0, msg=0x641d60, ctx=0x7fffe939b500) at mama/c_cpp/src/c/listenermsgcallback.c:471
#4 0x00007ffff78cd7e5 in mamaSubscription_processMsg (subscription=0x7fffe939b2e0, msg=0x641d60) at mama/c_cpp/src/c/subscription.c:2260
#5 0x00007ffff51fdaf3 in RMDSBridgeSubscription::OnMessage(mamaMsgImpl_*, mamaMsgType) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#6 0x00007ffff5258492 in UPASubscription::NotifyListenersRefreshMessage(mamaMsgImpl_*, boost::shared_ptr<RMDSBridgeSubscription>, bool) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#7 0x00007ffff5259032 in UPASubscription::InternalProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#8 0x00007ffff525e785 in UPASubscription::ProcessMarketPriceResponse(RsslMsg*, RsslDecIterator*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#9 0x00007ffff522ccc8 in UPAConsumer::ProcessResponse(RsslChannel*, RwfBuffer*) ()
from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#10 0x00007ffff522d2b0 in UPAConsumer::ReadFromChannel(RsslChannel*) () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#11 0x00007ffff522e62d in UPAConsumer::Run() () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#12 0x00007ffff52146cc in RMDSSubscriber::threadFunc(void*) () from /home/gedeapp/baydkon/apps/dbmama/dbmama-api-1.7.1462_dev/lib/libmamatick42rmdsimpl.so
#13 0x00007ffff6733806 in start_thread () from /lib64/libpthread.so.0
#14 0x00007ffff5af59bd in clone () from /lib64/libc.so.6
#15 0x0000000000000000 in ?? ()

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 8:12 PM
To: Tom Doust <tom.doust@...>; Konstantin Baydarov <konstantin.baydarov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: [Openmama-dev] [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Tom,

We are using version 1.3.
As I see from latest github code the problem still exists. See RMDSBridgeSubscription::OnMessage method:

if (isShutdown_ || ((0 != source_) && source_->IsPausedUpdates()))
{
return;
}

// ... Shutdown() may be called here
// And then MAMA can start destroying subscription_'s fields try
{
status = mamaSubscription_processMsg(subscription_, msg); // This function will be examining subscription_'s fields being destroyed
}


-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Tom Doust
Sent: Tuesday, June 20, 2017 4:09 PM
To: Konstantin Baydarov <konstantin.baydarov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-dev] [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury, Konstantin

Are you using the current github version of the bridge code? We looked at and fixed some of the issues around locking the subscription destroy some time back.

It would be good to know if we have missed something.

Best regards

Tom

-----Original Message-----
From: openmama-users-bounces@... [mailto:openmama-users-bounces@...] On Behalf Of Konstantin Baydarov
Sent: 20 June 2017 11:32
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi, guys.

I'm working on the issue with Yury. I spotted the deadlock possibility during debugging tick42rmds bridge crash on unsubscribe, will be interested knowing the solution as well.

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 12:38 PM
To: Frank Quinn <fquinn@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Cc: Konstantin Baydarov <konstantin.baydarov@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Frank,

Let me answer your question in random order :) 2. It looks like a designed behavior of RMDS bridge - callbacks are invoked in a thread servicing transport events from a server. One thread per mamaTransport is created.
1. Therefore race conditions are possible. Our case is: mute is called for a subscription, then mamaSubscription_cleanup frees self->mInitialRequest but concurrent mamaSubscription_processMsg call tries to access self->mInitialRequest because the message it processes is initial message from a server.
3. Taking in account p.2 we cannot process mute request asynchronously as MAMA starts freeing subscription resources immediately after mute is called. Do you think it is possible for MAMA not to invoke bridgeMamaSubscriptionMute() under mamaSubscription locks? Thus the contract for this function would be:
- This call should be synchronous and no events should be processed after it returned (like before)
- It should be reentrant and synchronize it's operations by itself (quite sensible requirement)

-----Original Message-----
From: Frank Quinn [mailto:fquinn@...]
Sent: Tuesday, June 20, 2017 10:05 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury,

Thanks for the detailed query, I have a few outstanding questions and suggestions on this one:

1. I would question whether or not mamaSubscription_processMsg should crash for a muted subscription. Muting is a state which exists to attempt to stop new events coming in. However if you're in the dispatcher thread and you just received an object, it's too late this time - mute should only be invoked prior to prevent the *next* read. So perhaps (knowing nothing about the RMDS bridge) the straightforward solution would be simply to do the checks which may cause muting after the callback?
2. Should the thread which processes events from RMDS server invoke the callback method directly (inline)? Is it on the same thread as is assigned to the MAMA Subscription object? It should be to match the application's expected concurrency behaviour.
3. Rather than muting immediately, you could consider creating a muting callback event which gets enqueued onto your subscription thread. That way the mute event will always be synchronous with the subscription thread and you don't need to worry about locking any associated resources. Locking with subscription objects (particularly MAMA core objects) is hairy stuff and you should avoid it where possible to avoid race conditions.

Cheers,
Frank



-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Yury Batrakov
Sent: 19 June 2017 17:42
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi guys,

I've faced the following issue when using OpenMAMA and tick42rmds bridge.
The bridge internally creates a thread to process events from RMDS server, once a message is received that thread invokes mamaSubscription_processMsg. While the message is processed user may want to destroy the subscription (obviously in other thread). To avoid corruption of mamaSubscription object, mamaSubscription_destroy() function calls bridge->mute for the bridge to stop calling mamaSubscription_processMsg() and only then deallocates mamaSubscription. The problem with this approach is the following: here's the pseudo code for RMDS dispatching thread


if(muted) {
// Do not dispatch
return;
}

// Do some other checks <-- mute() may be invoked here
mamaSubscription_processMsg() // processMsg for muted subscription, may crash


The solution for that is to change RMDS bridge to block in bridge->mute() call until mamaSubscription_processMsg() returns but there's another problem: mamaSubscription_processMsg and mamaSubscription_deactivate may deadlock on wombatThrottle. Consider the following scenario:
1. RMDS bridge thread invokes mamaSubscription_processMsg() for message of type initial
2. User thread invokes mamaSubscription_destroy() which acquires wombat throttle lock:
if (impl->mTransport)
throttle = mamaTransportImpl_getThrottle(impl->mTransport,
MAMA_THROTTLE_DEFAULT);

if(NULL != throttle)
{
wombatThrottle_lock(throttle);
}
3. Then mamaSubscription_destroy calls mamaSubscription_deactivate_internal which calls our new version of bridge->mute() which waits for RMDS bridge thread to finish message processing
if (impl->mSubscBridge)
{
impl->mBridgeImpl->bridgeMamaSubscriptionMute (impl->mSubscBridge);
}
4. RMDS bridge handles initial message and tries to acquire the same throttle:
#5 0x00007ffff78d4f32 in wombatThrottle_lock (throttle=0x6298e0) at mama/c_cpp/src/c/throttle.c:441
#6 0x00007ffff78a34e2 in imageRequest_stopWaitForResponse (request=0x14d1a20) at mama/c_cpp/src/c/imagerequest.c:774
#7 0x00007ffff78cbe06 in mamaSubscription_stopWaitForResponse (subscription=0xe36280, ctx=0xe364a0) at mama/c_cpp/src/c/subscription.c:1262
#8 0x00007ffff78a38fe in processPointToPointMessage (callback=0x1527e50, msg=0x642460, msgType=6, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:169
#9 0x00007ffff78a3f9c in listenerMsgCallback_processMsg (callback=0x1527e50, msg=0x642460, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:480
#10 0x00007ffff78cd825 in mamaSubscription_processMsg (subscription=0xe36280, msg=0x642460) at mama/c_cpp/src/c/subscription.c:2259

What do you think is the best way to avoid this?


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-users mailing list
Openmama-users@...
https://lists.openmama.org/mailman/listinfo/openmama-users
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

Re: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Tom Doust
 

Hi Yury, Konstantin

Are you using the current github version of the bridge code? We looked at and fixed some of the issues around locking the subscription destroy some time back.

It would be good to know if we have missed something.

Best regards

Tom

-----Original Message-----
From: openmama-users-bounces@... [mailto:openmama-users-bounces@...] On Behalf Of Konstantin Baydarov
Sent: 20 June 2017 11:32
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: Re: [Openmama-users] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi, guys.

I'm working on the issue with Yury. I spotted the deadlock possibility during debugging tick42rmds bridge crash on unsubscribe, will be interested knowing the solution as well.

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 12:38 PM
To: Frank Quinn <fquinn@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Cc: Konstantin Baydarov <konstantin.baydarov@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Frank,

Let me answer your question in random order :) 2. It looks like a designed behavior of RMDS bridge - callbacks are invoked in a thread servicing transport events from a server. One thread per mamaTransport is created.
1. Therefore race conditions are possible. Our case is: mute is called for a subscription, then mamaSubscription_cleanup frees self->mInitialRequest but concurrent mamaSubscription_processMsg call tries to access self->mInitialRequest because the message it processes is initial message from a server.
3. Taking in account p.2 we cannot process mute request asynchronously as MAMA starts freeing subscription resources immediately after mute is called. Do you think it is possible for MAMA not to invoke bridgeMamaSubscriptionMute() under mamaSubscription locks? Thus the contract for this function would be:
- This call should be synchronous and no events should be processed after it returned (like before)
- It should be reentrant and synchronize it's operations by itself (quite sensible requirement)

-----Original Message-----
From: Frank Quinn [mailto:fquinn@...]
Sent: Tuesday, June 20, 2017 10:05 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury,

Thanks for the detailed query, I have a few outstanding questions and suggestions on this one:

1. I would question whether or not mamaSubscription_processMsg should crash for a muted subscription. Muting is a state which exists to attempt to stop new events coming in. However if you're in the dispatcher thread and you just received an object, it's too late this time - mute should only be invoked prior to prevent the *next* read. So perhaps (knowing nothing about the RMDS bridge) the straightforward solution would be simply to do the checks which may cause muting after the callback?
2. Should the thread which processes events from RMDS server invoke the callback method directly (inline)? Is it on the same thread as is assigned to the MAMA Subscription object? It should be to match the application's expected concurrency behaviour.
3. Rather than muting immediately, you could consider creating a muting callback event which gets enqueued onto your subscription thread. That way the mute event will always be synchronous with the subscription thread and you don't need to worry about locking any associated resources. Locking with subscription objects (particularly MAMA core objects) is hairy stuff and you should avoid it where possible to avoid race conditions.

Cheers,
Frank



-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Yury Batrakov
Sent: 19 June 2017 17:42
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi guys,

I've faced the following issue when using OpenMAMA and tick42rmds bridge.
The bridge internally creates a thread to process events from RMDS server, once a message is received that thread invokes mamaSubscription_processMsg. While the message is processed user may want to destroy the subscription (obviously in other thread). To avoid corruption of mamaSubscription object, mamaSubscription_destroy() function calls bridge->mute for the bridge to stop calling mamaSubscription_processMsg() and only then deallocates mamaSubscription. The problem with this approach is the following: here's the pseudo code for RMDS dispatching thread


if(muted) {
// Do not dispatch
return;
}

// Do some other checks <-- mute() may be invoked here
mamaSubscription_processMsg() // processMsg for muted subscription, may crash


The solution for that is to change RMDS bridge to block in bridge->mute() call until mamaSubscription_processMsg() returns but there's another problem: mamaSubscription_processMsg and mamaSubscription_deactivate may deadlock on wombatThrottle. Consider the following scenario:
1. RMDS bridge thread invokes mamaSubscription_processMsg() for message of type initial
2. User thread invokes mamaSubscription_destroy() which acquires wombat throttle lock:
if (impl->mTransport)
throttle = mamaTransportImpl_getThrottle(impl->mTransport,
MAMA_THROTTLE_DEFAULT);

if(NULL != throttle)
{
wombatThrottle_lock(throttle);
}
3. Then mamaSubscription_destroy calls mamaSubscription_deactivate_internal which calls our new version of bridge->mute() which waits for RMDS bridge thread to finish message processing
if (impl->mSubscBridge)
{
impl->mBridgeImpl->bridgeMamaSubscriptionMute (impl->mSubscBridge);
}
4. RMDS bridge handles initial message and tries to acquire the same throttle:
#5 0x00007ffff78d4f32 in wombatThrottle_lock (throttle=0x6298e0) at mama/c_cpp/src/c/throttle.c:441
#6 0x00007ffff78a34e2 in imageRequest_stopWaitForResponse (request=0x14d1a20) at mama/c_cpp/src/c/imagerequest.c:774
#7 0x00007ffff78cbe06 in mamaSubscription_stopWaitForResponse (subscription=0xe36280, ctx=0xe364a0) at mama/c_cpp/src/c/subscription.c:1262
#8 0x00007ffff78a38fe in processPointToPointMessage (callback=0x1527e50, msg=0x642460, msgType=6, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:169
#9 0x00007ffff78a3f9c in listenerMsgCallback_processMsg (callback=0x1527e50, msg=0x642460, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:480
#10 0x00007ffff78cd825 in mamaSubscription_processMsg (subscription=0xe36280, msg=0x642460) at mama/c_cpp/src/c/subscription.c:2259

What do you think is the best way to avoid this?


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-users mailing list
Openmama-users@...
https://lists.openmama.org/mailman/listinfo/openmama-users

Re: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Konstantin Baydarov
 

Classification: Public

Hi, guys.

I'm working on the issue with Yury. I spotted the deadlock possibility during debugging tick42rmds bridge crash on unsubscribe, will be interested knowing the solution as well.

BR,
Konstantin Baydarov

-----Original Message-----
From: Yury Batrakov
Sent: Tuesday, June 20, 2017 12:38 PM
To: Frank Quinn <fquinn@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Cc: Konstantin Baydarov <konstantin.baydarov@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi Frank,

Let me answer your question in random order :)
2. It looks like a designed behavior of RMDS bridge - callbacks are invoked in a thread servicing transport events from a server. One thread per mamaTransport is created.
1. Therefore race conditions are possible. Our case is: mute is called for a subscription, then mamaSubscription_cleanup frees self->mInitialRequest but concurrent mamaSubscription_processMsg call tries to access self->mInitialRequest because the message it processes is initial message from a server.
3. Taking in account p.2 we cannot process mute request asynchronously as MAMA starts freeing subscription resources immediately after mute is called. Do you think it is possible for MAMA not to invoke bridgeMamaSubscriptionMute() under mamaSubscription locks? Thus the contract for this function would be:
- This call should be synchronous and no events should be processed after it returned (like before)
- It should be reentrant and synchronize it's operations by itself (quite sensible requirement)

-----Original Message-----
From: Frank Quinn [mailto:fquinn@...]
Sent: Tuesday, June 20, 2017 10:05 AM
To: Yury Batrakov <yury.batrakov@...>; openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: RE: Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Hi Yury,

Thanks for the detailed query, I have a few outstanding questions and suggestions on this one:

1. I would question whether or not mamaSubscription_processMsg should crash for a muted subscription. Muting is a state which exists to attempt to stop new events coming in. However if you're in the dispatcher thread and you just received an object, it's too late this time - mute should only be invoked prior to prevent the *next* read. So perhaps (knowing nothing about the RMDS bridge) the straightforward solution would be simply to do the checks which may cause muting after the callback?
2. Should the thread which processes events from RMDS server invoke the callback method directly (inline)? Is it on the same thread as is assigned to the MAMA Subscription object? It should be to match the application's expected concurrency behaviour.
3. Rather than muting immediately, you could consider creating a muting callback event which gets enqueued onto your subscription thread. That way the mute event will always be synchronous with the subscription thread and you don't need to worry about locking any associated resources. Locking with subscription objects (particularly MAMA core objects) is hairy stuff and you should avoid it where possible to avoid race conditions.

Cheers,
Frank



-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Yury Batrakov
Sent: 19 June 2017 17:42
To: openmama-dev <openmama-dev@...>; openmama-users <openmama-users@...>
Subject: [Openmama-dev] Concurrent subscription.destroy() ? Crash when using tick42rmds transport.

Classification: Public

Hi guys,

I've faced the following issue when using OpenMAMA and tick42rmds bridge.
The bridge internally creates a thread to process events from RMDS server, once a message is received that thread invokes mamaSubscription_processMsg. While the message is processed user may want to destroy the subscription (obviously in other thread). To avoid corruption of mamaSubscription object, mamaSubscription_destroy() function calls bridge->mute for the bridge to stop calling mamaSubscription_processMsg() and only then deallocates mamaSubscription. The problem with this approach is the following: here's the pseudo code for RMDS dispatching thread


if(muted) {
// Do not dispatch
return;
}

// Do some other checks <-- mute() may be invoked here
mamaSubscription_processMsg() // processMsg for muted subscription, may crash


The solution for that is to change RMDS bridge to block in bridge->mute() call until mamaSubscription_processMsg() returns but there's another problem: mamaSubscription_processMsg and mamaSubscription_deactivate may deadlock on wombatThrottle. Consider the following scenario:
1. RMDS bridge thread invokes mamaSubscription_processMsg() for message of type initial
2. User thread invokes mamaSubscription_destroy() which acquires wombat throttle lock:
if (impl->mTransport)
throttle = mamaTransportImpl_getThrottle(impl->mTransport,
MAMA_THROTTLE_DEFAULT);

if(NULL != throttle)
{
wombatThrottle_lock(throttle);
}
3. Then mamaSubscription_destroy calls mamaSubscription_deactivate_internal which calls our new version of bridge->mute() which waits for RMDS bridge thread to finish message processing
if (impl->mSubscBridge)
{
impl->mBridgeImpl->bridgeMamaSubscriptionMute (impl->mSubscBridge);
}
4. RMDS bridge handles initial message and tries to acquire the same throttle:
#5 0x00007ffff78d4f32 in wombatThrottle_lock (throttle=0x6298e0) at mama/c_cpp/src/c/throttle.c:441
#6 0x00007ffff78a34e2 in imageRequest_stopWaitForResponse (request=0x14d1a20) at mama/c_cpp/src/c/imagerequest.c:774
#7 0x00007ffff78cbe06 in mamaSubscription_stopWaitForResponse (subscription=0xe36280, ctx=0xe364a0) at mama/c_cpp/src/c/subscription.c:1262
#8 0x00007ffff78a38fe in processPointToPointMessage (callback=0x1527e50, msg=0x642460, msgType=6, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:169
#9 0x00007ffff78a3f9c in listenerMsgCallback_processMsg (callback=0x1527e50, msg=0x642460, ctx=0xe364a0) at mama/c_cpp/src/c/listenermsgcallback.c:480
#10 0x00007ffff78cd825 in mamaSubscription_processMsg (subscription=0xe36280, msg=0x642460) at mama/c_cpp/src/c/subscription.c:2259

What do you think is the best way to avoid this?


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

OpenMAMA 6.2.1 Released

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

We are pleased to announce the final release of OpenMAMA 6.2.1 is now available:


This release exists mainly to address several issues coming out of the recent MAMA Datetime changes:

  • New explicit mamaDateTime_[gs]etEpochTimeExt methods to allow bridges and applications to directly set the underlying timestamp value regardless of whether or not time_t on the target system has sufficient resolution
  • Fixed issue with extended datetime representation on 32-bit systems (Windows and Linux)
  • Fixed issue with multiple subscribers for the same topic on qpid
  • Fixed crash in conflated order book processing
  • New explicit public accessors for datetime precision and hints
  • Release distributions will now include dependent libraries inside the target package and related license information is included (including the new Apache APR dependency)

NB: This release includes the removal of the legacy _USE_32BIT_TIME_T compile time macro for 32 bit windows. Please ensure that third party application and bridges are not compiled using this macro to avoid potential corruption of data.

For a complete list of all 17 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/7?closed=1

A special thanks to all developers, contributors and testers who helped is getting this out door.

Cheers,
Frank

OpenMAMA-6.2.1-rc2 Now Available

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

The second release candidate for OpenMAMA 6.2.1 has just been cut and contains a few fixes put in since RC1 - see https://github.com/OpenMAMA/OpenMAMA/releases/tag/OpenMAMA-6.2.1-rc2.

Once again, we appreciate the continued assistance from the community. Assuming no major issues are found, we'll aim to release the GA version on 15th June.

If you find any issues, please see here for details on how to report it to us:


Cheers,
Frank

Last call for OpenMAMA 6.2.1 RC2 bugs

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

I will soon be preparing a release for the second release candidate for OpenMAMA 6.2.1.

Note that there are a fair few changes and bug fixes to make extended datetime work on both 32 and 64 bit platforms:

1. New explicit mamaDateTime_[gs]etFromEpochExt methods to allow bridges to explicitly set the underlying value regardless of whether or not time_t on the target system has sufficient resolution
2. New explicit getters and setters for datetime precision and hints
3. Release distributions will now include dependent libraries inside the target package and related license information will be included (changes about to land).

If anyone has any further issues they would like to report please do so before the end of the day, otherwise we'll be cutting RC2 tomorrow morning.

I'll send a full notification after the cut is made.

Cheers,
Frank

OpenMAMA-6.2.1-rc1 Now Available

Frank Quinn <fquinn.ni@...>
 

Hi Folks,

We are pleased to announce the first release candidate for openMAMA 6.2.1 is now available:


This is a bugfix release to address a few main issues:
  • Fixed issue with extended datetime representation on 32-bit systems (Windows and Linux)
  • Fixed issue with multiple subscribers for the same topic on qpid
  • Fixed crash in conflated order book processing
For a complete list of all 9 issues included in this release, please see here: https://github.com/OpenMAMA/OpenMAMA/milestone/7?closed=1

Thank you all in advance for your help in testing - if you spot any issues, please follow our guidelines for raising an issue, or even better, follow our guidelines for raising a patch.

As this release is fairly small bugfix release, we're not expecting a large volume of bugs to fall out, so I will be acting on a reasonably brief test window of 1 week commencing today. If no issues have been found, we can aim to release on 29th May. If critical issues are found, we will continue to go through weekly release candidates until have a stable release ready.

Cheers,
Frank