Hi all,

We have noticed that both mamda/c_cpp/src/cpp/MamdaQuoteListener.cpp and mamda/c_cpp/src/orderbooks/MamdaQuoteToBookListener have implementations of a QuoteCache struct in the Wombat namespace. Therefore, both libmamda and libmamdabook are exporting the same symbol.

This is an issue when linking against the two libraries since the order they are dynamically linked determines which one is used. If the wrong QuoteCache struct is used this causes a segfault during construction of the object as the two implementations are different.

To me this seems like a bug and one of the two QuoteCache structs should be renamed.

Has anyone had an issue with this before / can anyone shed some light on this?

Thanks,

Mike Slade | Corporate & Investment Bank | Market Data Services | J.P. Morgan

This message is confidential and subject to terms at: https://www.jpmorgan.com/emaildisclaimer including on confidentiality, legal privilege, viruses and monitoring of electronic messages. If you are not the intended recipient, please delete this message and notify the sender immediately. Any unauthorized use is strictly prohibited.

 

Classification: Public

 

Hi openmama team,

 

 

 

I see that on linux platform openmama uses dlopen() with RTLD_GLOBAL flag to load bridge implementations.

 

What do you think about changing that dlopen() call to use RTLD_LOCAL instead?

 

Using dlopen() with RTLD_LOCAL naturally provides named symbol isolation behavior one would expect from a system which dynamically loads implementations.

 

 

 

I cannot post a patch here yet but otherwise it is a trivial change to src/common/c_cpp/src/c/linux/platform.c line 65

 

 

 

Using RTLD_GLOBAL creates a problem if any two bridge implementations provide the same global symbol name. If loaded simultaneously these will share the same underlying object resolved through the library loaded first.

 

For instance a crash can be reproduced with just loading tick42blp and tick42rmds bridges, and then closing mama. While mama is closing first bridge shutting down will destroy gTimerHeap object, and then last bridge shutting down will attempt to destroy the same gTimerHeap once again.

 

 

 

A workaround would be to carefully examine non-local symbols from each bridge implementation and fix accordingly if bridge sources are available.

 

We can amend either blp or rmds bridge implementation - but this is not always possible and is obviously very inconvenient.

 

 

 

Note that on windows platform implementation already is behaving as non-global due to the fact that you always need explicitly loaded library handle to resolve a symbol.

 

 

 

Thank you!

 

 

 

--

 

Kind regards,

Igor Kovalenko

____________________________________________________

 

Igor Kovalenko

 

OOO "Deutsche Bank TechCentre"

Global Technology

Business Center Aquamarine, Ozerkovskaya naberezhnaya 24, 115184 Moscow, Russia

Tel. +7 495 783-0313

Email igor.kovalenko@...

 

 

 

---

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

 

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

 

Classification: Public

Hi openmama team,

 

I see that on linux platform openmama uses dlopen() with RTLD_GLOBAL flag to load bridge implementations.

What do you think about changing that dlopen() call to use RTLD_LOCAL instead?

Using dlopen() with RTLD_LOCAL naturally provides named symbol isolation behavior one would expect from a system which dynamically loads implementations.

 

I cannot post a patch here yet but otherwise it is a trivial change to src/common/c_cpp/src/c/linux/platform.c line 65

 

Using RTLD_GLOBAL creates a problem if any two bridge implementations provide the same global symbol name. If loaded simultaneously these will share the same underlying object resolved through the library loaded first.

For instance a crash can be reproduced with just loading tick42blp and tick42rmds bridges, and then closing mama. While mama is closing first bridge shutting down will destroy gTimerHeap object, and then last bridge shutting down will attempt to destroy the same gTimerHeap once again.

 

A workaround would be to carefully examine non-local symbols from each bridge implementation and fix accordingly if bridge sources are available.

We can amend either blp or rmds bridge implementation - but this is not always possible and is obviously very inconvenient.

 

Note that on windows platform implementation already is behaving as non-global due to the fact that you always need explicitly loaded library handle to resolve a symbol.

 

Thank you!

 

--

Kind regards,

Igor Kovalenko

____________________________________________________

Igor Kovalenko

OOO "Deutsche Bank TechCentre"

Global Technology

Business Center Aquamarine, Ozerkovskaya naberezhnaya 24, 115184 Moscow, Russia

Tel. +7 495 783-0313

Email igor.kovalenko@...

 

---

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.

Hey Igor,

I find I am unable to reproduce your issue. I am using group subscriptions (to a feed handler) and have disabled dq.

I tried the various config options to reproduce the issue (as there are transport specific and global config options for disabling dq strategy). With regards to the missing conditional:  I can see that around the line of the change we have taken out the following conditional which I assume you are referring to:

I would just point out that this conditional statement hasn’t been removed, but was relocated to mamaPlugin_fireSubscriptionPreMsgHook.

Certainly, the return statement activated by that conditional now returns you to this function, from mamaPlugin_fireSubscriptionPreMsgHook. With previous functionality, this conditional would ensure you skip the code that you say causes the problem you are seeing, and that makes sense.

I cannot recommend duplicating the conditional statement, as this would be unnecessary and as I cannot reproduce your issue I can only make a suggestion that hopefully you could try?

At present mamaPlugin_fireSubscriptionPreMsgHook returns MAMA_STATUS_OK in all cases. My suggestion is, in dqstrategyplugin.c, at line 478, change MAMA_STATUS_OK to MAMA_STATUS_INVALID_ARG (for example).

Then, in listenermsgcallback.c, on line 380, add in a check to the return type, and if you do not receive MAMA_STATUS_OK, return from this function. This would look like:
    if (MAMA_STATUS_OK != mamaPlugin_fireSubscriptionPreMsgHook(subscription, msgType, msg))

    {

        return;

    }

This would now emulate the previous functionality (but does not duplicate the conditional).

I have done this myself and performed a quick check that it does not break anything and it looks fine. But as I say, I cannot replicate your issue and so cannot do a full check. I would therefore say that if it suits, you could change that single line in dqstrategyplugin.c and add the check in listenermsgcallback, and run a test to see if this fixes your issue.

I am still concerned as to why I cannot reproduce the same issue, but please let me know what you think.

Regards,

Aaron

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Systems LLC

Thanks Igor,

This was part of Vela's DQ strategy changes - Aaron could you have a look please?

Cheers,

Frank

On Mon, Nov 12, 2018 at 1:32 PM Igor Kovalenko <igor.kovalenko@...> wrote:

Classification: Public

Hi openmama team,

 

I see that commit 6de60350f40b4d9b0f181b7e64e0d192798b9f97 accidentally removed one conditional expression from around current line 393 of listenermsgcallbacks.c

 

Now if group subscription, or in fact any allowing multiple initials, receives first initial, then

- with dqstrategy disabled listenerMsgCallback_processMsg will unconditionally call mamaSubscription_stopWaitForResponse

- with dqstrategy enabled dqstrategy plugin implementation would only call mamaSubscription_stopWaitForResponse if mamaSubscription_getAcceptMultipleInitials() returns false

 

The code that stops waiting for response destroys related inbox object. This leads to easy crash with group subscriptions and dqstrategy disabled.

 

Hope it will be not hard to restore the conditional for openmama-6.2.x and later.

 

Thank you!

 

Kind regards,

Igor Kovalenko

 

--

 

---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to https://www.db.com/disclosures for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.