Openmama-dev@lists.openmama.org

[PATCH] QPIDMSG: Modified payload to work on other middlewares

Frank Quinn <fquinn.ni@...>

#1577

The byte buffer functions in the Qpid Proton payload implementation
only recognized the handles provided by the qpid proton middleware
bridge rather than real raw byte arrays. This made the payload
incompatible with other middlewares. This change keeps the
efficiency gains held by using these handles while also enabling
compatibility with other middleware bridges.

Signed-off-by: Frank Quinn <fquinn.ni@...>
---
 mama/c_cpp/src/c/payload/qpidmsg/payload.c | 27 +++++++++++++++++++++++----
 1 file changed, 23 insertions(+), 4 deletions(-)

diff --git a/mama/c_cpp/src/c/payload/qpidmsg/payload.c b/mama/c_cpp/src/c/payload/qpidmsg/payload.c
index 2e69769..34823a3 100644
--- a/mama/c_cpp/src/c/payload/qpidmsg/payload.c
+++ b/mama/c_cpp/src/c/payload/qpidmsg/payload.c
@@ -979,6 +979,7 @@ qpidmsgPayload_setByteBuffer (const msgPayload    msg,
                               mama_size_t         bufferLength)
 {
     qpidmsgPayloadImpl*  impl       = (qpidmsgPayloadImpl*) msg;
+    mama_status          status     = MAMA_STATUS_OK;
 
     if (   NULL == msg
         || NULL == buffer
@@ -987,10 +988,19 @@ qpidmsgPayload_setByteBuffer (const msgPayload    msg,
         return MAMA_STATUS_NULL_ARG;
     }
 
-    impl->mQpidMsg = (pn_message_t*) buffer;
-    impl->mBody    = pn_message_body (impl->mQpidMsg);
+    if (bufferLength == sizeof(pn_message_t*))
+    {
+        impl->mQpidMsg = (pn_message_t*) buffer;
+        impl->mBody    = pn_message_body (impl->mQpidMsg);
+    }
+    else
+    {
+        status = qpidmsgPayload_unSerialize (msg,
+                                             (const void**)buffer,
+                                             bufferLength);
+    }
 
-    return MAMA_STATUS_OK;
+    return status;
 }
 
 mama_status
@@ -999,13 +1009,22 @@ qpidmsgPayload_createFromByteBuffer (msgPayload*         msg,
                                      const void*         buffer,
                                      mama_size_t         bufferLength)
 {
-    mama_status status = qpidmsgPayloadImpl_createImplementationOnly (msg);
+    mama_status status = MAMA_STATUS_OK;
 
     if (0 == bufferLength)
     {
     	return MAMA_STATUS_INVALID_ARG;
     }
 
+    // If this is a byte handle
+    if (bufferLength == sizeof(pn_message_t*))
+    {
+        status = qpidmsgPayloadImpl_createImplementationOnly (msg);
+    }
+    else
+    {
+        status = qpidmsgPayload_create (msg);
+    }
     if (MAMA_STATUS_OK == status)
     {
         status = qpidmsgPayload_setByteBuffer (*msg,
-- 
2.4.3

0001-QPIDMSG-Modified-payload-to-work-on-other-middleware.patch

Re: OM 2.3.3 and entitlements

Glenn McClements <g.mcclements@...>

#1576

Yeah send on the patch and we can have a look.

Cheers,

Glenn

Glenn McClements, SVP Engineering

Cell: +44 78259 58270

Adelaide Exchange, 24-26 Adelaide Street, Belfast, UK, BT2 8GD

www.srtechlabs.com

SR.LABS Proven High Speed Electronic Trading Solutions

From: "Alpert, Reed"
Date: Wednesday, 12 August 2015 18:12
To: Glenn McClements, "openmama-dev@..."
Subject: RE: [Openmama-dev] OM 2.3.3 and entitlements

Hi,

OK if I submit the patch that for Sconscripts that allows build w/ entitlements?

It can all be removed when the entitle plugin is ready.

The patch adds entitle_home to the command line parms (location of oeac lib) and then adds the path and lib itself to the links for the examples and test tools.

Thanks,

Reed.

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

From: Glenn McClements [mailto:g.mcclements@...]
Sent: Wednesday, August 12, 2015 4:54 AM
To: Alpert, Reed; openmama-dev@...
Subject: Re: [Openmama-dev] OM 2.3.3 and entitlements

Hi Reed,

No, the open source release is not meant to be built with entitlements (though the option is there is the customer has licensed OEA).

To do it properly I want to decouple OEA from from the core OpenMAMA code and make it available via a plugin but this is still on the roadmap.

Thanks,

Glenn

Glenn McClements, SVP Engineering

Cell: +44 78259 58270

Adelaide Exchange, 24-26 Adelaide Street, Belfast, UK, BT2 8GD

www.srtechlabs.com

SR.LABS Proven High Speed Electronic Trading Solutions

From: <openmama-dev-bounces@...> on behalf of "Alpert, Reed"
Date: Tuesday, 11 August 2015 22:02
To: "openmama-dev@..."
Subject: [Openmama-dev] OM 2.3.3 and entitlements

Hi,

When building OM 2.3.3 with entitlements enabled there are some needed changes to SConscripts.

I have prepared a patch (mostly adding in the oeac library), and can send that in, but wanted to check if OM is intended to be built with entitlements, since that requires that the oeac library be available, along with the header files.

Thanks,

Reed.

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.

Re: OM 2.3.3 and entitlements

Alpert, Reed <reed.alpert@...>

#1575

Hi,

OK if I submit the patch that for Sconscripts that allows build w/ entitlements?

It can all be removed when the entitle plugin is ready.

The patch adds entitle_home to the command line parms (location of oeac lib) and then adds the path and lib itself to the links for the examples and test tools.

Thanks,

Reed.

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

From: Glenn McClements [mailto:g.mcclements@...]
Sent: Wednesday, August 12, 2015 4:54 AM
To: Alpert, Reed; openmama-dev@...
Subject: Re: [Openmama-dev] OM 2.3.3 and entitlements

Hi Reed,

No, the open source release is not meant to be built with entitlements (though the option is there is the customer has licensed OEA).

To do it properly I want to decouple OEA from from the core OpenMAMA code and make it available via a plugin but this is still on the roadmap.

Thanks,

Glenn

Glenn McClements, SVP Engineering

Cell: +44 78259 58270

Adelaide Exchange, 24-26 Adelaide Street, Belfast, UK, BT2 8GD

www.srtechlabs.com

SR.LABS Proven High Speed Electronic Trading Solutions

From: <openmama-dev-bounces@...> on behalf of "Alpert, Reed"
Date: Tuesday, 11 August 2015 22:02
To: "openmama-dev@..."
Subject: [Openmama-dev] OM 2.3.3 and entitlements

Hi,

When building OM 2.3.3 with entitlements enabled there are some needed changes to SConscripts.

I have prepared a patch (mostly adding in the oeac library), and can send that in, but wanted to check if OM is intended to be built with entitlements, since that requires that the oeac library be available, along with the header files.

Thanks,

Reed.

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.

Re: OM 2.3.3 and entitlements

Glenn McClements <g.mcclements@...>

#1574

Hi Reed,

No, the open source release is not meant to be built with entitlements (though the option is there is the customer has licensed OEA).

To do it properly I want to decouple OEA from from the core OpenMAMA code and make it available via a plugin but this is still on the roadmap.

Thanks,

Glenn

Glenn McClements, SVP Engineering

Cell: +44 78259 58270

Adelaide Exchange, 24-26 Adelaide Street, Belfast, UK, BT2 8GD

www.srtechlabs.com

SR.LABS Proven High Speed Electronic Trading Solutions

From: <openmama-dev-bounces@...> on behalf of "Alpert, Reed"
Date: Tuesday, 11 August 2015 22:02
To: "openmama-dev@..."
Subject: [Openmama-dev] OM 2.3.3 and entitlements

Hi,

When building OM 2.3.3 with entitlements enabled there are some needed changes to SConscripts.

I have prepared a patch (mostly adding in the oeac library), and can send that in, but wanted to check if OM is intended to be built with entitlements, since that requires that the oeac library be available, along with the header files.

Thanks,

Reed.

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.

OM 2.3.3 and entitlements

Alpert, Reed <reed.alpert@...>

#1573

Hi,

When building OM 2.3.3 with entitlements enabled there are some needed changes to SConscripts.

I have prepared a patch (mostly adding in the oeac library), and can send that in, but wanted to check if OM is intended to be built with entitlements, since that requires that the oeac library be available, along with the header files.

Thanks,

Reed.

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.

Re: Timer race conditions

Frank Quinn <fquinn.ni@...>

#1572

Hi Duane,

OK I understand now, thanks - got confused between 'the thread on which timer events are firing' and the 'timer thread'.

I have given consideration in the past to simply ripping out this timer code (it's literally not used by anything other than avis / qpid bridges) and using a libevent evtimer or something like that instead directly from the bridge - I even have a mock implementation for that lying around somewhere, but that's a matter for another day.

In the interim, yeah it would be great if you could raise a bugzilla ticket with a patch for timers.c and the qpid bridge (don't worry about avis because the qpid implementation is about to replace it anyway) and we'll look to get that merged.

Cheers,

Frank

Show quoted text

On Thu, Jul 30, 2015 at 10:01 PM, Duane Pauls <Duane.Pauls@...> wrote:

Hi Frank,

The mamaTimer is only being destroyed from one thread. The problem is that the wombat timerElement is being destroyed from two threads at once, but not the mamaTimer.

One of the timerElement destroys is occurring from within the bridge’s timerElement callback, which is necessary to implement a reoccurring timer with a one-shot timer primitive. The second destroy is occurring from an application call to mamaTimer_destroy(). The application has no way of knowing the timer is currently expiring.

The natural solution is to put locks in the bridge implementation to prevent certain things from occurring at the same time in different threads (which I believe should be done in the qpid and avis bridges), but then you end up with deadlock because the heap is holding the lock while calling the bridge’s callback.

In our experience with messaging API’s, you need to be very careful about holding locks while calling application callbacks. We try to avoid this in our API’s. I don’t see any reason the timer heap needs to hold a lock during the callback here.

Cheers,

Duane

From: Frank Quinn [mailto:fquinn.ni@...]
Sent: Thursday, July 30, 2015 3:52 PM
To: Duane Pauls
Cc: openmama-dev@...
Subject: Re: [Openmama-dev] Timer race conditions

Hi Duane,

I read this as the problem being triggered by the timer being destroyed from multiple threads is that right? If so, that is an illegal sequence of events and would constitute a bug in the application itself. From the MAMA Developer's guide section on timers:

"A timer can be destroyed from within a timer callback or any other callback on the queue. This function must be called from the same thread dispatching on the associated event queue unless both the default queue and dispatch queue are not actively dispatching."

Cheers,

Frank

On Thu, Jul 30, 2015 at 6:54 PM, Duane Pauls <Duane.Pauls@...> wrote:

At Solace our timer implementation in our middleware bridge is very similar to that of qpid. So although my comments below are based primarily on our experience with our own middleware bridge, I believe the problems described apply equally to qpid. The natural fix for the problem, as I’ll explain below, leads to the possibility of deadlock, which I believe should be fixed in common/c_cpp/src/c/timers.c.

I’d like to get feedback on my assessment, to see if we agree that the timers module should be modified in OpenMAMA. Or, if anyone has a suggestion for another way to solve the problem, I’d be interested in that as well.

We discovered rare cases of crashes when destroying timers. By looking at the dumps and inspecting the code, I believe the problem is that the bridgeImpl’s timerElement is not protected against multi-threaded access. Consider:

1.       A timer expires. In the timer thread, the bridge’s callback calls ‘BridgeMamaTimer_reset’, which destroys (i.e. frees) the timerElement.

2.       Before BridgeMamaTimer_reset recreates the timerElement, an application thread decides to destroy the timer (i.e. it is being destroyed as it is expiring). The destroy flows through to the bridge, which is asked to destroy the timer. The bridge then frees the timerElement. This timerElement has already been freed in #1 above, leading to a crash.

The natural fix here is to protect the bridge’s timerElement with a lock. In the BridgeMamaTimer_reset function, use the lock to make the destroy/create an atomic operation, and also protect accesses to the timerElement.

If this is done however, there is now a possibility for deadlock. When a timer expires:

1.       timers.c ‘dispatchEntry’ locks the timer heap.

2.       The bridge timer callback is called.

3.       The bridge callback needs to reset the timer, which means taking the timer lock.

Now suppose an application (for example MamaTimerTestCPP_ForTimer) decides that when a timer expires it wants to destroy the timer (i.e. implement a one-shot timer). The following sequence would occur:

1.       The application timer callback is called on the application’s queue dispatch thread.

2.       The application calls mamaTimer_destroy().

3.       mamaTimer_destroy() calls BridgeMamaTimer_destroy. This involves taking the timerElement lock.

4.       BridgeMamaTimer_destroy calls destroyTimer, which involves taking the timer heap lock.

Because the above sequences take the same locks, but in different orders, we have the opportunity for deadlock.

A patch that I would propose to solve this issue is:

Index: timers.c

===================================================================

@@ -176,7 +176,9 @@

                 while (!timercmp(&ele->mTimeout, &now, >))

                 {

                     RB_REMOVE (orderedTimeRBTree_, &heap->mTimeTree, ele);

+                    wthread_mutex_unlock (&heap->mLock);

                     ele->mCb (ele, ele->mClosure);

+                    wthread_mutex_lock (&heap->mLock);

                     ele = RB_MIN (orderedTimeRBTree_, &heap->mTimeTree);

                     /* No timers left so break */

                     if (ele == NULL)

This means the timer heap lock is not held for the duration of an application callback. It should always be safe to re-acquire the timer heap lock after the callback since something that could have been invalidated, such as an iterator, is not being used – the function calls RB_MIN to look at the first item on the heap.



Does this seem like a reasonable approach to the problem I’ve described? Or does someone have an alternate proposal for how to handle this in the bridge? Or another solution with OpenMAMA code?

Should a bug be raised for this issue?

Note qpid is not currently susceptible to this issue, but I believe it is susceptible to the double-free described at the beginning.

Cheers,

Duane

_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

Re: Timer race conditions

Duane Pauls <Duane.Pauls@...>

#1571

Hi Frank,

The mamaTimer is only being destroyed from one thread. The problem is that the wombat timerElement is being destroyed from two threads at once, but not the mamaTimer.

One of the timerElement destroys is occurring from within the bridge’s timerElement callback, which is necessary to implement a reoccurring timer with a one-shot timer primitive. The second destroy is occurring from an application call to mamaTimer_destroy(). The application has no way of knowing the timer is currently expiring.

The natural solution is to put locks in the bridge implementation to prevent certain things from occurring at the same time in different threads (which I believe should be done in the qpid and avis bridges), but then you end up with deadlock because the heap is holding the lock while calling the bridge’s callback.

In our experience with messaging API’s, you need to be very careful about holding locks while calling application callbacks. We try to avoid this in our API’s. I don’t see any reason the timer heap needs to hold a lock during the callback here.

Cheers,

Duane

From: Frank Quinn [mailto:fquinn.ni@...]
Sent: Thursday, July 30, 2015 3:52 PM
To: Duane Pauls
Cc: openmama-dev@...
Subject: Re: [Openmama-dev] Timer race conditions

Hi Duane,

I read this as the problem being triggered by the timer being destroyed from multiple threads is that right? If so, that is an illegal sequence of events and would constitute a bug in the application itself. From the MAMA Developer's guide section on timers:

"A timer can be destroyed from within a timer callback or any other callback on the queue. This function must be called from the same thread dispatching on the associated event queue unless both the default queue and dispatch queue are not actively dispatching."

Cheers,

Frank

Show quoted text

On Thu, Jul 30, 2015 at 6:54 PM, Duane Pauls <Duane.Pauls@...> wrote:

At Solace our timer implementation in our middleware bridge is very similar to that of qpid. So although my comments below are based primarily on our experience with our own middleware bridge, I believe the problems described apply equally to qpid. The natural fix for the problem, as I’ll explain below, leads to the possibility of deadlock, which I believe should be fixed in common/c_cpp/src/c/timers.c.

I’d like to get feedback on my assessment, to see if we agree that the timers module should be modified in OpenMAMA. Or, if anyone has a suggestion for another way to solve the problem, I’d be interested in that as well.

We discovered rare cases of crashes when destroying timers. By looking at the dumps and inspecting the code, I believe the problem is that the bridgeImpl’s timerElement is not protected against multi-threaded access. Consider:

1. A timer expires. In the timer thread, the bridge’s callback calls ‘BridgeMamaTimer_reset’, which destroys (i.e. frees) the timerElement.

2. Before BridgeMamaTimer_reset recreates the timerElement, an application thread decides to destroy the timer (i.e. it is being destroyed as it is expiring). The destroy flows through to the bridge, which is asked to destroy the timer. The bridge then frees the timerElement. This timerElement has already been freed in #1 above, leading to a crash.

The natural fix here is to protect the bridge’s timerElement with a lock. In the BridgeMamaTimer_reset function, use the lock to make the destroy/create an atomic operation, and also protect accesses to the timerElement.

If this is done however, there is now a possibility for deadlock. When a timer expires:

1. timers.c ‘dispatchEntry’ locks the timer heap.

2. The bridge timer callback is called.

3. The bridge callback needs to reset the timer, which means taking the timer lock.

Now suppose an application (for example MamaTimerTestCPP_ForTimer) decides that when a timer expires it wants to destroy the timer (i.e. implement a one-shot timer). The following sequence would occur:

1. The application timer callback is called on the application’s queue dispatch thread.

2. The application calls mamaTimer_destroy().

3. mamaTimer_destroy() calls BridgeMamaTimer_destroy. This involves taking the timerElement lock.

4. BridgeMamaTimer_destroy calls destroyTimer, which involves taking the timer heap lock.

Because the above sequences take the same locks, but in different orders, we have the opportunity for deadlock.

A patch that I would propose to solve this issue is:

Index: timers.c

===================================================================

@@ -176,7 +176,9 @@

while (!timercmp(&ele->mTimeout, &now, >))

{

RB_REMOVE (orderedTimeRBTree_, &heap->mTimeTree, ele);

+ wthread_mutex_unlock (&heap->mLock);

ele->mCb (ele, ele->mClosure);

+ wthread_mutex_lock (&heap->mLock);

ele = RB_MIN (orderedTimeRBTree_, &heap->mTimeTree);

/* No timers left so break */

if (ele == NULL)

This means the timer heap lock is not held for the duration of an application callback. It should always be safe to re-acquire the timer heap lock after the callback since something that could have been invalidated, such as an iterator, is not being used – the function calls RB_MIN to look at the first item on the heap.

Does this seem like a reasonable approach to the problem I’ve described? Or does someone have an alternate proposal for how to handle this in the bridge? Or another solution with OpenMAMA code?

Should a bug be raised for this issue?

Note qpid is not currently susceptible to this issue, but I believe it is susceptible to the double-free described at the beginning.

Cheers,

Duane

_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

Re: Timer race conditions

Frank Quinn <fquinn.ni@...>

#1570

Hi Duane,

I read this as the problem being triggered by the timer being destroyed from multiple threads is that right? If so, that is an illegal sequence of events and would constitute a bug in the application itself. From the MAMA Developer's guide section on timers:

"A timer can be destroyed from within a timer callback or any other callback on the queue. This function must be called from the same thread dispatching on the associated event queue unless both the default queue and dispatch queue are not actively dispatching."

Cheers,

Frank

Show quoted text

On Thu, Jul 30, 2015 at 6:54 PM, Duane Pauls <Duane.Pauls@...> wrote:

At Solace our timer implementation in our middleware bridge is very similar to that of qpid. So although my comments below are based primarily on our experience with our own middleware bridge, I believe the problems described apply equally to qpid. The natural fix for the problem, as I’ll explain below, leads to the possibility of deadlock, which I believe should be fixed in common/c_cpp/src/c/timers.c.

I’d like to get feedback on my assessment, to see if we agree that the timers module should be modified in OpenMAMA. Or, if anyone has a suggestion for another way to solve the problem, I’d be interested in that as well.

We discovered rare cases of crashes when destroying timers. By looking at the dumps and inspecting the code, I believe the problem is that the bridgeImpl’s timerElement is not protected against multi-threaded access. Consider:

1.       A timer expires. In the timer thread, the bridge’s callback calls ‘BridgeMamaTimer_reset’, which destroys (i.e. frees) the timerElement.

2.       Before BridgeMamaTimer_reset recreates the timerElement, an application thread decides to destroy the timer (i.e. it is being destroyed as it is expiring). The destroy flows through to the bridge, which is asked to destroy the timer. The bridge then frees the timerElement. This timerElement has already been freed in #1 above, leading to a crash.

The natural fix here is to protect the bridge’s timerElement with a lock. In the BridgeMamaTimer_reset function, use the lock to make the destroy/create an atomic operation, and also protect accesses to the timerElement.

If this is done however, there is now a possibility for deadlock. When a timer expires:

1.       timers.c ‘dispatchEntry’ locks the timer heap.

2.       The bridge timer callback is called.

3.       The bridge callback needs to reset the timer, which means taking the timer lock.

Now suppose an application (for example MamaTimerTestCPP_ForTimer) decides that when a timer expires it wants to destroy the timer (i.e. implement a one-shot timer). The following sequence would occur:

1.       The application timer callback is called on the application’s queue dispatch thread.

2.       The application calls mamaTimer_destroy().

3.       mamaTimer_destroy() calls BridgeMamaTimer_destroy. This involves taking the timerElement lock.

4.       BridgeMamaTimer_destroy calls destroyTimer, which involves taking the timer heap lock.

Because the above sequences take the same locks, but in different orders, we have the opportunity for deadlock.

A patch that I would propose to solve this issue is:

Index: timers.c

===================================================================

@@ -176,7 +176,9 @@

                 while (!timercmp(&ele->mTimeout, &now, >))

                 {

                     RB_REMOVE (orderedTimeRBTree_, &heap->mTimeTree, ele);

+                    wthread_mutex_unlock (&heap->mLock);

                     ele->mCb (ele, ele->mClosure);

+                    wthread_mutex_lock (&heap->mLock);

                     ele = RB_MIN (orderedTimeRBTree_, &heap->mTimeTree);

                     /* No timers left so break */

                     if (ele == NULL)

This means the timer heap lock is not held for the duration of an application callback. It should always be safe to re-acquire the timer heap lock after the callback since something that could have been invalidated, such as an iterator, is not being used – the function calls RB_MIN to look at the first item on the heap.



Does this seem like a reasonable approach to the problem I’ve described? Or does someone have an alternate proposal for how to handle this in the bridge? Or another solution with OpenMAMA code?

Should a bug be raised for this issue?

Note qpid is not currently susceptible to this issue, but I believe it is susceptible to the double-free described at the beginning.

Cheers,

Duane

_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

Timer race conditions

Duane Pauls <Duane.Pauls@...>

#1569

At Solace our timer implementation in our middleware bridge is very similar to that of qpid. So although my comments below are based primarily on our experience with our own middleware bridge, I believe the problems described apply equally to qpid. The natural fix for the problem, as I’ll explain below, leads to the possibility of deadlock, which I believe should be fixed in common/c_cpp/src/c/timers.c.

I’d like to get feedback on my assessment, to see if we agree that the timers module should be modified in OpenMAMA. Or, if anyone has a suggestion for another way to solve the problem, I’d be interested in that as well.

We discovered rare cases of crashes when destroying timers. By looking at the dumps and inspecting the code, I believe the problem is that the bridgeImpl’s timerElement is not protected against multi-threaded access. Consider:

1. A timer expires. In the timer thread, the bridge’s callback calls ‘BridgeMamaTimer_reset’, which destroys (i.e. frees) the timerElement.

2. Before BridgeMamaTimer_reset recreates the timerElement, an application thread decides to destroy the timer (i.e. it is being destroyed as it is expiring). The destroy flows through to the bridge, which is asked to destroy the timer. The bridge then frees the timerElement. This timerElement has already been freed in #1 above, leading to a crash.

The natural fix here is to protect the bridge’s timerElement with a lock. In the BridgeMamaTimer_reset function, use the lock to make the destroy/create an atomic operation, and also protect accesses to the timerElement.

If this is done however, there is now a possibility for deadlock. When a timer expires:

1. timers.c ‘dispatchEntry’ locks the timer heap.

2. The bridge timer callback is called.

3. The bridge callback needs to reset the timer, which means taking the timer lock.

Now suppose an application (for example MamaTimerTestCPP_ForTimer) decides that when a timer expires it wants to destroy the timer (i.e. implement a one-shot timer). The following sequence would occur:

1. The application timer callback is called on the application’s queue dispatch thread.

2. The application calls mamaTimer_destroy().

3. mamaTimer_destroy() calls BridgeMamaTimer_destroy. This involves taking the timerElement lock.

4. BridgeMamaTimer_destroy calls destroyTimer, which involves taking the timer heap lock.

Because the above sequences take the same locks, but in different orders, we have the opportunity for deadlock.

A patch that I would propose to solve this issue is:

Index: timers.c

===================================================================

@@ -176,7 +176,9 @@

while (!timercmp(&ele->mTimeout, &now, >))

{

RB_REMOVE (orderedTimeRBTree_, &heap->mTimeTree, ele);

+ wthread_mutex_unlock (&heap->mLock);

ele->mCb (ele, ele->mClosure);

+ wthread_mutex_lock (&heap->mLock);

ele = RB_MIN (orderedTimeRBTree_, &heap->mTimeTree);

/* No timers left so break */

if (ele == NULL)

This means the timer heap lock is not held for the duration of an application callback. It should always be safe to re-acquire the timer heap lock after the callback since something that could have been invalidated, such as an iterator, is not being used – the function calls RB_MIN to look at the first item on the heap.

Does this seem like a reasonable approach to the problem I’ve described? Or does someone have an alternate proposal for how to handle this in the bridge? Or another solution with OpenMAMA code?

Should a bug be raised for this issue?

Note qpid is not currently susceptible to this issue, but I believe it is susceptible to the double-free described at the beginning.

Cheers,

Duane

Re: QPID Proton bridge: multiple subscribers, one publisher

Frank Quinn <fquinn.ni@...>

#1568

Hi Nestor,

You definitely need a broker in the middle for that sort of implementation - point to point will not support that.

You should be able to publish data to the same MAMA source (topic namespace) from multiple instances if you're using a broker and collect all the data from a single client, but you need to be careful about publishing to the same topic across multiple instances if you're using market data subscriptions.

Market data subscriptions check the MamaSenderId and MdSeqNum on every update to detect gaps, so if the client constantly receives overlapping values for these fields on the same subscription, you'll get recap storms.

Cheers,

Frank

Show quoted text

On Wed, Jul 29, 2015 at 7:11 PM, Macrux <kmacrux@...> wrote:

Hi Frank,

Just to solve a similar question, is it possible to have the opposite behaviour in which multiple applications (separated instances) publish data into the same topic (or source), and only one subscriber receives all that data?

Thanks in advance,

On 21 July 2015 at 13:27, Frank Quinn <fquinn.ni@...> wrote:
Hi Nestor,

If you modify each new subscriber to use a different incoming URL and reply URL, you can actually do this today, though as you can imagine it becomes a bit of a brute to configure very quickly.

The better solution though is probably the currently experimental feature-qpid-broker branch that I'm hoping will make it into OpenMAMA 2.3.4. It allows for a much easier to manage pub / sub infrastructure where in terms of transport configuration, all you do is point it to the broker - there's an example mama.properties on that branch to give you an idea: http://git.openmama.org/?p=OpenMAMA.git;a=blob;f=mama/c_cpp/src/examples/mama.properties;h=86c7b1ce3933a10b0022b89837094bdb5904b974;hb=feature-qpid-broker

Cheers,
Frank

On Tue, 21 Jul 2015 16:23 Macrux <kmacrux@...> wrote:
Hi guys,

I'd like to know if there was or there will be some change to the Qpid Proton Bridge such that I can have a publish/subscribe connection in which only one advanced publisher can have multiple subscribers, because according to the wiki, "the current QPID Proton bridge is a point-to-point implementation" and I've had to run multiple publishers (always the same) each one with a different transport, and to subscribe each client with only one publisher.

Maybe, if someone knows some technique to achieve this behavior, it would be very helpful for me.

Thanks in advance.

Kind regards,

Nestor.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

Re: SNAPSHOTS Messages

Frank Quinn <fquinn.ni@...>

#1567

Hi Eduardo,

As discussed on IRC, this query is related to what bridge components would need wired to facilitate a traditional exchange's broadcast / refresh / gap fill requirements and there are really two parts to the story for each component - what OpenMAMA does, and how it would be implemented in a bridge. So let's go through your points and try and provide a little clarity.

1. Snapshots / Intra day restart

The OpenMAMA DQ Publisher has a concept of 'initial' requests which greets new intraday clients with a full image of the instrument to which they're subscribing. To implement this usually involves a bridge implementation for 'inbox' related functions to work to facilitate request / reply.

2. Retransmission / Recap

If using market data subscriptions, the message is injected with a sequence number which OpenMAMA then checks to detect gaps. If a gap is detected at all (at a OpenMAMA layer), OpenMAMA will then request a snapshot image with what it calls a 'recap'. I know what you're thinking - this would be a nightmare if the bridge was receiving multicast data and data is coming in gappy or out of sequence, which is why I would suggest that line arbitrage and possibly minor gap filling (if you're going to support that) is done underneath the bridge layer and invisible to MAMA. That way gaps are only reported up to MAMA when they are something that would require an image refresh to recover from. Again, this would require an inbox implementation. It's worth noting that OpenMAMA itself has no concept of 'full tick by tick gap fill', and in existing implementations of MAMA, that sort of functionality is handled exclusively at the middleware layer.

Hope this helps,

Cheers,

Frank

Show quoted text

On Wed, Jul 29, 2015 at 5:43 PM, eduardo noe rodriguez franco <enoerodriguezf@...> wrote:

Hi everyone,

Could you help me with some questions, please?

1.- If one client or broker house made the connection when the market had been working for a short while, I mean, the client made the connection lately to the open of the market. can the client ask for a SNAPSHOT or summary of the operations? I need to build one instance to attend that kind of requests.

2.- In a multicast connection, when the client ask for a re-transmission or recap, that is point to point, but how its works? I mean, what i need to do to attend that kind of request.

Thanks in advance, and I am glad to be part of your group.

Regards

Eduardo Rodriguez

_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

Re: QPID Proton bridge: multiple subscribers, one publisher

Macrux <kmacrux@...>

#1566

Hi Frank,

Just to solve a similar question, is it possible to have the opposite behaviour in which multiple applications (separated instances) publish data into the same topic (or source), and only one subscriber receives all that data?

Thanks in advance,

Show quoted text

On 21 July 2015 at 13:27, Frank Quinn <fquinn.ni@...> wrote:

Hi Nestor,

If you modify each new subscriber to use a different incoming URL and reply URL, you can actually do this today, though as you can imagine it becomes a bit of a brute to configure very quickly.

The better solution though is probably the currently experimental feature-qpid-broker branch that I'm hoping will make it into OpenMAMA 2.3.4. It allows for a much easier to manage pub / sub infrastructure where in terms of transport configuration, all you do is point it to the broker - there's an example mama.properties on that branch to give you an idea: http://git.openmama.org/?p=OpenMAMA.git;a=blob;f=mama/c_cpp/src/examples/mama.properties;h=86c7b1ce3933a10b0022b89837094bdb5904b974;hb=feature-qpid-broker

Cheers,
Frank

On Tue, 21 Jul 2015 16:23 Macrux <kmacrux@...> wrote:
Hi guys,

I'd like to know if there was or there will be some change to the Qpid Proton Bridge such that I can have a publish/subscribe connection in which only one advanced publisher can have multiple subscribers, because according to the wiki, "the current QPID Proton bridge is a point-to-point implementation" and I've had to run multiple publishers (always the same) each one with a different transport, and to subscribe each client with only one publisher.

Maybe, if someone knows some technique to achieve this behavior, it would be very helpful for me.

Thanks in advance.

Kind regards,

Nestor.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

SNAPSHOTS Messages

eduardo noe rodriguez franco <enoerodriguezf@...>

#1565

Hi everyone,

Could you help me with some questions, please?

1.- If one client or broker house made the connection when the market had been working for a short while, I mean, the client made the connection lately to the open of the market. can the client ask for a SNAPSHOT or summary of the operations? I need to build one instance to attend that kind of requests.

2.- In a multicast connection, when the client ask for a re-transmission or recap, that is point to point, but how its works? I mean, what i need to do to attend that kind of request.

Thanks in advance, and I am glad to be part of your group.

Regards

Eduardo Rodriguez

Re: Dropping support for Qpid Proton <= 0.6

Damian Maguire <d.maguire@...>

#1564

No objections here, removing support for the older versions makes sense and should help reduce the overhead of supporting the middleware. I also think we should be pretty aggressive in deprecating support in the future, especially for versions which are known to have major issues.

Cheers,

D

Damian Maguire

Senior Sales Engineer

SR.LABS Proven High Speed Electronic Trading Solutions

Adelaide Exchange | 24-26 Adelaide Street | Belfast | UK |BT2 8GD

d.maguire@...

+44 7835 844770

From: <openmama-dev-bounces@...> on behalf of Frank Quinn
Reply-To: "fquinn.ni@..."
Date: Tuesday, July 28, 2015 at 9:45 AM
To: openmama-dev
Subject: Re: [Openmama-dev] Dropping support for Qpid Proton <= 0.6

Hi Folks,

Last call on this - if I hear no objections within a week, I'll be removing 0.5 and 0.6 support from CI and removing the backwards compatibility macros from the code.

Cheers,

Frank

On Wed, Jul 22, 2015 at 9:28 AM, Frank Quinn <fquinn.ni@...> wrote:

Show quoted text

Hi Folks,

I am planning on dropping support for Qpid Proton 0.5 and 0.6 in the next OpenMAMA release and instead sticking to versions 0.7+. Those guys move quickly and make small changes to interfaces and header names fairly regularly so keeping up with new releases without breaking older versions is fairly tricky, and something I'd like to at least limit.

Also, they had some compiler warnings in one of their headers when our strict parsing was enabled in 0.5 that wasn't fixed until 0.7 (see https://issues.apache.org/jira/browse/PROTON-420), so dropping support for all versions prior to 0.7 will mean we can turn -Werror back on without breaking older builds.

If anyone has any objections to this, please respond.

Cheers,

Frank

Re: Dropping support for Qpid Proton <= 0.6

Frank Quinn <fquinn.ni@...>

#1563

Hi Folks,

Last call on this - if I hear no objections within a week, I'll be removing 0.5 and 0.6 support from CI and removing the backwards compatibility macros from the code.

Cheers,

Frank

Show quoted text

On Wed, Jul 22, 2015 at 9:28 AM, Frank Quinn <fquinn.ni@...> wrote:

Hi Folks,

I am planning on dropping support for Qpid Proton 0.5 and 0.6 in the next OpenMAMA release and instead sticking to versions 0.7+. Those guys move quickly and make small changes to interfaces and header names fairly regularly so keeping up with new releases without breaking older versions is fairly tricky, and something I'd like to at least limit.

Also, they had some compiler warnings in one of their headers when our strict parsing was enabled in 0.5 that wasn't fixed until 0.7 (see https://issues.apache.org/jira/browse/PROTON-420), so dropping support for all versions prior to 0.7 will mean we can turn -Werror back on without breaking older builds.

If anyone has any objections to this, please respond.

Cheers,
Frank

Re: OpenMAMA to OpenMAMDA Proxy App

Frank Quinn <f.quinn@...>

#1562

Hi Nestor,

Order book instruments are typically prefixed with a lower case ‘b’, for an example, see the bookticker example in http://www.openmama.org/blog/getting-started-part-2

Cheers,

Frank

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Macrux
Sent: 27 July 2015 16:22
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA to OpenMAMDA Proxy App

Hello everybody,

I'm trying to figure this: I want to built an OpenMAMA proxy application, similar to MamaProxy, which listen to a source for data and then publish that data in an topic again, but the difference is that I want to publish that data as OpenMAMDA messages, I mean, as an orderbook, just as the MamdaBookPublisher example, but I haven't been able to build the orderbook from the OpenMAMA data.

Let me to explain my setup:

I run the capture replay as in the example:

capturereplayc -S TEST -m qpid -tport pub -dictionary data/data.dict -f data/openmama_utpcasheuro_capture.5000.10.qpid.mplay -r

Then, I run an application (similar to the MamaProxy listener) to listen to:

-S TEST -s PTBIZJYE0064.EUR.ENXL -tport sub -m qpid

And I have been trying to extract the information to build the book, but I can't find the data because I don't know the fields to extract from or the way to do it, the only thing I've recognized are the two first prices, times and sizes from each side. I'm working with the Java API.

I apologize if it's a silly question, but I'm stuck on this problem.

Thanks in advance,

Néstor.

OpenMAMA to OpenMAMDA Proxy App

Macrux <kmacrux@...>

#1561

Hello everybody,

I'm trying to figure this: I want to built an OpenMAMA proxy application, similar to MamaProxy, which listen to a source for data and then publish that data in an topic again, but the difference is that I want to publish that data as OpenMAMDA messages, I mean, as an orderbook, just as the MamdaBookPublisher example, but I haven't been able to build the orderbook from the OpenMAMA data.

Let me to explain my setup:

I run the capture replay as in the example:

capturereplayc -S TEST -m qpid -tport pub -dictionary data/data.dict -f data/openmama_utpcasheuro_capture.5000.10.qpid.mplay -r

Then, I run an application (similar to the MamaProxy listener) to listen to:

-S TEST -s PTBIZJYE0064.EUR.ENXL -tport sub -m qpid

And I have been trying to extract the information to build the book, but I can't find the data because I don't know the fields to extract from or the way to do it, the only thing I've recognized are the two first prices, times and sizes from each side. I'm working with the Java API.

I apologize if it's a silly question, but I'm stuck on this problem.

Thanks in advance,

Néstor.

[PATCH 14/14] AVIS: Fixed corruption bug in realloc

Frank Quinn <fquinn.ni@...>

#1559

There was a realloc bug in AVIS when serializing / deserializing
that should be fixed with this patch. Also added a getByteBuffer
implementation.

Signed-off-by: Frank Quinn <fquinn.ni@...>
---
mama/c_cpp/src/c/payload/avismsg/avispayload.c | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)

diff --git a/mama/c_cpp/src/c/payload/avismsg/avispayload.c b/mama/c_cpp/src/c/payload/avismsg/avispayload.c
index 20063f5..82aca1a 100644
--- a/mama/c_cpp/src/c/payload/avismsg/avispayload.c
+++ b/mama/c_cpp/src/c/payload/avismsg/avispayload.c
@@ -406,14 +406,16 @@ avismsgPayload_serialize     (const msgPayload    msg,
                 {
                     void*vp=realloc (impl->mBuffer, impl->mBufferLen+200);
                     impl->mBuffer = vp;
-                    buffPos=&impl->mBuffer;
+                    buffPos=impl->mBuffer;
                     buffPos+=currLen;
                     impl->mBufferLen+=200;
                 }
                 *(int8_t *)(buffPos) = 2;   buffPos+=1;     currLen+=1;
                 *(int16_t *)(buffPos) = len; buffPos+=2; currLen+=2;
                 memcpy (buffPos, currField->mName, len); buffPos+=len; currLen+=len;
-                *(int64_t *)(buffPos) = currField->mValue->value.int64; buffPos+=sizeof(int64_t); currLen+=sizeof(int64_t);
+                *(int64_t *)(buffPos) = (int64_t)currField->mValue->value.int64;
+                buffPos+=sizeof(int64_t);
+                currLen+=sizeof(int64_t);
                 break;
             case TYPE_REAL64:
                 len=+ strlen(currField->mName);;
@@ -421,7 +423,7 @@ avismsgPayload_serialize     (const msgPayload    msg,
                 {
                     void*vp=realloc (impl->mBuffer, impl->mBufferLen+200);
                     impl->mBuffer = vp;
-                    buffPos=&impl->mBuffer;
+                    buffPos=impl->mBuffer;
                     buffPos+=currLen;
                     impl->mBufferLen+=200;
                 }
@@ -436,11 +438,11 @@ avismsgPayload_serialize     (const msgPayload    msg,
                 {
                     void*vp=realloc (impl->mBuffer, impl->mBufferLen+200);
                     impl->mBuffer = vp;
-                    buffPos=&impl->mBuffer;
+                    buffPos=impl->mBuffer;
                     buffPos+=currLen;
                     impl->mBufferLen+=200;
                 }
-                *(int8_t *)(buffPos) = 4;   buffPos+=1;     currLen+=1;
+                *(int8_t *)(buffPos) = TYPE_STRING;   buffPos+=1;     currLen+=1;
                 *(int16_t *)(buffPos) = len; buffPos+=2; currLen+=2;
                 memcpy (buffPos, currField->mName, len); buffPos+=len; currLen+=len;
                 *(int16_t *)(buffPos) = strlen(currField->mValue->value.str); buffPos+=2; currLen+=2;
@@ -503,7 +505,7 @@ avismsgPayload_getByteBuffer     (const msgPayload    msg,
     CHECK_NULL(bufferLength);

     *buffer = impl->mAvisMsg;
-
+    *bufferLength = sizeof(impl->mAvisMsg);

     return MAMA_STATUS_OK;
}
@@ -2034,6 +2036,7 @@ avismsgPayloadIter_get          (msgPayloadIter iter,
         return NULL;
     }

+    /* If this is a special meta field, do not consider during iteration */
     if ((strcmp(SUBJECT_FIELD_NAME, avisField(field)->mName) == 0) ||
         (strcmp(INBOX_FIELD_NAME, avisField(field)->mName)== 0))
             return (avismsgPayloadIter_next(iter,field,msg));
--
2.4.3

0014-AVIS-Fixed-corruption-bug-in-realloc.patch

[PATCH 13/14] UNITTEST: Remove presumptuous isTportDisconnected test

Frank Quinn <fquinn.ni@...>

#1558

I don't think it's reasonable to expect a unit test around OpenMAMA
to support this function call as the nature of transport connectivity
will vary across all middlewares. In the instance of Avis, it is
never really in a disconnected state and there are doubtless other
middleware implementations that are not connection oriented at all
and for which this test doesn't really make any sense, so I'm
taking it out.

Signed-off-by: Frank Quinn <fquinn.ni@...>
---
.../c/middleware/middlewareSubscriptionTests.cpp       | 18 ------------------
1 file changed, 18 deletions(-)

diff --git a/mama/c_cpp/src/gunittest/c/middleware/middlewareSubscriptionTests.cpp b/mama/c_cpp/src/gunittest/c/middleware/middlewareSubscriptionTests.cpp
index 6f19c2e..e6eccbf 100644
--- a/mama/c_cpp/src/gunittest/c/middleware/middlewareSubscriptionTests.cpp
+++ b/mama/c_cpp/src/gunittest/c/middleware/middlewareSubscriptionTests.cpp
@@ -424,24 +424,6 @@ TEST_F (MiddlewareSubscriptionTests, getPlatformErrorInvalidSubBridge)
                status);
}

-TEST_F (MiddlewareSubscriptionTests, isTportDisconnected)
-{
-    int res = NULL;
-    ASSERT_EQ(MAMA_STATUS_OK,
-              mamaSubscription_create(parent, queue, &callbacks, source, sourceName, closure));
-
-    ASSERT_EQ(MAMA_STATUS_OK,
-              mBridge->bridgeMamaSubscriptionCreate(&subscriber, sourceName, symbol,
-                                                    tport, queue, callbacks,
-                                                    parent, closure));
-
-    res=mBridge->bridgeMamaSubscriptionIsTportDisconnected(subscriber);
-    ASSERT_TRUE(res != NULL);
-
-    ASSERT_EQ(MAMA_STATUS_OK,
-              mBridge->bridgeMamaSubscriptionDestroy(subscriber));
-}
-
TEST_F (MiddlewareSubscriptionTests, isTportDisconnectedInvalid)
{
     ASSERT_EQ (MAMA_STATUS_NULL_ARG,
--
2.4.3

0013-UNITTEST-Remove-presumptuous-isTportDisconnected-tes.patch

[PATCH 12/14] UNITTEST: Removed duplicateReplyHandle test

Frank Quinn <fquinn.ni@...>

#1557

This test doesn't really make sense without the use of an inbox
so it serves no useful function at the moment, therefore I'm taking
it out.

Signed-off-by: Frank Quinn <fquinn.ni@...>
---
.../src/gunittest/c/middleware/middlewareMsgTests.cpp | 19 -------------------
1 file changed, 19 deletions(-)

diff --git a/mama/c_cpp/src/gunittest/c/middleware/middlewareMsgTests.cpp b/mama/c_cpp/src/gunittest/c/middleware/middlewareMsgTests.cpp
index fc12c55..8630bb1 100644
--- a/mama/c_cpp/src/gunittest/c/middleware/middlewareMsgTests.cpp
+++ b/mama/c_cpp/src/gunittest/c/middleware/middlewareMsgTests.cpp
@@ -290,25 +290,6 @@ TEST_F (MiddlewareMsgTests, getNativeHandleInvalidResult)
                mBridge->bridgeMamaMsgGetNativeHandle(msg,NULL));
}

-TEST_F (MiddlewareMsgTests, duplicateReplyHandle)
-{
-    msgBridge msg        = NULL;
-    mamaMsg   parent     = NULL;
-    void*     result     = NULL;
-    int       destroyMsg = 1;
-
-    ASSERT_EQ(MAMA_STATUS_OK,mamaMsg_create(&parent));
-
-    ASSERT_EQ(MAMA_STATUS_OK,
-              mBridge->bridgeMamaMsgCreate(&msg,parent));
-
-    ASSERT_EQ(MAMA_STATUS_OK,
-              mBridge->bridgeMamaMsgDuplicateReplyHandle(msg,&result));
-
-    ASSERT_EQ(MAMA_STATUS_OK,
-              mBridge->bridgeMamaMsgDestroy(msg,destroyMsg));
-}
-
TEST_F (MiddlewareMsgTests, duplicateReplyHandleInvalidMsgBridge)
{
     void* result = NOT_NULL;
--
2.4.3

0012-UNITTEST-Removed-duplicateReplyHandle-test.patch