Date   

Re: OpenMAMA-2.3.2-rc1

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

Gary,

 

Is it possible to include the fix for the following issues in the 2.3.2? There is a patch available for each of these.

 

Bug 176 - MAMAC: Missing actions for snapshot subscriptions transition to deactivate state

Bug 169 - Wombat queue has no separate deallocate method

Bug 166 - Wombat: wInterlocked_set inconsistent return value

 

Thanks.

 

--Alireza

 

From: Gary Molloy [mailto:g.molloy@...]
Sent: Tuesday, January 06, 2015 10:02 AM
To: Alireza Assadzadeh
Cc: openmama-dev@...
Subject: RE: OpenMAMA-2.3.2-rc1

 

Hi Alireza,

 

Yes, it will include the current contents of the next branch.

 

Thanks,

Gary

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 

From: Alireza Assadzadeh [mailto:Alireza.Assadzadeh@...]
Sent: 06 January 2015 14:10
To: Gary Molloy; openmama-dev@...
Subject: RE: OpenMAMA-2.3.2-rc1

 

Hi Gary,

 

Is the current content of ‘next’ branch what you had in mind for the 2.3.2 release?

 

Regards,

 

--Alireza

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Gary Molloy
Sent: Tuesday, January 06, 2015 8:57 AM
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA-2.3.2-rc1

 

Hi Guys,

 

I will be cutting RC1 for the next release, 2.3.2, this afternoon.

 

This will contain a handful of minor issues, unit tests updates, enterprise issues etc…

 

If there are any issues in particular you would like to see included in the next release, please feel free to reply and we can review any issue for inclusion.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


Re: OpenMAMA-2.3.2-rc1

Gary Molloy <g.molloy@...>
 

Hi Guys,

 

I have cut the new OpenMAMA-2.3.2 branch and created the OpenMAMA-2.3.2-rc1 tag, and this is currently available for testing.

 

The RPM’s will be delayed unfortunately.  It seems that the Fedora repository has updated the default version of proton, to 0.8, and this doesn’t contain the “proton/util.h” header file anymore.  This is causing the mock RPM’s to fail as we look for this include file in a few places.  I don’t believe that we actually use any of the functions that were in the util.h header, but I will investigate this further, correct it and get the RPMs out as soon as possible.

 

Thanks,

Gary

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 

From: Gary Molloy
Sent: 06 January 2015 13:57
To: openmama-dev@...
Subject: OpenMAMA-2.3.2-rc1

 

Hi Guys,

 

I will be cutting RC1 for the next release, 2.3.2, this afternoon.

 

This will contain a handful of minor issues, unit tests updates, enterprise issues etc…

 

If there are any issues in particular you would like to see included in the next release, please feel free to reply and we can review any issue for inclusion.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


Re: OpenMAMA-2.3.2-rc1

Gary Molloy <g.molloy@...>
 

Hi Alireza,

 

Yes, it will include the current contents of the next branch.

 

Thanks,

Gary

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 

From: Alireza Assadzadeh [mailto:Alireza.Assadzadeh@...]
Sent: 06 January 2015 14:10
To: Gary Molloy; openmama-dev@...
Subject: RE: OpenMAMA-2.3.2-rc1

 

Hi Gary,

 

Is the current content of ‘next’ branch what you had in mind for the 2.3.2 release?

 

Regards,

 

--Alireza

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Gary Molloy
Sent: Tuesday, January 06, 2015 8:57 AM
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA-2.3.2-rc1

 

Hi Guys,

 

I will be cutting RC1 for the next release, 2.3.2, this afternoon.

 

This will contain a handful of minor issues, unit tests updates, enterprise issues etc…

 

If there are any issues in particular you would like to see included in the next release, please feel free to reply and we can review any issue for inclusion.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


Re: OpenMAMA-2.3.2-rc1

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

Hi Gary,

 

Is the current content of ‘next’ branch what you had in mind for the 2.3.2 release?

 

Regards,

 

--Alireza

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Gary Molloy
Sent: Tuesday, January 06, 2015 8:57 AM
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA-2.3.2-rc1

 

Hi Guys,

 

I will be cutting RC1 for the next release, 2.3.2, this afternoon.

 

This will contain a handful of minor issues, unit tests updates, enterprise issues etc…

 

If there are any issues in particular you would like to see included in the next release, please feel free to reply and we can review any issue for inclusion.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


OpenMAMA-2.3.2-rc1

Gary Molloy <g.molloy@...>
 

Hi Guys,

 

I will be cutting RC1 for the next release, 2.3.2, this afternoon.

 

This will contain a handful of minor issues, unit tests updates, enterprise issues etc…

 

If there are any issues in particular you would like to see included in the next release, please feel free to reply and we can review any issue for inclusion.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


[PATCH 2.3.1 1/1] TESTTOOLS: capturereplayc does not need to send reply when the received mama subscription request is not from inbox.

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

If subscription message is not from inbox, then there is no need to
publish the initial image when processing onNewRequest or onRequest.
For example, in the case of a real time market data subscription that
has disabled the receipt of initial image.

Test Plan:
- Checked with capturereplayc and mamalistenc.
- Used qpid middleware bridge for regression testing.
- Used solace middleware bridge for the case of real time market
data subscription with receipt of initial image disabled.

Signed-off-by: Alireza Assadzadeh <Alireza.Assadzadeh@...>
---

The patches are against 'next' branch. They are tested by using sample and
testtool programs: capturereplayc and mamalistenc.

.../src/testtools/capturereplay/c/capturereplayc.c | 26 +++++++++++++++++-----
1 file changed, 21 insertions(+), 5 deletions(-)

diff --git a/mama/c_cpp/src/testtools/capturereplay/c/capturereplayc.c b/mama/c_cpp/src/testtools/capturereplay/c/capturereplayc.c
index e4b1ed0..c420384 100644
--- a/mama/c_cpp/src/testtools/capturereplay/c/capturereplayc.c
+++ b/mama/c_cpp/src/testtools/capturereplay/c/capturereplayc.c
@@ -478,8 +478,16 @@ subscriptionHandlerOnNewRequestCb (mamaDQPublisherManager manager,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_INITIAL);
}
- mamaDQPublisher_sendReply (gSubscriptionList[index].pub, msg,
- gSubscriptionList[index].cachedMsg);
+ /* If subscription message is from inbox, then send reply to inbox.
+ * Otherwise, there is no need to publish the initial image. For
+ * example, in the case of a real time market data subscription that
+ * has disabled the receipt of initial image.
+ */
+ if (mamaMsg_isFromInbox (msg))
+ {
+ mamaDQPublisher_sendReply (gSubscriptionList[index].pub, msg,
+ gSubscriptionList[index].cachedMsg);
+ }
break;
default:
mama_log (MAMA_LOG_LEVEL_NORMAL, "Publishing MAMA_MSG_TYPE_RECAP");
@@ -529,9 +537,17 @@ subscriptionHandlerOnRequestCb (mamaDQPublisherManager manager,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_INITIAL);
}
- mamaDQPublisher_sendReply (gSubscriptionList[index].pub,
- msg,
- gSubscriptionList[index].cachedMsg);
+ /* If subscription message is from inbox, then send reply to inbox.
+ * Otherwise, there is no need to publish the initial image. For
+ * example, in the case of a real time market data subscription that
+ * has disabled the receipt of initial image.
+ */
+ if (mamaMsg_isFromInbox (msg))
+ {
+ mamaDQPublisher_sendReply (gSubscriptionList[index].pub,
+ msg,
+ gSubscriptionList[index].cachedMsg);
+ }
break;
case MAMA_SUBSC_DQ_SUBSCRIBER:
case MAMA_SUBSC_DQ_PUBLISHER:
--
1.9.3


[PATCH 2.3.1 1/1] MAMAC: Missing actions for snapshot subscriptions transition to deactivate state

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

For snapshot subscriptions, there is no bridge subscription callback
to be called later with mamaSubscriptionImpl_onSubscriptionDestroyed.
Therefore, add code to perform the additional actions for deactivated
state in mamaSubscription_deactivate. Namely, add code to decrement
the queue object count.

Test Plan:
- Check with capturereplayc and mamalistenc using -shutdown option.
Use mama.properties: mama.queue.object_lock_tracking=1 and
mama.logging.level=finest.
- Build and run unit tests (UnitTestCommon*, UnitTestsMama*).

Signed-off-by: Alireza Assadzadeh <Alireza.Assadzadeh@...>
---

I have raised bug http://bugs.openmama.org/show_bug.cgi?id=176 to track
this issue and attached the patches to the bug.

The patches are against 'next' branch. They are tested by using gunittests,
and sample programs (capturereplayc, mamalistenc and bookticker).

Before this fix the problem is observable when calling mama_close after
earlier having created and destroyed a snapshot subscription (either
dictionary snapshot normal/book snapshot subscription). The mamaQueue_destory
for default queue times out since there are left over open objects in the queue
due to the missing call to mamaQueue_decrementObjectCount.
For example, using the mamalistenc with -shutdown option with qpid, the mama_close
takes 5 seconds to complete with the following finest log "Entering
mamaQueue_destroy for queue 0x<pointer>" repeated every 10ms (500 times) and
finally timing out with log "mamaQueue_destroy(): timed out destroying queue.".

mama/c_cpp/src/c/subscription.c | 12 ++++++++++++
1 file changed, 12 insertions(+)

diff --git a/mama/c_cpp/src/c/subscription.c b/mama/c_cpp/src/c/subscription.c
index c809257..1f78746 100644
--- a/mama/c_cpp/src/c/subscription.c
+++ b/mama/c_cpp/src/c/subscription.c
@@ -2616,7 +2616,19 @@ mama_status mamaSubscription_deactivate(mamaSubscription subscription)
ret = mamaSubscription_deactivate_internal(impl);
if (impl->mSubscMsgType == MAMA_SUBSC_DDICT_SNAPSHOT ||
impl->mSubscMsgType == MAMA_SUBSC_SNAPSHOT)
+ {
+ /* Snapshot subscriptions don't have a bridge and are transitioned to deactivated
+ * immediately after deactivating.
+ * Also, since there is no bridge subscription callback to be called later with
+ * mamaSubscriptionImpl_onSubscriptionDestroyed, the necessary actions for deactivated
+ * state are performed here. Namely the queue object count is decremented.
+ */
+ if(NULL != impl->mQueue)
+ {
+ mamaQueue_decrementObjectCount(&impl->mLockHandle, impl->mQueue);
+ }
mamaSubscriptionImpl_setState(impl, MAMA_SUBSCRIPTION_DEACTIVATED);
+ }
break;

case MAMA_SUBSCRIPTION_DEACTIVATING:
--
1.9.3


Re: OpenMAMA and Enterprise Mama

Lee Skillen <lskillen@...>
 

Quick addendum ...

If you can't access Google Docs (seems to be banned everywhere), I've
also prepared a "document pack" zip-file with all of the
aforementioned docs. Available at the following URI :-

http://static.vulcanft.com/prod/files/openmama_librarymanager.zip

On 15 December 2014 at 22:19, Lee Skillen <lskillen@...> wrote:
Hi Reed,

I'm not sure about the current ongoings/plans with OpenMAMA or
EnterpriseMAMA but VFT released a draft for an extensible library
manager earlier this year.

We also have an unreleased draft of an entitlement manager based upon
the draft library manager work which is incomplete but in a usable
state.

The GitHub branch and some documentation for the library manager :-

1. Library Manager Branch :- feature-librarymanager
2. First Draft (0.1) :-
https://docs.google.com/document/d/1sg2i84NnhJO90z8DECYeGf799wDUzY4B_uXNsCpFuac/edit?usp=sharing
3. Second Draft (0.2) :-
https://docs.google.com/document/d/1ZLZjpiu3A7MPNauOQWLFKVxnoR6jdAp3Vas91PqWhnY/edit?usp=sharing
4. Developer Docs (Incomplete) :-
https://docs.google.com/document/d/10Y9ebCeCDPeBrVgaqE4MrcOc6zgittJGVchMSO2lKrs/edit?usp=sharing

The GitHub branch and some documentation for the entitlement manager :-

1. Entitlement Bridge Branch :- feature-entitlementbridge
2. Brief Overview :-
https://docs.google.com/document/d/1Nw_TetxADez5PR1COaOBhbkQhQcdc0BgH7HEnfQyBjI/edit?usp=sharing

It doesn't support it atm but it wouldn't be difficult to extend the
entitlement bridge code to apply different bridges to different
transports.

Unfortunately the projects have stalled for the moment but we'd be
happy to pick it back up again if some momentum is gained.

We'd love to hear any feedback about any aspects of the proposed code
- We'd also accept patches, pull requests and issues.

If you have any problems viewing any of the documents I could also
render them as PDF and send them out separately.

Cheers,
Lee

On 15 December 2014 at 21:21, Alpert, Reed <reed.alpert@...> wrote:
Hi,



What is the plan for OM and EM?

The entitlements in EM are not available in OM, and it looks like turning on
entitlements in mama layer is for all of the loaded bridges.

Are patches to OM made available in EM (and vice-versa) – how long does the
dev cycle take for that?



Our concern is that we run 3 bridges : wombat, tick42/rmds, and solace.

For entitlements we use dart for wombat (our DF5/6/mamacache env), and dacs
for tick42/rmds and solace.

We’d like to run OM for all of this (or EM if that is the best way to go).



Thanks,



Reed.



________________________________

Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P.
Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T:
718.242.5198 | M: 917.414.4613 | reed.alpert@...



Alternate Contact: CIB PIM Trading Technology Solutions NA |
CIB_PIM_Trading_Technology_Solutions_NA@...



This communication is for informational purposes only. It is not intended as
an offer or solicitation for the purchase or sale of any financial
instrument or as an official confirmation of any transaction. All market
prices, data and other information are not warranted as to completeness or
accuracy and are subject to change without notice. Any comments or
statements made herein do not necessarily reflect those of JPMorgan Chase &
Co., its subsidiaries and affiliates. This transmission may contain
information that is proprietary, privileged, confidential and/or exempt from
disclosure under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or use of
the information contained herein (including any reliance thereon) is
STRICTLY PROHIBITED. Although this transmission and any attachments are
believed to be free of any virus or other defect that might affect any
computer system into which it is received and opened, it is the
responsibility of the recipient to ensure that it is virus free and no
responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and
affiliates, as applicable, for any loss or damage arising in any way from
its use. If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety, whether in
electronic or hard copy format. Thank you. Please refer to
http://www.jpmorgan.com/pages/disclosures for disclosures relating to
European legal entities.


_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


--
Lee Skillen

Vulcan Financial Technologies
1st Floor, 47 Malone Road, Belfast, BT9 6RY

Office: +44 (0)28 95 817888
Web: www.vulcanft.com
--
Lee Skillen

Vulcan Financial Technologies
1st Floor, 47 Malone Road, Belfast, BT9 6RY

Office: +44 (0)28 95 817888
Web: www.vulcanft.com


Re: OpenMAMA and Enterprise Mama

Lee Skillen <lskillen@...>
 

Hi Reed,

I'm not sure about the current ongoings/plans with OpenMAMA or
EnterpriseMAMA but VFT released a draft for an extensible library
manager earlier this year.

We also have an unreleased draft of an entitlement manager based upon
the draft library manager work which is incomplete but in a usable
state.

The GitHub branch and some documentation for the library manager :-

1. Library Manager Branch :- feature-librarymanager
2. First Draft (0.1) :-
https://docs.google.com/document/d/1sg2i84NnhJO90z8DECYeGf799wDUzY4B_uXNsCpFuac/edit?usp=sharing
3. Second Draft (0.2) :-
https://docs.google.com/document/d/1ZLZjpiu3A7MPNauOQWLFKVxnoR6jdAp3Vas91PqWhnY/edit?usp=sharing
4. Developer Docs (Incomplete) :-
https://docs.google.com/document/d/10Y9ebCeCDPeBrVgaqE4MrcOc6zgittJGVchMSO2lKrs/edit?usp=sharing

The GitHub branch and some documentation for the entitlement manager :-

1. Entitlement Bridge Branch :- feature-entitlementbridge
2. Brief Overview :-
https://docs.google.com/document/d/1Nw_TetxADez5PR1COaOBhbkQhQcdc0BgH7HEnfQyBjI/edit?usp=sharing

It doesn't support it atm but it wouldn't be difficult to extend the
entitlement bridge code to apply different bridges to different
transports.

Unfortunately the projects have stalled for the moment but we'd be
happy to pick it back up again if some momentum is gained.

We'd love to hear any feedback about any aspects of the proposed code
- We'd also accept patches, pull requests and issues.

If you have any problems viewing any of the documents I could also
render them as PDF and send them out separately.

Cheers,
Lee

On 15 December 2014 at 21:21, Alpert, Reed <reed.alpert@...> wrote:
Hi,



What is the plan for OM and EM?

The entitlements in EM are not available in OM, and it looks like turning on
entitlements in mama layer is for all of the loaded bridges.

Are patches to OM made available in EM (and vice-versa) – how long does the
dev cycle take for that?



Our concern is that we run 3 bridges : wombat, tick42/rmds, and solace.

For entitlements we use dart for wombat (our DF5/6/mamacache env), and dacs
for tick42/rmds and solace.

We’d like to run OM for all of this (or EM if that is the best way to go).



Thanks,



Reed.



________________________________

Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P.
Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T:
718.242.5198 | M: 917.414.4613 | reed.alpert@...



Alternate Contact: CIB PIM Trading Technology Solutions NA |
CIB_PIM_Trading_Technology_Solutions_NA@...



This communication is for informational purposes only. It is not intended as
an offer or solicitation for the purchase or sale of any financial
instrument or as an official confirmation of any transaction. All market
prices, data and other information are not warranted as to completeness or
accuracy and are subject to change without notice. Any comments or
statements made herein do not necessarily reflect those of JPMorgan Chase &
Co., its subsidiaries and affiliates. This transmission may contain
information that is proprietary, privileged, confidential and/or exempt from
disclosure under applicable law. If you are not the intended recipient, you
are hereby notified that any disclosure, copying, distribution, or use of
the information contained herein (including any reliance thereon) is
STRICTLY PROHIBITED. Although this transmission and any attachments are
believed to be free of any virus or other defect that might affect any
computer system into which it is received and opened, it is the
responsibility of the recipient to ensure that it is virus free and no
responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and
affiliates, as applicable, for any loss or damage arising in any way from
its use. If you received this transmission in error, please immediately
contact the sender and destroy the material in its entirety, whether in
electronic or hard copy format. Thank you. Please refer to
http://www.jpmorgan.com/pages/disclosures for disclosures relating to
European legal entities.


_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
--
Lee Skillen

Vulcan Financial Technologies
1st Floor, 47 Malone Road, Belfast, BT9 6RY

Office: +44 (0)28 95 817888
Web: www.vulcanft.com


OpenMAMA and Enterprise Mama

Alpert, Reed <reed.alpert@...>
 

Hi,

 

What is the plan for OM and EM?

The entitlements in EM are not available in OM, and it looks like turning on entitlements in mama layer is for all of the loaded bridges.

Are patches to OM made available in EM (and vice-versa) – how long does the dev cycle take for that?

 

Our concern is that we run 3 bridges : wombat, tick42/rmds, and solace.

For entitlements we use dart for wombat (our DF5/6/mamacache env), and dacs for tick42/rmds and solace.

We’d like to run OM for all of this (or EM if that is the best way to go).

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

 

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Re: Error handling in market data subscription activation and error notification to application

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

Hi Gary,

 

Thanks a lot for your response on this. Your suggestions sound good.

 

Please see my comments to your mail inline marked with [AA:].

 

--Alireza

 

From: Gary Molloy [mailto:g.molloy@...]
Sent: Friday, December 12, 2014 12:50 PM
To: Alireza Assadzadeh; openmama-dev@...
Subject: RE: [Openmama-dev] Error handling in market data subscription activation and error notification to application

 

Hi Alireza,

 

Thanks for your email.

 

I think that you have touched upon 2 issues here:

 

1.       Synchronous Fails – if the create fails within the mamaSubscription_initialize() function it should be handled at this point.  We should probably add in a return code and check what is being passed back from the call to self->mBridgeImpl->bridgeMamaSubscriptionCreate() where we could then invoked the onError() callback and stop the creation of the subscription continuing (and prevent onCreate() from being invoked).  Some things that may need consideration:

a.       Return codes – would the common set of MAMA return codes cover this or would they require expanding?

 

[AA:] For the MW Bridge error conditions, I think the MAMA_STATUS_PLATFORM_ERROR in conjunction with mamaSubscription_getPlatformError should be fine. I think we need to check any additional error cases that are not from the MW-Bridge and see if the existing codes properly cover the failure case. Overall, there may be value in providing more specific or additional mama_status codes.

 

b.      Threading - this will cause the onError() to be invoked from the default thread as opposed to the subscription thread.  However this may not be a problem as the subscription will not be active…

 

[AA:] This seems ok to me, since also the onCreate() was called from the same thread and also subscription is not active (as you pointed out).

 

2.       Asynchronous Fails – I agree with you that mamaSubscription_processErr() is the correct function to use here.  We have used this previously for in-house bridges for the same scenario.  There is a transport function, mamaTransport_getDeactivateSubscriptionOnError(), that can be used to determine whether or not to deactivate the subscription on an error.  A few additional things to consider also:

a.       These events must be fired from the correct queue by the middleware bridge.

b.      There is an issue with the mamaSubscription_processErr() function in that it always sends out a timeout with status ok.  Again may need to look at the available return codes?

 

[AA:] Agreed, I noticed that issue too with mamaSubscription_processErr(). I think the status code and platform error need to be changed to provide other errors that may occur.

 

Would you agree with this take on your proposed solution(s)?

 

[AA:] Yes. that sounds good to me.

 

Specially for the synchronous fail, I believe it is a better approach for the Mama layer to examine error conditions and call onError versus onCreate. This helps in providing a common error notification for the sync case to the application independent of the various MW-Bridge implementations.

 

Should there be anything else to consider?

 

[AA:] Other items that come to mind

a)      I am wondering about other cases that constitute a subscription activation failure. We have covered the MW Bridge bridgeMamaSubscription_create case here. There is the case of the subscription request for initial (if requires initial is set) failing which may fail in the MW-Bridge mamaPublisher_sendFromInboxByIndex. So, we need to examine other failure cases and determine for each case if it should result in the subscription activation failure and invoking onError.

b)      Backward compatibility for changing the synchronous failure notification mechanism also needs some thought. For synchronous failure, since OpenMAMA 2.3.1 does not provide the error notification of MW-Bridge errors, should the MW-Bridge provide the error notification to the application itself to work around this? If so, then when OpenMAMA new version provides the notification to the application, then the MW-Bridge should not anymore (to avoid having the onError() callback called twice).

 

Thanks,

Gary

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

Tel: +44 28 9099 7580  Ext 3397

g.molloy@...

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Alireza Assadzadeh
Sent: 09 December 2014 19:21
To: openmama-dev@...
Subject: [Openmama-dev] Error handling in market data subscription activation and error notification to application

 

Hi Folks,

 

I have noticed that in Mama C subscription, if a market data subscription activation callback (i.e. createAction) fails the error is not propagated in Mama subscription layer to the application through the onError user callback for the subscription. For example, a throttled market data subscription may fail to activate due to Middleware Bridge create subscription failure.

 

Specifically for the Middleware Bridge failure case, looking at mamaSubscriptionImpl_completeMarketDataInitialisation, the call to mamaSubscription_initialize may have failed for example in birdgeMamaSubscriptionCreate call. The return code is not examined in the Mama Subscription and the subscription is always transitioned to activated state and the onCreate user callback is called. The onCreate user callback is called because the subscription creation (activation) is completed from the creation throttle.

 

This seems to be the design intent and there is no expectation that the Mama subscription would be calling the onError user callback in such failure cases. Can you please confirm and/or let me know your opinion on this?

 

Assuming this is the design intent, I think in the Middleware Bridge create subscription failure case, it follows that it is the responsibility of Middleware Bridge to notify the Mama C layer and the application about such errors asynchronously (beyond the function return status code for birdgeMamaSubscriptionCreate). The Middleware Bridge can provide the error notification through a subscription queue event error callback which will then perform the Mama subscription deactivation (optionally) and the call to the user onError callback. In other words, the Middleware Bridge may enqueue an event for mamaSubscription_processErr or a similar callback of its own to deactivate the Mama subscription (optionally) and call the user onError callback.

 

Note that there are cases where the Middleware Bridge may determine immediately in the bridge subscription create call that it is going to fail the subscription create call. There are also other cases where the Middleware Bridge (depending on its capabilities) may determine at later point in time, in an asynchronously fashion, that the Middleware Bridge subscription creation  process encountered an error subsequently. So, for such cases, the Middleware Bridge may already have a mechanism in place to deactivate the Mama subscription and call the user onError callback.

 

Regards,

 

--Alireza


Re: Error handling in market data subscription activation and error notification to application

Gary Molloy <g.molloy@...>
 

Hi Alireza,

 

Thanks for your email.

 

I think that you have touched upon 2 issues here:

 

1.       Synchronous Fails – if the create fails within the mamaSubscription_initialize() function it should be handled at this point.  We should probably add in a return code and check what is being passed back from the call to self->mBridgeImpl->bridgeMamaSubscriptionCreate() where we could then invoked the onError() callback and stop the creation of the subscription continuing (and prevent onCreate() from being invoked).  Some things that may need consideration:

a.       Return codes – would the common set of MAMA return codes cover this or would they require expanding?

b.      Threading - this will cause the onError() to be invoked from the default thread as opposed to the subscription thread.  However this may not be a problem as the subscription will not be active…

 

2.       Asynchronous Fails – I agree with you that mamaSubscription_processErr() is the correct function to use here.  We have used this previously for in-house bridges for the same scenario.  There is a transport function, mamaTransport_getDeactivateSubscriptionOnError(), that can be used to determine whether or not to deactivate the subscription on an error.  A few additional things to consider also:

a.       These events must be fired from the correct queue by the middleware bridge.

b.      There is an issue with the mamaSubscription_processErr() function in that it always sends out a timeout with status ok.  Again may need to look at the available return codes?

 

Would you agree with this take on your proposed solution(s)?

 

Should there be anything else to consider?

 

Thanks,

Gary

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

Tel: +44 28 9099 7580  Ext 3397

g.molloy@...

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Alireza Assadzadeh
Sent: 09 December 2014 19:21
To: openmama-dev@...
Subject: [Openmama-dev] Error handling in market data subscription activation and error notification to application

 

Hi Folks,

 

I have noticed that in Mama C subscription, if a market data subscription activation callback (i.e. createAction) fails the error is not propagated in Mama subscription layer to the application through the onError user callback for the subscription. For example, a throttled market data subscription may fail to activate due to Middleware Bridge create subscription failure.

 

Specifically for the Middleware Bridge failure case, looking at mamaSubscriptionImpl_completeMarketDataInitialisation, the call to mamaSubscription_initialize may have failed for example in birdgeMamaSubscriptionCreate call. The return code is not examined in the Mama Subscription and the subscription is always transitioned to activated state and the onCreate user callback is called. The onCreate user callback is called because the subscription creation (activation) is completed from the creation throttle.

 

This seems to be the design intent and there is no expectation that the Mama subscription would be calling the onError user callback in such failure cases. Can you please confirm and/or let me know your opinion on this?

 

Assuming this is the design intent, I think in the Middleware Bridge create subscription failure case, it follows that it is the responsibility of Middleware Bridge to notify the Mama C layer and the application about such errors asynchronously (beyond the function return status code for birdgeMamaSubscriptionCreate). The Middleware Bridge can provide the error notification through a subscription queue event error callback which will then perform the Mama subscription deactivation (optionally) and the call to the user onError callback. In other words, the Middleware Bridge may enqueue an event for mamaSubscription_processErr or a similar callback of its own to deactivate the Mama subscription (optionally) and call the user onError callback.

 

Note that there are cases where the Middleware Bridge may determine immediately in the bridge subscription create call that it is going to fail the subscription create call. There are also other cases where the Middleware Bridge (depending on its capabilities) may determine at later point in time, in an asynchronously fashion, that the Middleware Bridge subscription creation  process encountered an error subsequently. So, for such cases, the Middleware Bridge may already have a mechanism in place to deactivate the Mama subscription and call the user onError callback.

 

Regards,

 

--Alireza


Re: OpenMAMA RPM Release Scripts

Gary Molloy <g.molloy@...>
 

Hi Reed,

 

Thanks for your feedback, we appreciate your input with this.

 

The name of the directory was just arbitrarily chosen and we can certainly change it to be something else.

 

Thanks,

Gary

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

Tel: +44 28 9099 7580  Ext 3397

g.molloy@...

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Alpert, Reed
Sent: 18 November 2014 16:55
To: Damian Maguire; openmama-dev@...
Subject: Re: [Openmama-dev] OpenMAMA RPM Release Scripts

 

Hi Damian,

 

Just a note: the release subdir is also used by Vlsual Studio for its output, so there will be the rpm scripts and lib/dll/pdb files from the builds.

Our Windows git looks like this after running both Visual Studio and scons builds:

 

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

 

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Damian Maguire
Sent: Wednesday, November 12, 2014 9:31 AM
To: openmama-dev@...
Subject: [Openmama-dev] OpenMAMA RPM Release Scripts

 

Folks,

 

Some of you may have noticed a few commits which have just landed in next which relate to the OpenMAMA RPM builds. These are tools which I've been using internally to help facilitate the OpenMAMA release process by automating some of the steps involved. At present they are pretty rough around the edges, however after talking to a few people about them I believe there is some value in having them out there for others to review/enhance. 

 

The primary one of interest to most people is likely to be the openmama.spec file, which resides under the 'release' directory. This is the specification file used by rpmbuild to generate the base RPM and SRPMs, which can then used to generate other RPMs for alternative platforms. 

 

If you clone down next (and assuming you have met the base requirements - running RHEL6 or derivative, have all the OpenMAMA pre-reqs, have rpmbuild and mock installed etc), you should be able to perform a full build yourself by simply entering the release directory and executing:

 

    $ ./openmama-rpm.sh

 

As I say, it's all a little rough, and still quite heavily tailored to my own build environment, but I would appreciate any feedback/bug fixes/issues etc.

 

Cheers, 

 

D

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Error handling in market data subscription activation and error notification to application

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

Hi Folks,

 

I have noticed that in Mama C subscription, if a market data subscription activation callback (i.e. createAction) fails the error is not propagated in Mama subscription layer to the application through the onError user callback for the subscription. For example, a throttled market data subscription may fail to activate due to Middleware Bridge create subscription failure.

 

Specifically for the Middleware Bridge failure case, looking at mamaSubscriptionImpl_completeMarketDataInitialisation, the call to mamaSubscription_initialize may have failed for example in birdgeMamaSubscriptionCreate call. The return code is not examined in the Mama Subscription and the subscription is always transitioned to activated state and the onCreate user callback is called. The onCreate user callback is called because the subscription creation (activation) is completed from the creation throttle.

 

This seems to be the design intent and there is no expectation that the Mama subscription would be calling the onError user callback in such failure cases. Can you please confirm and/or let me know your opinion on this?

 

Assuming this is the design intent, I think in the Middleware Bridge create subscription failure case, it follows that it is the responsibility of Middleware Bridge to notify the Mama C layer and the application about such errors asynchronously (beyond the function return status code for birdgeMamaSubscriptionCreate). The Middleware Bridge can provide the error notification through a subscription queue event error callback which will then perform the Mama subscription deactivation (optionally) and the call to the user onError callback. In other words, the Middleware Bridge may enqueue an event for mamaSubscription_processErr or a similar callback of its own to deactivate the Mama subscription (optionally) and call the user onError callback.

 

Note that there are cases where the Middleware Bridge may determine immediately in the bridge subscription create call that it is going to fail the subscription create call. There are also other cases where the Middleware Bridge (depending on its capabilities) may determine at later point in time, in an asynchronously fashion, that the Middleware Bridge subscription creation  process encountered an error subsequently. So, for such cases, the Middleware Bridge may already have a mechanism in place to deactivate the Mama subscription and call the user onError callback.

 

Regards,

 

--Alireza


Re: Asynchronous publisher events

Gary Molloy <g.molloy@...>
 

Hi Guys,

Thanks for the feedback on the proposed dates.

From the feedback I think the best date that suits most would be Tuesday 16th December @ 4pm UTC, so let's schedule the call for then.

The connection details once again are:

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664

Looking forward to the call.

Thanks,
Gary


Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@...

-----Original Message-----
From: Gary Molloy
Sent: 08 December 2014 11:16
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@...
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@...

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@...
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@...



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@...
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@...> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@...> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@...
[mailto:openmama-dev-bounces@...] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@...
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Asynchronous publisher events

Alpert, Reed <reed.alpert@...>
 

16th is better for me.

Thanks,

Reed.


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Gary Molloy
Sent: Monday, December 08, 2014 6:16 AM
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@...
Subject: Re: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@...

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@...
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@...



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@...
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@...> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@...> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@...
[mailto:openmama-dev-bounces@...] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@...
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.

This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Re: Asynchronous publisher events

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

Either one works for me. Looking forward to the meeting.

--Alireza

-----Original Message-----
From: openmama-dev-bounces@... [mailto:openmama-
dev-bounces@...] On Behalf Of Gary Molloy
Sent: Monday, December 08, 2014 6:16 AM
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-
dev@...
Subject: Re: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the
discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of
December @ 4pm UTC. I would appreciate it if you could let us know which
date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@...

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-
dev@...
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher
event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-
y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so
we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions
g.molloy@...



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@...
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have
been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher
is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional
and extensible. One gotcha though is the question of which thread/queue
the callback should be fired on? Currently the mamaPublisher doesn’t have
a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events
and put together a proposal/appraisal, then we can figure out who wants to
implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@...>
wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox
messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is
currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though
we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@...> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message
structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1,
MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others.
Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@...
[mailto:openmama-dev-bounces@...] On Behalf Of
Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@...
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns
immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT
or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error.
Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Asynchronous publisher events

Tom Doust
 

Hi.

Either is OK by me at the moment

Tom

-----Original Message-----
From: Gary Molloy [mailto:g.molloy@...]
Sent: 08 December 2014 11:16 AM
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@...
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@...

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@...
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@...



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@...
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@...> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@...> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@...
[mailto:openmama-dev-bounces@...] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@...
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Asynchronous publisher events

Gary Molloy <g.molloy@...>
 

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@...

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@...
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@...



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@...
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@...> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@...> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@...
[mailto:openmama-dev-bounces@...] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@...
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Asynchronous publisher events

Gary Molloy <g.molloy@...>
 

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions
g.molloy@...

-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@...
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@...> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@...> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@...
[mailto:openmama-dev-bounces@...] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@...
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

901 - 920 of 2312