Asynchronous publisher events


Sam Wilson <Sam.Wilson@...>
 

Hey all,

We're looking for a way to notify a mama application asynchronously of
errors while publishing, specifically when the published message was
rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the connection,
giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since the
application or the environment is incorrectly configured, there's not
much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support these
kind of events, what would such extensions look like? Perhaps adding a
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the mamaTransportTopicEvent
enumeration?

Thanks,
Sam


Tom Doust
 

Hi

In the Tick42 RMDS bridge we address this problem by allowing applications to use the sendFromInbox methods on a publisher and where applicable converting the an asynchronous response from the RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post" publishing model that will ACK (or NAK) every message I think it will work for any middleware that has an asynchronous response mechanism; other publishing models on RMDS do not have a message reponse.

The message sent to the inbox is arbitrary but it would make sense if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent mama code, or use an OK status in the case of an ACK. These include states like MAMA_MSG_STATUS_NOT_ENTITLED and MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text string which could carry more information relevant to the client application.

We would propose that everyone addressing the issue of processing asynchronous publishing responses takes this approach and that as a community we agree on a convention for the message content.


Best Regards

Tom Doust

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously of errors while publishing, specifically when the published message was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since the application or the environment is incorrectly configured, there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support these kind of events, what would such extensions look like? Perhaps adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Glenn McClements <g.mcclements@...>
 

Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a solution
to Sam’s query but I wasn’t overly keen on it. The reason being that being
rejected by the broker felt very much like a middleware level event, not
an application or market data event so it feels better to me that the
error comes from the middleware as a callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might actually
prevent this from happening. It could be that you form the message on the
client side in the bridge, but this itself feels incorrect as it’s
translating from a middleware level even into a data message. (Also a
minor point is that the MAMA_MSG_TYPE_SEC_STATUS relates to security
status events on a exchange). This is not to say that a message response
is incorrect for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently the
best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to inform
the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would need
to consider how this would be used/interact with the higher level concept
of market data entitlements.

Another related concept is guaranteed or acknowledged messaging, which
would generate other asynchronous event on a publishers. Like the original
issue, this *could* be done at a high level with MamaMsgs being passed
back to acknowledge delivery, but generally this is implemented at the
middleware level so again a publisher callback would be better (with a
handle to the original message as well for context).

Glenn

On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and where
applicable converting the an asynchronous response from the RMDS into a
mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post" publishing
model that will ACK (or NAK) every message I think it will work for any
middleware that has an asynchronous response mechanism; other publishing
models on RMDS do not have a message reponse.

The message sent to the inbox is arbitrary but it would make sense if
other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent mama
code, or use an OK status in the case of an ACK. These include states
like MAMA_MSG_STATUS_NOT_ENTITLED and MAMA_MSG_STATUS_BAD_SYMBOL amongst
a number of others. Obviously the set of code and the mappings are
platform specific

The one thing I think is missing here is a field containing a text string
which could carry more information relevant to the client application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously of
errors while publishing, specifically when the published message was
rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback informing
the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the connection,
giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since the
application or the environment is incorrectly configured, there's not
much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support these
kind of events, what would such extensions look like? Perhaps adding a
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the mamaTransportTopicEvent
enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Tom Doust
 

Hi Glenn

In many cases publishing a message is transactional and can succeed or fail. Failure does not mean that the connection (for the transport or the item is at fault) in which case connection level call-backs are not the appropriate way to signal a transaction failure.

Even with the ACL failure condition described by Sam could recover without a reconnect from the client if the Solace server supports dynamic changes to the ACL.

On RMDS/TREP a common cause for receiving a NAK is that the user is not permissioned to publish this particular symbol on this source. The may be entitled on other symbols

I'm inclined to agree with you about the use of callbacks over messages but in practical terms the "send-from-inbox / reply-to-inbox" mechanism is in place and is consistently available in the c/cpp, java and .net apis; that’s not true of call-backs. It's also true that messages are generally easier to manage in an asynchronous, multi-threaded environment.

Best regards

Tom

-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: 07 November 2014 3:57 PM
To: Tom Doust; Sam Wilson; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Asynchronous publisher events

Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a solution to Sam’s query but I wasn’t overly keen on it. The reason being that being rejected by the broker felt very much like a middleware level event, not an application or market data event so it feels better to me that the error comes from the middleware as a callback.

Also, responding with a MamaMsg to the sender means assuming the connection is up and alive, whereas a middleware ACL policy might actually prevent this from happening. It could be that you form the message on the client side in the bridge, but this itself feels incorrect as it’s translating from a middleware level even into a data message. (Also a minor point is that the MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a exchange). This is not to say that a message response is incorrect for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to inform the client asynchronously of events like this, and MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would need to consider how this would be used/interact with the higher level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging, which would generate other asynchronous event on a publishers. Like the original issue, this *could* be done at a high level with MamaMsgs being passed back to acknowledge delivery, but generally this is implemented at the middleware level so again a publisher callback would be better (with a handle to the original message as well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and where
applicable converting the an asynchronous response from the RMDS into a
mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post" publishing
model that will ACK (or NAK) every message I think it will work for any
middleware that has an asynchronous response mechanism; other
publishing models on RMDS do not have a message reponse.

The message sent to the inbox is arbitrary but it would make sense if
other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent mama
code, or use an OK status in the case of an ACK. These include states
like MAMA_MSG_STATUS_NOT_ENTITLED and MAMA_MSG_STATUS_BAD_SYMBOL
amongst a number of others. Obviously the set of code and the mappings
are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously of
errors while publishing, specifically when the published message was
rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the connection,
giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since the
application or the environment is incorrectly configured, there's not
much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Sam Wilson <Sam.Wilson@...>
 

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages as
well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before moving
on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the specific
publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a solution
to Sam’s query but I wasn’t overly keen on it. The reason being that being
rejected by the broker felt very much like a middleware level event, not
an application or market data event so it feels better to me that the
error comes from the middleware as a callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might actually
prevent this from happening. It could be that you form the message on the
client side in the bridge, but this itself feels incorrect as it’s
translating from a middleware level even into a data message. (Also a
minor point is that the MAMA_MSG_TYPE_SEC_STATUS relates to security
status events on a exchange). This is not to say that a message response
is incorrect for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently the
best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to inform
the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would need
to consider how this would be used/interact with the higher level concept
of market data entitlements.

Another related concept is guaranteed or acknowledged messaging, which
would generate other asynchronous event on a publishers. Like the original
issue, this *could* be done at a high level with MamaMsgs being passed
back to acknowledge delivery, but generally this is implemented at the
middleware level so again a publisher callback would be better (with a
handle to the original message as well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and where
applicable converting the an asynchronous response from the RMDS into a
mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post" publishing
model that will ACK (or NAK) every message I think it will work for any
middleware that has an asynchronous response mechanism; other publishing
models on RMDS do not have a message reponse.

The message sent to the inbox is arbitrary but it would make sense if
other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent mama
code, or use an OK status in the case of an ACK. These include states
like MAMA_MSG_STATUS_NOT_ENTITLED and MAMA_MSG_STATUS_BAD_SYMBOL amongst
a number of others. Obviously the set of code and the mappings are
platform specific

The one thing I think is missing here is a field containing a text string
which could carry more information relevant to the client application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously of
errors while publishing, specifically when the published message was
rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback informing
the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the connection,
giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since the
application or the environment is incorrectly configured, there's not
much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support these
kind of events, what would such extensions look like? Perhaps adding a
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the mamaTransportTopicEvent
enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Tom Doust
 

Hi Sam (and Glenn)

Its interesting to hear the Solace perspective on this.

In the RMDS case we are dealing with real messages, they come from the intended destination of the published message (or some routing/authentication element encountered en route to the destination) and they do come in on the wire. And they are associated with a publisher. Almost completely complementary to the case you describe

I don’t really care how we deliver this kind of notification to the client. I agree though that callbacks are (in some respects) easier to implement. If publisher call-backs are added to the api I will modify our bridge to give client apps the option of using them. In the meantime inbox messages work just fine.

I think one thing that this issue demonstrates is that it is important to look at the behaviour of a range of platforms as part of defining interfaces.

Best regards

Tom

-----Original Message-----
From: Sam Wilson [mailto:Sam.Wilson@solacesystems.com]
Sent: 11 November 2014 4:46 PM
To: Glenn McClements; Tom Doust; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Asynchronous publisher events

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from a publisher, they wouldn't travel along the wire, and they probably would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback mechanism is much more general, and would work for non-inbox messages as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous publisher events, what needs to be thought about/discussed before moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks; and a pair of new methods mamaPublisher_createWithCB and/or mamaPublisher_setCallbacks. That way we report the error on the specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a data
message. (Also a minor point is that the MAMA_MSG_TYPE_SEC_STATUS
relates to security status events on a exchange). This is not to say
that a message response is incorrect for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher level
concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging, which
would generate other asynchronous event on a publishers. Like the
original issue, this *could* be done at a high level with MamaMsgs
being passed back to acknowledge delivery, but generally this is
implemented at the middleware level so again a publisher callback
would be better (with a handle to the original message as well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it will
work for any middleware that has an asynchronous response mechanism;
other publishing models on RMDS do not have a message reponse.

The message sent to the inbox is arbitrary but it would make sense if
other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent mama
code, or use an OK status in the case of an ACK. These include states
like MAMA_MSG_STATUS_NOT_ENTITLED and MAMA_MSG_STATUS_BAD_SYMBOL
amongst a number of others. Obviously the set of code and the
mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since the
application or the environment is incorrectly configured, there's not
much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Glenn McClements <g.mcclements@...>
 

Yeah I think something analogous to mamaMsgCallbacks (which should have
been
called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the
nicest
fit here.

The advantage of doing it this was is that the set of callbacks can be
optional
and extensible. One gotcha though is the question of which thread/queue
the
callback should be fired on? Currently the mamaPublisher doesn’t have a
queue
associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events
and
put together a proposal/appraisal, then we can figure out who wants to
implement
and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to
discuss.

Cheers,
Glenn

On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@solacesystems.com> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages as
well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before moving
on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the specific
publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a solution
to Sam’s query but I wasn’t overly keen on it. The reason being that
being
rejected by the broker felt very much like a middleware level event, not
an application or market data event so it feels better to me that the
error comes from the middleware as a callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually
prevent this from happening. It could be that you form the message on
the
client side in the bridge, but this itself feels incorrect as it’s
translating from a middleware level even into a data message. (Also a
minor point is that the MAMA_MSG_TYPE_SEC_STATUS relates to security
status events on a exchange). This is not to say that a message response
is incorrect for RMDS posts however, it’s just a different type of
event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently the
best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform
the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need
to consider how this would be used/interact with the higher level
concept
of market data entitlements.

Another related concept is guaranteed or acknowledged messaging, which
would generate other asynchronous event on a publishers. Like the
original
issue, this *could* be done at a high level with MamaMsgs being passed
back to acknowledge delivery, but generally this is implemented at the
middleware level so again a publisher callback would be better (with a
handle to the original message as well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and where
applicable converting the an asynchronous response from the RMDS into a
mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post" publishing
model that will ACK (or NAK) every message I think it will work for any
middleware that has an asynchronous response mechanism; other
publishing
models on RMDS do not have a message reponse.

The message sent to the inbox is arbitrary but it would make sense if
other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent mama
code, or use an OK status in the case of an ACK. These include states
like MAMA_MSG_STATUS_NOT_ENTITLED and MAMA_MSG_STATUS_BAD_SYMBOL
amongst
a number of others. Obviously the set of code and the mappings are
platform specific

The one thing I think is missing here is a field containing a text
string
which could carry more information relevant to the client application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously of
errors while publishing, specifically when the published message was
rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing
the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the connection,
giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since the
application or the environment is incorrectly configured, there's not
much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these
kind of events, what would such extensions look like? Perhaps adding a
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the mamaTransportTopicEvent
enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Gary Molloy <g.molloy@...>
 

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions
g.molloy@srtechlabs.com

-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@solacesystems.com> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Gary Molloy <g.molloy@...>
 

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@srtechlabs.com

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@srtechlabs.com



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@solacesystems.com> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Tom Doust
 

Hi.

Either is OK by me at the moment

Tom

-----Original Message-----
From: Gary Molloy [mailto:g.molloy@srtechlabs.com]
Sent: 08 December 2014 11:16 AM
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@srtechlabs.com

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@srtechlabs.com



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@solacesystems.com> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

Either one works for me. Looking forward to the meeting.

--Alireza

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-
dev-bounces@lists.openmama.org] On Behalf Of Gary Molloy
Sent: Monday, December 08, 2014 6:16 AM
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-
dev@lists.openmama.org
Subject: Re: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the
discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of
December @ 4pm UTC. I would appreciate it if you could let us know which
date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@srtechlabs.com

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-
dev@lists.openmama.org
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher
event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-
y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so
we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions
g.molloy@srtechlabs.com



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have
been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher
is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional
and extensible. One gotcha though is the question of which thread/queue
the callback should be fired on? Currently the mamaPublisher doesn’t have
a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events
and put together a proposal/appraisal, then we can figure out who wants to
implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@solacesystems.com>
wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox
messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is
currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though
we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message
structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1,
MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others.
Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of
Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns
immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT
or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error.
Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Alpert, Reed <reed.alpert@...>
 

16th is better for me.

Thanks,

Reed.


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@jpmorgan.com

Alternate Contact: CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@jpmorgan.com

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Gary Molloy
Sent: Monday, December 08, 2014 6:16 AM
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@srtechlabs.com

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@srtechlabs.com



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@solacesystems.com> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates.

This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you.

Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


Gary Molloy <g.molloy@...>
 

Hi Guys,

Thanks for the feedback on the proposed dates.

From the feedback I think the best date that suits most would be Tuesday 16th December @ 4pm UTC, so let's schedule the call for then.

The connection details once again are:

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664

Looking forward to the call.

Thanks,
Gary


Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@srtechlabs.com

-----Original Message-----
From: Gary Molloy
Sent: 08 December 2014 11:16
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I would like to propose a call this week (or next week) to kick off the discussion for the new mamaPublisher event handling feature.

I was hoping to schedule a call either Thursday 11th, or Tuesday 16th of December @ 4pm UTC. I would appreciate it if you could let us know which date would suit most?

Conference call details:

UK Number: +44-28-90998719
US Number: +646-828-7349

Conference ID: 4741664


Thanks
Gary



Gary Molloy – SR Labs
Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD
Tel: +44 28 9099 7580 Ext 3397
g.molloy@srtechlabs.com

-----Original Message-----
From: Gary Molloy
Sent: 02 December 2014 17:44
To: Glenn McClements; Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Asynchronous publisher events

Hi Guys,

I have created an appraisal / proposal document for the new publisher event handling feature on Google Docs which you can find here:

https://docs.google.com/document/d/1bxy0Muxit4uc8Bz-y6CS3wFq5I7sJ0qAkA8-zr5vLec/edit?usp=sharing

If you would like to review the document and we can then schedule a call so we can go over the proposal to discuss it in further detail.

Thanks,
Gary


Gary Molloy
SR.LABS Proven High Speed Electronic Trading Solutions g.molloy@srtechlabs.com



-----Original Message-----
From: Glenn McClements
Sent: 13 November 2014 12:15
To: Sam Wilson; Tom Doust; openmama-dev@lists.openmama.org
Cc: Gary Molloy
Subject: Re: [Openmama-dev] Asynchronous publisher events

Yeah I think something analogous to mamaMsgCallbacks (which should have been called mamaSubscriberCallbacks in hindsight) for the mamaPublisher is the nicest fit here.

The advantage of doing it this was is that the set of callbacks can be optional and extensible. One gotcha though is the question of which thread/queue the callback should be fired on? Currently the mamaPublisher doesn’t have a queue associated with it so we’ll probably need to add this as well.

For next steps I propose we collate the requirements for publisher events and put together a proposal/appraisal, then we can figure out who wants to implement and test it. Possible events are:

- undeliverable (with a reason/status)
- delivered
- …. anything else?

Gary on our side will pull these together and we can set up a call to discuss.

Cheers,
Glenn




On 11/11/2014 16:45, "Sam Wilson" <Sam.Wilson@solacesystems.com> wrote:

Hey Glenn, Tom,

I have to agree with Glenn here. I don't think these kind of events
belong alongside regular messages, for a couple of reasons.

For one, simply enough, they aren't messages. They don't originate from
a publisher, they wouldn't travel along the wire, and they probably
would have to be synthesized in the middleware bridge.

Secondly, Tom's solution only works for sendFromInbox. The callback
mechanism is much more general, and would work for non-inbox messages
as well, which is important for our use case.

Assuming we do go ahead with implementing callbacks for asynchronous
publisher events, what needs to be thought about/discussed before
moving on to writing/accepting a patch?

I have two ideas on how to implement this, but keep in mind that I'm
rather new to OpenMAMA and market data in general.

We could use a mamaTransportTopic event, and report the errors with a
callback on the transport.

The other option that I see, which Glenn hinted at, is that we create a
structure similar to mamaMsgCallbacks, called mamaPublisherCallbacks;
and a pair of new methods mamaPublisher_createWithCB and/or
mamaPublisher_setCallbacks. That way we report the error on the
specific publisher.

What are your thoughts?

Regards,
Sam

On 14-11-07 10:57 AM, Glenn McClements wrote:
Hi Tom, Sam,
Before you responded I was considering the use of an inbox as a
solution to Sam’s query but I wasn’t overly keen on it. The reason
being that being rejected by the broker felt very much like a
middleware level event, not an application or market data event so it
feels better to me that the error comes from the middleware as a
callback.

Also, responding with a MamaMsg to the sender means assuming the
connection is up and alive, whereas a middleware ACL policy might
actually prevent this from happening. It could be that you form the
message on the client side in the bridge, but this itself feels
incorrect as it’s translating from a middleware level even into a
data message. (Also a minor point is that the
MAMA_MSG_TYPE_SEC_STATUS relates to security status events on a
exchange). This is not to say that a message response is incorrect
for RMDS posts however, it’s just a different type of event.


For Sam’s point I do prefer the idea of currently returning a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and the transport is currently
the best/only place to do this.

Going forward we could add callbacks to the MamaPublisher object to
inform the client asynchronously of events like this, and
MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED seems reasonable though we would
need to consider how this would be used/interact with the higher
level concept of market data entitlements.

Another related concept is guaranteed or acknowledged messaging,
which would generate other asynchronous event on a publishers. Like
the original issue, this *could* be done at a high level with
MamaMsgs being passed back to acknowledge delivery, but generally
this is implemented at the middleware level so again a publisher
callback would be better (with a handle to the original message as
well for context).

Glenn



On 07/11/2014 14:20, "Tom Doust" <tom.doust@tick42.com> wrote:

Hi

In the Tick42 RMDS bridge we address this problem by allowing
applications to use the sendFromInbox methods on a publisher and
where applicable converting the an asynchronous response from the
RMDS into a mama message which is delivered to the inbox.

Although this is intended to support the RMDS message "post"
publishing model that will ACK (or NAK) every message I think it
will work for any middleware that has an asynchronous response
mechanism; other publishing models on RMDS do not have a message
reponse.

The message sent to the inbox is arbitrary but it would make sense
if other bridges that used this technique used the same message structure.

The message we use is formed as follows

mamaMsg_addI32(msg, "MdMsgType", 1, MAMA_MSG_TYPE_SEC_STATUS);

mamaMsgStatus status = MAMA_MSG_STATUS_OK;
if(nakCode != RSSL_NAKC_NONE)
{
status = RsslNakCode2MamaMsgStatus(nakCode);
}

mamaMsg_addI32(msg, "MdMsgStatus", 2, status);
mamaMsg_addString(msg, "wSymbol",470, symbol_.c_str());

As you can see we just convert an rssl NAK code to an equivalent
mama code, or use an OK status in the case of an ACK. These include
states like MAMA_MSG_STATUS_NOT_ENTITLED and
MAMA_MSG_STATUS_BAD_SYMBOL amongst a number of others. Obviously the
set of code and the mappings are platform specific

The one thing I think is missing here is a field containing a text
string which could carry more information relevant to the client
application.

We would propose that everyone addressing the issue of processing
asynchronous publishing responses takes this approach and that as a
community we agree on a convention for the message content.


Best Regards

Tom Doust




-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Sam
Wilson
Sent: 06 November 2014 7:45 PM
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Asynchronous publisher events

Hey all,

We're looking for a way to notify a mama application asynchronously
of errors while publishing, specifically when the published message
was rejected by the message broker's ACL.

Our API assumes the messages were accepted, and returns immediately.
After some time you might get a connection-level event callback
informing the application that its messages were rejected.

I have two questions.

First off, what is the best way to represent this situation in 2.3.1?
We've been tossing around the idea of just tearing down the
connection, giving the application a MAMA_TRANSPORT_DISCONNECT or a
MAMA_TRANSPORT_PUBLISHER_DISCONNECT, and logging the error. Since
the application or the environment is incorrectly configured,
there's not much that can be done by the application anyways.

Secondly, if openmama were to be extended in the future to support
these kind of events, what would such extensions look like? Perhaps
adding a MAMA_TRANSPORT_TOPIC_PUBLISH_DENIED to the
mamaTransportTopicEvent enumeration?

Thanks,
Sam
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev