Openmama-dev@lists.openmama.org | RFC DQ Pluggability

Aaron Sneddon

#2146

Hey everyone,

You may have noticed that a new RFC has been merged into master of https://github.com/OpenMAMA/openmama.github.io .

This is just to make you aware, if you weren’t already. Feel free to have a look at it under /pages/openmama/openmama_rfc_dq_pluggability.md and
let me know what you think. Comments and suggestions will be happily considered.

Regards,
Aaron

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC

Frank Quinn

#2148

Thanks Aaron,

Apologies for the delay in getting to read this properly - for reference to the group this is the link to the document:

https://openmama.github.io/openmama_rfc_source_discovery.html

From my point of view I think it would be useful to get some visibility on what the proposed function prototypes will look like at the hook points.

There are several method names in there which is good but proposed arguments which will be passed to the plugin methods will probably be the thing of most interest to bridge developers who will want to take advantage of this functionality and may want a series of objects to be made available which may be more than what is required for the DQ use case alone.

Consider the fact that these plugin methods once created will be fairly set in stone, otherwise we'll end up going down the functionNameEx road again or mandatory bridge changes which nobody wants. It may be a lot easier to add additional arguments (if necessary) at this point even if they aren't required for this use case.

Cheers,

Frank

Show quoted text

On Wed, Jan 17, 2018 at 10:01 AM, Aaron Sneddon <asneddon@...> wrote:

Hey everyone,

You may have noticed that a new RFC has been merged into master of https://github.com/OpenMAMA/openmama.github.io .

This is just to make you aware, if you weren’t already. Feel free to have a look at it under /pages/openmama/openmama_rfc_dq_pluggability.md and
let me know what you think. Comments and suggestions will be happily considered.

Regards,
Aaron

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC

_______________________________________________
Openmama-dev mailing list
Openmama-dev@....org
https://lists.openmama.org/mailman/listinfo/openmama-dev

Frank Quinn

#2149

Thanks Aaron,

Put in a few changes to the document to make the prototypes "pop" a little bit more. A few suggestions / talking points though:

mama_status dqstrategyMamaPlugin_transportPostCreateHook (mamaPluginInfo pluginInfo)

I think it would be useful if this at least included the mamaTransport that has just been created.

mama_status dqstrategyMamaPlugin_transportEventHook(mamaPluginInfo pluginInfo,
                                                    mamaTransport  transport,
                                                    int            setStale)

It reads like setStale will represent something related to the staleness but it's not clear what possible values they may have. I also wonder if the use of mamaTransportEvent might be more useful here? Is it also feasible that this may not be necessary at all if the postCreateHook could set up a standard transport callback?

I'm not sure about what's meant by the Plugin Changes section when it talks about the overriding of the default dq plugin in the plugin code. Instead might it be more straightforward if:

1. Additional plugins for dq could simply be loaded and live alongside each other. On-hook the plugin can then decide if it wants to set itself up based on its own configuration standards.

2. The "default" dq plugin could do something similar and decide itself if it wants to take action. This would make it more like a standard plugin rather than a special case for DQ.

3. The transportCreateHook for the dq plugin could then *itself* check a property *there* or on init to decide whether or not it should take action on this transport for the bridge as updates are fired.

Cheers,

Frank

Show quoted text

On Wed, Jan 31, 2018 at 11:30 AM, Aaron Sneddon <asneddon@...> wrote:

Thanks for your feedback, I have now added the proposed prototypes to the RFC as requested.
https://openmama.github.io/openmama_rfc_dq_pluggability.html

Just so the group is aware, there is until Thursday now to give feedback before development kicks off so today is your last chance.
You might want to take a look at the document once more, even just to look at the function prototypes since they were only added today.

Thanks,

Aaron

From: Frank Quinn [mailto:frank@...]
Sent: 30 January 2018 21:04
To: Aaron Sneddon <asneddon@...>
Cc: openmama-dev@....org
Subject: Re: [Openmama-dev] RFC DQ Pluggability

Thanks Aaron,

Apologies for the delay in getting to read this properly - for reference to the group this is the link to the document:

https://openmama.github.io/openmama_rfc_source_discovery.html

From my point of view I think it would be useful to get some visibility on what the proposed function prototypes will look like at the hook points.

There are several method names in there which is good but proposed arguments which will be passed to the plugin methods will probably be the thing of most interest to bridge developers who will want to take advantage of this functionality and may want a series of objects to be made available which may be more than what is required for the DQ use case alone.

Consider the fact that these plugin methods once created will be fairly set in stone, otherwise we'll end up going down the functionNameEx road again or mandatory bridge changes which nobody wants. It may be a lot easier to add additional arguments (if necessary) at this point even if they aren't required for this use case.

Cheers,

Frank

On Wed, Jan 17, 2018 at 10:01 AM, Aaron Sneddon <asneddon@...> wrote:

Hey everyone,

You may have noticed that a new RFC has been merged into master of https://github.com/OpenMAMA/openmama.github.io .

This is just to make you aware, if you weren’t already. Feel free to have a look at it under /pages/openmama/openmama_rfc_dq_pluggability.md and
let me know what you think. Comments and suggestions will be happily considered.

Regards,
Aaron

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC

_______________________________________________
Openmama-dev mailing list
Openmama-dev@....org
https://lists.openmama.org/mailman/listinfo/openmama-dev

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC

Frank Quinn

#2152

Thanks Aaron,

Just on the plugin-only front, I'll probably appreciate more what you mean when a pull request comes in, but it's my opinion that if the plugin code isn't flexible enough to allow us to handle this, we should consider being more flexible with the plugin code.

Maybe that's another task but again it goes into setting aside these prototypes now since I think i'd like to add it soon afterwards if it's not in the PR. I'm not sure what the rest of the dev list things but opinions are welcome?

To explain, I'm not sure hooks are much use if you can't attach context specific data to them, because then you'll *always* need to do lookups on firing. For example, I could see something like this being more useful (same for subscription):

mama_status dqstrategyMamaPlugin_transportPostCreateHook (mamaPluginInfo pluginInfo,

mamaTransport transport,

void** closure) {

*closure = (void*) new Whatever();

}

mama_status dqstrategyMamaPlugin_transportEventHook(mamaPluginInfo pluginInfo,

mamaTransport transport,

int setStale

void* closure) {

Whatever whatever = (Whatever)closure;

}

That way you can initialize and attach objects on postcreate hooks which can then be retrieved later in subsequent hooks without requiring any lookups. There are a number of ways to implement this but the easiest is probably an array of pointers for each plugin instance attached to each object.

You could then have member variables attached to transports, subscriptions etc to allow plugins to fire data around without having to do its own out of band lookups.

Cheers,

Frank

Show quoted text

On 1 Feb 2018 17:26, "Aaron Sneddon" <asneddon@...> wrote:

Hey Frank,

I have taken on board the suggested changes to function signatures, and will have the transport passed to transportPostCreateHook, and mamaTransportEvent to transportEventHook.

With regards to setstale, the transport callback will generally call the transportEventHook, and will take care of the setPossiblyStale stuff.

So, the setStale is there because ‘event’ on its own isn’t enough to differentiate between current disconnect/disconnectNoStale logic.

Also, I’ll update the RFC for this to be clearer.

With regards to:

I'm not sure about what's meant by the Plugin Changes section when it talks about the overriding of the default dq plugin in the plugin code. Instead might it be more straightforward if:

1. Additional plugins for dq could simply be loaded and live alongside each other. On-hook the plugin can then decide if it wants to set itself up based on its own configuration standards.

2. The "default" dq plugin could do something similar and decide itself if it wants to take action. This would make it more like a standard plugin rather than a special case for DQ.

3. The transportCreateHook for the dq plugin could then *itself* check a property *there* or on init to decide whether or not it should take action on this transport for the bridge as updates are fired.

Ideally we would like to make the DQ plugin stuff as much like normal plugins as possible. But fitting plugins which may only need to act on particular transports in is a little awkward. Configuration wise and conceptually it is easier to treat them in the same way as normal plugins, but point 3) is the problem here – the decision about whether or not to take action would surely need to be made on a per-update basis and so would need some sort of lookup to be performed every time.

Hope this helps..

Thanks,

Aaron

From: Frank Quinn [mailto:frank@...]
Sent: 31 January 2018 23:13
To: Aaron Sneddon <asneddon@...>
Cc: openmama-dev@...g
Subject: Re: [Openmama-dev] RFC DQ Pluggability
Thanks Aaron,

Put in a few changes to the document to make the prototypes "pop" a little bit more. A few suggestions / talking points though:
mama_status dqstrategyMamaPlugin_transportPostCreateHook (mamaPluginInfo pluginInfo)
I think it would be useful if this at least included the mamaTransport that has just been created.
mama_status dqstrategyMamaPlugin_transportEventHook(mamaPluginInfo pluginInfo,
                                                    mamaTransport  transport,
                                                    int            setStale)
It reads like setStale will represent something related to the staleness but it's not clear what possible values they may have. I also wonder if the use of mamaTransportEvent might be more useful here? Is it also feasible that this may not be necessary at all if the postCreateHook could set up a standard transport callback?

I'm not sure about what's meant by the Plugin Changes section when it talks about the overriding of the default dq plugin in the plugin code. Instead might it be more straightforward if:

1. Additional plugins for dq could simply be loaded and live alongside each other. On-hook the plugin can then decide if it wants to set itself up based on its own configuration standards.

2. The "default" dq plugin could do something similar and decide itself if it wants to take action. This would make it more like a standard plugin rather than a special case for DQ.

3. The transportCreateHook for the dq plugin could then *itself* check a property *there* or on init to decide whether or not it should take action on this transport for the bridge as updates are fired.

Cheers,

Frank
On Wed, Jan 31, 2018 at 11:30 AM, Aaron Sneddon <asneddon@...> wrote:

Thanks for your feedback, I have now added the proposed prototypes to the RFC as requested.
https://openmama.github.io/openmama_rfc_dq_pluggability.html

Just so the group is aware, there is until Thursday now to give feedback before development kicks off so today is your last chance.
You might want to take a look at the document once more, even just to look at the function prototypes since they were only added today.

Thanks,

Aaron

From: Frank Quinn [mailto:frank@...]
Sent: 30 January 2018 21:04
To: Aaron Sneddon <asneddon@...>
Cc: openmama-dev@...g
Subject: Re: [Openmama-dev] RFC DQ Pluggability

Thanks Aaron,

Apologies for the delay in getting to read this properly - for reference to the group this is the link to the document:

https://openmama.github.io/openmama_rfc_source_discovery.html

From my point of view I think it would be useful to get some visibility on what the proposed function prototypes will look like at the hook points.

There are several method names in there which is good but proposed arguments which will be passed to the plugin methods will probably be the thing of most interest to bridge developers who will want to take advantage of this functionality and may want a series of objects to be made available which may be more than what is required for the DQ use case alone.

Consider the fact that these plugin methods once created will be fairly set in stone, otherwise we'll end up going down the functionNameEx road again or mandatory bridge changes which nobody wants. It may be a lot easier to add additional arguments (if necessary) at this point even if they aren't required for this use case.

Cheers,

Frank

On Wed, Jan 17, 2018 at 10:01 AM, Aaron Sneddon <asneddon@...> wrote:

Hey everyone,

You may have noticed that a new RFC has been merged into master of https://github.com/OpenMAMA/openmama.github.io .

This is just to make you aware, if you weren’t already. Feel free to have a look at it under /pages/openmama/openmama_rfc_dq_pluggability.md and
let me know what you think. Comments and suggestions will be happily considered.

Regards,
Aaron

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC

_______________________________________________
Openmama-dev mailing list
Openmama-dev@...g
https://lists.openmama.org/mailman/listinfo/openmama-dev

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC
The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. Vela Trading Technologies LLC