Date   

Re: OpenMAMA-2.3.3

Gary Molloy <g.molloy@...>
 

Hi Keith,

 

Thanks for your email.

 

The plan is to try to finish up the issues for the next release by the end of this week (beginning of next week) and then cut the next branch so that the community can begin testing it with the view to have it completed by the end of May.

 

Thanks,

Gary

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 

From: Keith Rudd [mailto:keith.rudd@...]
Sent: 27 April 2015 14:58
To: Gary Molloy; Openmama-dev@...
Subject: RE: OpenMAMA-2.3.3

 

Classification: Public

Is there a planned/target release date for OpenMAMA 2.3.3 ?

 

Regards,

Keith

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Gary Molloy
Sent: 24 February 2015 12:16
To: Openmama-dev@...
Subject: Re: [Openmama-dev] OpenMAMA-2.3.3

 

Hi Guys,

 

We have updated some of the tickets being considered for this release asking for some addition information. 

 

If you would like to have other issues to be considered for this upcoming release we will require a Bugzilla ticket to be submitted first (following the submission guidelines, http://wiki.openmama.org/index.php/Patch_submission)

 

We would appreciate your timely responses on these.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 

From: Gary Molloy
Sent: 23 February 2015 15:47
To: Openmama-dev@...
Subject: OpenMAMA-2.3.3

 

Hi Guys,

 

We are currently planning the next release of OpenMAMA - OpenMAMA-2.3.3.

 

Please find below the initial list of issues that are currently scheduled for inclusion in this release:

 

BZ166    Make wInterlocked_set return the prior value

BZ164    MAMAJNI: MamaPublisher: Overload the MamaPublisher Create Method

BZ169    Wombat queue has no separate deallocate method

BZ176    MAMAC: Missing actions for snapshot subscriptions transition to deactivate state

BZ168    MAMA: Complete support for Vector Bool and Vector Char field types

BZ156    COMMON: Variable expansion in property value on the last line of properties file fails

BZ125    Payload and Middleware Unit-test in Visual Studio

BZ-114  MAMA JNI crash in MamaListen.java when using platformInfo.toString()

BZ-178  Problem with mamaDictionary_getDictionaryMessage when multiple bridges are loaded

BZ-179  OpenMAMA mock RPM's fail to build

 

If there are any issues in particular you would like to see included in the next release, please feel free to reply (with a prioritised list) and we can review them for inclusion.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 



---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


Re: OpenMAMA-2.3.3

Keith Rudd
 

Classification: Public

Is there a planned/target release date for OpenMAMA 2.3.3 ?

 

Regards,

Keith

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Gary Molloy
Sent: 24 February 2015 12:16
To: Openmama-dev@...
Subject: Re: [Openmama-dev] OpenMAMA-2.3.3

 

Hi Guys,

 

We have updated some of the tickets being considered for this release asking for some addition information. 

 

If you would like to have other issues to be considered for this upcoming release we will require a Bugzilla ticket to be submitted first (following the submission guidelines, http://wiki.openmama.org/index.php/Patch_submission)

 

We would appreciate your timely responses on these.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 

From: Gary Molloy
Sent: 23 February 2015 15:47
To: Openmama-dev@...
Subject: OpenMAMA-2.3.3

 

Hi Guys,

 

We are currently planning the next release of OpenMAMA - OpenMAMA-2.3.3.

 

Please find below the initial list of issues that are currently scheduled for inclusion in this release:

 

BZ166    Make wInterlocked_set return the prior value

BZ164    MAMAJNI: MamaPublisher: Overload the MamaPublisher Create Method

BZ169    Wombat queue has no separate deallocate method

BZ176    MAMAC: Missing actions for snapshot subscriptions transition to deactivate state

BZ168    MAMA: Complete support for Vector Bool and Vector Char field types

BZ156    COMMON: Variable expansion in property value on the last line of properties file fails

BZ125    Payload and Middleware Unit-test in Visual Studio

BZ-114  MAMA JNI crash in MamaListen.java when using platformInfo.toString()

BZ-178  Problem with mamaDictionary_getDictionaryMessage when multiple bridges are loaded

BZ-179  OpenMAMA mock RPM's fail to build

 

If there are any issues in particular you would like to see included in the next release, please feel free to reply (with a prioritised list) and we can review them for inclusion.

 

Thanks,

Gary

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 



---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


Re: [PATCH 1/3] [MAMAC] mamaPlugin Feature

Glenn McClements <g.mcclements@...>
 

Thanks Gary.

To give some background on this change, it’s a first pass at a plugin framework that I hope to build on to allow OpenMAMA to become much more of a flexible framework.

The initial hooks were there because of an immediate need, but I’d like to get feedback on where else people would find hooks useful. One area which I’d like to explore is for pluggable entitlements. Using the plugin framework rather than the bridge mechanism:

- the plugins are already dynamic in the sense that any can be loaded at runtime 
- there is the concept of optional functions, so that a plugin can implement as many or as few hooks as they wish. We can also add new hooks without breaking compatibility. 
- the hooks are multi purpose, and multiple plugins can be called from the same hook

Thanks,
Glenn 


From: Gary Molloy
Date: Thursday, 23 April 2015 17:53
To: "openmama-dev@..."
Subject: [Openmama-dev] [PATCH 1/3] [MAMAC] mamaPlugin Feature

Testing Strategy:-

Not middleware or O/S specific. 
 

Using the attached template you can create your own plugin and load it in via your mama.properties, for example:

 

mama.plugin.name_0=template

 

You can run up a mamaPublisher to test your plugin, for example:

 

mamapublisherc -tport pub -m wmw -l MAMA_INBOUND_TOPIC -v -v -v -v -v

 

You can expect to see output similar this if your plugin was loaded successfully:

 

2015-04-09 13:45:02: (f9018280) : mama_initPlugins(): Initialising [mama.plugin.name_0] template 

2015-04-09 13:45:02: (f9018280) : mama_loadPlugin(): Sucessfully registered plugin functions for [template] 

2015-04-09 13:45:02: (f9018280) : mama_loadPlugin(): Successfully run the init hook for mama plugin [template]  

 

If your plugin was not loaded successful you can expect to see output similar to this:

 

2015-04-09 13:45:02: (f9018280) : mama_initPlugins(): Initialising [mama.plugin.name_0] template 

2015-04-09 13:45:02: (f9018280) : mamaPlugin_registerFunctions(): Cannot load plugin, does not implement required function: [templateMamaPlugin_shutdownHook] 

2015-04-09 13:45:02: (f9018280) : mama_loadPlugin(): Failed to register plugin functions for [template] 

 

Further documentation on the mamaPlugin feature will be available shortly which will provide more details on the feature which will include the various hooks available etc...

 

 

=========================================================================================================

 

From e2a6a3f9c5750cee780cb6185a1310b4e4b70629 Mon Sep 17 00:00:00 2001

From: Gary Molloy <g.molloy@...>

Date: Thu, 23 Apr 2015 11:22:14 -0400

Subject: [PATCH 1/3] [MAMAC] mamaPlugin Feature

 

A new feature, mamaPlugin has been introduced that will allow you to

create and run your own code within 'hooks' into the MAMA codebase.

 

A mamaPlugin is essentially a shared object.

 

Signed-off-by: Gary Molloy <g.molloy@...>

---

mama/c_cpp/src/c/Makefile.am    |    1 +

mama/c_cpp/src/c/SConscript     |    1 +

mama/c_cpp/src/c/SConscript.win |    1 +

mama/c_cpp/src/c/mama.c         |   10 +-

mama/c_cpp/src/c/mama/types.h   |   10 +

mama/c_cpp/src/c/plugin.c       |  472 +++++++++++++++++++++++++++++++++++++++

mama/c_cpp/src/c/plugin.h       |   59 +++++

mama/c_cpp/src/c/publisher.c    |   10 +

mama/c_cpp/src/c/transport.c    |   21 ++-

9 files changed, 579 insertions(+), 6 deletions(-)

create mode 100644 mama/c_cpp/src/c/plugin.c

create mode 100644 mama/c_cpp/src/c/plugin.h

 

diff --git a/mama/c_cpp/src/c/Makefile.am b/mama/c_cpp/src/c/Makefile.am

index c831ce4..318c353 100644

--- a/mama/c_cpp/src/c/Makefile.am

+++ b/mama/c_cpp/src/c/Makefile.am

@@ -131,6 +131,7 @@ libmama_la_SOURCES = \

     mamaStrUtils.h \

     mamaStrUtils.c \

               marketdata.c \

+    plugin.c \

     middleware.c \

               msg.c \

               msgfield.c \

diff --git a/mama/c_cpp/src/c/SConscript b/mama/c_cpp/src/c/SConscript

index 4362f50..f47e18c 100644

--- a/mama/c_cpp/src/c/SConscript

+++ b/mama/c_cpp/src/c/SConscript

@@ -98,6 +98,7 @@ libmama_sources = \

     mama.c

     mamaStrUtils.c

               marketdata.c

+    plugin.c

     middleware.c

               msg.c

               msgfield.c

diff --git a/mama/c_cpp/src/c/SConscript.win b/mama/c_cpp/src/c/SConscript.win

index 77e3e18..9a6ff05 100644

--- a/mama/c_cpp/src/c/SConscript.win

+++ b/mama/c_cpp/src/c/SConscript.win

@@ -50,6 +50,7 @@ dqpublishermanager.c

inbox.c

msgtype.c

msgutils.c

+plugin.c

senderId.c

reservedfields.c

subscription.c

diff --git a/mama/c_cpp/src/c/mama.c b/mama/c_cpp/src/c/mama.c

index 33a0768..5e31856 100644

--- a/mama/c_cpp/src/c/mama.c

+++ b/mama/c_cpp/src/c/mama.c

@@ -35,6 +35,7 @@

#include <payloadbridge.h>

#include <property.h>

#include <platform.h>

+#include <plugin.h>

 #include "fileutils.h"

#include "reservedfieldsimpl.h"

@@ -42,7 +43,6 @@

#include <mama/stat.h>

#include <mama/statfields.h>

#include <statsgeneratorinternal.h>

-#include <statsgeneratorinternal.h>

#include <mama/statscollector.h>

#include "transportimpl.h"

@@ -740,6 +740,9 @@ mama_openWithPropertiesCount (const char* path,

                               }

     }

+    /* This will initialise all plugins */

+    mama_initPlugins();

+

     prop = properties_Get (gProperties, "mama.catchcallbackexceptions.enable");

     if (prop != NULL && strtobool(prop))

     {

@@ -1253,9 +1256,12 @@ mama_closeCount (unsigned int* count)

                 gImpl.myPayloadLibraries[(uint8_t)payload] = NULL;

             }

         }

-       

+

        gDefaultPayload = NULL;

+       /* This will shutdown all plugins */

+       mama_shutdownPlugins();

+

         /* Look for a bridge for each of the middlewares and close them */

         for (middleware = 0; middleware != MAMA_MIDDLEWARE_MAX; ++middleware)

         {

diff --git a/mama/c_cpp/src/c/mama/types.h b/mama/c_cpp/src/c/mama/types.h

index 50d6608..03f6da8 100644

--- a/mama/c_cpp/src/c/mama/types.h

+++ b/mama/c_cpp/src/c/mama/types.h

@@ -74,6 +74,16 @@ typedef struct mamaBridgeImpl_* mamaBridge;

typedef struct mamaPayloadBridgeImpl_* mamaPayloadBridge;

 /**

+ * @brief Container for holding a mama plugin object

+ */

+typedef struct mamaPluginImpl_*     mamaPlugin;

+

+/**

+ * @brief Container for holding a mama plugin info

+ */

+typedef void*       mamaPluginInfo;

+

+/**

  * Flexible date/time format

  */

typedef   mama_u64_t*   mamaDateTime;

diff --git a/mama/c_cpp/src/c/plugin.c b/mama/c_cpp/src/c/plugin.c

new file mode 100644

index 0000000..b6475fe

--- /dev/null

+++ b/mama/c_cpp/src/c/plugin.c

@@ -0,0 +1,472 @@

+/* $Id$

+ *

+ * OpenMAMA: The open middleware agnostic messaging API

+ * Copyright (C) 2011 NYSE Technologies, Inc.

+ *

+ * This library is free software; you can redistribute it and/or

+ * modify it under the terms of the GNU Lesser General Public

+ * License as published by the Free Software Foundation; either

+ * version 2.1 of the License, or (at your option) any later version.

+ *

+ * This library is distributed in the hope that it will be useful,

+ * but WITHOUT ANY WARRANTY; without even the implied warranty of

+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU

+ * Lesser General Public License for more details.

+ *

+ * You should have received a copy of the GNU Lesser General Public

+ * License along with this library; if not, write to the Free Software

+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA

+ * 02110-1301 USA

+ */

+

+#include <string.h>

+#include <stdio.h>

+#include <limits.h>

+

+#include "wombat/port.h"

+#include "wombat/environment.h"

+#include "wombat/strutils.h"

+#include <wombat/wtable.h>

+

+#include <mama/mama.h>

+#include <mama/error.h>

+#include <mamainternal.h>

+#include <mama/version.h>

+#include <property.h>

+#include <plugin.h>

+#include <platform.h>

+

+#define PLUGIN_PROPERTY "mama.plugin.name_"

+#define PLUGIN_NAME "mamaplugin"

+

+#define MAX_PLUGINS 100

+#define MAX_PLUGIN_STRING 1024

+

+#define MAX_FUNC_STRING 256

+

+/**

+ * @brief Mechanism for registering required plugin functions.

+ *

+ * Taking a function string name search the shared library handle, using the

+ * loadLibFunc portability method, for the function. If it is found, set the

+ * appropriate function pointer in the plugin struct to the result. If not,

+ * log as an error the fact that the funciton cannot be found, and return

+ * MAMA_STATUS_PLATFORM

+ *

+ * @param FUNCSTRINGNAME The string function name.

+ * @param FUNCIMPLNAME The name of the function pointer in the plugin struct

+ * @param FUNCIMPLTYPE The type of the function pointer expected.

+ */

+#define REGISTER_PLUGIN_FUNCTION(FUNCSTRINGNAME, FUNCIMPLNAME, FUNCIMPLTYPE)    \

+do {                                                                            \

+    snprintf (functionName, MAX_FUNC_STRING, "%s"#FUNCSTRINGNAME, name);        \

+    result = loadLibFunc (pluginLib, functionName);                             \

+                                                                                \

+    if (NULL != result) {                                                       \

+        (pluginImpl)->FUNCIMPLNAME = *(FUNCIMPLTYPE*)&result;                   \

+        result = NULL;                                                          \

+    } else {                                                                    \

+        mama_log (MAMA_LOG_LEVEL_ERROR,                                         \

+                  "mamaPlugin_registerFunctions(): "                            \

+                  "Cannot load plugin, does not implement required function: [%s]",\

+                  functionName);                                                \

+        status = MAMA_STATUS_PLATFORM;                                          \

+        return status;                                                          \

+    }                                                                           \

+} while (0)

+

+/**

+ * @brief Mechanism for registering required plugin functions.

+ *

+ * Taking a function string name search the shared library handle, using the

+ * loadLibFunc portability method, for the function. If it is found, set the

+ * appropriate function pointer in the plugin struct to the result. If not

+ * log the fact that the function has not been found, and continue.

+ *

+ * @param FUNCSTRINGNAME The string function name.

+ * @param FUNCIMPLNAME The name of the function pointer in the plugin struct

+ * @param FUNCIMPLTYPE The type of the function pointer expected.

+ */

+#define REGISTER_OPTIONAL_PLUGIN_FUNCTION(FUNCSTRINGNAME,                       \

+                                          FUNCIMPLNAME,                         \

+                                          FUNCIMPLTYPE)                         \

+do {                                                                            \

+    snprintf (functionName, MAX_FUNC_STRING, "%s"#FUNCSTRINGNAME, name);        \

+    result = loadLibFunc (pluginLib, functionName);                             \

+                                                                                \

+    if (NULL != result) {                                                       \

+        (pluginImpl)->FUNCIMPLNAME = *(FUNCIMPLTYPE*)&result;                   \

+        result = NULL;                                                          \

+    } else {                                                                    \

+        mama_log (MAMA_LOG_LEVEL_FINE,                                          \

+                  "mamaPlugin_registerFunctions(): "                            \

+                  "Optional plugin function [%s] not found. Unavailable.",      \

+                  functionName);                                                \

+    }                                                                           \

+} while (0)

+

+typedef struct mamaPluginImpl_

+{

+    LIB_HANDLE          mPluginHandle;

+    char*               mPluginName;

+    mamaPluginInfo      mPluginInfo;

+

+    mamaPlugin_publisherPreSendHook     mamaPluginPublisherPreSendHook;

+    mamaPlugin_transportPostCreateHook  mamaPluginTransportPostCreateHook;

+    mamaPlugin_shutdownHook             mamaPluginShutdownHook;

+    mamaPlugin_initHook                 mamaPluginInitHook;

+

+} mamaPluginImpl;

+

+static mamaPluginImpl*      gPlugins[MAX_PLUGINS];

+static volatile int         gPluginNo = 0;

+

+/**

+ * @brief Used to load register all possible plugin functions to be

+ *        used within Mama.

+ *

+ * param[in] pluginLib

+ * param[in] name

+ * param[in] pluginImpl The plugin impl to be used

+ *

+ * @return mama_status return code can be one of:

+ *          MAMA_STATUS_OK

+ */

+mama_status

+mamaPlugin_registerFunctions   (LIB_HANDLE      pluginLib,

+                                const char*     name,

+                                mamaPluginInfo  pluginInfo,

+                                mamaPluginImpl* pluginImpl);

+

+/**

+ * @brief Used find a plugin using the library name

+ *

+ * param[in] name

+ *

+ * @return a valid mamaPluginImpl if found

+ */

+mamaPluginImpl*

+mamaPlugin_findPlugin (const char* name);

+

+/**

+ * @brief Used find a plugin using the library name

+ *

+ * param[in] pluginName

+ *

+ * @return mama_status return code can be one of:

+ *          MAMA_STATUS_OK

+ */

+mama_status

+mama_loadPlugin (const char* pluginName);

+

+

+/**

+ * Register function pointers associated with a specific plugin.

+ */

+mama_status

+mamaPlugin_registerFunctions (LIB_HANDLE      pluginLib,

+                              const char*     name,

+                              mamaPluginInfo  pluginInfo,

+                              mamaPluginImpl* pluginImpl)

+{

+    mama_status  status        = MAMA_STATUS_OK;

+    void*        result        = NULL;

+    char         functionName[MAX_FUNC_STRING];

+

+    /* Save off some informatin on the plugin */

+    pluginImpl->mPluginHandle = pluginLib;

+    pluginImpl->mPluginName = strdup(name);

+    pluginImpl->mPluginInfo = pluginInfo;

+

+

+    /* Required fuctions */

+    REGISTER_PLUGIN_FUNCTION (MamaPlugin_initHook, mamaPluginInitHook,

+                              mamaPlugin_initHook);

+    REGISTER_PLUGIN_FUNCTION (MamaPlugin_shutdownHook, mamaPluginShutdownHook,

+                              mamaPlugin_shutdownHook);

+

+

+    /* Optional fuctions */

+    REGISTER_OPTIONAL_PLUGIN_FUNCTION (MamaPlugin_publisherPreSendHook, mamaPluginPublisherPreSendHook,

+                                       mamaPlugin_publisherPreSendHook);

+    REGISTER_OPTIONAL_PLUGIN_FUNCTION (MamaPlugin_transportPostCreateHook, mamaPluginTransportPostCreateHook,

+                                       mamaPlugin_transportPostCreateHook);

+

+   return status;

+}

+

+mama_status

+mama_initPlugins(void)

+{

+    int             pluginCount      = 0;

+    const char*     prop             = NULL;

+    char            propString[MAX_PLUGIN_STRING];

+

+    for (pluginCount = 0; pluginCount < MAX_PLUGINS; pluginCount++)

+    {

+        snprintf(propString, MAX_PLUGIN_STRING,

+            PLUGIN_PROPERTY"%d",

+            pluginCount);

+

+        prop = properties_Get (mamaInternal_getProperties (), propString);

+        if (prop != NULL && strlen(prop)!= 0)

+        {

+            mama_log (MAMA_LOG_LEVEL_FINE, "mama_initPlugins(): Initialising [%s] %s", propString, prop);

+            mama_loadPlugin (prop);

+        }

+    }

+

+    mama_log (MAMA_LOG_LEVEL_FINE, "mama_initPlugins(): Initialising mamacenterprise");

+    mama_loadPlugin ("mamacenterprise");

+

+    return MAMA_STATUS_OK;

+}

+

+mama_status

+mama_loadPlugin (const char* pluginName)

+{

+    LIB_HANDLE              pluginLib       = NULL;

+    mamaPluginImpl*         pluginImpl      = NULL;

+    mama_status             status          = MAMA_STATUS_OK;

+    mamaPluginInfo          pluginInfo      = NULL;

+    char                    loadPluginName  [MAX_PLUGIN_STRING];

+    mamaPluginImpl*         aPluginImpl     = NULL;

+

+    if (!pluginName)

+        return MAMA_STATUS_NULL_ARG;

+

+    pluginImpl = mamaPlugin_findPlugin(pluginName);

+

+    /*

+     * Check to see if pluginImpl has already been loaded

+     */

+    if (pluginImpl == NULL)

+    {

+       /* The plugin name should be of the format mamaplugin<name> */

+        if (strncmp(pluginName, "mamacenterprise", MAX_PLUGIN_STRING) == 0)

+            snprintf(loadPluginName, MAX_PLUGIN_STRING,

+                "%s",

+                pluginName);

+        else

+            snprintf(loadPluginName, MAX_PLUGIN_STRING,

+                "%s%s",

+                PLUGIN_NAME,

+                pluginName);

+

+        pluginLib = openSharedLib (loadPluginName, NULL);

+

+        if (!pluginLib)

+        {

+

+           mama_log (MAMA_LOG_LEVEL_ERROR,

+                    "mama_loadPlugin(): "

+                    "Could not open plugin library [%s] [%s]",

+                    pluginName,

+                    getLibError());

+            return MAMA_STATUS_PLATFORM;

+        }

+

+        /* Create structure to hold plugin information */

+        aPluginImpl = (mamaPluginImpl*)calloc (1, sizeof(mamaPluginImpl));

+

+        status = mamaPlugin_registerFunctions (pluginLib,

+                                               pluginName,

+                                               pluginInfo,

+                                               aPluginImpl);

+

+        if (MAMA_STATUS_OK == status)

+        {

+            mama_log (MAMA_LOG_LEVEL_NORMAL,

+                     "mama_loadPlugin(): "

+                     "Sucessfully registered plugin functions for [%s]",

+                     pluginName);

+

+        }

+        else

+        {

+            mama_log (MAMA_LOG_LEVEL_WARN,

+                     "mama_loadPlugin(): "

+                     "Failed to register plugin functions for [%s]",

+                     pluginName);

+

+            closeSharedLib (aPluginImpl->mPluginHandle);

+

+            free ((char*)aPluginImpl->mPluginName);

+            free ((mamaPluginImpl*)aPluginImpl);

+

+            return status;

+        }

+

+        /* Invoke the init function */

+        status = aPluginImpl->mamaPluginInitHook (aPluginImpl->mPluginInfo);

+

+        if (MAMA_STATUS_OK == status)

+        {

+            mama_log (MAMA_LOG_LEVEL_NORMAL,

+                      "mama_loadPlugin(): Successfully run the init hook for mama plugin [%s]",

+                       aPluginImpl->mPluginName);

+        }

+        else

+        {

+            mama_log (MAMA_LOG_LEVEL_WARN,

+                      "mama_loadPlugin(): Init hook failed for mama plugin [%s]",

+                       aPluginImpl->mPluginName);

+

+            closeSharedLib (aPluginImpl->mPluginHandle);

+

+            free ((char*)aPluginImpl->mPluginName);

+            free ((mamaPluginImpl*)aPluginImpl);

+

+            return status;

+        }

+

+        /* Save off the plugin impl and increment the plugin counter */

+        gPlugins[gPluginNo] = aPluginImpl;

+        gPluginNo++;

+

+    }

+    else

+    {

+        mama_log (MAMA_LOG_LEVEL_NORMAL,

+                 "mama_loadPlugin(): "

+                 "Plugin [%s] has already been loaded and initialised",

+                 pluginName);

+    }

+

+    return MAMA_STATUS_OK;

+}

+

+mama_status

+mama_shutdownPlugins (void)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+    int          ret    = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin]->mPluginHandle != NULL)

+        {

+            /* Fire the user shutdown hook first - if one is present */

+            if (gPlugins[plugin]->mamaPluginShutdownHook != NULL)

+            {

+                gPlugins[plugin]->mamaPluginShutdownHook (gPlugins[plugin]->mPluginInfo);

+            }

+

+            ret = closeSharedLib (gPlugins[plugin]->mPluginHandle);

+            if (0!=ret)

+            {

+                 mama_log (MAMA_LOG_LEVEL_WARN,

+                            "mama_shutdownPlugins(): Failed to close Mama Plugin [%s]",

+                            gPlugins[plugin]->mPluginName);

+            }

+            else

+            {

+                 mama_log (MAMA_LOG_LEVEL_WARN,

+                            "mama_shutdownPlugins(): Mama Plugin [%s] successfully closed",

+                            gPlugins[plugin]->mPluginName);

+            }

+

+            free ((char*)gPlugins[plugin]->mPluginName);

+            free (gPlugins[plugin]);

+            gPlugins[plugin] = NULL;

+        }

+    }

+    return status;

+}

+

+mama_status

+mamaPlugin_firePublisherPreSendHook (mamaPublisher publisher, mamaMsg message)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin] != NULL)

+        {

+            if (gPlugins[plugin]->mamaPluginPublisherPreSendHook != NULL)

+            {

+                status = gPlugins[plugin]->mamaPluginPublisherPreSendHook (gPlugins[plugin]->mPluginInfo, publisher, message);

+

+                if (MAMA_STATUS_OK != status)

+                {

+                     mama_log (MAMA_LOG_LEVEL_WARN,

+                                "mamaPlugin_firePublisherPreSendHook(): Publisher pre send hook failed for mama plugin [%s]",

+                                gPlugins[plugin]->mPluginName);

+                }

+            }

+        }

+    }

+    return status;

+}

+

+mama_status

+mamaPlugin_fireTransportPostCreateHook (mamaTransport transport)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin] != NULL)

+        {

+            if (gPlugins[plugin]->mamaPluginTransportPostCreateHook != NULL)

+            {

+                status = gPlugins[plugin]->mamaPluginTransportPostCreateHook (gPlugins[plugin]->mPluginInfo, transport);

+

+                if (MAMA_STATUS_OK != status)

+                {

+                     mama_log (MAMA_LOG_LEVEL_WARN,

+                                "mamaPlugin_fireTransportPostCreateHook(): Transport post create hook failed for mama plugin [%s]",

+                                gPlugins[plugin]->mPluginName);

+                }

+            }

+        }

+    }

+    return status;

+}

+

+mama_status

+mamaPlugin_fireShutdownHook (void)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin] != NULL)

+        {

+            if (gPlugins[plugin]->mamaPluginShutdownHook != NULL)

+            {

+                status = gPlugins[plugin]->mamaPluginShutdownHook (gPlugins[plugin]->mPluginInfo);

+

+                if (MAMA_STATUS_OK != status)

+                {

+                     mama_log (MAMA_LOG_LEVEL_WARN,

+                                "mamaPlugin_fireShutdownHook(): Shutdown hook failed for mama plugin [%s]",

+                                gPlugins[plugin]->mPluginName);

+                }

+            }

+        }

+    }

+    return status;

+}

+

+mamaPluginImpl*

+mamaPlugin_findPlugin (const char* name)

+{

+    int plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin])

+        {

+            if ((strncmp(gPlugins[plugin]->mPluginName, name, MAX_PLUGIN_STRING) == 0))

+            {

+                return gPlugins[plugin];

+            }

+        }

+    }

+    return NULL;

+}

diff --git a/mama/c_cpp/src/c/plugin.h b/mama/c_cpp/src/c/plugin.h

new file mode 100644

index 0000000..df43b27

--- /dev/null

+++ b/mama/c_cpp/src/c/plugin.h

@@ -0,0 +1,59 @@

+/* $Id$

+ *

+ * OpenMAMA: The open middleware agnostic messaging API

+ * Copyright (C) 2011 NYSE Technologies, Inc.

+ *

+ * This library is free software; you can redistribute it and/or

+ * modify it under the terms of the GNU Lesser General Public

+ * License as published by the Free Software Foundation; either

+ * version 2.1 of the License, or (at your option) any later version.

+ *

+ * This library is distributed in the hope that it will be useful,

+ * but WITHOUT ANY WARRANTY; without even the implied warranty of

+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU

+ * Lesser General Public License for more details.

+ *

+ * You should have received a copy of the GNU Lesser General Public

+ * License along with this library; if not, write to the Free Software

+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA

+ * 02110-1301 USA

+ */

+

+#ifndef PluginH__

+#define PluginH__

+

+

+/**

+ * @brief Containers used within the mama plugin

+ */

+typedef mama_status (*mamaPlugin_publisherPreSendHook) (mamaPluginInfo pluginInfo, mamaPublisher publisher, mamaMsg message);

+typedef mama_status (*mamaPlugin_transportPostCreateHook) (mamaPluginInfo pluginInfo, mamaTransport transport);

+typedef mama_status (*mamaPlugin_shutdownHook) (mamaPluginInfo pluginInfo);

+typedef mama_status (*mamaPlugin_initHook) (mamaPluginInfo* pluginInfo);

+

+/**

+ * Initialize the internal plugin interface

+ *

+ * @return mama status code

+ */

+extern mama_status

+mama_initPlugins (void);

+

+/**

+ * Shutdown the internal plugin interface

+ *

+ * @return mama status code

+ */

+extern mama_status

+mama_shutdownPlugins (void);

+

+extern mama_status

+mamaPlugin_firePublisherPreSendHook (mamaPublisher publisher, mamaMsg message);

+

+extern mama_status

+mamaPlugin_fireTransportPostCreateHook (mamaTransport transport);

+

+extern mama_status

+mamaPlugin_fireShutdownHook (void);

+

+#endif /* PluginH__ */

diff --git a/mama/c_cpp/src/c/publisher.c b/mama/c_cpp/src/c/publisher.c

index 4612d17..6a7f92d 100644

--- a/mama/c_cpp/src/c/publisher.c

+++ b/mama/c_cpp/src/c/publisher.c

@@ -23,6 +23,7 @@

#include "mama/publisher.h"

 #include "bridge.h"

+#include "plugin.h"

#include "throttle.h"

#include "transportimpl.h"

@@ -245,6 +246,15 @@ mamaPublisher_send (mamaPublisher publisher,

     if (!impl->mMamaPublisherBridgeImpl) return MAMA_STATUS_INVALID_ARG;

     if (!impl->mBridgeImpl) return MAMA_STATUS_NO_BRIDGE_IMPL;

+    /* Calling plugin hook */

+    status = mamaPlugin_firePublisherPreSendHook (publisher, msg);

+    if (MAMA_STATUS_OK != status)

+    {

+        mama_log (MAMA_LOG_LEVEL_ERROR,

+                  "mamaPublisher_send(): PublisherPreSendHook failed. Not sending message.");

+        return status;

+    }

+

     status = impl->mBridgeImpl->bridgeMamaPublisherSend

         (impl->mMamaPublisherBridgeImpl,

          msg);

diff --git a/mama/c_cpp/src/c/transport.c b/mama/c_cpp/src/c/transport.c

index 339dbc0..622f47d 100644

--- a/mama/c_cpp/src/c/transport.c

+++ b/mama/c_cpp/src/c/transport.c

@@ -25,6 +25,7 @@

 #include "mama/mama.h"

#include "throttle.h"

+#include "plugin.h"

#include "list.h"

#include "transportimpl.h"

#include "bridge.h"

@@ -876,10 +877,22 @@ mamaTransport_create (mamaTransport transport,

     if ((!self->mDisableRefresh) && (!mamaTransportInternal_disableRefreshes(name)))

     {

-        return refreshTransport_create (&self->mRefreshTransport,

-                                    (mamaTransport)self,

-                                    self->mListeners,

-                                    self->mBridgeImpl);

+        status = refreshTransport_create (&self->mRefreshTransport,

+                                          (mamaTransport)self,

+                                          self->mListeners,

+                                          self->mBridgeImpl);

+

+        if (MAMA_STATUS_OK != status)

+            return status;

+    }

+

+    /* Calling plugin hook*/

+    status = mamaPlugin_fireTransportPostCreateHook (transport);

+    if (MAMA_STATUS_OK != status)

+    {

+        mama_log (MAMA_LOG_LEVEL_ERROR,

+                  "mamaTransport_create(): TransportPostCreateHook failed with a status of %s",

+                   mamaStatus_stringForStatus(status));

     }

     return MAMA_STATUS_OK;

--

1.7.1

 

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


[PATCH 3/3] [MAMAC] mamaPublisherImpl_getTransportImpl() accessor

Gary Molloy <g.molloy@...>
 

Testing Strategy:-

Not middleware specific. 

 

To test this simply add the following lines to the mamapublisherc.c example application within the createPublisher() function:


mamaTransport tport = mamaPublisherImpl_getTransportImpl (gPublisher);
const char* myTportName = NULL;
mamaTransport_getName (tport, &myTportName);
printf ("TEST: mamaPublisherImpl_getTransportImpl: %s\n", myTportName);


Running the modified mamapublisherc, for example:


./mamapublisherc -m wmw -tport tcp_sub

 

You can expect to see the following output:

 

Created inbound subscription.
TEST: mamaPublisherImpl_getTransportImpl: tcp_sub <------- returns transport

Publishing message 0 to MAMA_TOPIC. msg={MdMsgStatus[2]=0,MdSeqNum[10]=0,PublisherTopic[10002]=MAMA_TOPIC}
Publishing message 1 to MAMA_TOPIC. msg={MdMsgStatus[2]=0,MdSeqNum[10]=1,PublisherTopic[10002]=MAMA_TOPIC}

 

 

=========================================================================================================

 

 

From 8d3b6b92deacc0aae262984245fca5c70923d2b3 Mon Sep 17 00:00:00 2001

From: Gary Molloy <g.molloy@...>

Date: Tue, 21 Apr 2015 10:09:53 -0400

Subject: [PATCH 3/3] [MAMAC] mamaPublisherImpl_getTransportImpl() accessor

 

Added a new mamaPublisherImpl_getTransportImpl() accessor method

that will return the mamaTransport from the mamaPublisher.

 

Signed-off-by: Gary Molloy <g.molloy@...>

---

mama/c_cpp/src/c/publisher.c     |    7 +++++++

mama/c_cpp/src/c/publisherimpl.h |    3 +++

2 files changed, 10 insertions(+), 0 deletions(-)

 

diff --git a/mama/c_cpp/src/c/publisher.c b/mama/c_cpp/src/c/publisher.c

index 281b054..e09017c 100644

--- a/mama/c_cpp/src/c/publisher.c

+++ b/mama/c_cpp/src/c/publisher.c

@@ -600,4 +600,11 @@ mama_status mamaPublisherImpl_clearTransport (mamaPublisher publisher)

     return mamaPublisherImpl_destroy((mamaPublisherImpl *)publisher);

}

+mamaTransport

+mamaPublisherImpl_getTransportImpl (mamaPublisher publisher)

+{

+    mamaPublisherImpl* impl   = (mamaPublisherImpl*)publisher;

+    if (!impl) return NULL;

+    return impl->mTport;

+}

diff --git a/mama/c_cpp/src/c/publisherimpl.h b/mama/c_cpp/src/c/publisherimpl.h

index f336fd8..b9b9c43 100644

--- a/mama/c_cpp/src/c/publisherimpl.h

+++ b/mama/c_cpp/src/c/publisherimpl.h

@@ -44,6 +44,9 @@ mamaPublisher_sendFromInboxByIndex (mamaPublisher publisher,

 mama_status mamaPublisherImpl_clearTransport (mamaPublisher publisher);

+extern mamaTransport

+mamaPublisherImpl_getTransportImpl (mamaPublisher publisher);

+

#if defined(__cplusplus)

}

#endif

--

1.7.1

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


[PATCH 2/3] [MAMAC] Add ability to turn on/off entitlements on a per bridge basis

Gary Molloy <g.molloy@...>
 

Testing Strategy:-

Not middleware or O/S specific. 

This will require an entitled build and an entitlement server.

 

There are 4 tests that you can run using a standard mamalistenc. 

For each test you will have to add/modify the following within your middleware bridge code (when you create your bridge):

 

1) mamaBridgeImpl_setReadOnlyProperty ((mamaBridge)impl, "mama.<middleware>.entitlements.deferred", "true", 1);
- Deferred true, will run will addition entitlement logging.


2) mamaBridgeImpl_setReadOnlyProperty ((mamaBridge)impl, "mama.<middleware>.entitlements.deferred", "false", 1); <------default now in <middleware> bridge (This will be the standard setting)
- Deferred false, will connect to the normal entitlement server.


3) mamaBridgeImpl_setReadOnlyProperty ((mamaBridge)impl, "mama.TEST.entitlements.deferred", "true", 1); <---- Invalid property string (will not matter if deferred or not, will log error at very top & fail to connect to entitlements)
- This will fail whether set to deferred or not, logging seen at top is will look like this:-

mamaBridgeImpl_setProperty(): Unknown property string [mama.TEST.entitlements.deferred] entered.

 

4) mamaBridgeImpl_setReadOnlyProperty ((mamaBridge)impl, "mama.<middleware>.entitlements.deferred", "true", 1);
      mamaBridgeImpl_setReadOnlyProperty ((mamaBridge)impl, "mama.<middleware>.entitlements.deferred", "false", 1); <-----property called twice (true/false, or false/true).


With false/true -
Should follow the standard logic of it not being deferred - logging at top will read
“mamaBridgeImpl_setProperty(): Bridge property is read only, property can not be set. “
But this should connect to the normal entitlement server.

With true/false -
Should follow the standard logic if deferred is true - logging at top will look like this:
“mamaBridgeImpl_setProperty(): Bridge property is read only, property can not be set. “

But will continue.

 

Further documentation on the this feature will be available shortly which will provide more details on the feature which will include the various logging you can expect etc...

 

 

=========================================================================================================

 

From d016afce1b1cc4ffb6d65c1d032ddea41d56b62e Mon Sep 17 00:00:00 2001

From: Gary Molloy <g.molloy@...>

Date: Tue, 21 Apr 2015 09:56:52 -0400

Subject: [PATCH 2/3] [MAMAC] Add ability to turn on/off entitlements on a per bridge basis

 

Signed-off-by: Gary Molloy <g.molloy@...>

---

mama/c_cpp/src/c/bridge.c              |   67 ++++++++++++++++++++++++++++++++

mama/c_cpp/src/c/bridge.h              |   20 +++++++++

mama/c_cpp/src/c/bridge/qpid/bridge.c  |    2 +

mama/c_cpp/src/c/listenermsgcallback.c |   15 ++++++-

mama/c_cpp/src/c/mama.c                |   26 ++++++++++++

mama/c_cpp/src/c/subscription.c        |   38 ++++++++++++++----

6 files changed, 157 insertions(+), 11 deletions(-)

 

diff --git a/mama/c_cpp/src/c/bridge.c b/mama/c_cpp/src/c/bridge.c

index 01cbcc5..dd42a55 100644

--- a/mama/c_cpp/src/c/bridge.c

+++ b/mama/c_cpp/src/c/bridge.c

@@ -20,8 +20,12 @@

  */

 #include <mama/mama.h>

+#include <wombat/strutils.h>

#include "bridge.h"

+#define MAX_PROP_STRING                   1000

+#define PROP_NAME_ENTITLEMENTS_DEFERRED   "entitlements.deferred"

+

int mamaBridgeImpl_getDefaultQueueTimeout(void)

{

     /* Returns. */

@@ -143,3 +147,66 @@ mamaBridgeImpl_stopInternalEventQueue (mamaBridge bridgeImpl)

    

     return MAMA_STATUS_OK;

}

+

+mama_status

+mamaBridgeImpl_setReadOnlyProperty (mamaBridge bridgeImpl, const char* property, const char* value, mama_bool_t readOnly)

+{

+    mamaBridgeImpl_setProperty(bridgeImpl, property, value);

+    bridgeImpl->mEntitleReadOnly = 1;

+    return MAMA_STATUS_OK;

+}

+

+mama_status

+mamaBridgeImpl_setProperty (mamaBridge bridgeImpl, const char* property, const char* value)

+{

+    char propString[MAX_PROP_STRING];

+    int retVal;

+

+    mamaBridgeImpl* impl = (mamaBridgeImpl*)bridgeImpl;

+

+    /* Check for mama.middleware.entitlements_deferred first */

+    retVal = snprintf(propString, MAX_PROP_STRING,

+        "mama.%s.%s",

+        impl->bridgeGetName(),

+        PROP_NAME_ENTITLEMENTS_DEFERRED);

+

+    if(0 == strcmp(property, propString))

+    {

+        if (1 == bridgeImpl->mEntitleReadOnly)

+        {

+            mama_log (MAMA_LOG_LEVEL_WARN, "mamaBridgeImpl_setProperty(): "

+                      "Bridge is read only, property can not be set.");

+            return MAMA_STATUS_INVALID_ARG;

+        }

+        else

+        {

+            if (strtobool(value))

+                bridgeImpl->mEntitleDeferred = 1;

+            else

+                bridgeImpl->mEntitleDeferred = 0;

+        }

+    }

+    else

+    {

+        mama_log (MAMA_LOG_LEVEL_WARN, "mamaBridgeImpl_setProperty(): "

+            "Unknown property string [%s] entered.", property);

+        return MAMA_STATUS_INVALID_ARG;

+    }

+    return MAMA_STATUS_OK;

+}

+

+const char*

+mamaBridgeImpl_getProperty (mamaBridge bridgeImpl, const char* property)

+{

+    return NULL;

+}

+

+mama_bool_t

+mamaBridgeImpl_areEntitlementsDeferred (mamaBridge bridgeImpl)

+{

+    if (bridgeImpl)

+    {

+        return bridgeImpl->mEntitleDeferred;

+    }

+    return 0;

+}

diff --git a/mama/c_cpp/src/c/bridge.h b/mama/c_cpp/src/c/bridge.h

index c079819..9bcd204 100644

--- a/mama/c_cpp/src/c/bridge.h

+++ b/mama/c_cpp/src/c/bridge.h

@@ -715,6 +715,10 @@ typedef struct mamaBridgeImpl_

        used when getting the default queue */

     void*     mCppCallback;

+    /*Used in bridge.c*/

+    mama_bool_t  mEntitleDeferred;

+    mama_bool_t  mEntitleReadOnly;

+

     /*Used in mama.c*/

     bridge_open                                                 bridgeOpen;

     bridge_close                                                 bridgeClose;

@@ -873,6 +877,22 @@ MAMAExpDLL

mama_status

mamaBridgeImpl_stopInternalEventQueue (mamaBridge bridgeImpl);

+MAMAExpDLL

+extern mama_status

+mamaBridgeImpl_setReadOnlyProperty (mamaBridge bridgeImpl, const char* property, const char* value, mama_bool_t readOnly);

+

+MAMAExpDLL

+extern mama_status

+mamaBridgeImpl_setProperty (mamaBridge bridgeImpl, const char* property, const char* value);

+

+MAMAExpDLL

+extern const char*

+mamaBridgeImpl_getProperty (mamaBridge bridgeImpl, const char* property);

+

+MAMAExpDLL

+extern mama_bool_t

+mamaBridgeImpl_areEntitlementsDeferred (mamaBridge bridgeImpl);

+

#if defined(__cplusplus)

}

#endif

diff --git a/mama/c_cpp/src/c/bridge/qpid/bridge.c b/mama/c_cpp/src/c/bridge/qpid/bridge.c

index 4ab74c7..233c8ec 100644

--- a/mama/c_cpp/src/c/bridge/qpid/bridge.c

+++ b/mama/c_cpp/src/c/bridge/qpid/bridge.c

@@ -85,6 +85,8 @@ void qpidBridge_createImpl (mamaBridge* result)

     /* Return the newly created bridge */

     *result = (mamaBridge) bridge;

+

+    mamaBridgeImpl_setReadOnlyProperty ((mamaBridge)bridge, "mama.qpid.entitlements.deferred", "false", 1);

}

 mama_status

diff --git a/mama/c_cpp/src/c/listenermsgcallback.c b/mama/c_cpp/src/c/listenermsgcallback.c

index 85c3bf2..caeaa56 100644

--- a/mama/c_cpp/src/c/listenermsgcallback.c

+++ b/mama/c_cpp/src/c/listenermsgcallback.c

@@ -86,7 +86,8 @@ listenerMsgCallback_create( listenerMsgCallback *result,

     msgCallback* callback = (msgCallback*)calloc( 1, sizeof( msgCallback ) );

 #ifdef WITH_ENTITLEMENTS  /* No listener creation without a client. */

-    if( gEntitlementClient == 0 )

+    mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+    if( gEntitlementClient == 0 && !(mamaBridgeImpl_areEntitlementsDeferred(bridge)))

     {

         return MAMA_ENTITLE_NO_SERVERS_SPECIFIED;

     }

@@ -625,7 +626,6 @@ static void handleNoSubscribers (msgCallback *callback,

static int

checkEntitlement( msgCallback *callback, mamaMsg msg, SubjectContext* ctx )

{

-

#ifdef WITH_ENTITLEMENTS

     int result = 0;

     int32_t value;

@@ -634,6 +634,17 @@ checkEntitlement( msgCallback *callback, mamaMsg msg, SubjectContext* ctx )

         return 1;

     }

+    mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(self->mSubscription);

+

+    if (bridge && (mamaBridgeImpl_areEntitlementsDeferred(bridge)))

+    {

+        mama_log (MAMA_LOG_LEVEL_FINER,

+                       "Deferred checking injected entitlement to %s bridge [%p]",

+                        bridge->bridgeGetName(), bridge);

+        ctx->mEntitlementAlreadyVerified = 1;

+        return 1;

+    }

+

     if( MAMA_STATUS_OK == mamaMsg_getEntitleCode( msg,

                                                   &value ) )

     {

diff --git a/mama/c_cpp/src/c/mama.c b/mama/c_cpp/src/c/mama.c

index 5e31856..1815bc0 100644

--- a/mama/c_cpp/src/c/mama.c

+++ b/mama/c_cpp/src/c/mama.c

@@ -1707,6 +1707,8 @@ enableEntitlements (const char **servers)

     const char* altUserId;

     const char* altIp;

     const char* site;

+    mamaMiddleware middleware = 0;

+    int entitlementsRequired = 0; /*boolean*/

 

     if (gEntitlementClient != 0)

@@ -1715,6 +1717,30 @@ enableEntitlements (const char **servers)

         gEntitlementClient = 0;

     }

+    for (middleware=0; middleware != MAMA_MIDDLEWARE_MAX; ++middleware)

+    {

+        mamaBridgeImpl* impl = (mamaBridgeImpl*) gImpl.myBridges [middleware];

+        if (impl)

+        {

+            /* Check if entitlements are deferred to bridge */

+            if (mamaBridgeImpl_areEntitlementsDeferred(impl) == 1)

+            {

+                mama_log (MAMA_LOG_LEVEL_WARN,

+                    "Entitlements deferred on %s bridge.",

+                    mamaMiddleware_convertToString (middleware));

+            }

+            else

+            {

+                /* Entitlements are not deferred, continue with entitlement checking */

+                entitlementsRequired = 1;

+            }

+        }

+    }

+

+    /* Entitlements are deferred, do not continue with entitlement checking */

+    if (entitlementsRequired==0)

+        return MAMA_STATUS_OK;

+

     if (servers == NULL)

     {

         if (NULL == (servers = mdrvImpl_ParseServersProperty()))

diff --git a/mama/c_cpp/src/c/subscription.c b/mama/c_cpp/src/c/subscription.c

index 1f78746..42ad347 100644

--- a/mama/c_cpp/src/c/subscription.c

+++ b/mama/c_cpp/src/c/subscription.c

@@ -450,11 +450,19 @@ mamaSubscription_setupBasic (

                   "Could not get bridge impl from transport.");

         return MAMA_STATUS_NO_BRIDGE_IMPL;

     }

-       

+

#ifdef WITH_ENTITLEMENTS

-    self->mSubjectContext.mOeaSubscription = oeaClient_newSubscription (&entitlementStatus, gEntitlementClient);

+    mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+    if (gEntitlementClient == 0 || mamaBridgeImpl_areEntitlementsDeferred(bridge))

+    {

+        mama_log (MAMA_LOG_LEVEL_FINER,

+                    "Entitlements checking at subscription creation deferred to %s bridge [%p]",

+                    bridge->bridgeGetName(), bridge);

+    }

+    else

+        self->mSubjectContext.mOeaSubscription = oeaClient_newSubscription (&entitlementStatus, gEntitlementClient);

#endif

-   

+

     /*Up from entitlement check based on string compare on symbol*/

     if (!isEntitledToSymbol (source, symbol, self))

     {

@@ -476,7 +484,9 @@ mamaSubscription_setupBasic (

             if (!self->mRequiresInitial) return MAMA_STATUS_INVALID_ARG;

             subscMsgType = MAMA_SUBSC_SNAPSHOT;

#ifdef WITH_ENTITLEMENTS

-            oeaSubscription_setIsSnapshot (self->mSubjectContext.mOeaSubscription, 1);

+            mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+            if (!(gEntitlementClient == 0 || mamaBridgeImpl_areEntitlementsDeferred(bridge)))

+                oeaSubscription_setIsSnapshot (self->mSubjectContext.mOeaSubscription, 1);

#endif

             break;

         case MAMA_SERVICE_LEVEL_CONFLATED:/*fall through*/

@@ -1184,7 +1194,9 @@ mamaSubscription_getSubjectContext (mamaSubscription subscription,

         msgUtils_getIssueSymbol (msg, &issueSymbol);

         context->mSymbol = copyString (issueSymbol);

         #ifdef WITH_ENTITLEMENTS

-        context->mOeaSubscription = oeaClient_newSubscription (&entitlementStatus, gEntitlementClient);

+        mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+        if (!(gEntitlementClient == 0 || mamaBridgeImpl_areEntitlementsDeferred(bridge)))

+            context->mOeaSubscription = oeaClient_newSubscription (&entitlementStatus, gEntitlementClient);

         #endif

 

         wtable_insert (self->mSubjects, (char*)sendSubject, (void*)context);

@@ -2048,7 +2060,9 @@ mamaSubscription_processTportMsg( mamaSubscription subscription,

     }

 #ifdef WITH_ENTITLEMENTS

-    mamaMsg_getEntitleCode (msg, &entitleCode);

+    mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+    if (!(gEntitlementClient == 0 || mamaBridgeImpl_areEntitlementsDeferred(bridge)))

+        mamaMsg_getEntitleCode (msg, &entitleCode);

#endif

     if (entitleCode == 0)

     {

@@ -2100,7 +2114,9 @@ mamaSubscription_processWildCardMsg( mamaSubscription subscription,

     }

 #ifdef WITH_ENTITLEMENTS

-    mamaMsg_getEntitleCode (msg, &entitleCode);

+    mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+    if (!(gEntitlementClient == 0 || mamaBridgeImpl_areEntitlementsDeferred(bridge)))

+        mamaMsg_getEntitleCode (msg, &entitleCode);

#endif

     if (entitleCode == 0)

     {

@@ -2173,7 +2189,9 @@ mamaSubscription_processMsg (mamaSubscription subscription, mamaMsg msg)

     {

         int32_t entitleCode = 0;

#ifdef WITH_ENTITLEMENTS

-        mamaMsg_getEntitleCode (msg, &entitleCode);

+        mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+        if (!(gEntitlementClient == 0 || mamaBridgeImpl_areEntitlementsDeferred(bridge)))

+            mamaMsg_getEntitleCode (msg, &entitleCode);

#endif

         if (entitleCode == 0)

         {

@@ -2412,7 +2430,9 @@ isEntitledToSymbol (const char *source, const char*symbol, mamaSubscription subs

     snprintf (subject, WOMBAT_SUBJECT_MAX, "%s.%s", source, symbol);

-    if (gEntitlementClient == 0) /* Not enforcing entitlements. */

+    mamaBridgeImpl* bridge = mamaSubscription_getBridgeImpl(subscription);

+

+    if (gEntitlementClient == 0 || mamaBridgeImpl_areEntitlementsDeferred(bridge)) /* Not enforcing entitlements. */

     {

         return 1;

     }

--

1.7.1

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


[PATCH 1/3] [MAMAC] mamaPlugin Feature

Gary Molloy <g.molloy@...>
 

Testing Strategy:-

Not middleware or O/S specific. 
 

Using the attached template you can create your own plugin and load it in via your mama.properties, for example:

 

mama.plugin.name_0=template

 

You can run up a mamaPublisher to test your plugin, for example:

 

mamapublisherc -tport pub -m wmw -l MAMA_INBOUND_TOPIC -v -v -v -v -v

 

You can expect to see output similar this if your plugin was loaded successfully:

 

2015-04-09 13:45:02: (f9018280) : mama_initPlugins(): Initialising [mama.plugin.name_0] template 

2015-04-09 13:45:02: (f9018280) : mama_loadPlugin(): Sucessfully registered plugin functions for [template] 

2015-04-09 13:45:02: (f9018280) : mama_loadPlugin(): Successfully run the init hook for mama plugin [template]  

 

If your plugin was not loaded successful you can expect to see output similar to this:

 

2015-04-09 13:45:02: (f9018280) : mama_initPlugins(): Initialising [mama.plugin.name_0] template 

2015-04-09 13:45:02: (f9018280) : mamaPlugin_registerFunctions(): Cannot load plugin, does not implement required function: [templateMamaPlugin_shutdownHook] 

2015-04-09 13:45:02: (f9018280) : mama_loadPlugin(): Failed to register plugin functions for [template] 

 

Further documentation on the mamaPlugin feature will be available shortly which will provide more details on the feature which will include the various hooks available etc...

 

 

=========================================================================================================

 

From e2a6a3f9c5750cee780cb6185a1310b4e4b70629 Mon Sep 17 00:00:00 2001

From: Gary Molloy <g.molloy@...>

Date: Thu, 23 Apr 2015 11:22:14 -0400

Subject: [PATCH 1/3] [MAMAC] mamaPlugin Feature

 

A new feature, mamaPlugin has been introduced that will allow you to

create and run your own code within 'hooks' into the MAMA codebase.

 

A mamaPlugin is essentially a shared object.

 

Signed-off-by: Gary Molloy <g.molloy@...>

---

mama/c_cpp/src/c/Makefile.am    |    1 +

mama/c_cpp/src/c/SConscript     |    1 +

mama/c_cpp/src/c/SConscript.win |    1 +

mama/c_cpp/src/c/mama.c         |   10 +-

mama/c_cpp/src/c/mama/types.h   |   10 +

mama/c_cpp/src/c/plugin.c       |  472 +++++++++++++++++++++++++++++++++++++++

mama/c_cpp/src/c/plugin.h       |   59 +++++

mama/c_cpp/src/c/publisher.c    |   10 +

mama/c_cpp/src/c/transport.c    |   21 ++-

9 files changed, 579 insertions(+), 6 deletions(-)

create mode 100644 mama/c_cpp/src/c/plugin.c

create mode 100644 mama/c_cpp/src/c/plugin.h

 

diff --git a/mama/c_cpp/src/c/Makefile.am b/mama/c_cpp/src/c/Makefile.am

index c831ce4..318c353 100644

--- a/mama/c_cpp/src/c/Makefile.am

+++ b/mama/c_cpp/src/c/Makefile.am

@@ -131,6 +131,7 @@ libmama_la_SOURCES = \

     mamaStrUtils.h \

     mamaStrUtils.c \

               marketdata.c \

+    plugin.c \

     middleware.c \

               msg.c \

               msgfield.c \

diff --git a/mama/c_cpp/src/c/SConscript b/mama/c_cpp/src/c/SConscript

index 4362f50..f47e18c 100644

--- a/mama/c_cpp/src/c/SConscript

+++ b/mama/c_cpp/src/c/SConscript

@@ -98,6 +98,7 @@ libmama_sources = \

     mama.c

     mamaStrUtils.c

               marketdata.c

+    plugin.c

     middleware.c

               msg.c

               msgfield.c

diff --git a/mama/c_cpp/src/c/SConscript.win b/mama/c_cpp/src/c/SConscript.win

index 77e3e18..9a6ff05 100644

--- a/mama/c_cpp/src/c/SConscript.win

+++ b/mama/c_cpp/src/c/SConscript.win

@@ -50,6 +50,7 @@ dqpublishermanager.c

inbox.c

msgtype.c

msgutils.c

+plugin.c

senderId.c

reservedfields.c

subscription.c

diff --git a/mama/c_cpp/src/c/mama.c b/mama/c_cpp/src/c/mama.c

index 33a0768..5e31856 100644

--- a/mama/c_cpp/src/c/mama.c

+++ b/mama/c_cpp/src/c/mama.c

@@ -35,6 +35,7 @@

#include <payloadbridge.h>

#include <property.h>

#include <platform.h>

+#include <plugin.h>

 #include "fileutils.h"

#include "reservedfieldsimpl.h"

@@ -42,7 +43,6 @@

#include <mama/stat.h>

#include <mama/statfields.h>

#include <statsgeneratorinternal.h>

-#include <statsgeneratorinternal.h>

#include <mama/statscollector.h>

#include "transportimpl.h"

@@ -740,6 +740,9 @@ mama_openWithPropertiesCount (const char* path,

                               }

     }

+    /* This will initialise all plugins */

+    mama_initPlugins();

+

     prop = properties_Get (gProperties, "mama.catchcallbackexceptions.enable");

     if (prop != NULL && strtobool(prop))

     {

@@ -1253,9 +1256,12 @@ mama_closeCount (unsigned int* count)

                 gImpl.myPayloadLibraries[(uint8_t)payload] = NULL;

             }

         }

-       

+

        gDefaultPayload = NULL;

+       /* This will shutdown all plugins */

+       mama_shutdownPlugins();

+

         /* Look for a bridge for each of the middlewares and close them */

         for (middleware = 0; middleware != MAMA_MIDDLEWARE_MAX; ++middleware)

         {

diff --git a/mama/c_cpp/src/c/mama/types.h b/mama/c_cpp/src/c/mama/types.h

index 50d6608..03f6da8 100644

--- a/mama/c_cpp/src/c/mama/types.h

+++ b/mama/c_cpp/src/c/mama/types.h

@@ -74,6 +74,16 @@ typedef struct mamaBridgeImpl_* mamaBridge;

typedef struct mamaPayloadBridgeImpl_* mamaPayloadBridge;

 /**

+ * @brief Container for holding a mama plugin object

+ */

+typedef struct mamaPluginImpl_*     mamaPlugin;

+

+/**

+ * @brief Container for holding a mama plugin info

+ */

+typedef void*       mamaPluginInfo;

+

+/**

  * Flexible date/time format

  */

typedef   mama_u64_t*   mamaDateTime;

diff --git a/mama/c_cpp/src/c/plugin.c b/mama/c_cpp/src/c/plugin.c

new file mode 100644

index 0000000..b6475fe

--- /dev/null

+++ b/mama/c_cpp/src/c/plugin.c

@@ -0,0 +1,472 @@

+/* $Id$

+ *

+ * OpenMAMA: The open middleware agnostic messaging API

+ * Copyright (C) 2011 NYSE Technologies, Inc.

+ *

+ * This library is free software; you can redistribute it and/or

+ * modify it under the terms of the GNU Lesser General Public

+ * License as published by the Free Software Foundation; either

+ * version 2.1 of the License, or (at your option) any later version.

+ *

+ * This library is distributed in the hope that it will be useful,

+ * but WITHOUT ANY WARRANTY; without even the implied warranty of

+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU

+ * Lesser General Public License for more details.

+ *

+ * You should have received a copy of the GNU Lesser General Public

+ * License along with this library; if not, write to the Free Software

+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA

+ * 02110-1301 USA

+ */

+

+#include <string.h>

+#include <stdio.h>

+#include <limits.h>

+

+#include "wombat/port.h"

+#include "wombat/environment.h"

+#include "wombat/strutils.h"

+#include <wombat/wtable.h>

+

+#include <mama/mama.h>

+#include <mama/error.h>

+#include <mamainternal.h>

+#include <mama/version.h>

+#include <property.h>

+#include <plugin.h>

+#include <platform.h>

+

+#define PLUGIN_PROPERTY "mama.plugin.name_"

+#define PLUGIN_NAME "mamaplugin"

+

+#define MAX_PLUGINS 100

+#define MAX_PLUGIN_STRING 1024

+

+#define MAX_FUNC_STRING 256

+

+/**

+ * @brief Mechanism for registering required plugin functions.

+ *

+ * Taking a function string name search the shared library handle, using the

+ * loadLibFunc portability method, for the function. If it is found, set the

+ * appropriate function pointer in the plugin struct to the result. If not,

+ * log as an error the fact that the funciton cannot be found, and return

+ * MAMA_STATUS_PLATFORM

+ *

+ * @param FUNCSTRINGNAME The string function name.

+ * @param FUNCIMPLNAME The name of the function pointer in the plugin struct

+ * @param FUNCIMPLTYPE The type of the function pointer expected.

+ */

+#define REGISTER_PLUGIN_FUNCTION(FUNCSTRINGNAME, FUNCIMPLNAME, FUNCIMPLTYPE)    \

+do {                                                                            \

+    snprintf (functionName, MAX_FUNC_STRING, "%s"#FUNCSTRINGNAME, name);        \

+    result = loadLibFunc (pluginLib, functionName);                             \

+                                                                                \

+    if (NULL != result) {                                                       \

+        (pluginImpl)->FUNCIMPLNAME = *(FUNCIMPLTYPE*)&result;                   \

+        result = NULL;                                                          \

+    } else {                                                                    \

+        mama_log (MAMA_LOG_LEVEL_ERROR,                                         \

+                  "mamaPlugin_registerFunctions(): "                            \

+                  "Cannot load plugin, does not implement required function: [%s]",\

+                  functionName);                                                \

+        status = MAMA_STATUS_PLATFORM;                                          \

+        return status;                                                          \

+    }                                                                           \

+} while (0)

+

+/**

+ * @brief Mechanism for registering required plugin functions.

+ *

+ * Taking a function string name search the shared library handle, using the

+ * loadLibFunc portability method, for the function. If it is found, set the

+ * appropriate function pointer in the plugin struct to the result. If not

+ * log the fact that the function has not been found, and continue.

+ *

+ * @param FUNCSTRINGNAME The string function name.

+ * @param FUNCIMPLNAME The name of the function pointer in the plugin struct

+ * @param FUNCIMPLTYPE The type of the function pointer expected.

+ */

+#define REGISTER_OPTIONAL_PLUGIN_FUNCTION(FUNCSTRINGNAME,                       \

+                                          FUNCIMPLNAME,                         \

+                                          FUNCIMPLTYPE)                         \

+do {                                                                            \

+    snprintf (functionName, MAX_FUNC_STRING, "%s"#FUNCSTRINGNAME, name);        \

+    result = loadLibFunc (pluginLib, functionName);                             \

+                                                                                \

+    if (NULL != result) {                                                       \

+        (pluginImpl)->FUNCIMPLNAME = *(FUNCIMPLTYPE*)&result;                   \

+        result = NULL;                                                          \

+    } else {                                                                    \

+        mama_log (MAMA_LOG_LEVEL_FINE,                                          \

+                  "mamaPlugin_registerFunctions(): "                            \

+                  "Optional plugin function [%s] not found. Unavailable.",      \

+                  functionName);                                                \

+    }                                                                           \

+} while (0)

+

+typedef struct mamaPluginImpl_

+{

+    LIB_HANDLE          mPluginHandle;

+    char*               mPluginName;

+    mamaPluginInfo      mPluginInfo;

+

+    mamaPlugin_publisherPreSendHook     mamaPluginPublisherPreSendHook;

+    mamaPlugin_transportPostCreateHook  mamaPluginTransportPostCreateHook;

+    mamaPlugin_shutdownHook             mamaPluginShutdownHook;

+    mamaPlugin_initHook                 mamaPluginInitHook;

+

+} mamaPluginImpl;

+

+static mamaPluginImpl*      gPlugins[MAX_PLUGINS];

+static volatile int         gPluginNo = 0;

+

+/**

+ * @brief Used to load register all possible plugin functions to be

+ *        used within Mama.

+ *

+ * param[in] pluginLib

+ * param[in] name

+ * param[in] pluginImpl The plugin impl to be used

+ *

+ * @return mama_status return code can be one of:

+ *          MAMA_STATUS_OK

+ */

+mama_status

+mamaPlugin_registerFunctions   (LIB_HANDLE      pluginLib,

+                                const char*     name,

+                                mamaPluginInfo  pluginInfo,

+                                mamaPluginImpl* pluginImpl);

+

+/**

+ * @brief Used find a plugin using the library name

+ *

+ * param[in] name

+ *

+ * @return a valid mamaPluginImpl if found

+ */

+mamaPluginImpl*

+mamaPlugin_findPlugin (const char* name);

+

+/**

+ * @brief Used find a plugin using the library name

+ *

+ * param[in] pluginName

+ *

+ * @return mama_status return code can be one of:

+ *          MAMA_STATUS_OK

+ */

+mama_status

+mama_loadPlugin (const char* pluginName);

+

+

+/**

+ * Register function pointers associated with a specific plugin.

+ */

+mama_status

+mamaPlugin_registerFunctions (LIB_HANDLE      pluginLib,

+                              const char*     name,

+                              mamaPluginInfo  pluginInfo,

+                              mamaPluginImpl* pluginImpl)

+{

+    mama_status  status        = MAMA_STATUS_OK;

+    void*        result        = NULL;

+    char         functionName[MAX_FUNC_STRING];

+

+    /* Save off some informatin on the plugin */

+    pluginImpl->mPluginHandle = pluginLib;

+    pluginImpl->mPluginName = strdup(name);

+    pluginImpl->mPluginInfo = pluginInfo;

+

+

+    /* Required fuctions */

+    REGISTER_PLUGIN_FUNCTION (MamaPlugin_initHook, mamaPluginInitHook,

+                              mamaPlugin_initHook);

+    REGISTER_PLUGIN_FUNCTION (MamaPlugin_shutdownHook, mamaPluginShutdownHook,

+                              mamaPlugin_shutdownHook);

+

+

+    /* Optional fuctions */

+    REGISTER_OPTIONAL_PLUGIN_FUNCTION (MamaPlugin_publisherPreSendHook, mamaPluginPublisherPreSendHook,

+                                       mamaPlugin_publisherPreSendHook);

+    REGISTER_OPTIONAL_PLUGIN_FUNCTION (MamaPlugin_transportPostCreateHook, mamaPluginTransportPostCreateHook,

+                                       mamaPlugin_transportPostCreateHook);

+

+   return status;

+}

+

+mama_status

+mama_initPlugins(void)

+{

+    int             pluginCount      = 0;

+    const char*     prop             = NULL;

+    char            propString[MAX_PLUGIN_STRING];

+

+    for (pluginCount = 0; pluginCount < MAX_PLUGINS; pluginCount++)

+    {

+        snprintf(propString, MAX_PLUGIN_STRING,

+            PLUGIN_PROPERTY"%d",

+            pluginCount);

+

+        prop = properties_Get (mamaInternal_getProperties (), propString);

+        if (prop != NULL && strlen(prop)!= 0)

+        {

+            mama_log (MAMA_LOG_LEVEL_FINE, "mama_initPlugins(): Initialising [%s] %s", propString, prop);

+            mama_loadPlugin (prop);

+        }

+    }

+

+    mama_log (MAMA_LOG_LEVEL_FINE, "mama_initPlugins(): Initialising mamacenterprise");

+    mama_loadPlugin ("mamacenterprise");

+

+    return MAMA_STATUS_OK;

+}

+

+mama_status

+mama_loadPlugin (const char* pluginName)

+{

+    LIB_HANDLE              pluginLib       = NULL;

+    mamaPluginImpl*         pluginImpl      = NULL;

+    mama_status             status          = MAMA_STATUS_OK;

+    mamaPluginInfo          pluginInfo      = NULL;

+    char                    loadPluginName  [MAX_PLUGIN_STRING];

+    mamaPluginImpl*         aPluginImpl     = NULL;

+

+    if (!pluginName)

+        return MAMA_STATUS_NULL_ARG;

+

+    pluginImpl = mamaPlugin_findPlugin(pluginName);

+

+    /*

+     * Check to see if pluginImpl has already been loaded

+     */

+    if (pluginImpl == NULL)

+    {

+       /* The plugin name should be of the format mamaplugin<name> */

+        if (strncmp(pluginName, "mamacenterprise", MAX_PLUGIN_STRING) == 0)

+            snprintf(loadPluginName, MAX_PLUGIN_STRING,

+                "%s",

+                pluginName);

+        else

+            snprintf(loadPluginName, MAX_PLUGIN_STRING,

+                "%s%s",

+                PLUGIN_NAME,

+                pluginName);

+

+        pluginLib = openSharedLib (loadPluginName, NULL);

+

+        if (!pluginLib)

+        {

+

+           mama_log (MAMA_LOG_LEVEL_ERROR,

+                    "mama_loadPlugin(): "

+                    "Could not open plugin library [%s] [%s]",

+                    pluginName,

+                    getLibError());

+            return MAMA_STATUS_PLATFORM;

+        }

+

+        /* Create structure to hold plugin information */

+        aPluginImpl = (mamaPluginImpl*)calloc (1, sizeof(mamaPluginImpl));

+

+        status = mamaPlugin_registerFunctions (pluginLib,

+                                               pluginName,

+                                               pluginInfo,

+                                               aPluginImpl);

+

+        if (MAMA_STATUS_OK == status)

+        {

+            mama_log (MAMA_LOG_LEVEL_NORMAL,

+                     "mama_loadPlugin(): "

+                     "Sucessfully registered plugin functions for [%s]",

+                     pluginName);

+

+        }

+        else

+        {

+            mama_log (MAMA_LOG_LEVEL_WARN,

+                     "mama_loadPlugin(): "

+                     "Failed to register plugin functions for [%s]",

+                     pluginName);

+

+            closeSharedLib (aPluginImpl->mPluginHandle);

+

+            free ((char*)aPluginImpl->mPluginName);

+            free ((mamaPluginImpl*)aPluginImpl);

+

+            return status;

+        }

+

+        /* Invoke the init function */

+        status = aPluginImpl->mamaPluginInitHook (aPluginImpl->mPluginInfo);

+

+        if (MAMA_STATUS_OK == status)

+        {

+            mama_log (MAMA_LOG_LEVEL_NORMAL,

+                      "mama_loadPlugin(): Successfully run the init hook for mama plugin [%s]",

+                       aPluginImpl->mPluginName);

+        }

+        else

+        {

+            mama_log (MAMA_LOG_LEVEL_WARN,

+                      "mama_loadPlugin(): Init hook failed for mama plugin [%s]",

+                       aPluginImpl->mPluginName);

+

+            closeSharedLib (aPluginImpl->mPluginHandle);

+

+            free ((char*)aPluginImpl->mPluginName);

+            free ((mamaPluginImpl*)aPluginImpl);

+

+            return status;

+        }

+

+        /* Save off the plugin impl and increment the plugin counter */

+        gPlugins[gPluginNo] = aPluginImpl;

+        gPluginNo++;

+

+    }

+    else

+    {

+        mama_log (MAMA_LOG_LEVEL_NORMAL,

+                 "mama_loadPlugin(): "

+                 "Plugin [%s] has already been loaded and initialised",

+                 pluginName);

+    }

+

+    return MAMA_STATUS_OK;

+}

+

+mama_status

+mama_shutdownPlugins (void)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+    int          ret    = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin]->mPluginHandle != NULL)

+        {

+            /* Fire the user shutdown hook first - if one is present */

+            if (gPlugins[plugin]->mamaPluginShutdownHook != NULL)

+            {

+                gPlugins[plugin]->mamaPluginShutdownHook (gPlugins[plugin]->mPluginInfo);

+            }

+

+            ret = closeSharedLib (gPlugins[plugin]->mPluginHandle);

+            if (0!=ret)

+            {

+                 mama_log (MAMA_LOG_LEVEL_WARN,

+                            "mama_shutdownPlugins(): Failed to close Mama Plugin [%s]",

+                            gPlugins[plugin]->mPluginName);

+            }

+            else

+            {

+                 mama_log (MAMA_LOG_LEVEL_WARN,

+                            "mama_shutdownPlugins(): Mama Plugin [%s] successfully closed",

+                            gPlugins[plugin]->mPluginName);

+            }

+

+            free ((char*)gPlugins[plugin]->mPluginName);

+            free (gPlugins[plugin]);

+            gPlugins[plugin] = NULL;

+        }

+    }

+    return status;

+}

+

+mama_status

+mamaPlugin_firePublisherPreSendHook (mamaPublisher publisher, mamaMsg message)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin] != NULL)

+        {

+            if (gPlugins[plugin]->mamaPluginPublisherPreSendHook != NULL)

+            {

+                status = gPlugins[plugin]->mamaPluginPublisherPreSendHook (gPlugins[plugin]->mPluginInfo, publisher, message);

+

+                if (MAMA_STATUS_OK != status)

+                {

+                     mama_log (MAMA_LOG_LEVEL_WARN,

+                                "mamaPlugin_firePublisherPreSendHook(): Publisher pre send hook failed for mama plugin [%s]",

+                                gPlugins[plugin]->mPluginName);

+                }

+            }

+        }

+    }

+    return status;

+}

+

+mama_status

+mamaPlugin_fireTransportPostCreateHook (mamaTransport transport)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin] != NULL)

+        {

+            if (gPlugins[plugin]->mamaPluginTransportPostCreateHook != NULL)

+            {

+                status = gPlugins[plugin]->mamaPluginTransportPostCreateHook (gPlugins[plugin]->mPluginInfo, transport);

+

+                if (MAMA_STATUS_OK != status)

+                {

+                     mama_log (MAMA_LOG_LEVEL_WARN,

+                                "mamaPlugin_fireTransportPostCreateHook(): Transport post create hook failed for mama plugin [%s]",

+                                gPlugins[plugin]->mPluginName);

+                }

+            }

+        }

+    }

+    return status;

+}

+

+mama_status

+mamaPlugin_fireShutdownHook (void)

+{

+    mama_status  status = MAMA_STATUS_OK;

+    int          plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin] != NULL)

+        {

+            if (gPlugins[plugin]->mamaPluginShutdownHook != NULL)

+            {

+                status = gPlugins[plugin]->mamaPluginShutdownHook (gPlugins[plugin]->mPluginInfo);

+

+                if (MAMA_STATUS_OK != status)

+                {

+                     mama_log (MAMA_LOG_LEVEL_WARN,

+                                "mamaPlugin_fireShutdownHook(): Shutdown hook failed for mama plugin [%s]",

+                                gPlugins[plugin]->mPluginName);

+                }

+            }

+        }

+    }

+    return status;

+}

+

+mamaPluginImpl*

+mamaPlugin_findPlugin (const char* name)

+{

+    int plugin = 0;

+

+    for (plugin = 0; plugin <= gPluginNo; plugin++)

+    {

+        if (gPlugins[plugin])

+        {

+            if ((strncmp(gPlugins[plugin]->mPluginName, name, MAX_PLUGIN_STRING) == 0))

+            {

+                return gPlugins[plugin];

+            }

+        }

+    }

+    return NULL;

+}

diff --git a/mama/c_cpp/src/c/plugin.h b/mama/c_cpp/src/c/plugin.h

new file mode 100644

index 0000000..df43b27

--- /dev/null

+++ b/mama/c_cpp/src/c/plugin.h

@@ -0,0 +1,59 @@

+/* $Id$

+ *

+ * OpenMAMA: The open middleware agnostic messaging API

+ * Copyright (C) 2011 NYSE Technologies, Inc.

+ *

+ * This library is free software; you can redistribute it and/or

+ * modify it under the terms of the GNU Lesser General Public

+ * License as published by the Free Software Foundation; either

+ * version 2.1 of the License, or (at your option) any later version.

+ *

+ * This library is distributed in the hope that it will be useful,

+ * but WITHOUT ANY WARRANTY; without even the implied warranty of

+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU

+ * Lesser General Public License for more details.

+ *

+ * You should have received a copy of the GNU Lesser General Public

+ * License along with this library; if not, write to the Free Software

+ * Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA

+ * 02110-1301 USA

+ */

+

+#ifndef PluginH__

+#define PluginH__

+

+

+/**

+ * @brief Containers used within the mama plugin

+ */

+typedef mama_status (*mamaPlugin_publisherPreSendHook) (mamaPluginInfo pluginInfo, mamaPublisher publisher, mamaMsg message);

+typedef mama_status (*mamaPlugin_transportPostCreateHook) (mamaPluginInfo pluginInfo, mamaTransport transport);

+typedef mama_status (*mamaPlugin_shutdownHook) (mamaPluginInfo pluginInfo);

+typedef mama_status (*mamaPlugin_initHook) (mamaPluginInfo* pluginInfo);

+

+/**

+ * Initialize the internal plugin interface

+ *

+ * @return mama status code

+ */

+extern mama_status

+mama_initPlugins (void);

+

+/**

+ * Shutdown the internal plugin interface

+ *

+ * @return mama status code

+ */

+extern mama_status

+mama_shutdownPlugins (void);

+

+extern mama_status

+mamaPlugin_firePublisherPreSendHook (mamaPublisher publisher, mamaMsg message);

+

+extern mama_status

+mamaPlugin_fireTransportPostCreateHook (mamaTransport transport);

+

+extern mama_status

+mamaPlugin_fireShutdownHook (void);

+

+#endif /* PluginH__ */

diff --git a/mama/c_cpp/src/c/publisher.c b/mama/c_cpp/src/c/publisher.c

index 4612d17..6a7f92d 100644

--- a/mama/c_cpp/src/c/publisher.c

+++ b/mama/c_cpp/src/c/publisher.c

@@ -23,6 +23,7 @@

#include "mama/publisher.h"

 #include "bridge.h"

+#include "plugin.h"

#include "throttle.h"

#include "transportimpl.h"

@@ -245,6 +246,15 @@ mamaPublisher_send (mamaPublisher publisher,

     if (!impl->mMamaPublisherBridgeImpl) return MAMA_STATUS_INVALID_ARG;

     if (!impl->mBridgeImpl) return MAMA_STATUS_NO_BRIDGE_IMPL;

+    /* Calling plugin hook */

+    status = mamaPlugin_firePublisherPreSendHook (publisher, msg);

+    if (MAMA_STATUS_OK != status)

+    {

+        mama_log (MAMA_LOG_LEVEL_ERROR,

+                  "mamaPublisher_send(): PublisherPreSendHook failed. Not sending message.");

+        return status;

+    }

+

     status = impl->mBridgeImpl->bridgeMamaPublisherSend

         (impl->mMamaPublisherBridgeImpl,

          msg);

diff --git a/mama/c_cpp/src/c/transport.c b/mama/c_cpp/src/c/transport.c

index 339dbc0..622f47d 100644

--- a/mama/c_cpp/src/c/transport.c

+++ b/mama/c_cpp/src/c/transport.c

@@ -25,6 +25,7 @@

 #include "mama/mama.h"

#include "throttle.h"

+#include "plugin.h"

#include "list.h"

#include "transportimpl.h"

#include "bridge.h"

@@ -876,10 +877,22 @@ mamaTransport_create (mamaTransport transport,

     if ((!self->mDisableRefresh) && (!mamaTransportInternal_disableRefreshes(name)))

     {

-        return refreshTransport_create (&self->mRefreshTransport,

-                                    (mamaTransport)self,

-                                    self->mListeners,

-                                    self->mBridgeImpl);

+        status = refreshTransport_create (&self->mRefreshTransport,

+                                          (mamaTransport)self,

+                                          self->mListeners,

+                                          self->mBridgeImpl);

+

+        if (MAMA_STATUS_OK != status)

+            return status;

+    }

+

+    /* Calling plugin hook*/

+    status = mamaPlugin_fireTransportPostCreateHook (transport);

+    if (MAMA_STATUS_OK != status)

+    {

+        mama_log (MAMA_LOG_LEVEL_ERROR,

+                  "mamaTransport_create(): TransportPostCreateHook failed with a status of %s",

+                   mamaStatus_stringForStatus(status));

     }

     return MAMA_STATUS_OK;

--

1.7.1

 

 

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

g.molloy@...

 


Re: [Patch 2.3.2] Scons: fixes for Windows and Linux

Gary Molloy <g.molloy@...>
 

Hey Reed,

 

Thank you for your patch.  I have logged this as BZ-188 for you.

 

Your patch looks good.  I noticed just a few minor things:

 

The following can just be “./src/c/mama”:

 

-INPUT                  = ./src/c/mama ./src/enterprise/c/mama

+INPUT                  = ./src/c/mama ./src/c/mama

 

And the same could be applied to mama/c_cpp/doxyconfig-cpp.in

 

Thanks,

Gary

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

Tel: +44 28 9099 7580  Ext 3397

g.molloy@...

 

From: openmama-dev-bounces@... [mailto:openmama-dev-bounces@...] On Behalf Of Reed Alpert
Sent: 28 March 2015 21:19
To: openmama-dev@...
Subject: [Openmama-dev] [Patch 2.3.2] Scons: fixes for Windows and Linux

 

Hi,

 

These are against the next branch.

 

Fixes for Windows build using docs, testtools, and unittests.
Fixes for using DOS commands instead of GNU/Linux commands.
Remove reference to 'enterprise' in path names.
Add junit_home as a cmd line parm.
Add Visual Studio 11 and 12 as options (leave 10 as default).
Add MSVS_VERSION (along with existing MSVC_VERSION) to tell scons what Visual Studio version to use.

 

Thanks,

 

Reed.

 

 

diff --git a/mama/c_cpp/SConscript.win b/mama/c_cpp/SConscript.win

index 77e3157..ece0710 100644

--- a/mama/c_cpp/SConscript.win

+++ b/mama/c_cpp/SConscript.win

@@ -121,14 +121,13 @@ if env['with_testtools'] == True and 'dynamic' in env['build']:

 if (    not Media.has_key('mama/c_cpp/docs')

         and env['build'] == 'dynamic'

         and not env.GetOption('clean')

+        and env['product'] == 'mama'

         and env['with_docs'] == True ):

 

     cdoc = env.Doxygen('doxyconfig-c.in')

     cppdoc = env.Doxygen('doxyconfig-cpp.in')

 

-    env.Command( '$prefix/doc/mama/images', cdoc, 'mkdir $TARGET && cp -rf %s\mama\c_cpp\doc\images\* $TARGET' % ( env['TOPLEVEL'] ) )

-    env.Command( '$prefix/doc/mama/c/html', cdoc, 'mkdir $TARGET && cp -rf %s\mama\c_cpp\doc\c\html\* $TARGET' % ( env['TOPLEVEL'] ) )

-    env.Command( '$prefix/doc/mama/cpp/html', cppdoc, 'mkdir $TARGET && cp -rf %s\mama\c_cpp\doc\cpp\html\* $TARGET' % ( env['TOPLEVEL'] ) )

+    env.Command( '$prefix/doc/mama', '', 'mkdir $TARGET && xcopy /q /s /e /y %s\mama\c_cpp\doc\* $TARGET' % ( env['TOPLEVEL'] ) )

 

     env.Clean( cdoc, '%s/mama/c_cpp/doc/c' % (env['TOPLEVEL']) )

     env.Clean( cppdoc, '%s/mama/c_cpp/doc/cpp' % (env['TOPLEVEL']) )

diff --git a/mama/c_cpp/doxyconfig-c.in b/mama/c_cpp/doxyconfig-c.in

index 9e03518..b92b005 100644

--- a/mama/c_cpp/doxyconfig-c.in

+++ b/mama/c_cpp/doxyconfig-c.in

@@ -754,7 +754,7 @@ WARN_LOGFILE           =

 # spaces.

 # Note: If this tag is empty the current directory is searched.

 

-INPUT                  = ./src/c/mama ./src/enterprise/c/mama

+INPUT                  = ./src/c/mama ./src/c/mama

 

 # This tag can be used to specify the character encoding of the source files

 # that doxygen parses. Internally doxygen uses the UTF-8 encoding. Doxygen uses

diff --git a/mama/jni/SConscript b/mama/jni/SConscript

index 6d4173d..61a3570 100644

--- a/mama/jni/SConscript

+++ b/mama/jni/SConscript

@@ -20,6 +20,9 @@ testtools_source = Split("""

 

 testtools = []

 

+unittest_source = Glob('src/junittests')

+unittest = []

+

 jExamples = Glob('src/com/wombat/mama/examples/*java')

 

 version = env['versions']['mama']['releaseString']

@@ -43,14 +46,20 @@ if env['with_testtools'] == True:

     for t in testtools_source:

         testtools.append( env.Java( 'classes', t ) )

 

+if env['with_unittest'] == True:

+    for t in unittest_source:

+        unittest.append( env.Java( 'classes', t ) )

+        env.Append(JAVACLASSPATH = ('$junit_home/junit.jar'))

+

 env.Depends( mama_classes, common_classes )

 env.Depends( testtools, mama_classes )

+env.Depends( unittest, mama_classes )

 

 # Builds all of the header files which is unnecessary but no reason not to

 # do this

 headers = env.JavaH(target=Dir('src/c/mamajni').abspath,source= [ mama_classes, common_classes ])

 

-env.Depends( headers, [ mama_classes, common_classes, testtools ] )

+env.Depends( headers, [ mama_classes, common_classes, testtools, unittest ] )

 

 jar = env.Jar(target='mamajni.jar', source='classes', JARCHDIR = Dir('classes').abspath  )

 

diff --git a/mamda/c_cpp/SConscript.win b/mamda/c_cpp/SConscript.win

index 4c63b51..0a67d48 100644

--- a/mamda/c_cpp/SConscript.win

+++ b/mamda/c_cpp/SConscript.win

@@ -113,7 +113,7 @@ if env['with_unittest'] == True:

 if ( env['with_docs'] == True and not Media.has_key('mamda/c_cpp/docs') and env['build'] == 'dynamic' and not env.GetOption('clean') ):

    cppdoc = env.Doxygen('doxyconfig-cpp.in')

 

-   env.Command( '$prefix/doc/mamda/cpp/html', cppdoc, 'mkdir $TARGET && cp -rf %s\mamda\c_cpp\doc\cpp\html\* $TARGET' % ( env['TOPLEVEL'] ) )

+   env.Command( '$prefix/doc/mamda', cppdoc, 'mkdir $TARGET && xcopy /q /s /e /y  %s\mamda\c_cpp\doc\* $TARGET' % ( env['TOPLEVEL'] ) )

 

    env.Clean( cppdoc, '%s/mamda/c_cpp/doc/cpp' % (env['TOPLEVEL']) )

 

diff --git a/site_scons/community/command_line.py b/site_scons/community/command_line.py

index e1791e2..4bbacc5 100644

--- a/site_scons/community/command_line.py

+++ b/site_scons/community/command_line.py

@@ -26,6 +26,7 @@ def get_command_line_opts( host, products, VERSIONS ):

        BoolVariable('with_examples','Build with test tools',True),

        BoolVariable('entitled','Whether the build is entitled or unentitled',False),

        PathVariable('gtest_home','Path to Google Test home',None, PathVariable.PathIsDir),

+       PathVariable('junit_home','Path to Junit home',None, PathVariable.PathIsDir),

        ListVariable('middleware','Middleware(s) to be compiled in', 'avis', names = ['avis', 'qpid'] ),

        ('jobs', 'Number of scons threads to spawn, if n is passed the number of availabe cores is calculated and used', '1'),

 

@@ -39,7 +40,7 @@ def get_command_line_opts( host, products, VERSIONS ):

             PathVariable('qpid_home', 'Path to QPID Proton Libraries',

                           'c:\\proton', PathVariable.PathAccept),

             EnumVariable('vsver','Visual Studio Version to use', '10.0',

-                allowed_values=('8.0','9.0','10.0')),

+                allowed_values=('8.0','9.0','10.0','11.0','12.0')),

             EnumVariable('product', 'Product to be built', 'mamda',

                      allowed_values=( products )),

             EnumVariable('dotnet_version', 'Dotnet Version used to determine framework directory', '2.0',

@@ -63,6 +64,8 @@ def get_command_line_opts( host, products, VERSIONS ):

             EnumVariable('product', 'Product to be built', 'mamda',

                          #mamda all is a windows only build

                          allowed_values=( [ x for x in products if x != "mamdaall" ] )),

+            EnumVariable('target_arch', 'Specifies if the build should target 32 or 64 bit architectures.',

+                          host['arch'], allowed_values=['x86', 'x86_64']),

             EnumVariable( 'compiler', 'Compiler to use for building OpenMAMA',

                          'default', allowed_values=('default', 'gcc', 'clang', 'clang-analyzer')),

         )

diff --git a/site_scons/community/windows.py b/site_scons/community/windows.py

index 362f697..80c241d 100644

--- a/site_scons/community/windows.py

+++ b/site_scons/community/windows.py

@@ -64,8 +64,10 @@ class Windows:

             env = Environment(ENV={

                 'JAVA_HOME': '%s' % (optsEnv['java_home']),

                 'PATH': '%s:%s\\bin' % (os.environ['PATH'], optsEnv['java_home']),

-                'MSVC_VERSION' : '%s' %(optsEnv['vsver'])},

+                },

                 tools = tools,

+                MSVC_VERSION = optsEnv['vsver'],

+                MSVS_VERSION = optsEnv['vsver'],

                 TARGET_ARCH = optsEnv['target_arch'])

 

             #ConfigureJNI searches os.env for java_home not env['ENV']['JAVA_HOME'] 

@@ -78,7 +80,7 @@ class Windows:

             env['JAVAH'] = 'javah'

 

         else:

-            env = Environment(ENV={'PATH': '%s' % (os.environ['PATH'])}, MSVC_VERSION = optsEnv['vsver'], tools = tools, TARGET_ARCH = optsEnv['target_arch'])

+            env = Environment(ENV={'PATH': '%s' % (os.environ['PATH'])}, MSVC_VERSION = optsEnv['vsver'], MSVS_VERSION = optsEnv['vsver'], tools = tools, TARGET_ARCH = optsEnv['target_arch'])

 

         env['SPAWN'] = logger.log_output

         env['PRINT_CMD_LINE_FUNC'] = logger.log_command

@@ -196,7 +198,7 @@ class Windows:

     # Configures all of the appropriate environment variables for windows.

     def configure(self, env ):

         env.Append( CPPDEFINES = ['WIN32'] )

-        env.Append( CCFLAGS = ['-EHsc','/GR','/FIwombat\\targetsxs.h'] )

+        env.Append( CCFLAGS = ['/EHsc','/GR','/FIwombat\\targetsxs.h'] )

         env.Append( LINKFLAGS =['/MANIFEST'] )

         env.Append( CCPDBFLAGS = '/Fd${TARGET}.pdb' )

         env.Append( PDB = '${TARGET.base}.pdb')

 


[PATCH] [MAMAC] OpenMAMA mock RPM's fail to build

Gary Molloy <g.molloy@...>
 

Bugzilla Ticket :- Bug 179

 

Testing Strategy:-

Not middleware or O/S specific. 
This applies to building with QPID only
 

Try building OpenMAMA against the latest version of proton (v0.8) and the build will fail with the following error:

 

mama/c_cpp/src/c/bridge/qpid/qpiddefs.h:37:25: error: proton/util.h: No such file or directory

 

After applying the patch you will be able to build successfully.

 

=========================================================================================================

 

From fcaaf81364fceb05515c43c6e507f0d8a3c0f01a Mon Sep 17 00:00:00 2001

From: Gary Molloy <g.molloy@...>

Date: Fri, 17 Apr 2015 05:06:50 -0400

Subject: [PATCH] [MAMAC] OpenMAMA mock RPM's fail to build

 

The OpenMAMA mock RPM's are failing to build due to the Fedora

repository having updated the default version of proton to 0.8.

 

The problem is that it no longer contains the proton header file

"proton/util.h" which we try to include.

 

[OMAMA-342]

 

Signed-off-by: Gary Molloy <g.molloy@...>

---

mama/c_cpp/src/c/bridge/qpid/qpiddefs.h |    6 +++++-

1 files changed, 5 insertions(+), 1 deletions(-)

 

diff --git a/mama/c_cpp/src/c/bridge/qpid/qpiddefs.h b/mama/c_cpp/src/c/bridge/qpid/qpiddefs.h

index 8e686ea..cd52e96 100644

--- a/mama/c_cpp/src/c/bridge/qpid/qpiddefs.h

+++ b/mama/c_cpp/src/c/bridge/qpid/qpiddefs.h

@@ -34,7 +34,6 @@

/* Qpid include files */

#include <proton/driver.h>

#include <proton/message.h>

-#include <proton/util.h>

#include <proton/messenger.h>

 #include "endpointpool.h"

@@ -84,6 +83,11 @@ typedef enum qpidMsgType_

     qpidBridgeMamaTransportImpl_freeProtonMessenger (messenger)

#endif

+/* The proton header util.h was removed in version 0.8 */

+#if (PN_VERSION_MAJOR == 0 && PN_VERSION_MINOR <= 7)

+#include <proton/util.h>

+#endif

+

/* Place other version specific macros here */

 #else

--

1.7.1

 

 

Gary Molloy – SR Labs

Adelaide Exchange | 24-26 Adelaide Street | Belfast | BT2 8GD

Tel: +44 28 9099 7580  Ext 3397

g.molloy@...

 


Re: Handling REFRESH messages in DQ publisher manager

Glenn McClements <g.mcclements@...>
 

Yeah, as Ian points out, this entirely depends on whether it’s basic or market data subscriptions. For heartbeats I was originally assuming basic, but it’ll depend on the answer to my question in the previous mail. i.e. are the heartbeats

- at a transport or source level, which would imply basic subscription or a native middleware level subscription
- at the topic level, which would imply they are *part* of a market data subscription.

Glenn

On 16/04/2015 15:15, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi

Just to clairify.

If it’s a marketdata subscription (ie MamaSubscription) then STATUS and TYPE are required fields. If they are not in the message the message will be dropped.

For basic subscriptions no fields are mandatory and everything passes through.

Thanks
Ian

-----Original Message-----
From: Tom Doust [mailto:tom.doust@tick42.com]
Sent: 16 April 2015 15:08
To: Glenn McClements; Alireza Assadzadeh; Yury Batrakov; Ian Bell; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Hi Glenn

Are you saying that if a message doesn't carry the mdMsgType field then the OpenMAMA layer just passes it through to the client application. I hadn’t actually realised that ! I sort of assumed it was required and there would be some kind of error raised if it was missing.

Tom


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: 16 April 2015 14:29
To: Tom Doust; Alireza Assadzadeh; Yury Batrakov; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Hi Tom,
I’m not sure what distinguishes the messages you suggest from any other kind of application messages, which don’t even need to have a message type. The ANK/NAK messages you suggest look like they’re dealt with at a custom block at the application level, so OpenMAMA just needs to be dumb in this respect and pass those messages though.

In summary the stack is the below:

1) Middleware
------------
2) Bridge

------------
3) OpenMAMA
-------------
4) Application



(Hopefully the above comes out the mail server formatted OK)

And the messages we’ve been talking about terminate at the levels:

1) Middleware heartbeats (generally) other internal middleware messages

2) Not too much currently terminates at this level, some middlewares pass back info messages or callbacks which may or may not be propagated up to OpenMAMA. Yesterday when I mentioned the possibility of adding a new INTERNAL message type for the use case mentioned, I was considering it terminating here, or possibly up to OpenMAMA which would ignore it.

3) Some OpenMAMA internal messages, such as REFRESHs.

4) Everything else, including NULL messages. Sorry Alireza, I missed your mail first time round

Heartbeats could theoretically live in any level, but For Yury’s use case it’s specifically 1 & 2, and therefore I don’t think these messages should be propagated up. NULL messages could be used, but then you would be adding expected behaviour onto the application level e.g. they should be ignored, and this is something I try to avoid.

I think there is room here to make this more explicit, possibly though a couple of new message types or namespaces but this could be complicating things.

Yury - it actually just occurred to me, are these heartbeats on a per transport, publisher or topic level? If they’re of the transport or publisher, then another possibility would be to have a separate middleware level subscription which you could send whatever you wanted over, and OpenMAMA wouldn’t be affected at all.

Regards,
Glenn





On 16/04/2015 10:21, "Tom Doust" <tom.doust@tick42.com> wrote:

Although I can think of many reasons why this would be a bad idea, it might be worth considering defining some MAMA_MSG_TYPE_PLATFORM

This would allow bridge developers to deliver custom stuff to client code with the intention of formalising into the api when it turns out to become useful.

An example of this would be the ACK/NAK messages our RMDS bridge delivers in response to message posts on TREP. We implemented these in the absence of any publisher response mechanism in OpenMAMA and when the new publisher callbacks that are currently under development appear in a version of OpenMAMA we will update the bridge to use those instead


Tom

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Alireza Assadzadeh
Sent: 14 April 2015 17:06
To: Yury Batrakov; Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

BTW, I noticed the following description for MAMA_MSG_TYPE_NULL in mamaMsgType. I did not see this message type having special logic associated with it in OpenMAMA MAMC/MAMACPP.

From msgtype.h:

/** Keep alive message. */
MAMA_MSG_TYPE_NULL = 175,

Is this message type suitable for this case?

--Alireza

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-
dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: Tuesday, April 14, 2015 11:03 AM
To: Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what
is to return for mamaMsg_get(MdMsgType, 1), any integer which is not
in standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a
middleware level mechanism using OpenMAMA messages, so the solution is
not to modify dqpublishermanager to handle the new use of REFRESHs,
but to make sure that whatever messages are used for heartbeats are
never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for
this use case as would imply that heartbeats would be
sent/received/processed the OpenMAMA level (which is something we may
add at some point) and *not* at the middleware level. That’s not to
say you can’t add a new specific message type for your own heartbeats,
but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies
replying to an mamaInbox, and for refreshes there is (currently) no
inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware
heartbeat functionality. Specially they operate at a higher level
although they can be similar in nature.

Normal operation would be to have the transport clean up the client
context information as the clients went away using its own heartbeat
functionality and let MAMA inform the client application of
subscribers using its own refresh messages. The refresh timeout is an
hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to
reduce
the heartbeat traffic as well as to allow for transports which don't
maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh
message once
an hour less a random period of time, however the client refresh
timeout is reset on receiving a refresh response from a publisher, so
what actually happens is that the first client to timeout sends a
refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher.
So the publisher only processes a single refresh message per topic as
opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question
about Advanced publishing. In my middleware a publisher keeps track of
all subscribers connected to it. A publisher and subscribers exchange
with heartbeat messages posted to a single topic providing a string
with session id in heartbeats' payload. When a publisher receives a
heartbeat it sends a reply with the same session ID. Once heartbeat is
not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages
but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl-
mRefreshResponseMsg);
impl-
mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of
communication i.e. session id as mRefreshResponseMsg doesn't have any
fields set. Thus we force users to implement a part of underlying
protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl-
mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information.
If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and delete this e-mail. Any
unauthorized copying, disclosure or distribution of the material in
this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and delete this e-mail.
Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Handling REFRESH messages in DQ publisher manager

Ian Bell
 

Hi

Just to clairify.

If it’s a marketdata subscription (ie MamaSubscription) then STATUS and TYPE are required fields. If they are not in the message the message will be dropped.

For basic subscriptions no fields are mandatory and everything passes through.

Thanks
Ian

-----Original Message-----
From: Tom Doust [mailto:tom.doust@tick42.com]
Sent: 16 April 2015 15:08
To: Glenn McClements; Alireza Assadzadeh; Yury Batrakov; Ian Bell; openmama-dev@lists.openmama.org
Subject: RE: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Hi Glenn

Are you saying that if a message doesn't carry the mdMsgType field then the OpenMAMA layer just passes it through to the client application. I hadn’t actually realised that ! I sort of assumed it was required and there would be some kind of error raised if it was missing.

Tom


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: 16 April 2015 14:29
To: Tom Doust; Alireza Assadzadeh; Yury Batrakov; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Hi Tom,
I’m not sure what distinguishes the messages you suggest from any other kind of application messages, which don’t even need to have a message type. The ANK/NAK messages you suggest look like they’re dealt with at a custom block at the application level, so OpenMAMA just needs to be dumb in this respect and pass those messages though.

In summary the stack is the below:

1) Middleware
------------
2) Bridge

------------
3) OpenMAMA
-------------
4) Application



(Hopefully the above comes out the mail server formatted OK)

And the messages we’ve been talking about terminate at the levels:

1) Middleware heartbeats (generally) other internal middleware messages

2) Not too much currently terminates at this level, some middlewares pass back info messages or callbacks which may or may not be propagated up to OpenMAMA. Yesterday when I mentioned the possibility of adding a new INTERNAL message type for the use case mentioned, I was considering it terminating here, or possibly up to OpenMAMA which would ignore it.

3) Some OpenMAMA internal messages, such as REFRESHs.

4) Everything else, including NULL messages. Sorry Alireza, I missed your mail first time round

Heartbeats could theoretically live in any level, but For Yury’s use case it’s specifically 1 & 2, and therefore I don’t think these messages should be propagated up. NULL messages could be used, but then you would be adding expected behaviour onto the application level e.g. they should be ignored, and this is something I try to avoid.

I think there is room here to make this more explicit, possibly though a couple of new message types or namespaces but this could be complicating things.

Yury - it actually just occurred to me, are these heartbeats on a per transport, publisher or topic level? If they’re of the transport or publisher, then another possibility would be to have a separate middleware level subscription which you could send whatever you wanted over, and OpenMAMA wouldn’t be affected at all.

Regards,
Glenn





On 16/04/2015 10:21, "Tom Doust" <tom.doust@tick42.com> wrote:

Although I can think of many reasons why this would be a bad idea, it might be worth considering defining some MAMA_MSG_TYPE_PLATFORM

This would allow bridge developers to deliver custom stuff to client code with the intention of formalising into the api when it turns out to become useful.

An example of this would be the ACK/NAK messages our RMDS bridge delivers in response to message posts on TREP. We implemented these in the absence of any publisher response mechanism in OpenMAMA and when the new publisher callbacks that are currently under development appear in a version of OpenMAMA we will update the bridge to use those instead


Tom

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Alireza Assadzadeh
Sent: 14 April 2015 17:06
To: Yury Batrakov; Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

BTW, I noticed the following description for MAMA_MSG_TYPE_NULL in mamaMsgType. I did not see this message type having special logic associated with it in OpenMAMA MAMC/MAMACPP.

From msgtype.h:

/** Keep alive message. */
MAMA_MSG_TYPE_NULL = 175,

Is this message type suitable for this case?

--Alireza

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-
dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: Tuesday, April 14, 2015 11:03 AM
To: Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what
is to return for mamaMsg_get(MdMsgType, 1), any integer which is not
in standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a
middleware level mechanism using OpenMAMA messages, so the solution is
not to modify dqpublishermanager to handle the new use of REFRESHs,
but to make sure that whatever messages are used for heartbeats are
never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for
this use case as would imply that heartbeats would be
sent/received/processed the OpenMAMA level (which is something we may
add at some point) and *not* at the middleware level. That’s not to
say you can’t add a new specific message type for your own heartbeats,
but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies
replying to an mamaInbox, and for refreshes there is (currently) no
inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware
heartbeat functionality. Specially they operate at a higher level
although they can be similar in nature.

Normal operation would be to have the transport clean up the client
context information as the clients went away using its own heartbeat
functionality and let MAMA inform the client application of
subscribers using its own refresh messages. The refresh timeout is an
hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to
reduce
the heartbeat traffic as well as to allow for transports which don't
maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh
message once
an hour less a random period of time, however the client refresh
timeout is reset on receiving a refresh response from a publisher, so
what actually happens is that the first client to timeout sends a
refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher.
So the publisher only processes a single refresh message per topic as
opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question
about Advanced publishing. In my middleware a publisher keeps track of
all subscribers connected to it. A publisher and subscribers exchange
with heartbeat messages posted to a single topic providing a string
with session id in heartbeats' payload. When a publisher receives a
heartbeat it sends a reply with the same session ID. Once heartbeat is
not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages
but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl-
mRefreshResponseMsg);
impl-
mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of
communication i.e. session id as mRefreshResponseMsg doesn't have any
fields set. Thus we force users to implement a part of underlying
protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl-
mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information.
If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and delete this e-mail. Any
unauthorized copying, disclosure or distribution of the material in
this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and delete this e-mail.
Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Handling REFRESH messages in DQ publisher manager

Tom Doust
 

Hi Glenn

Are you saying that if a message doesn't carry the mdMsgType field then the OpenMAMA layer just passes it through to the client application. I hadn’t actually realised that ! I sort of assumed it was required and there would be some kind of error raised if it was missing.

Tom

-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: 16 April 2015 14:29
To: Tom Doust; Alireza Assadzadeh; Yury Batrakov; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Hi Tom,
I’m not sure what distinguishes the messages you suggest from any other kind of application messages, which don’t even need to have a message type. The ANK/NAK messages you suggest look like they’re dealt with at a custom block at the application level, so OpenMAMA just needs to be dumb in this respect and pass those messages though.

In summary the stack is the below:

1) Middleware
------------
2) Bridge

------------
3) OpenMAMA
-------------
4) Application



(Hopefully the above comes out the mail server formatted OK)

And the messages we’ve been talking about terminate at the levels:

1) Middleware heartbeats (generally) other internal middleware messages

2) Not too much currently terminates at this level, some middlewares pass back info messages or callbacks which may or may not be propagated up to OpenMAMA. Yesterday when I mentioned the possibility of adding a new INTERNAL message type for the use case mentioned, I was considering it terminating here, or possibly up to OpenMAMA which would ignore it.

3) Some OpenMAMA internal messages, such as REFRESHs.

4) Everything else, including NULL messages. Sorry Alireza, I missed your mail first time round

Heartbeats could theoretically live in any level, but For Yury’s use case it’s specifically 1 & 2, and therefore I don’t think these messages should be propagated up. NULL messages could be used, but then you would be adding expected behaviour onto the application level e.g. they should be ignored, and this is something I try to avoid.

I think there is room here to make this more explicit, possibly though a couple of new message types or namespaces but this could be complicating things.

Yury - it actually just occurred to me, are these heartbeats on a per transport, publisher or topic level? If they’re of the transport or publisher, then another possibility would be to have a separate middleware level subscription which you could send whatever you wanted over, and OpenMAMA wouldn’t be affected at all.

Regards,
Glenn





On 16/04/2015 10:21, "Tom Doust" <tom.doust@tick42.com> wrote:

Although I can think of many reasons why this would be a bad idea, it might be worth considering defining some MAMA_MSG_TYPE_PLATFORM

This would allow bridge developers to deliver custom stuff to client code with the intention of formalising into the api when it turns out to become useful.

An example of this would be the ACK/NAK messages our RMDS bridge delivers in response to message posts on TREP. We implemented these in the absence of any publisher response mechanism in OpenMAMA and when the new publisher callbacks that are currently under development appear in a version of OpenMAMA we will update the bridge to use those instead


Tom

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Alireza Assadzadeh
Sent: 14 April 2015 17:06
To: Yury Batrakov; Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

BTW, I noticed the following description for MAMA_MSG_TYPE_NULL in mamaMsgType. I did not see this message type having special logic associated with it in OpenMAMA MAMC/MAMACPP.

From msgtype.h:

/** Keep alive message. */
MAMA_MSG_TYPE_NULL = 175,

Is this message type suitable for this case?

--Alireza

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-
dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: Tuesday, April 14, 2015 11:03 AM
To: Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what
is to return for mamaMsg_get(MdMsgType, 1), any integer which is not
in standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a
middleware level mechanism using OpenMAMA messages, so the solution is
not to modify dqpublishermanager to handle the new use of REFRESHs,
but to make sure that whatever messages are used for heartbeats are
never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for
this use case as would imply that heartbeats would be
sent/received/processed the OpenMAMA level (which is something we may
add at some point) and *not* at the middleware level. That’s not to
say you can’t add a new specific message type for your own heartbeats,
but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies
replying to an mamaInbox, and for refreshes there is (currently) no
inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware
heartbeat functionality. Specially they operate at a higher level
although they can be similar in nature.

Normal operation would be to have the transport clean up the client
context information as the clients went away using its own heartbeat
functionality and let MAMA inform the client application of
subscribers using its own refresh messages. The refresh timeout is an
hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to
reduce
the heartbeat traffic as well as to allow for transports which don't
maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh
message once
an hour less a random period of time, however the client refresh
timeout is reset on receiving a refresh response from a publisher, so
what actually happens is that the first client to timeout sends a
refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher.
So the publisher only processes a single refresh message per topic as
opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question
about Advanced publishing. In my middleware a publisher keeps track of
all subscribers connected to it. A publisher and subscribers exchange
with heartbeat messages posted to a single topic providing a string
with session id in heartbeats' payload. When a publisher receives a
heartbeat it sends a reply with the same session ID. Once heartbeat is
not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages
but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl-
mRefreshResponseMsg);
impl-
mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of
communication i.e. session id as mRefreshResponseMsg doesn't have any
fields set. Thus we force users to implement a part of underlying
protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl-
mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information.
If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and delete this e-mail. Any
unauthorized copying, disclosure or distribution of the material in
this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and delete this e-mail.
Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Handling REFRESH messages in DQ publisher manager

Glenn McClements <g.mcclements@...>
 

Hi Tom,
I’m not sure what distinguishes the messages you suggest from any other kind of application messages, which don’t even need to have a message type. The ANK/NAK messages you suggest look like they’re dealt with at a custom block at the application level, so OpenMAMA just needs to be dumb in this respect and pass those messages though.

In summary the stack is the below:

1) Middleware
------------
2) Bridge

------------
3) OpenMAMA
-------------
4) Application



(Hopefully the above comes out the mail server formatted OK)

And the messages we’ve been talking about terminate at the levels:

1) Middleware heartbeats (generally) other internal middleware messages

2) Not too much currently terminates at this level, some middlewares pass back info messages or callbacks which may or may not be propagated up to OpenMAMA. Yesterday when I mentioned the possibility of adding a new INTERNAL message type for the use case mentioned, I was considering it terminating here, or possibly up to OpenMAMA which would ignore it.

3) Some OpenMAMA internal messages, such as REFRESHs.

4) Everything else, including NULL messages. Sorry Alireza, I missed your mail first time round

Heartbeats could theoretically live in any level, but For Yury’s use case it’s specifically 1 & 2, and therefore I don’t think these messages should be propagated up. NULL messages could be used, but then you would be adding expected behaviour onto the application level e.g. they should be ignored, and this is something I try to avoid.

I think there is room here to make this more explicit, possibly though a couple of new message types or namespaces but this could be complicating things.

Yury - it actually just occurred to me, are these heartbeats on a per transport, publisher or topic level? If they’re of the transport or publisher, then another possibility would be to have a separate middleware level subscription which you could send whatever you wanted over, and OpenMAMA wouldn’t be affected at all.

Regards,
Glenn

On 16/04/2015 10:21, "Tom Doust" <tom.doust@tick42.com> wrote:

Although I can think of many reasons why this would be a bad idea, it might be worth considering defining some MAMA_MSG_TYPE_PLATFORM

This would allow bridge developers to deliver custom stuff to client code with the intention of formalising into the api when it turns out to become useful.

An example of this would be the ACK/NAK messages our RMDS bridge delivers in response to message posts on TREP. We implemented these in the absence of any publisher response mechanism in OpenMAMA and when the new publisher callbacks that are currently under development appear in a version of OpenMAMA we will update the bridge to use those instead


Tom

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Alireza Assadzadeh
Sent: 14 April 2015 17:06
To: Yury Batrakov; Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

BTW, I noticed the following description for MAMA_MSG_TYPE_NULL in mamaMsgType. I did not see this message type having special logic associated with it in OpenMAMA MAMC/MAMACPP.

From msgtype.h:

/** Keep alive message. */
MAMA_MSG_TYPE_NULL = 175,

Is this message type suitable for this case?

--Alireza

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-
dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: Tuesday, April 14, 2015 11:03 AM
To: Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what
is to return for mamaMsg_get(MdMsgType, 1), any integer which is not
in standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a
middleware level mechanism using OpenMAMA messages, so the solution is
not to modify dqpublishermanager to handle the new use of REFRESHs,
but to make sure that whatever messages are used for heartbeats are
never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for
this use case as would imply that heartbeats would be
sent/received/processed the OpenMAMA level (which is something we may
add at some point) and *not* at the middleware level. That’s not to
say you can’t add a new specific message type for your own heartbeats,
but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies
replying to an mamaInbox, and for refreshes there is (currently) no
inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware
heartbeat functionality. Specially they operate at a higher level
although they can be similar in nature.

Normal operation would be to have the transport clean up the client
context information as the clients went away using its own heartbeat
functionality and let MAMA inform the client application of
subscribers using its own refresh messages. The refresh timeout is an
hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to
reduce
the heartbeat traffic as well as to allow for transports which don't
maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh
message once
an hour less a random period of time, however the client refresh
timeout is reset on receiving a refresh response from a publisher, so
what actually happens is that the first client to timeout sends a
refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher.
So the publisher only processes a single refresh message per topic as
opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question
about Advanced publishing. In my middleware a publisher keeps track of
all subscribers connected to it. A publisher and subscribers exchange
with heartbeat messages posted to a single topic providing a string
with session id in heartbeats' payload. When a publisher receives a
heartbeat it sends a reply with the same session ID. Once heartbeat is
not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages
but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl-
mRefreshResponseMsg);
impl-
mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of
communication i.e. session id as mRefreshResponseMsg doesn't have any
fields set. Thus we force users to implement a part of underlying
protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl-
mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information.
If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and delete this e-mail. Any
unauthorized copying, disclosure or distribution of the material in
this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and delete this e-mail.
Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Handling REFRESH messages in DQ publisher manager

Tom Doust
 

Although I can think of many reasons why this would be a bad idea, it might be worth considering defining some MAMA_MSG_TYPE_PLATFORM

This would allow bridge developers to deliver custom stuff to client code with the intention of formalising into the api when it turns out to become useful.

An example of this would be the ACK/NAK messages our RMDS bridge delivers in response to message posts on TREP. We implemented these in the absence of any publisher response mechanism in OpenMAMA and when the new publisher callbacks that are currently under development appear in a version of OpenMAMA we will update the bridge to use those instead


Tom

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Alireza Assadzadeh
Sent: 14 April 2015 17:06
To: Yury Batrakov; Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

BTW, I noticed the following description for MAMA_MSG_TYPE_NULL in mamaMsgType. I did not see this message type having special logic associated with it in OpenMAMA MAMC/MAMACPP.

From msgtype.h:

/** Keep alive message. */
MAMA_MSG_TYPE_NULL = 175,

Is this message type suitable for this case?

--Alireza

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-
dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: Tuesday, April 14, 2015 11:03 AM
To: Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what
is to return for mamaMsg_get(MdMsgType, 1), any integer which is not
in standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a
middleware level mechanism using OpenMAMA messages, so the solution is
not to modify dqpublishermanager to handle the new use of REFRESHs,
but to make sure that whatever messages are used for heartbeats are
never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for
this use case as would imply that heartbeats would be
sent/received/processed the OpenMAMA level (which is something we may
add at some point) and *not* at the middleware level. That’s not to
say you can’t add a new specific message type for your own heartbeats,
but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies
replying to an mamaInbox, and for refreshes there is (currently) no
inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware
heartbeat functionality. Specially they operate at a higher level
although they can be similar in nature.

Normal operation would be to have the transport clean up the client
context information as the clients went away using its own heartbeat
functionality and let MAMA inform the client application of
subscribers using its own refresh messages. The refresh timeout is an
hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to
reduce
the heartbeat traffic as well as to allow for transports which don't
maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh
message once
an hour less a random period of time, however the client refresh
timeout is reset on receiving a refresh response from a publisher, so
what actually happens is that the first client to timeout sends a
refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher.
So the publisher only processes a single refresh message per topic as
opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question
about Advanced publishing. In my middleware a publisher keeps track of
all subscribers connected to it. A publisher and subscribers exchange
with heartbeat messages posted to a single topic providing a string
with session id in heartbeats' payload. When a publisher receives a
heartbeat it sends a reply with the same session ID. Once heartbeat is
not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages
but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl-
mRefreshResponseMsg);
impl-
mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of
communication i.e. session id as mRefreshResponseMsg doesn't have any
fields set. Thus we force users to implement a part of underlying
protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl-
mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information.
If you
are not the intended recipient (or have received this e-mail in error)
please notify the sender immediately and delete this e-mail. Any
unauthorized copying, disclosure or distribution of the material in
this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and delete this e-mail.
Any unauthorized copying, disclosure or distribution of the material
in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Handling REFRESH messages in DQ publisher manager

Ian Bell
 

Hi

I would have to agree with Glenn here, it seems that these kind of heartbeat messages should be internal to the bridge and not passed up to MAMA and as such probably should not be represented in the MsgType enum.

In the past I've seen heartbeat messages and this type of event handled within the bridge and ending up in transport level callbacks.
Eg heartbeats from a publisher to the subscriber can cause a disconnect callback to be invoked when the heartbeats cease.
Another example is the Topic callbacks which exist on the transport and can be used to show when a subscriber for a symbol appears and disappears.
Both of these cases are triggered by transport messages but the msg is not visible or passed up through the MAMA layer.

Thanks,
Ian

-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: 14 April 2015 18:28
To: Yury Batrakov; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Actually typically OpenMAMA messages are either:
- always passed back back the application Or
- for a small amount of types such as REFRESH, handled in the core OpenMAMA code.

Other message types are dealt with at the application level, as OpenMAMA is dumb with them and simple passes them on up, however you are asking for something different in that a message that shouldn’t really progress past the bridge/middleware level.

I can’t think of any implementation I’m familiar with that uses an OpenMAMA message type at the bridge/middleware level only but it’s something to think about a bit more. Possibly a new MAMA_BRIDGE_INTERNAL type would be required.There would also be a question to whether the messages terminate in the bridge/middleware level or the OpenMAMA level - the user application should never see these.

Other middlewares typically have their own internal messages types for the kind of use case you mention below so OpenMAMA messages never get to see them.

Regards,
Glenn



On 14/04/2015 16:02, "Yury Batrakov" <yury.batrakov@db.com> wrote:

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what is to return for mamaMsg_get(MdMsgType, 1), any integer which is not in standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a middleware level mechanism using OpenMAMA messages, so the solution is not to modify dqpublishermanager to handle the new use of REFRESHs, but to make sure that whatever messages are used for heartbeats are never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for this use case as would imply that heartbeats would be sent/received/processed the OpenMAMA level (which is something we may add at some point) and *not* at the middleware level. That’s not to say you can’t add a new specific message type for your own heartbeats, but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies replying to an mamaInbox, and for refreshes there is (currently) no inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware heartbeat functionality. Specially they operate at a higher level although they can be similar in nature.

Normal operation would be to have the transport clean up the client context information as the clients went away using its own heartbeat functionality and let MAMA inform the client application of subscribers using its own refresh messages. The refresh timeout is an hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to reduce the heartbeat traffic as well as to allow for transports which don't maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh message once an hour less a random period of time, however the client refresh timeout is reset on receiving a refresh response from a publisher, so what actually happens is that the first client to timeout sends a refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher. So the publisher only processes a single refresh message per topic as opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question about Advanced publishing. In my middleware a publisher keeps track of all subscribers connected to it. A publisher and subscribers exchange with heartbeat messages posted to a single topic providing a string with session id in heartbeats' payload. When a publisher receives a heartbeat it sends a reply with the same session ID. Once heartbeat is not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
impl->mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of communication i.e. session id as mRefreshResponseMsg doesn't have any fields set. Thus we force users to implement a part of underlying protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg,
+ impl->mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


Re: Handling REFRESH messages in DQ publisher manager

Glenn McClements <g.mcclements@...>
 

Actually typically OpenMAMA messages are either:
- always passed back back the application
Or
- for a small amount of types such as REFRESH, handled in the core OpenMAMA code.

Other message types are dealt with at the application level, as OpenMAMA is dumb with them and simple passes them on up, however you are asking for something different in that a message that shouldn’t really progress past the bridge/middleware level.

I can’t think of any implementation I’m familiar with that uses an OpenMAMA message type at the bridge/middleware level only but it’s something to think about a bit more. Possibly a new MAMA_BRIDGE_INTERNAL type would be required.There would also be a question to whether the messages terminate in the bridge/middleware level or the OpenMAMA level - the user application should never see these.

Other middlewares typically have their own internal messages types for the kind of use case you mention below so OpenMAMA messages never get to see them.

Regards,
Glenn

On 14/04/2015 16:02, "Yury Batrakov" <yury.batrakov@db.com> wrote:

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what is to return for mamaMsg_get(MdMsgType, 1), any integer which is not in standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a middleware level mechanism using OpenMAMA messages, so the solution is not to modify dqpublishermanager to handle the new use of REFRESHs, but to make sure that whatever messages are used for heartbeats are never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for this use case as would imply that heartbeats would be sent/received/processed the OpenMAMA level (which is something we may add at some point) and *not* at the middleware level. That’s not to say you can’t add a new specific message type for your own heartbeats, but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies replying to an mamaInbox, and for refreshes there is (currently) no inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware heartbeat functionality. Specially they operate at a higher level although they can be similar in nature.

Normal operation would be to have the transport clean up the client context information as the clients went away using its own heartbeat functionality and let MAMA inform the client application of subscribers using its own refresh messages. The refresh timeout is an hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to reduce the heartbeat traffic as well as to allow for transports which don't maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh message once an hour less a random period of time, however the client refresh timeout is reset on receiving a refresh response from a publisher, so what actually happens is that the first client to timeout sends a refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher. So the publisher only processes a single refresh message per topic as opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question about Advanced publishing. In my middleware a publisher keeps track of all subscribers connected to it. A publisher and subscribers exchange with heartbeat messages posted to a single topic providing a string with session id in heartbeats' payload. When a publisher receives a heartbeat it sends a reply with the same session ID. Once heartbeat is not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
impl->mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of communication i.e. session id as mRefreshResponseMsg doesn't have any fields set. Thus we force users to implement a part of underlying protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl->mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


Re: Handling REFRESH messages in DQ publisher manager

Alireza Assadzadeh <Alireza.Assadzadeh@...>
 

BTW, I noticed the following description for MAMA_MSG_TYPE_NULL in mamaMsgType. I did not see this message type having special logic associated with it in OpenMAMA MAMC/MAMACPP.

From msgtype.h:

/** Keep alive message. */
MAMA_MSG_TYPE_NULL = 175,

Is this message type suitable for this case?

--Alireza

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-
dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: Tuesday, April 14, 2015 11:03 AM
To: Glenn McClements; Ian Bell; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what is
to return for mamaMsg_get(MdMsgType, 1), any integer which is not in
standard list of update types?


-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a
middleware level mechanism using OpenMAMA messages, so the solution is
not to modify dqpublishermanager to handle the new use of REFRESHs, but
to make sure that whatever messages are used for heartbeats are never
passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for
this use case as would imply that heartbeats would be
sent/received/processed the OpenMAMA level (which is something we may
add at some point) and *not* at the middleware level. That’s not to say you
can’t add a new specific message type for your own heartbeats, but it should
be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies
replying to an mamaInbox, and for refreshes there is (currently) no inbox to
reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware
heartbeat functionality. Specially they operate at a higher level although
they can be similar in nature.

Normal operation would be to have the transport clean up the client
context information as the clients went away using its own heartbeat
functionality and let MAMA inform the client application of subscribers using
its own refresh messages. The refresh timeout is an hour so this is generally
a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to reduce
the heartbeat traffic as well as to allow for transports which don't maintain
subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh message once
an hour less a random period of time, however the client refresh timeout is
reset on receiving a refresh response from a publisher, so what actually
happens is that the first client to timeout sends a refresh and then the rest of
the clients reset their timeouts on the refresh response from the publisher.
So the publisher only processes a single refresh message per topic as
opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question
about Advanced publishing. In my middleware a publisher keeps track of all
subscribers connected to it. A publisher and subscribers exchange with
heartbeat messages posted to a single topic providing a string with session id
in heartbeats' payload. When a publisher receives a heartbeat it sends a
reply with the same session ID. Once heartbeat is not received by publisher it
stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages
but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl-
mRefreshResponseMsg);
impl-
mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of
communication i.e. session id as mRefreshResponseMsg doesn't have any
fields set. Thus we force users to implement a part of underlying protocol by
handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl-
mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information. If you
are not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and delete this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is strictly
forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If you are
not the intended recipient (or have received this e-mail in error) please
notify the sender immediately and delete this e-mail. Any unauthorized
copying, disclosure or distribution of the material in this e-mail is strictly
forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for
additional EU corporate and regulatory disclosures and to
http://www.db.com/unitedkingdom/content/privacy.htm for information
about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Handling REFRESH messages in DQ publisher manager

Yury Batrakov
 

Classification: Public

Thanks Glenn,

BTW what is best practice of implementing custom messages? I mean what is to return for mamaMsg_get(MdMsgType, 1), any integer which is not in standard list of update types?

-----Original Message-----
From: Glenn McClements [mailto:g.mcclements@srtechlabs.com]
Sent: Tuesday, April 14, 2015 4:53 PM
To: Ian Bell; Yury Batrakov; openmama-dev@lists.openmama.org
Subject: Re: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a middleware level mechanism using OpenMAMA messages, so the solution is not to modify dqpublishermanager to handle the new use of REFRESHs, but to make sure that whatever messages are used for heartbeats are never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for this use case as would imply that heartbeats would be sent/received/processed the OpenMAMA level (which is something we may add at some point) and *not* at the middleware level. That’s not to say you can’t add a new specific message type for your own heartbeats, but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies replying to an mamaInbox, and for refreshes there is (currently) no inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware heartbeat functionality. Specially they operate at a higher level although they can be similar in nature.

Normal operation would be to have the transport clean up the client context information as the clients went away using its own heartbeat functionality and let MAMA inform the client application of subscribers using its own refresh messages. The refresh timeout is an hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to reduce the heartbeat traffic as well as to allow for transports which don't maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh message once an hour less a random period of time, however the client refresh timeout is reset on receiving a refresh response from a publisher, so what actually happens is that the first client to timeout sends a refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher. So the publisher only processes a single refresh message per topic as opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org
[mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury
Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher
manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question about Advanced publishing. In my middleware a publisher keeps track of all subscribers connected to it. A publisher and subscribers exchange with heartbeat messages posted to a single topic providing a string with session id in heartbeats' payload. When a publisher receives a heartbeat it sends a reply with the same session ID. Once heartbeat is not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages but
the following problem occurs: in dqpublishermanager.c there is a
handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid,
MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
impl->mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType,
msgType, msg );


The problem is that in the bridge we cannot restore the context of communication i.e. session id as mRefreshResponseMsg doesn't have any fields set. Thus we force users to implement a part of underlying protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl->mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


Re: Handling REFRESH messages in DQ publisher manager

Glenn McClements <g.mcclements@...>
 

Thanks Ian, you beat me to it!

First of all Yury, welcome to the list and thanks for your question.

As Ian mentions one of the issues here is that you are implementing a middleware level mechanism using OpenMAMA messages, so the solution is not to modify dqpublishermanager to handle the new use of REFRESHs, but to make sure that whatever messages are used for heartbeats are never passed up to the core OpenMAMA level.


I’d also be against adding a new HEARTBEAT OpenMAMA message type for this use case as would imply that heartbeats would be sent/received/processed the OpenMAMA level (which is something we may add at some point) and *not* at the middleware level. That’s not to say you can’t add a new specific message type for your own heartbeats, but it should be middleware/bridge specific.

Also a minor point is that mamaDQPublisher_sendReply() to me implies replying to an mamaInbox, and for refreshes there is (currently) no inbox to reply to.

Thanks,
Glenn

On 14/04/2015 14:34, "Ian Bell" <ian.bell@hapsoftconsulting.co.uk> wrote:

Hi Yury

The MAMA level refreshes are generally separate from the middleware heartbeat functionality. Specially they operate at a higher level although they can be similar in nature.

Normal operation would be to have the transport clean up the client context information as the clients went away using its own heartbeat functionality and let MAMA inform the client application of subscribers using its own refresh messages. The refresh timeout is an hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to reduce the heartbeat traffic as well as to allow for transports which don't maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh message once an hour less a random period of time, however the client refresh timeout is reset on receiving a refresh response from a publisher, so what actually happens is that the first client to timeout sends a refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher. So the publisher only processes a single refresh message per topic as opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question about Advanced publishing. In my middleware a publisher keeps track of all subscribers connected to it. A publisher and subscribers exchange with heartbeat messages posted to a single topic providing a string with session id in heartbeats' payload. When a publisher receives a heartbeat it sends a reply with the same session ID. Once heartbeat is not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages but the following problem occurs: in dqpublishermanager.c there is a handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid, MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
impl->mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType, msgType, msg );


The problem is that in the bridge we cannot restore the context of communication i.e. session id as mRefreshResponseMsg doesn't have any fields set. Thus we force users to implement a part of underlying protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl->mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


Re: Handling REFRESH messages in DQ publisher manager

Yury Batrakov
 

Classification: Public

Hi Ian,

Thanks for the explanation. Will do that on the application level.

-----Original Message-----
From: Ian Bell [mailto:ian.bell@hapsoftconsulting.co.uk]
Sent: Tuesday, April 14, 2015 4:34 PM
To: Yury Batrakov; openmama-dev@lists.openmama.org
Subject: RE: Handling REFRESH messages in DQ publisher manager

Hi Yury

The MAMA level refreshes are generally separate from the middleware heartbeat functionality. Specially they operate at a higher level although they can be similar in nature.

Normal operation would be to have the transport clean up the client context information as the clients went away using its own heartbeat functionality and let MAMA inform the client application of subscribers using its own refresh messages. The refresh timeout is an hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to reduce the heartbeat traffic as well as to allow for transports which don't maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh message once an hour less a random period of time, however the client refresh timeout is reset on receiving a refresh response from a publisher, so what actually happens is that the first client to timeout sends a refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher. So the publisher only processes a single refresh message per topic as opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk



-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question about Advanced publishing. In my middleware a publisher keeps track of all subscribers connected to it. A publisher and subscribers exchange with heartbeat messages posted to a single topic providing a string with session id in heartbeats' payload. When a publisher receives a heartbeat it sends a reply with the same session ID. Once heartbeat is not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages but the following problem occurs: in dqpublishermanager.c there is a handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid, MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
impl->mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType, msgType, msg );


The problem is that in the bridge we cannot restore the context of communication i.e. session id as mRefreshResponseMsg doesn't have any fields set. Thus we force users to implement a part of underlying protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl->mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.


Re: Handling REFRESH messages in DQ publisher manager

Ian Bell
 

Hi Yury

The MAMA level refreshes are generally separate from the middleware heartbeat functionality. Specially they operate at a higher level although they can be similar in nature.

Normal operation would be to have the transport clean up the client context information as the clients went away using its own heartbeat functionality and let MAMA inform the client application of subscribers using its own refresh messages. The refresh timeout is an hour so this is generally a lot higher than a transport heartbeat timeout.

The dqpublisher specifically does a send rather than a sendReply to reduce the heartbeat traffic as well as to allow for transports which don't maintain subscriber lists.

I.e. If you have 4 clients on 1 symbol each will send a refresh message once an hour less a random period of time, however the client refresh timeout is reset on receiving a refresh response from a publisher, so what actually happens is that the first client to timeout sends a refresh and then the rest of the clients reset their timeouts on the refresh response from the publisher. So the publisher only processes a single refresh message per topic as opposed to 1 for every client on every topic.

Thanks,
Ian Bell

Hapsoft Consulting Ltd
138 Belmont Road, Belfast, BT4 2AQ
Office: +447481866051
Web: www.hapsoftconsulting.co.uk

-----Original Message-----
From: openmama-dev-bounces@lists.openmama.org [mailto:openmama-dev-bounces@lists.openmama.org] On Behalf Of Yury Batrakov
Sent: 14 April 2015 13:43
To: openmama-dev@lists.openmama.org
Subject: [Openmama-dev] Handling REFRESH messages in DQ publisher manager

Classification: Public

Hi team,

I'm implementing new OpenMAMA transport bridge and have a question about Advanced publishing. In my middleware a publisher keeps track of all subscribers connected to it. A publisher and subscribers exchange with heartbeat messages posted to a single topic providing a string with session id in heartbeats' payload. When a publisher receives a heartbeat it sends a reply with the same session ID. Once heartbeat is not received by publisher it stops sending data and cleans up the resources allocated for a subscriber.
I tried implementing heartbeats with MAMA_SUBSC_REFRESH messages but the following problem occurs: in dqpublishermanager.c there is a handler for REFRESHES

switch (msgType)
{
case MAMA_SUBSC_REFRESH:
if (!impl->mRefreshResponseMsg)
{
*Here MAMA creates empty reply message*
mamaMsg_create(&impl->mRefreshResponseMsg);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgStatus.mFid, MAMA_MSG_STATUS_MISC);
mamaMsg_addU8(impl->mRefreshResponseMsg, NULL,
MamaFieldMsgType.mFid, MAMA_MSG_TYPE_REFRESH);

}
*And sends it via the bridge*
mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
impl->mUserCallbacks.onRefresh((mamaDQPublisherManager)impl,
info, subType, msgType, msg );


The problem is that in the bridge we cannot restore the context of communication i.e. session id as mRefreshResponseMsg doesn't have any fields set. Thus we force users to implement a part of underlying protocol by handling refreshes themselves.

I'd propose the following change to fix that:

- mamaDQPublisher_send(info->pub, impl->mRefreshResponseMsg);
+ mamaDQPublisher_sendReply(info->pub, msg, impl->mRefreshResponseMsg);

Or could you advise how to workaround it?

Thanks,
Yury.


---
This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and delete this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

Please refer to http://www.db.com/en/content/eu_disclosures.htm for additional EU corporate and regulatory disclosures and to http://www.db.com/unitedkingdom/content/privacy.htm for information about privacy.
_______________________________________________
Openmama-dev mailing list
Openmama-dev@lists.openmama.org
https://lists.openmama.org/mailman/listinfo/openmama-dev

821 - 840 of 2305