Re: Enforcing field type on publish


Alpert, Reed <reed.alpert@...>
 

Yes, this is C++, but we also have Java and DotNet clients (some C# and some Excel).

We do publish best practices, but our app dev groups sometimes do not follow them, and we want to prevent the SolCache from getting out of sync with the dictionary.

We also provide them with sample apps that have this type of control, but sometimes … J

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

 

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

From: Frank Quinn [mailto:fquinn.ni@...]
Sent: Thursday, September 04, 2014 4:56 PM
To: Glenn McClements
Cc: Alpert, Reed; Openmama-dev@...
Subject: Re: [Openmama-dev] Enforcing field type on publish

 

Guessing by your capitalization, I'm guessing this is C++? If so, have you considered mandating that all function calls use MamaFieldDescriptors pulled from a standardized dictionary rather than using string / fid pairs?

Cheers,
Frank

 

On Thu, Sep 4, 2014 at 9:33 PM, Glenn McClements <gmcclements@...> wrote:

There are a number of places where this *could* be done, but not all make sense to me:

 

- In the payload library - this doesn't feel right as I feel the payload should be 'dumb' and just trust the caller. Also it would need to be done for each supported payload. 

 

- In the payload bridge - same points as above

 

- In MamaMsg - this would work across all supported payloads, but again I think the payload should be trust the caller as to what fields should be what type and act a a simple wrapper 

 

- In a field cache (e.g. the C & C++ MamaFieldCache) - this makes the most sense to me as the logic for maintaining state about fields and if they have changed is already there, so extra rules and checks would belong here

 

With the current MamaFieldCache you could do this already if you primed the fields by creating them with the correct type on startup, this would depend on having a well known set of fields though because you don't want to be adding everything in the dictionary. You could also try this technique with the MamaMsg object, but it will be payload specific because some will throw an error if you try to update a field with a new type, and others may replace it.

 

Glenn 

 

 

 

From: Alpert, Reed [reed.alpert@...]
Sent: 03 September 2014 15:28
To: Glenn McClements; Openmama-dev@...


Subject: RE: Enforcing field type on publish

 

Hi,

 

Yes, the addString() is from MamaMsg.

 

The cache is our own market data infrastructure, not the MamaFieldCache.

In this case it is either Solace/SolCache or TREP RTIC.

 

We want business app dev groups to be able to look at the dictionary, see a field that is MamaPrice, and be guaranteed that MamaMsg.getPrice() will always work (although MamaPrice.getIsValidPrice() may return false).

A certain number of the type mismatches come from Excel, where a failed formula may populate a cell as #NA, and that gets published into a price/float field.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

 

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

From: Glenn McClements [mailto:gmcclements@...]
Sent: Wednesday, September 03, 2014 3:10 PM
To: Alpert, Reed; Openmama-dev@...
Subject: RE: Enforcing field type on publish

 

Hi Guy,

When you refer to 'addString' below you mention a cache, but this is not a method of the C/C++MamaFieldCache

are you refering to your own cache object or actually a MamaMsg object?

 

Regards,

Glenn 

 


From: openmama-dev-bounces@... [openmama-dev-bounces@...] on behalf of Alpert, Reed [reed.alpert@...]
Sent: 03 September 2014 13:59
To: Openmama-dev@...
Subject: [Openmama-dev] Enforcing field type on publish

Hi,

 

This is regarding a way to check that the field data type being published matches (closely) the dictionary data type.

 

The motivation for this is to prevent our internally published data caches from invalid data.

For example, if a field (name=MARK_PRICE fid=300) is a MamaPrice in the dictionary, and an app uses this:

                addString("MARK_PRICE", 300, "invalid price");

then we have a field that is now a string in the cache, and that causes problems with subscribing apps that use getPrice().

 

Currently we are looking to the bridges that we use (Tick42/TREP & Solace) to enforce these restrictions.

 

Adding this to the OpenMAMA layer is tricky, but allows a single policy to be enforced across multiple bridges. The main question from my point of view being how to set the policy. They can range from none (default), to most strict (published field's type must match dictionary), to common conversions supported (e.g., F32 -> F64), to a custom grid setting exactly what conversions are allowed.

 

I welcome any input from others who have run into this same issue, or found other ways to solve it.

 

Thanks,

 

Reed.

 


Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

 

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of Intercontinental Exchange, Inc. (ICE), Euronext or any of their subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.


This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.


This message may contain confidential information and is intended for specific recipients unless explicitly noted otherwise. If you have reason to believe you are not an intended recipient of this message, please delete it and notify the sender. This message may not represent the opinion of Intercontinental Exchange, Inc. (ICE), Euronext or any of their subsidiaries or affiliates, and does not constitute a contract or guarantee. Unencrypted electronic mail is not secure and the recipient of this message is expected to provide safeguards from viruses and pursue alternate means of communication where privacy or a binding message is desired.



_______________________________________________
Openmama-dev mailing list
Openmama-dev@...
https://lists.openmama.org/mailman/listinfo/openmama-dev

 

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates. This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMorgan Chase & Co., its subsidiaries and affiliates, as applicable, for any loss or damage arising in any way from its use. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Thank you. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.

Join Openmama-dev@lists.openmama.org to automatically receive all group messages.