Open MAMA DateTime object


Frank Quinn <f.quinn@...>
 

Hi Folks,

Please see thread below - interesting point raised about date time and the Year 2038 problem. I’ve weighed in with my thoughts, further opinions welcome…

Cheers,
Frank

From: Frank Quinn
Sent: 08 September 2015 14:10
To: 'Phelan, Nigel' <nigel.phelan@...>
Cc: Glenn McClements <g.mcclements@...>; Alpert, Reed <reed.alpert@...>
Subject: RE: Open MAMA DateTime object

 

Hi Nigel,

In short, yes – your understanding aligns with mine. Using a uint32 rather than an int32 has doubled our date range, but sooner or later it looks like we will need to face this.

It’s interesting because when we discussed this last week, I had the uint64 bitmap in my head and I had a pretty good idea that there were a few bits going spare that we could use if we needed to (even if they weren’t sequential), but I had overlooked the function prototypes themselves – yes some of those would need changed or new implementations added to use 64 bit values.

There is a nibble going spare to store some additional data, and we should be able to use a bit in the hints for letting MAMA know which date format to use for encode / decode… so yes at a glance I think we can extend in a backwards compatible way with some creative bit shifting.

I’m in two minds about whether it’s best in OpenMAMA 6 or OpenMAMA 7 – I’m sure Glenn will weigh in with an opinion here, but the only way I can think of that we can add it to OpenMAMA 6 is if we create new MAMA function calls and don’t modify any existing ones (e.g. mamaDateTime_setEpochTimeEx), and then deprecate that method when OpenMAMA 7 is released. If I had to choose I would prefer to just go big bang with OpenMAMA 7 rather than create an interim method in OpenMAMA 6 which is destined to be retired in the next major release anyway, but it will all depend on how urgent a requirement this is for Year 2106+.

Cheers,
Frank

 

From: Phelan, Nigel [mailto:nigel.phelan@...]
Sent: 07 September 2015 14:17
To: Frank Quinn <f.quinn@...>
Cc: Glenn McClements <g.mcclements@...>; Alpert, Reed <reed.alpert@...>
Subject: Open MAMA DateTime object

 

Hi Frank – following on from our discussion last week I spoke to Reed in our New York team, and he thought:

·         The range of dates you can store in a MamaDateTime is from 1970 to 2106 (further out than I thought, but still a problem for us).

·         The underlying representation has 4 bytes of seconds and 20 bits of microseconds; he thought the type could be extended in a backward compatible way, but that new methods might be needed to handle an extended range as some of them currently return a uint_32 seconds value.

How does that fit with your observations?  Do you think this is something that could be addressed in Open MAMA 6, or are we going to have to defer it to Open MAMA 7?

 

Nigel

 


Nigel Phelan | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 25 Bank Street, London E14 5JP| T: +44-(0)20-7134-9558 | nigel.phelan@...

 

This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email


The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC


Alpert, Reed <reed.alpert@...>
 

Hi,

 

Some notes on the datetime 2038 & 2106 issues.

 

·         Today

o   OM holds seconds in unsigned 4 bytes (as part of 8 byte struct).

§  4,294,967,295 secs, ~136 years.

o   This is the same for 32- and 64-bit builds.

o   Max stored date is Jan 28, 2106.

§  32 bits.

§  4,294,967,295 seconds.

§  49,710.26961805556 days.

§  136 years.

·         Why

o   Some bonds have issue dates before 1970.

§  Perpetual bonds issued a long time ago.

§  Don’t think we can fix this since Linux/C standard is 1970.

o   Some bonds mature after 2106.

§  This is fixed with expanding the seconds storage.

 

·         However

o   Windows has fixed the 2038 issue (VS8 or later).

o   Linux it is still there:

§  On 32-bit build time_t is 4 bytes.

§  On 64-bit build time_t is 8 bytes.

§  Calling gmtime_r gives: (used in utcTm, which is called ~7 times in datetime.c):

·         2038 for 32-bit.

·         2106 for 64-bit.

 

·         Possible solution:

o   There are 4 bits available in the data struct, but they are not contiguous to existing seconds.

o   Adding 4 more bits goes to the year 4147.

§  97,982,352,000 seconds from 1970.

o   Need to redefine the masks so the newly-used 4 bits are contiguous.

o   Need full rebuild: OM, bridges, apps.

o   Update methods to use u64 for seconds.

o   Potential break of existing apps.

§  But it may just be a cast to u32 or change to use u64.

§  Or truncate will occur depending upon the compiler; this causes the date to stop at the same as now.

o   Upsides:

§  Keeps datetime 4-bytes for better performance.

o   Downsides:

§  All bits in struct are used, no room for any other expansion.

§  Big-bang, the bridge, OM, and apps all need to rebuild at once.

·         But that is mostly common with a major release.

§  Capture data will not work if stored as binary.

 

Thanks,

 

Reed.

 

________________________________________

Reed Alpert | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 4 Metrotech Center, 23rd Floor, Brooklyn, NY 11245 | T: 718.242.5198  | M: 917.414.4613 | reed.alpert@...

Alternate Contact:  CIB PIM Trading Technology Solutions NA | CIB_PIM_Trading_Technology_Solutions_NA@...

 

 

From: Frank Quinn [mailto:f.quinn@...]

Sent: Tuesday, September 08, 2015 9:25 AM

To: openmama-dev@...

Cc: Glenn McClements; Alpert, Reed; Phelan, Nigel

Subject: RE: Open MAMA DateTime object

 

Hi Folks,

Please see thread below - interesting point raised about date time and the Year 2038 problem. I’ve weighed in with my thoughts, further opinions welcome…

Cheers,

Frank

From: Frank Quinn

Sent: 08 September 2015 14:10

To: 'Phelan, Nigel' <nigel.phelan@...>

Cc: Glenn McClements <g.mcclements@...>; Alpert, Reed <reed.alpert@...>

Subject: RE: Open MAMA DateTime object

 

Hi Nigel,

 

In short, yes – your understanding aligns with mine. Using a uint32 rather than an int32 has doubled our date range, but sooner or later it looks like we will need to face this.

 

It’s interesting because when we discussed this last week, I had the uint64 bitmap in my head and I had a pretty good idea that there were a few bits going spare that we could use if we needed to (even if they weren’t sequential), but I had overlooked the function prototypes themselves – yes some of those would need changed or new implementations added to use 64 bit values.

There is a nibble going spare to store some additional data, and we should be able to use a bit in the hints for letting MAMA know which date format to use for encode / decode… so yes at a glance I think we can extend in a backwards compatible way with some creative bit shifting.

 

I’m in two minds about whether it’s best in OpenMAMA 6 or OpenMAMA 7 – I’m sure Glenn will weigh in with an opinion here, but the only way I can think of that we can add it to OpenMAMA 6 is if we create new MAMA function calls and don’t modify any existing ones (e.g. mamaDateTime_setEpochTimeEx), and then deprecate that method when OpenMAMA 7 is released. If I had to choose I would prefer to just go big bang with OpenMAMA 7 rather than create an interim method in OpenMAMA 6 which is destined to be retired in the next major release anyway, but it will all depend on how urgent a requirement this is for Year 2106+.

 

Cheers,

Frank

 

From: Phelan, Nigel [mailto:nigel.phelan@...]

Sent: 07 September 2015 14:17

To: Frank Quinn <f.quinn@...>

Cc: Glenn McClements <g.mcclements@...>; Alpert, Reed <reed.alpert@...>

Subject: Open MAMA DateTime object

 

Hi Frank – following on from our discussion last week I spoke to Reed in our New York team, and he thought:

•         The range of dates you can store in a MamaDateTime is from 1970 to 2106 (further out than I thought, but still a problem for us).

•         The underlying representation has 4 bytes of seconds and 20 bits of microseconds; he thought the type could be extended in a backward compatible way, but that new methods might be needed to handle an extended range as some of them currently return a uint_32 seconds value.

How does that fit with your observations?  Do you think this is something that could be addressed in Open MAMA 6, or are we going to have to defer it to Open MAMA 7?

 

Nigel

 

________________________________________

Nigel Phelan | Corporate & Investment Bank | Market Data Services | J.P. Morgan | 25 Bank Street, London E14 5JP| T: +44-(0)20-7134-9558 | nigel.phelan@...

 

This email is confidential and subject to important disclaimers and conditions including on offers for the purchase or sale of securities, accuracy and completeness of information, viruses, confidentiality, legal privilege, and legal entity disclaimers, available at http://www.jpmorgan.com/pages/disclosures/email

 

The information contained in this message may be privileged and confidential and protected from disclosure. If the reader of this message is not the intended recipient, you are hereby notified that any dissemination, distribution or copying of this communication is strictly prohibited. If you have received this communication in error, please notify us immediately by replying to this message and deleting it from your computer. Thank you. SR Labs LLC

This communication is for informational purposes only. It is not intended as an offer or solicitation for the purchase or sale of any financial instrument or as an official confirmation of any transaction. All market prices, data and other information are not warranted as to completeness or accuracy and are subject to change without notice. Any comments or statements made herein do not necessarily reflect those of JPMorgan Chase & Co., its subsidiaries and affiliates (collectively, "JPMC"). This transmission may contain information that is proprietary, privileged, confidential and/or exempt from disclosure under applicable law. If you are not the intended recipient, you are hereby notified that any disclosure, copying, distribution, or use of the information contained herein (including any reliance thereon) is STRICTLY PROHIBITED. If you received this transmission in error, please immediately contact the sender and destroy the material in its entirety, whether in electronic or hard copy format. Although this transmission and any attachments are believed to be free of any virus or other defect that might affect any computer system into which it is received and opened, it is the responsibility of the recipient to ensure that it is virus free and no responsibility is accepted by JPMC for any loss or damage arising in any way from its use. Please note that any electronic communication that is conducted within or through JPMC's systems is subject to interception, monitoring, review, retention and external production in accordance with JPMC's policy and local laws, rules and regulations; may be stored or otherwise processed in countries other than the country in which you are located; and will be treated in accordance with JPMC policies and applicable laws and regulations. Please refer to http://www.jpmorgan.com/pages/disclosures for disclosures relating to European legal entities.