Draft to handle negative time scale units

[ronaldtse]

As discussed in tc39/proposal-temporal#819 , CC 18011 did not provide a mechanism for resolving negative time scale units, and as a result ISO 8601-2:2019 lacked this content as well.

Given there is now a need for this, we should update both CC 18011 and ISO 8601-2:2019. I've written up the following content to be added to both (using 8601-2 numbering).

Ping @CalConnect/tc-datetime : let's discuss!

---

(insert) ISO 8601-2:2019, 14.4

In date-time arithmetic, some time scale units can be converted into one another through deterministic relationships, which are transitive. These freely convertible relationships include:

• century and decade (a calendar century contains 10 calendar decades)
• decade and year (a calendar decade contains 10 calendar years)
• year and week (a calendar year contains 52 calendar weeks)
• year and month (a calendar year contains 12 calendar months)
• week and day (a calendar week contains 7 calendar days)
• day and hour (a calendar day contains 24 clock hours)
• hour and minute (a clock hour contains 60 clock minutes)

The following time scale unit conversions are not deterministic in abstract, the actual durations of these time scale units depend on contextual information:

• year and day (a calendar year may contain 365 or 366 calendar days);
• month and day (a calendar month may contain 28, 29, 30, 31 calendar days);
• minute and second (a clock minute may contain 59, 60, 61 clock seconds, in UTC).

These rules are considered as the "time scale unit conversion boundaries".

The result of a date time arithmetic expression does not change with time scale unit conversion as long as the conversion boundaries are not violated.

When the full context of a date time expression is provided, these boundaries can be relaxed accordingly.

EXAMPLE 1 The date time expression '2023Y59O' can be freely converted into '2023Y2M28D', because the 59th ordinal day of a Gregorian year is always February 28th.

EXAMPLE 2 The date time expression '2020Y60O' can be freely converted into '2020Y2M29D', because the 60th ordinal day of the Gregorian year 2020 is February 29th.

EXAMPLE 3 The date time expression '2016Y12M31DT61S' can be freely converted into '2017Y1M1DT-1S', because a leap second was inserted into the last clock minute on 2016-12-31.

(add) ISO 8601-2, D.3.3 Borrow to resolve underflow in time scale components

The composite duration form was meant to delay the resolution of date time duration until application because some time scale units require contextual information.

In some cases, it is possible to convert a duration expression that contains a negative time scale component into an equivalent duration expression where the previously negative time scale component becomes positive.

This process of conversion involves two operations:

• the "borrow" operation, the counterpart to the "carry-over" operation;
• time scale unit conversion.

NOTE: If exact duration is not necessary or is considered too complex to calculate, one could simplify to omit some boundaries to adopt "nominal duration", such as ignoring the leap second (i.e. ignore rule 3) and treat 1 minute = 60 seconds.

EXAMPLE 4 The expression 'P1Y10M3D − P2Y5MT10M' results in 'P3Y15M3DT-10M'. The negative sign at '-10M' can be converted into a positive expression by "borrowing" from a time scale unit of a higher order. Since the "hour" time scale unit is 0, the next time scale unit to borrow from is the "day" unit. One "day" unit expands to 24H, resulting in 'P3Y15M2DT24H-10M'. One "hour" unit can expand into 60M, resulting in 'P3Y15M2DT23H(60-10)M', which ends up being 'P3Y15M2DT23H50M'.

EXAMPLE 5 'P1Y-10M3D + P2Y-5M' results in 'P-1Y-15M3D'. The negative sign at '-1Y' cannot be resolved because there is no higher order unit to borrow from. The negative sign at '-15M' can be resolved by borrowing from the "year" unit. In order to pad the "month" unit to a positive number, the minimum number of years to borrow from "year" is 2, resulting in 'P-3Y(24-15)M3D', which equals 'P-3Y9M3D'.

EXAMPLE 6 'PT1H60S - PT122M' results in 'PT1H-122M60S'. The negative sign at '-122M' can be resolved, because the conversion between "hour" and "minute" does not cross any conversion boundaries. In order to resolve the negative sign at '-122M', a total of 3 hours need to be borrowed, resulting in 'PT(1-3)H(180-122)M60S', which concludes as 'PT-2H58M60S'. This expression can also be represented as 'PT-62M60S', since "hour" and "minute" can be freely converted (-2H = -120M; -120M + 58M = -62M).

EXAMPLE 7 'PT5H120S - PT1M' results in 'PT5H-1M120S'. The negative sign at '-1M' can be resolved by borrowing from the "hour" unit without crossing conversion boundaries. The borrowing of 1 hour to minutes results in 'PT4H(60-1)M120S', which concludes as 'PT4H59M120S'. The '59M120S' portion cannot be simplified as the "minute to seconds" conversion requires contextual information.

(new) ISO 8601-2, D.4.6 Resolving negative time scale units in an expression ("underflows")

The steps are as follows:

1. Starting at the rightmost side of the composite duration expression, process every negative durational unit one by one as follows:
   a) From the negative time scale component (the "underflowed component"), find in the expression a higher order time scale component that is freely convertible (i.e. does not violate conversion boundaries) and has a positive value. There could be multiple higher order units that can be freely convertible, such as minute <> hour and minute <> day.
   b) Borrow units from the higher order value until the current time scale is positive. Ensure the time scale components being modified are properly updated.
   c) Once the underflowed component becomes positive, or if there are no more freely convertible components (even if the underflowed component is still negative), continue with step 1 on the resulting duration expression.
2. Once all durational units have been processed, the resulting date and time representation is complete.

EXAMPLE 1 Calculation of '2022Y2M2D - P1Y10M3D':

• Calculate in accordance to Annex D, compute the intermediate expression '(2022-1)Y(2-10)M(2-3)D' which results in '2021Y-8M-1D'.
• Begin resolving negative time scale units.
  • Process the first underflow component from the right: '-1D'.
  • The "day" unit does not have a freely convertible higher-order component in this expression.
  • Process the second durational unit from the right: '-8M'
  • The "month" unit can be freely convertible with the higher order component "year" in this expression.
  • The "month" unit borrows 1 year, which is equal to 12 months. The partial expression under consideration
    of '2021Y-8M' becomes '2020Y(12-8)M', which resolves to '2020Y4M'.
  • The intermediate expression becomes '2020Y4M-1D'.
  • There are no more underflow components.
• The resulting date and time expression after underflow resolution is '2020Y4M-1D'.

EXAMPLE 2 Calculation of '2025Y59O - P20DT1H30M':

• Notice that the former and latter expressions use 'O' and 'D' to express the same time scale unit of calendar day respectively. This means that their values can be calculated within the same time scale unit. To preserve the meaning of the "ordinal day" time scale component, we select 'O' as the component to perform calculations on.

• Calculate in accordance to Annex D, compute the intermediate expression '2025Y(59-20)OT(-1)H(-30)M' which results in '2025Y39OT-1H-30M'.

• Begin resolving negative time scale units.

  • Process the first underflow component from the right: '-30M'.
  • The "minute" unit can be freely convertible with the higher order component "hour" in this expression.
  • The "minute" unit borrows 1 hour, which is equal to 60 minutes. The partial expression under consideration
    of 'T-1H-30M' becomes 'T(-1-1)H(60-30)M', which resolves to 'T-2H30M'.
  • Process the second durational unit from the right: '-2H'
  • The "hour" unit can be freely convertible with the higher order component "day" in this expression.
  • The "hour" unit borrows 1 day, which is equal to 24 hours. The partial expression under consideration
    of '39OT-2H' becomes '(39-1)OT(24-2)H', which resolves to '38OT22H'.
  • The intermediate expression becomes '2025Y38OT22H'.
  • There are no more underflow components.

• The resulting date and time expression after underflow resolution is '2025Y38OT22H'.

EXAMPLE 3 Calculation of 'T10H10M10S - PT5H30M20S':

• Calculate in accordance to Annex D, compute the intermediate expression 'T(10-5)H(10-30)M(10-20)S' which results in 'T5H-20M-10S'.
• Begin resolving negative time scale units.
  • Process the first underflow component from the right: '-10S'.
  • The "second" unit does not have a freely convertible higher-order component in this expression.
  • Process the second durational unit from the right: '-20M'
  • The "minute" unit can be freely convertible with the higher order component "hour" in this expression.
  • The "minute" unit borrows 1 hour, which is equal to 60 minutes. The partial expression under consideration
    of 'T5H-20M' becomes 'T(5-1)H(60-20)M', which resolves to 'T4H40M'.
  • The intermediate expression becomes 'T4H40M-10S'.
  • There are no more underflow components.
• The resulting date and time expression after underflow resolution is 'T4H40M-10S'.

[sffc]

Thanks for writing this up!

EXAMPLE 3 The date time expression '2016Y12M31DT61S' can be freely converted into '2017Y1M1DT-1S', because a leap second was inserted into the last clock minute on 2016-12-31.

Should the start date time expression end with 60S instead of 61S, or else should the resulting date time expression end in 0S instead of -1S? If there were 61 seconds in the last clock minute of 2016-12-31, then the last second of that year was T60S, not T61S.

D.4.6 ... EXAMPLE 1 ... The resulting date and time expression after underflow resolution is '2020Y4M-1D'.

Is the idea that you would then convert this to 2020Y3M30D in accordance with the new section 14.4? (Note: since days are 1-indexed, -1D is actually 2 days before the first of the month, which is why I ended up with March 30th instead of March 31st.)

I like this model of considering multiple time scale expressions as equivalent. What would be even better would be an algorithm to go from any time scale expression to the "canonical" expression in its equivalence class. For example, all of the following time scale expressions are in the same equivalence class:

• 2020Y3M30D
• 2020Y4M-1D
• 2020Y2M59D
• 2019Y3M396D

The first is the canonical form. Can we have a simple algorithm that maps a time scale expression to its canonical form?

---

What happens in the following example: PT180S - PT2M? You get PT-2M180S. There are no fields from which the -2M can borrow. However, it could borrow from the right, resulting in 60S, or, equivalently, 1M.

[ronaldtse]

Should the start date time expression end with 60S instead of 61S, or else should the resulting date time expression end in 0S instead of -1S? If there were 61 seconds in the last clock minute of 2016-12-31, then the last second of that year was T60S, not T61S.

You're absolutely right. It should be 2016Y12M31DT60S. Blaming lack of sleep...

EXAMPLE` 3 The date time expression '2016Y12M31DT60S' can be freely converted into '2017Y1M1DT-1S', because a leap second was inserted into the last clock minute on 2016-12-31.

Since this is clearly a point of confusion, we'll also insert that information somewhere.

NOTE: In the explicit form of a date time expression, the "clock second" component typically goes from 'T0S' (representing the first second of the minute) to 'T59S' (representing the 60th second of the minute). The last second of a minute is represented by 'T-1S'. If a minute does not contain a leap second, 'T59S' is identical to 'T-1S' in the same minute.
When a minute contains a positive leap second, 'T60S' represents the unit of the 61st second of a minute, and therefore corresponds to 'T-1S' in the same minute. In the case of a negative leap second, 'T58S' represents the unit of the 59th second of a minute, which corresponds to 'T-1S' in this minute.

NOTE: In a duration expression, the component 'T0S' represents a duration of zero seconds, while 'T60S' represents a duration of 60 seconds. Implementers should be aware of the semantic differences between a seemingly identical syntax for a data time expression and a duration expression.

---

Is the idea that you would then convert this to 2020Y3M30D in accordance with the new section 14.4? (Note: since days are 1-indexed, -1D is actually 2 days before the first of the month, which is why I ended up with March 30th instead of March 31st.)

With regards to -1D, 8601-2 gives the following explanation.

While days are 1-indexed, the negative index notation always starts from -1 (-1 - last, -2 - last 2nd, ...). Perhaps I'm missing something here?

---

I like this model of considering multiple time scale expressions as equivalent. What would be even better would be an algorithm to go from any time scale expression to the "canonical" expression in its equivalence class. For example, all of the following time scale expressions are in the same equivalence class:

2020Y3M30D
2020Y4M-1D
2020Y2M59D
2019Y3M396D
The first is the canonical form. Can we have a simple algorithm that maps a time scale expression to its canonical form?

Note that these sample expressions are date time expressions with concrete context, so they are by definition resolvable to a canonical expression.

I think we could come up with a canonical form definition: the expression contains minimal underflow and no overflow.

The algorithm can be as simple as:

1. Reduce all underflows, if there are any negative components
2. Remove all overflows (is this called "balance" in your terminology?)

For expressions without context, e.g. ones like 'P3Y15M3DT-10M', we cannot cross the boundaries, meaning that it is possible to have at most 3 underflowing units we cannot further resolve (year <> day, month <> day, min <> sec).

i.e.:

• if sec underflows, we cannot resolve it. (1 unresolved unit)
• if day underflows, we cannot resolve it. (1 unresolved unit)
• if the largest time scale component underflows, we cannot resolve it (it is intended) (1 unresolved unit)

EXAMPLE 'P3Y15M-3DT120M'.

1. Reduce underflow. Can't cross "day" boundary. No action.
2. Remove overflow. 'P4Y3M-3DT2H'

EXAMPLE 'PT28H-122M60S'

1. Reduce underflow. 'PT25H58M60S'
2. Remove overflow. 'PT1D1H58M60S'

Sample of duration expression with 2 underflowing units:

EXAMPLE 'P15M-12DT30H-10S'

1. Reduce underflow. 'P15M-12DT30H-10S'. Can't cross day and second boundaries.
2. Remove overflow. 'P1Y3M-11DT6H-10S'

Sample of resolving multiple underflow units:

EXAMPLE 'P3Y-15M3DT-8H-30M9S'

1. Reduce underflow. 'P2Y9M2DT15H30M9S'. No need to cross day and second boundaries.
2. Remove overflow. 'P2Y9M2DT15H30M9S'. Not necessary.

Sample of having 3 underflowing units:

EXAMPLE 'P1Y-15M-3DT-8H-30M-9S'

1. Reduce underflow. 'P-1Y9M-4DT15H30M-9S'. Can't cross day and second boundaries.
2. Remove overflow. 'P-1Y9M-4DT15H30M-9S'. Not necessary.

The fortunate thing is the underflow algorithm naturally propagates the negative units as far left as possible (without trespassing the boundaries), so the result is canonical.

[ronaldtse]

Just realized I missed answering this:

What happens in the following example: PT180S - PT2M? You get PT-2M180S. There are no fields from which the -2M can borrow. However, it could borrow from the right, resulting in 60S, or, equivalently, 1M.

If we hold the conversion boundaries tight, PT-2M180S is the final expression. We are not supposed to borrow "minute" to "seconds" because "a minute" could be 59, 60 or 61 seconds.

Because we know that leap seconds are inserted either on Jun 30/Dec 31/Mar 31/Sept 30, we can assume that there in a consecutive two (or three) minutes there can only be 1 minute that has the leap second (positive or negative):

• Then amongst a period of 2M (minutes), we can always remove 60S. Amongst a period of 180S, we can always remove 2M. As long as we save 1 minute, we're safe.
• Then this results in 'PT60S'.

But this would require a more complex algorithm that understands the extent of boundaries.

[sffc]

While days are 1-indexed, the negative index notation always starts from -1 (-1 - last, -2 - last 2nd, ...). Perhaps I'm missing something here?

Got it. In that case, we just need to make sure the following behavior is well-defined: 2020Y4M2D - 5D. Naively it looks like the solution should be 2020Y4M-3D, but the correct solution is actually 2020Y4M-4D.

Note that these sample expressions are date time expressions with concrete context, so they are by definition resolvable to a canonical expression.

Right. I think there are two classes of algorithms:

1. When context is available and we can freely move data across all boundaries.
2. When context is unavailable and we cannot move data across certain boundaries.

For the second case, it would be good if the algorithm were general enough to allow additional boundaries beyond those required by the ISO-8601 proleptic Gregorian calendar system. For example, it would be nice to apply the algorithm to a Hebrew date (in which case there is a boundary between months and years) or a date with an IANA time zone like America/Chicago (in which case there is a boundary between hours and days).

If I understand correctly, ISO-8601-2 uses the term "time scale units" to mean datetime expressions with context.

If we hold the conversion boundaries tight, PT-2M180S is the final expression. We are not supposed to borrow "minute" to "seconds" because "a minute" could be 59, 60 or 61 seconds.

Right. Bad example. How about PT180M - PT2H? Intermediate is PT-2H180M. Is there an algorithm to represent this as PT60M?

As a general note, in ECMAScript Temporal, we've found that it is useful to keep a distinction between duration fields, even when they are convertible. For example, "90 minutes" and "1 hour 30 minutes" are useful to express as different concepts.

[ronaldtse]

Got it. In that case, we just need to make sure the following behavior is well-defined: 2020Y4M2D - 5D. Naively it looks like the solution should be 2020Y4M-3D, but the correct solution is actually 2020Y4M-4D.

Your example is good. If you don't mind I'd like to steal it for the standard readers... 😉

For the second case, it would be good if the algorithm were general enough to allow additional boundaries beyond those required by the ISO-8601 proleptic Gregorian calendar system. For example, it would be nice to apply the algorithm to a Hebrew date (in which case there is a boundary between months and years) or a date with an IANA time zone like America/Chicago (in which case there is a boundary between hours and days).

Agreed. There is nothing that stops us from applying the same algorithm to other calendar systems, but the potentially changing boundaries need to be declared and respected. Unfortunately I am not familiar with the Hebrew system, but presumably someone else is here...?

Timezones are can pose a problem because with DST or Winter Time it is possible to have overlapping time representation (disjoint is a much less serious issue). In these cases, the transition periods serve as boundaries and they should be respected. This requires some thinking (which is beyond me at this time of day).

Right. Bad example. How about PT180M - PT2H? Intermediate is PT-2H180M. Is there an algorithm to represent this as PT60M?

Yes, the "underflow then overflow" algorithm (#12 (comment)) resolves:

• 'PT-2H180M' => (resolve overflows) into 'PT1H'.

As a general note, in ECMAScript Temporal, we've found that it is useful to keep a distinction between duration fields, even when they are convertible. For example, "90 minutes" and "1 hour 30 minutes" are useful to express as different concepts.

Great tip, thanks!

[gilmoreorless]

I've come here via the ECMAScript Temporal proposal, and I have some questions. Forgive me if there's some context I'm missing, but some of the examples are confusing me, and I want to make sure I'm reading things correctly.

Section 14.4

These freely convertible relationships include:

• year and week (a calendar year contains 52 calendar weeks)
• week and day (a calendar week contains 7 calendar days)

Is that correct? I don't see how a year can freely convert to 52 weeks. It seems odd to me that year <-> day is a boundary, but year <-> week <-> day is freely convertable.

Section D.3.3

EXAMPLE 4 The expression 'P1Y10M3D − P2Y5MT10M' results in 'P3Y15M3DT-10M'.

This doesn't add up, but I think you already addressed this at tc39/proposal-temporal#819 (comment)

EXAMPLE 5 'P1Y-10M3D + P2Y-5M' results in 'P-1Y-15M3D'.

Shouldn't it result in 'P3Y-15M3D'?

Follow-up comment: #12 (comment)

EXAMPLE 'P3Y-15M3DT-8H-30M9S'

1. Reduce underflow. 'P2Y9M2DT15H30M9S'. No need to cross day and second boundaries.
2. Remove overflow. 'P2Y9M2DT15H30M9S'. Not necessary.

The year values don't look right to me. Should they be 'P1Y9M...'?

[ronaldtse]

Is that correct? I don't see how a year can freely convert to 52 weeks. It seems odd to me that year <-> day is a boundary, but year <-> week <-> day is freely convertable.

Good point, and you are right. The reason is slightly different from common sense -- in ISO 8601 the definition of a "ISO 8601 calendar week year" is 52 or 53 full weeks. Therefore "year <> week" is not freely convertible. (This has the side effect that the "ISO 8601 calendar week year" does not fully correspond with the "Gregorian calendar year").

EXAMPLE 4 The expression 'P1Y10M3D − P2Y5MT10M' results in 'P3Y15M3DT-10M'.
This doesn't add up, but I think you already addressed this at tc39/proposal-temporal#819 (comment)

Ugh I copied the non-corrected example straight out of the standard, so the error persisted. Will fix.

EXAMPLE 5 'P1Y-10M3D + P2Y-5M' results in 'P-1Y-15M3D'.
Shouldn't it result in 'P3Y-15M3D'?

Yes you are right.

EXAMPLE 'P3Y-15M3DT-8H-30M9S'
Reduce underflow. 'P2Y9M2DT15H30M9S'. No need to cross day and second boundaries.
Remove overflow. 'P2Y9M2DT15H30M9S'. Not necessary.
The year values don't look right to me. Should they be 'P1Y9M...'?

Again, you are correct.

@gilmoreorless Thanks for pointing out the errors!