Subject: [Lager] Fwd: When are sequences valid in an LGR

From: Asmus Freytag <[email-masked]>
Date: Fri, 29 Jan 2016 12:07:51 -0800

All,

in pursuing the root zone LGR work, we have stumbled across a small 
ambiguity in treating sequences of code points. We had resolved how to 
treat this case before in internal discussion, but apparently not 
updated the language of the specification.

This is the first of three forwarded messages that discusses how to 
remove the ambiguity from the specification by clarifying the text in 
section 7.

A./

-------- Forwarded Message --------
Subject: 	When are sequences valid in an LGR
Date: 	Tue, 26 Jan 2016 19:47:08 -0800
From: 	Asmus Freytag <[email-masked]>
To: 	Wil Tan <[email-masked]>, Kim Davies <[email-masked]>

Let's take the case of an LGR as follows:

a
b
ab when=x

and a label "ab".

How do we evaluate whether this label is valid:

We have, in Section 7.1 of XML-LGR the following:

    In order to test a specific label for membership in the LGR, a
    consumer of the LGR must iterate through each code point within a
    given label, and test that each code point *is a member* of the LGR.
    If any code point is not a member of the LGR, the label shall be
    deemed as invalid.

    An individual code point *is deemed a member* of the LGR when it is
    listed using a "char" element, or is part of a range defined with a
    "range" element, and all necessary condition in any "when" or
    "not-when" attributes are correctly satisfied.

    Alternatively, a code point is*also deemed a membe*r of the LGR when
    it forms part of a sequence that corresponds to a sequence listed
    using a "char" element for which the "cp" attribute defines a
    sequence, and all necessary condition in any "when" or "not-when"
    attributes are correctly satisfied.

Clearly, we have

a = member
b = member
ab:
      if ab satisfies x, then a and b are members
      else neither a or b are members

Now, what is not fully resolved in this specification is whether in the 
case of !x, the not-member status of the elements of a,b, overrides 
their individual statuses as members of the LGR by virtue of being 
listed by themselves.

What is also not resolved is how to determine the variants for this label.

a -> c
b -> d
ab -> ef

do we assume (a)(b) and map to (c),(d) (as well as to ad and cb)?
or do we assume (ab) and map to (ef),
or do we do both?

I believe what we want to add is this paragraph:

    "In determining eligibility, the longest possible sequence of code
    points is evaluated first. If it satisfies any required context, its
    constituent code points are deemed members of the LGR and evaluation
    proceeds with the next code point following the sequence. If the
    sequence does not satisfy the required context, successively shorter
    sequences are evaluated, until only a single code point remains, the
    eligibility of which is determined as described above for an
    individual code point."

For variants, we do need to process all possible partitions of the label:

    "In determining the permuted set of variant labels, all eligible
    partitions into sequences must be mapped. A label "ab", that matches
    a sequence "ab" defined in the LGR as well as the sequence of
    individual code points "a" and "b" (both defined in the LGR), must
    be permuted using any defined variant mappings for "ab", "a" and "b"."

This language needs to go in the specification of the variants 
processing. If we don't add this, the coexistence of sequences and 
individual constituents thereof will be undefined.

A./

---

From: Asmus Freytag <[email-masked]>
Date: Fri, 29 Jan 2016 12:08:21 -0800

Second forwarded message.

-------- Forwarded Message --------
Subject: 	Re: When are sequences valid in an LGR
Date: 	Wed, 27 Jan 2016 16:35:56 +1100
From: 	Wil Tan <[email-masked]>
To: 	Asmus Freytag <[email-masked]>
CC: 	Kim Davies <[email-masked]>

Hi Asmus,

I have run into this ambiguity as well. Please see my comments inline..

On Wed, Jan 27, 2016 at 2:47 PM, Asmus Freytag <[email-masked] 
<mailto:[email-masked]>> wrote:

    a
    b
    ab when=x

    and a label "ab".

        In order to test a specific label for membership in the LGR, a
        consumer of the LGR must iterate through each code point within
        a given label, and test that each code point *is a member* of
        the LGR. If any code point is not a member of the LGR, the label
        shall be deemed as invalid.

        An individual code point *is deemed a member* of the LGR when it
        is listed using a "char" element, or is part of a range defined
        with a "range" element, and all necessary condition in any
        "when" or "not-when" attributes are correctly satisfied.

        Alternatively, a code point is*also deemed a membe*r of the LGR
        when it forms part of a sequence that corresponds to a sequence
        listed using a "char" element for which the "cp" attribute
        defines a sequence, and all necessary condition in any "when" or
        "not-when" attributes are correctly satisfied.

Actually, I find that the phrase:

"individual code point is deemed a member of the LGR"

reads a bit like:

"individual code point is deemed part of the repertoire of the LGR"

Perhaps it would be better to say:

"individual code point /instance/ is deemed a member of the LGR"

to make it clear that we are talking about an /instance/ of the code 
point occurring within the label being tested.

    Clearly, we have

    a = member
    b = member
    ab:

    Now, what is not fully resolved in this specification is whether in
    the case of !x, the not-member status of the elements of a,b,
    overrides their individual statuses as members of the LGR by virtue
    of being listed by themselves.

    What is also not resolved is how to determine the variants for this
    label.

    a -> c
    b -> d
    ab -> ef

        "In determining eligibility, the longest possible sequence of
        code points is evaluated first. If it satisfies any required
        context, its constituent code points are deemed members of the
        LGR and evaluation proceeds with the next code point following
        the sequence. If the sequence does not satisfy the required
        context, successively shorter sequences are evaluated, until
        only a single code point remains, the eligibility of which is
        determined as described above for an individual code point."

This looks good, except "constituent code points" => "constituent code 
point instances"?

    For variants, we do need to process all possible partitions of the
    label:

        "In determining the permuted set of variant labels, all eligible
        partitions into sequences must be mapped. A label "ab", that
        matches a sequence "ab" defined in the LGR as well as the
        sequence of individual code points "a" and "b" (both defined in
        the LGR), must be permuted using any defined variant mappings
        for "ab", "a" and "b"."

Looks good too.

Agreed that this would tighten up the spec, and avoid interoperability 
issues.

.wil