Subject: [scim] Filtering against Null Attribute Values

From: Shelley <[email-masked]>
Date: Tue, 22 Sep 2020 13:10:55 -0500

Do the SCIM specifications prescribe any special behavior for filtering and
sorting by attributes that are null/unspecified?

For instance, given an optional user extension attribute for "age" and the
following filters:

   - filter=age lt 18
   - filter=age gt 18

Would either of these filters return a user resource with no specified age,
such that nulls have a prescribed ordering? Or, would both of these filters
exclude all users without specified ages? (And, if it's the latter, what
effect is negation intended to have on the filter?)

This question applies to filters, matching resources during PATCH, sorting,
and other attribute operators as well (e.g. would not (displayName co
"test") match users without display names?).

Thanks in advance for any clarifications!

---

From: Phil Hunt <[email-masked]>
Date: Tue, 22 Sep 2020 13:11:59 -0700

Shelley

It is indirectly specified in the last paragraph of 3.4.2.1
   For filtered attributes that are not part of a particular resource
   type, the service provider SHALL treat the attribute as if there is
   no attribute value.  For example, a presence or equality filter for
   an undefined attribute evaluates to false.

In your example below, both conditions would evaluate false as the value is undefined (and neither statement can not be said to lexicographically true) —> leading to a confusing situation where a resource is neither lt 18 or ge 18. A more qualified filter might be:
filter=age pr and age lt 18 
filter=age pr and age gt 18

Phil Hunt
@independentid
[email-masked]

> On Sep 22, 2020, at 11:10 AM, Shelley <[email-masked]> wrote:
> 
> Do the SCIM specifications prescribe any special behavior for filtering and sorting by attributes that are null/unspecified?
> 
> For instance, given an optional user extension attribute for "age" and the following filters:
> filter=age lt 18
> filter=age gt 18
> Would either of these filters return a user resource with no specified age, such that nulls have a prescribed ordering? Or, would both of these filters exclude all users without specified ages? (And, if it's the latter, what effect is negation intended to have on the filter?)
> 
> This question applies to filters, matching resources during PATCH, sorting, and other attribute operators as well (e.g. would not (displayName co "test") match users without display names?).
> 
> _______________________________________________
> scim mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/scim

---

From: Shelley <[email-masked]>
Date: Tue, 22 Sep 2020 16:40:12 -0500

Thanks. I agree that the more qualified filter you provided is clearer; I'm
trying to determine how to properly handle various filters as a service
provider.

I can understand the less than/greater than comparison operators (lt, gt,
ge, le) evaluating to false for unspecified attributes since there is no
defined order, but would assume that other direct equality and negation can
still evaluate to true? For example:

1. Given that the aforementioned filters would both evaluate to false, for
users with an unspecified age, would all of the following then evaluate to
true?

   - filter=not (age lt 18)
   - filter=not (age gt 18)
   - filter=age ne 18
   - filter=age eq null
   - filter=not (age pr)

2. Using another example, I assume that users without an "active" attribute
value specified would match the following filters?

   - active ne true
   - active ne false
   - not (active eq true)
   - not (active eq false)

3. Also, if there is no lexicographical ordering for undefined attributes,
would they be ordered first or last when sorting by that attribute?

Thanks again for the clarifications.

On Tue, Sep 22, 2020 at 3:12 PM Phil Hunt <[email-masked]>
wrote:

> In your example below, both conditions would evaluate false as the value
> is undefined (and neither statement can not be said to lexicographically
> true) —> leading to a confusing situation where a resource is neither lt 18
> or ge 18. A more qualified filter might be:
> Do the SCIM specifications prescribe any special behavior for filtering
> and sorting by attributes that are null/unspecified?
>
> Would either of these filters return a user resource with no specified
> age, such that nulls have a prescribed ordering? Or, would both of these
> filters exclude all users without specified ages? (And, if it's the latter,
> what effect is negation intended to have on the filter?)
>
> This question applies to filters, matching resources during PATCH,
> sorting, and other attribute operators as well (e.g. would not
> (displayName co "test") match users without display names?).
>