Subject: [scim] Pagination for large ​multi-valued attributes

From: Aleksey Chernoraenko <[email-masked]>
Date: Tue, 6 Feb 2018 14:05:55 -0600

​​
Hello,

SCIM defines "startIndex" and "count" pagination query parameters that
allow to control the amount of returned resources.

SCIM protocol seems to represent pagination only for top level resources
(e.g. /Users?startIndex=2&count=10) and does not address pagination for
multi-valued attributes, like group membership.

Are there any plans/draft or recommendations how to handle such use cases?

---
Thank you,
Alexei

---

From: Kelly Grizzle <[email-masked]>
Date: Wed, 7 Feb 2018 16:27:10 +0000

Hi Alexei – No there is a not a draft or plan to address this currently.  It has been discussed, and the working group decided for now that service providers can use a “returned” value of “request” for large attributes such as this.  This prevents the full value from being returned on most calls, but does not give a great solution if you want to retrieve the full list.

Also, this is one of the problems that the PATCH operation is trying to address.  This allows clients to update large attributes (adding or removing values) without having to read the entire attribute.

--Kelly

From: scim [mailto:[email-masked]] On Behalf Of Aleksey Chernoraenko
Sent: Tuesday, February 6, 2018 2:06 PM
To: [email-masked]
Subject: [scim] Pagination for large ​multi-valued attributes

​​
Hello,

SCIM defines "startIndex" and "count" pagination query parameters that allow to control the amount of returned resources.

SCIM protocol seems to represent pagination only for top level resources (e.g. /Users?startIndex=2&count=10) and does not address pagination for multi-valued attributes, like group membership.

---
Thank you,
Alexei

---

From: Phil Hunt <[email-masked]>
Date: Wed, 7 Feb 2018 08:44:18 -0800

+1

There is also a practical problem of consistency. Paging in scim is subject to change between requests. 

The group discussed supporting paging with a result set for consistency, but the scale of intended scim usage would become challenging and expensive for most implementers. It also opens the door to DoS attacks because of the resources such calls would consume. 

Then there is the why question. Enumerating large groups raises privacy concerns. This type of action is usually quite limited to things like audit systems. In these cases, because paging is not consistent, it is much better to eat the whole elephant and get all values in one call. 

Phil

> On Feb 7, 2018, at 8:27 AM, Kelly Grizzle <[email-masked]> wrote:
> 
> _______________________________________________
> scim mailing list
> [email-masked]
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_scim&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIrMUB65eapI_JnE&r=na5FVzBTWmanqWNy4DpctyXPpuYqPkAI1aLcLN4KZNA&m=85am0fsWIUdt_GtySTbIxdrAiBPR9Uj7m4NCxKasMoU&s=gBOvAbkEpQlYlLgz-Xn119F9mavBadvf5oNyg6fICXI&e=

---

From: "Matt Peterson (mpeterso)" <[email-masked]>
Date: Wed, 7 Feb 2018 21:19:58 +0000

Aleskey,

I wish there were a better answer to your question than the one from Kelly.  Representing group memberships as multi-valued attributes is a material flaw in SCIM.

I think that the “GetAll” scenario is much more common that the SCIM authors anticipated.   I would be very supportive of a draft proposing a better representation for group membership.  There are several well design patterns that could be followed.  

Is there anyone out there that thinks this is a problem worth addressing with a draft?  

--
Matt

From: Phil Hunt [mailto:[email-masked]] 
Sent: Wednesday, February 7, 2018 9:44 AM
To: Kelly Grizzle <[email-masked]>
Cc: Aleksey Chernoraenko <[email-masked]>; [email-masked]
Subject: Re: [scim] Pagination for large ​multi-valued attributes

+1

Phil

On Feb 7, 2018, at 8:27 AM, Kelly Grizzle <mailto:[email-masked]> wrote:
Hi Alexei – No there is a not a draft or plan to address this currently.  It has been discussed, and the working group decided for now that service providers can use a “returned” value of “request” for large attributes such as this.  This prevents the full value from being returned on most calls, but does not give a great solution if you want to retrieve the full list.

--Kelly
 
To: mailto:[email-masked]
 
​​
Hello,

---
Thank you,
Alexei
mailto:[email-masked]

---

From: Aleksey Chernoraenko <[email-masked]>
Date: Wed, 7 Feb 2018 22:59:42 -0600

Matt,

“request” value for “returned” will work, and I think this should be
default for large attributes.

My initial idea was to use SCIM as a general CRUD api for all our IAM
resources, not only for the ones that are allowed for provisioning - we
wouldn't want to support another interface (e.g. OData).
If pagination for such attributes is a problem, we will have to come up
with some kind of extension.

On the other hand, addressing this in more general terms would be helpful.

---
Alexei

On Wed, Feb 7, 2018 at 3:19 PM, Matt Peterson (mpeterso) <
[email-masked]> wrote:

> Aleskey,
>
> I wish there were a better answer to your question than the one from
> Kelly.  Representing group memberships as multi-valued attributes is a
> material flaw in SCIM.
>
> I think that the “GetAll” scenario is much more common that the SCIM
> authors anticipated.   I would be very supportive of a draft proposing a
> better representation for group membership.  There are several well design
> patterns that could be followed.
>
> The group discussed supporting paging with a result set for consistency,
> but the scale of intended scim usage would become challenging and expensive
> for most implementers. It also opens the door to DoS attacks because of the
> resources such calls would consume.
>
> Then there is the why question. Enumerating large groups raises privacy
> concerns. This type of action is usually quite limited to things like audit
> systems. In these cases, because paging is not consistent, it is much
> better to eat the whole elephant and get all values in one call.
>
> Phil
>
> Hi Alexei – No there is a not a draft or plan to address this currently.
> It has been discussed, and the working group decided for now that service
> providers can use a “returned” value of “request” for large attributes such
> as this.  This prevents the full value from being returned on most calls,
> but does not give a great solution if you want to retrieve the full list.
>
> Also, this is one of the problems that the PATCH operation is trying to
> address.  This allows clients to update large attributes (adding or
> removing values) without having to read the entire attribute.
>
> --Kelly
>
> https://urldefense.proofpoint.com/v2/url?u=https-3A__www.
> ietf.org_mailman_listinfo_scim&d=DwICAg&c=RoP1YumCXCgaWHvlZYR8PZh8Bv7qIr
> MUB65eapI_JnE&r=na5FVzBTWmanqWNy4DpctyXPpuYqPk
> AI1aLcLN4KZNA&m=85am0fsWIUdt_GtySTbIxdrAiBPR9Uj7m4NCxKasMoU
> &s=gBOvAbkEpQlYlLgz-Xn119F9mavBadvf5oNyg6fICXI&e=
>

-- 
Best regards,
Aleksey Chernoraenko