Subject: Re: [Rserpool] Data transmission

From: "Jose Luis Jimenez Ramirez ECE/TL/NB ID-000 Tlf.4061" <[email-masked]>
Date: Wed, 25 Jul 2001 15:13:07 +0200 (MEST)

Hi Randall,

Couldn't be more precise these sentences?:

A. When a PU wants to talk to a PE.. it
 
   1) ASAP Asks ENRP where it is (Name Resolution procedure).
  
   2) Gets a list back of PE's that are registered under the
      name... (NAME_RESOLUTION_REPONSE )the list has the IP address/ports. 

   3) ASAP layer must select one of the PEs in the Pool(Based on the server 		   
      pooling policy)

   4) The PU sets up an association with the PE and sends data
      directly to the PE with NO ASAP formatting or ANY INTERVENTION.

I can understand that once the ASAP layer selects the "best" PE for this PU,
the ASAP layer doesn't get involved in the transport of data, but there should be a 
"control" of this communication between PU<-->PE in the sense that for example a
SCTP SEND.FAILURE Notification could happen and a new PE should be selected.

So my question is:
When you say:

  "4) The PU sets up an association with the PE and sends data
    directly to the PE with NO ASAP formatting or ANY INTERVENTION."
    
Do you mean that the PU (i.e ASAP User) sets up an association directly to the PE:

+-------+            +----+----+
+ PU    +            +   PE    +
+-----  + SCTP assoc +    +----+
|ASAP|  |<---------->|    |ASAP| 
+-------+            +----+----+       
| SCTP  |            |  SCTP   |
+-------+            +---------+
|  IP   |            |   IP    |
+-------+            +---------+

Shouldn't  be this assoc established by ASAP, and then ASAP by using a 
"ASAP UserData message" (not defined yet), send this User data to the PE, without 
processing the content?
+-------+            +----+----+
+ PU    +            +   PE    +
+-------+ SCTP assoc +---------+
| ASAP  |<---------->|  ASAP   | 
+-------+            +----+----+       
| SCTP  |            |  SCTP   |
+-------+            +---------+
|  IP   |            |   IP    |
+-------+            +---------+

Maybe the solution is something like this:
ASAP bypass UserData (No ASAP UserData message should be defined),
+-------+            +----+----+
+ PU    +            +   PE    +
+-------+ SCTP assoc +    +----+
| ASAP  |<---------->|    |ASAP| 
+-------+            +----+----+       
| SCTP  |            |  SCTP   |
+-------+            +---------+
|  IP   |            |   IP    |
+-------+            +---------+

Can anybody clarify this.
I'm in a mess :-o

                   - -
Regards            º º
Jose      --oOOo---()--oOOo--

_______________________________________________
rserpool mailing list
[email-masked]
http://www.ietf.org/mailman/listinfo/rserpool

---

From: Randall Stewart at work <[email-masked]>
Date: Wed, 25 Jul 2001 09:17:23 -0500

> SCTP SEND.FAILURE Notification could happen and a new PE should be selected.

Yes.. of course.. I did not mention the failure scenarios... and of
course ASAP may reroute failed data based on the policy.. i.e. does
the server pool support failover.. it may not if it does not
do some state sharing or is a stateless type application...

> Do you mean that the PU (i.e ASAP User) sets up an association directly to the PE:

Yes but I think I would draw the picture more:

 +-------+            +----+----+
 + PU    +            +   PE    +
 +-----  +            +    +----+
 |ASAP|  |            |    |ASAP|
 +-------+  SCTP Assoc+----+----+
 | SCTP  |<---------->|  SCTP   |
 +-------+            +---------+
 |  IP   |            |   IP    |
 +-------+            +---------+

Since the association exists on the SCTP layer not the
ASAP layer.

Now we may choose to pass through a message from PU:ASAP to
PE:ASAP.. this is the business card I mentioned in my earlier
post. This way the PE that receives the request will be able
to form a handle using the name of the PU... this is of course
if the PU is actually a PE as well....

This is why the ASAP messages had the 8 byte identification on
them... 

> "ASAP UserData message" (not defined yet), send this User data to the PE, without

I like the idea of transparent pass through better than
having ASAP bundle everything inside a wrapper. We can
do it this way... but our original implementation never
needed this and that way the wire did not get extra
bytes on every message... Of course the trade off here
is that for every ASAP message that goes from PE to PU you
must put an identification (such as the 8 bytes that
is in all ASAP messages) to delineate a ASAP message
to be intercepted.. and of course ASAP has to watch the
first 8 bytes of every message to see if it recognizes
the "identity" mark. Either way is probably ok.. but I
do like the idea that would allow an application to
NOT have to have ASAP in the picture.. this allows a
bit more flexibility and makes ASAP less intrusive.

> ASAP bypass UserData (No ASAP UserData message should be defined),

The current ASAP, as defined, is as you drew it in your
first picture with my modification. A message may flow
from ASAP to ASAP (conceptually) but in general does not.
The current spec does not define any message that flows
PE <-> PU... I had one in the old implementation I worked
on but it is not yet in the spec. 

Now Once the "session" is established ASAP's job is really
done (unless there is a failure of course since the 
failure messages would come to ASAP to be redirected etc)...
Having this extra message I mention is an enhancement to better 
support PE's acting as PU's and wanting fault tolerant
service two ways. We don't have to add it back to the protocol
but it does add some benefit :-)

The way the implementation I worked with did this is that it
basically looked at messages arriving from the SCTP association,
if the magic cookie/identity "8 bytes" matched the expected ASAP message
type... i.e. the "ENRP Endpoint Message Identifier"  then
it intercepted the message and prevented it from going to
the application. If it did not see the 8 bytes.. then it
just went ahead and ignored the message coming in. 

In other words the message is considered an ASAP message
if the first 8 bytes holds: "0x1803868877734683"

And yes, I know you have a 1 in 2^64 chance that the inbound
user message MAY be mistaken for an ASAP message.. but this
was such a remote unlikely thing we did not feel it was worth
worrying about.. 

The benefit here of course is that:

1) An application server can get messages that are NOT ASAP
   at all and still service requests transparently.. This allows
   any application to talk to the server without having ASAP in
   the picture at all... if it is willing of course to give up
   the fault tolerant benefits.

2) The wire protocol is very lightweight and does not interfere with
   anything the application may be doing such as TLS security end2end.

3) Being lightweight means less bytes on the wire.

Has a SESSION layer protocol we really do not want many messages
that go PE<->PU. It is far better to let the applications talk and
do what a session layer is supposed to... 

R
-- 
Randall R. Stewart
[email-masked] 815-342-5222 (cell phone)

---

From: Manuel Hugo Angelina Rodriguez EEM/TD/LNB <[email-masked]>
Date: Wed, 25 Jul 2001 16:38:20 +0200

Hi Randall,

I think I don't understand your figure:

+-------+
+----+----+

+ PU    +
+   PE    +

+-----  +
+    +----+

|ASAP|  |
|    |ASAP|

+-------+
+---------+

|  IP   |
|   IP    |

+-------+
+---------+

As far as I understand, when a PU wants to request
a service (i.e. send a message) to a PE, it first sends a data.send.req
to ASAP. ASAP will get a list of PEs (or will have it in its cache)
and will select one PE. Once this PE is selected, ASAP (and not
the PU) will do sctp.send. This way, if there is any failure, ASAP (and
not the PU) will receive sctp.send.failure, and will pick another PE from
the list, thus giving more reliability to the requests of the PU. If it
were the PU who sends the data to the PE, then ASAP would only work
selecting one PE from the list and passing it to the PU, but then any possible
sctp.send.failure would be sent to PU instead of ASAP.

Wouldn't it be more like this:

+-------+
+----+----+

+ PU    +
+   PE    +

+-------+
+    +----+

| ASAP  |
|    |ASAP|

+-------+
+---------+

|  IP   |
|   IP    |

+-------+
+---------+

Mvh,

hugo.

Randall Stewart at work wrote:

"Jose Luis Jimenez Ramirez ECE/TL/NB ID-000 Tlf.4061"
wrote:

Yes.. of course.. I did not mention the failure scenarios... and
of

course ASAP may reroute failed data based on the policy.. i.e.
does

> +-------+
+----+----+

> + PU    +
+   PE    +

> +-------+
+----+----+

> | SCTP  |
|  SCTP   |

> +-------+
+---------+

> |  IP   |
|   IP    |

> +-------+
+---------+

+-------+
+----+----+

+ PU    +
+   PE    +

+-----  +
+    +----+

|ASAP|  |
|    |ASAP|

+-------+
+---------+

|  IP   |
|   IP    |

+-------+
+---------+

ASAP layer.

them...

> +-------+
+----+----+

> + PU    +
+   PE    +

> +-------+
+----+----+

> | SCTP  |
|  SCTP   |

> +-------+
+---------+

> |  IP   |
|   IP    |

> +-------+
+---------+

> +-------+
+----+----+

> + PU    +
+   PE    +

> +-------+
+----+----+

> | SCTP  |
|  SCTP   |

> +-------+
+---------+

> |  IP   |
|   IP    |

> +-------+
+---------+

if the magic cookie/identity "8 bytes" matched the expected ASAP
message

worrying about..

at all and still service requests transparently..
This allows

any application to talk to the server without having
ASAP in

the picture at all... if it is willing of course to
give up

2) The wire protocol is very lightweight and does not interfere
with

anything the application may be doing such as TLS
security end2end.

that go PE<->PU. It is far better to let the applications talk
and

R

--

Randall R. Stewart

rserpool mailing list

[email-masked]

---

From: Qiaobing Xie <[email-masked]>
Date: Wed, 25 Jul 2001 11:29:29 -0500

Jose,

The root of confusion here is the careless use of PU and PE terminology
:-)

See below...

> A. When a PU wants to talk to a PE.. it

This should read:

When the __app layer__ of a PU wants to talk to the __app layer__ of a
PE, it calls the SEND primitive specifying the pool name of the PE. This

>       directly to the PE with NO ASAP formatting or ANY INTERVENTION.

This should read: 

     4) the __ASAP layer__ of the PU sets up an association with the
__ASAP layer__ of the PE and sends data directly to the PE with NO ASAP
formatting or ANY INTERVENTION..

> SCTP SEND.FAILURE Notification could happen and a new PE should be selected.

In such case we will have:

     5) The __ASAP layer__ of the PU will intercept the SCTP
SEND.FAILURE Notification and select another PE from the list and
re-send the data to the __ASAP layer of the new PE. The __app layer__ of
the PU will normally not be notified about the occurrence of the
failover.

And finally for completeness:

     6) When the __ASAP layer__ of the PE receives the data from the
__SCTP layer__ of the PE, the __ASAP layer__ will simply generate a
DATA.ARRIVE notification and pass the data to its __app layer__.

-Qiaobing

---

From: Randall Stewart at work <[email-masked]>
Date: Wed, 25 Jul 2001 14:32:08 -0500

Hmm..

Well.. I guess it depends on how you look at it :)

From a conceptual view I guess ASAP is there
to intercept all of the SCTP notifications etc... but
it does not insert any data or wrappers around
the data... if you will once you have the address

ie

asap_send(char *name, u_char* data, int sz)
{
   struct sockaddr *addr;

   addr = nameTranslationGook(name);

   if(addr == NULL){
      return (error-not-found);
   }
   return(sctp_send(addr,data,sz));
}

Now the above is a very simpleton view of it all of course..
the "nameTranslationGook()" would be capable of queueing
and or blocking if it was not in a cache ... etc..

Now the other thing this presumes is that the failure notifications
are hooked up to the asap layer.

When we did our implementation we used a reactor model (termed
that by the Design Patterns book Doug Scmidt addision/wesely)
The concept being that you subscribe to events and get callbacks
for everything. If you look at the SCTP reference imnplementation
this is what runs it as well.. In fact it is a new and improved
version of what I was playing with in my past life... I re-wrote
it again (for about the third time) and now have a better main
reactor .. if you will :0

Right now the model in the ref-imple does not let you subscribe
for Failure events .. something I was going to add.. it will
come in when we re-write the ASAP/ENRP again. We had planned to
add this all to the reference impl so we had a ref impl for
SCTP and ASAP and ENRP..

I have delayed starting on this though since we are about to
start hacking up the message formats... :)

R

Manuel Hugo Angelina Rodriguez EEM/TD/LNB wrote:
>     As far as I understand, when a PU wants to request a service (i.e.
> send a message) to a PE, it first sends a data.send.req to ASAP.
> ASAP will get a list of PEs (or will have it in its cache) and will
> select one PE. Once this PE is selected, ASAP (and not the PU) will do
> sctp.send. This way, if there is any failure, ASAP (and not the PU)
> will receive sctp.send.failure, and will pick another PE from the
> list, thus giving more reliability to the requests of the PU. If it
> selecting one PE from the list and passing it to the PU, but then any
> possible sctp.send.failure would be sent to PU instead of ASAP.
> 
>  +-------+            +    +----+
>  | ASAP  |            |    |ASAP|

---

From: Miguel Angel Nieto Jimenez <[email-masked]>
Date: Thu, 26 Jul 2001 17:40:17 +0200

Hi,
	I have included a sequence diagram about how I think the ASAP
protocol should behave. It is in text-plain format. Could you look
at this and reply if it is ok?

Regards 

	Miguel

> > Do you mean that the PU (i.e ASAP User) sets up an association directly to the PE:
Hi,
   This is the way how I understand the ASAP protocol. In the primitives I only have
 indicate the most importants parameters. As the primitives were incompleted in the
draft, I have added some of them that I consider are necessaries. For example,
asap.bind to inform ASAP about their users. Maybe there are some points that could
be application specific, but they help me to understand the overall problem.

 These are some of the abbreviations I have used:

The addresses of the different elements are:
PU: IP=IP1, port=P1
PE: IP=IP2, port=P2
ASAP1: IP=IP1, port=P3 (well-known port)
ASAP2: IP=IP2, port=P3

I1, I2, I3 are the SCTP instance identifiers. These parameters are locals to the node
A1, A2, A3, ... are the association identifiers. 

AUI: Asap user identifier. It is used to by ASAP to identify its users. There is a table that maps 
"Asap user identifiers" to "Instance identifiers". ASAP doesn't return the Instance identifier to the 
user to avoid this uses SCTP directly.

CI: Communication identifier. It is passed to the ASAP user to identify associations. There is a table
 that maps "Communication identifiers" to "Associations identifiers". ASAP doesn't return the Instance 
identifier to the user to avoid this uses SCTP directly.

ph is the pool handle

I have supposed that ASAP@PU starts generating I,A,CI and AUI identifiers from 0 while ASAP@PE starts
from 50.

This is the sequence:

PU           ASAP@PU         SCTP                      SCTP           ASAP@PE         PE

1. ASAP notifies SCTP that it is an user (when the stack is initialized)
                sctp.initialize.req(IP1,P1)                sctp.initialize.req(IP2,P3)
               ---------------->                         <--------------
                sctp.initialize.conf(I1)                    sctp.initialize.conf(I50) 
               <----------------                         -------------->                 

2.- PE and PU bind to its underlying ASAP layer.
                                                                          asap.bind.req(IP2,P2)
                                                                        <--------------
                                                           sctp.initialize.req(IP2,P2)
                                                         <--------------              
                                                           sctp.initialize.conf(I2)
                                                         --------------> 
                                                                     (Map I2 to AUI50)
                                                                     asap.bind.conf(AUI50)
                                                                        -------------->
                                                                            
  asap.bind.req(IP1, P1)
-------------->
                sctp.initialize.req(IP1,P1)
               --------------->
                sctp.initialize.conf(I1)
               <---------------
               (Map I1 to AUI1)
  asap.bind.conf(AUI1)
<--------------

3.- PE register itself in the ENRP server
                                                                         asap.register.req(AUI50,ph)
                                                                        <--------------
                                                                        (ASAP@PE opens an association 
                                                                         to the ENRP node using P3 port 
                                                                         and get a register itself)
                                                                        
                                                                         asap.register.conf()
                                                                        -------------->

4.- Data transmission. 

  asap.send.req(AUI1,ph,data)
-------------->
  (ASAP@PU verifies that AU1 is its user. Maps
   AUI1 to I1)

  (ASAP@PU opens an association to the ENRP node
   using P3 port and get a list of PE for the
   pool requested. One of them is selected (IP2,P2)
   according to the policy)
 
                sctp.associate.req(I1,IP2,P2)
               --------------->
                sctp.associate.conf(A1)
               <---------------

                sctp.comm_up.ind(I1,A1,IP2)               sctp.comm_up.ind(I50,A50,IP1)
               <---------------                          -------------->
                sctp.send.req(A1, data)                 
               --------------->   
               (Map A1 to CI1)
  asap.send.conf(CI1)
<--------------
             
                                                          sctp.data_arrive.ind(A50,data)
                                                         -------------->
                                                                        (Map A50 to CI50)

                                                                        asap.data_received(CI50, data)
                                                                        -------------->
                                                                        asap.send(AUI50, CI50,data)     
       
                                                                        <--------------     
                                                                        (Verify that the user and
                                                                         communication exits)
                                                          sctp.send(A50,data)             
                                                          <------------                               

                sctp.data_arrive.ind(A1,data)     
               <---------------
               (Map A1 to CI1)
  asap.data_received(CI1,data)
<--------------

(PU can send again to the same pool element)
  asap.send.req(AUI1,CI1,data)
-------------->
...
...
...
...

Is this sequence OK?

Regards
    Miguel Angel

---

From: Randall Stewart <[email-masked]>
Date: Fri, 27 Jul 2001 12:01:04 -0500

Miguel:

Ok, I will attempt to answer your drawing/illustration with
a more codish text example based on the enrp/asap that Q and I
put together at my former employer :) 

Also to answer another comment you made, the ASAP layer
had a send flag to tell it that the application requests
restransmissions to alternates or NOT. If not then the
send.failure notification is passed back to the application... otherwise
it is automatically resent.

Now how things worked in what we put together.... its to bad
the ref-impl is not yet ready :<

First and formost... the model we developed uses a reactor model... or
dispatcher...

In the heart of things you have some object (it was c++ oo) doing a
select/poll and when a FD came ready for reading it would
do a callback to the registred object that had registered against
the FD.

The object had always a pointer to the single dispatcher and could
do a Message.Send() that basically sent a message to the stack.. so you
had

FD-Event->(dispatcher)->(subscribed Object)---X which read a message and
then
(subscribed Object forms message) send-> (dispatcher)-> send to message
subscribers.

An object could subscribe and/or get events for:

Messages  (subscribed)
FD events (subscribed)
audit/clock tick (subscribed)
callback (object to object path not subscribed)
timer event (specific to timer calls)
dispatcher start event (default subscribed but can be disabled)
dispatcher stop event (default subscribed but can be disabled)
failure notification (subscribe to failures sent by FD event handlers)

Now with this in mind here is what went on:

In main you would create:

1) Dispatcher
2) ASAP object
3) application objects

Dispatcher disp;
ASAP_object a("Name");
ApplicationObj app(&a);

All of the object class had a common base class that included
a pointer to the static dispatcher object instance.

So in app()'s constructor call it could subscribe to various
events through the dispachers public members and of course
save off a copy of "a" the asap object so that it could send
messages.

Now inside the ASAP object it created a

SCTP_Object sctp();

It COULD specify a port if it wanted but generally did not
need to since it would register its "NAME" with ENRP. Note
also we had some special flags we called on sctp() that 
redirected the failiure notifications to the ASAP object
and then the ASAP object could send them to the dispatcher
stack or do a retransmit... more on that in a moment..

After all objects in main were created the last line
would be:

disp.process();

Which would begin the event loop. The event loop start
would of course first, before entering the event loop,
call ASAP.startup() member (inherited from the base class).

Now ASAP.startup would:

Send off the registration event to ENRP ... it was
actually a small state machine that was always running in
ASAP doing a re-register every 10 or so minutes after initial
registration and return.

When a message arrived it arranged to be the first message
consumer ahead of all others... so the ASAP registration response
would come to it first. 

Note: the concept with messages was where your ASAP.message() member
was called and you could either:
 a) consume the message and thus stop any other object from seeing it
 b) ignore the message (its not mine) and let others see it
 c) modify the message and pass it on for others.

So in the case of ASAP messages it did (a). Once registered
the application object may all of a sudden get a request say,
"hello send me X". This would first hit ASAP who did not
see its 8 byte magic signature and thus be ignored and
passed to app(). This then responded to the message by sending
a reply to the from address. 

If the application instead wanted to send a message to a PE it
would call one of the 3 asap send methods

1) Send("name");
2) Send("handle");
3) Send(Transport Address);

(1) and (2) gained the attention of the fault tolerant ASAP layer.
Method (3) just was a pass through

int
ASAP_Object::Send(Transpor Address ...)
{
   return(sctp.send(Transpor Address ..));
}

Thus you lost the advantages of ASAP if you used it. In cases
like a server responding, this might be appropriate...

Now after sending to a name/handle if a failure came up the
context field of SCTP was used to transport the send mode.. so
if failover was allowed ASAP would take the upcoming failure
messages select an alternative and resend them. Otherwise if
fail-over was NOT allowed it notified the application via
the mechanims the dispatcher provided for sending failures. 

Now the above is a simplified flow there was some other interaction I
cut out aka the business card that enabled the from transport address
to actually be a handle instead of a transport address... but I hope
this clearifies things a small amount..

The strict seperation you layed out in your flows does not
match, necessarily, how I view it all working. They may be
correct.. I am not to sure, but I don't see the ASAP layer
as being anything but a service layer for the application.

R

---

From: Miguel Angel Nieto Jimenez <[email-masked]>
Date: Mon, 30 Jul 2001 12:06:44 +0200

Hi,
   Sorry, but after having a lot of mails with the same subject
"Data transmission", my original question has not be answered yet.
The heart of it is that I'm not sure how associations should be
set up. I can think different ways:
* Case 1:
  - 1 association for ASAP messages exchanges between two nodes.
It would use a well known port. If ASAP wants to register, make a
name-resolution, ... it'd use this association.
  - 1 association per PU and PE. As happens with PE's, PU's are
identified by their IP address and port. For example if we have
one PU communicating with three PE's, there would be 3 different
associations. Here I see a problem. As ASAP wants to get
load-sharing, if ASAP has to make round-robin and there are 10 PE,
it would be necessary to have 10 associations for only one
communication.

* Case 2:
  - Only 1 association between two nodes. All the PU's and PE's in
one node wanting to communicate with other PE's or PU's in other
node would use the same assoc. It would be necessary to identify
the origin and destination of the data messages within the same
node. It could be use the payloadProtocolId or use a stream for
each communication. In this case should be necessary an  ASAP
header for data transmission or it could be said that ASAP
messages go into stream 0 and data in the other streams. In this
case why should PE's be defined by their IP address and port when
this is not used.

Regards 
   Miguel

---

From: Randall Stewart <[email-masked]>
Date: Mon, 30 Jul 2001 08:57:42 -0500

Miguel:

Ok, sorry your questions has yet, not been answered :-0

let me see if I can give it an attempt..

> it would be necessary to have 10 associations for only one

I think this one above is more what we end up with. A PE registers
its transport address(es) with ENRP. A PU queries the ENRP deamon
for a list of addresses for a given name. Then as it needs it, it
sets up a association to the PE it wishes to talk to. 

I don't see having 10 associations is a problem due to a number
of facts:

1) Using the UDP model (see the sockets api draft) you neither
   need 10 sockets AND do not need to do any connect/accept
   for the association setup.

2) Normally you complete a transaction with the same server. In other
   words I load balance NEW service requests but continuation of
   the same service should go to the same PE. Think of a phone call
   example. I place a call, this involves several messages back and
   forth with my PE that is setting up my call.  When I begin the call
   I query for the name "callserver" and get back a list of PE's. I
choose
   one of these and I then use the Pool Element Handle of that one for
   the rest of my call. When it is time to make a new call, I then once
   again use the name so I load balance to possibly another call server.

   This yeilds:
    - A locality of reference, so that state information will not
      have to be constantly pulled from the state repository and
      reconstructed. This gives us minimal state expansion.
    - If a failure occurs to my call server, either I was the call
      that failed the server, which means the call is lost. Or I
      was NOT the call where the callserver failed and there is
      a high probablility that my last state update that my callserver
      made will have had a chance to propagate by the time I 
      fail-over to another callserver.
    - This allows the callservers to have the minimum amount of state,
      if you will a very small shared set of information that may take
      DB queries and other things to expand the state information into
      a useable form. This is ok, since most of the time (barring
failure)
      I stay with the same PE who does this at the beginning of my call
      only. If I have a fail-over, no big deal, so I pay a extra few
      milliseconds for my state to be reconstructed from a small piece
      of information.

    The above works for other things besides calls as well. It also for
    the most part lets you simplify state sharing (outside of the scope
    of rserpool) of course...
  
   3) The autoclose feature will shutdown idle associations so again
      the PU never needs to maintain any of the gory details on what
      it has open and needs to close.

I hope this answers your question :)

R