[AFS3-std] File Server/Cache Manager Extended Status Information

[Jeffrey Altman]

Matt Benjamin wrote:
> At the risk of getting really volatile discussion going, I should state
> that Jeff Altman's comments on IRC, raise topics not yet addressed here:
>
> 1.  Jeff's initial reaction in public IRC was that notifications cannot
> be asynchronous--we must require clients to poll for them.  Jeff A.
> appears to have changed his position, but asserted the previous position
> in very strong terms.  I think using the call back channel for
> notifications is highly preferable to another polling interface.  Has
> Jeff A.'s position in fact changed?  Is this in any way controversial?
Change notifications (aka callback breaks) are synchronous in AFS.
That is not a question available for debate.  The file server architecture
triggers callbacks synchronously with the operation that is producing
the change.  Callbacks can be delayed but must be delivered successfully
to the client before any other request can be processed from a client
with an outstanding callback.

This behavior is a core component of the AFS coherency model.

Any changes that are going to be integrated in the near term must
adhere to this model. 

While I will not rule out asynchronous notifications at some point
in the future after careful consideration of a cohesive architectural
design, I do not see such changes being accepted into production
branches anytime soon.

In addition, my comments to you were specifically regarding your
desire for the following locking exchange:

   1. client A requests blocking byte range write lock on fid 1.2.3
   2. server replies that the lock is not available but will send an
      asynchronous notification when it is
   3. another client drops the held lock
   4. server allocates lock to client A and sends asynchronous
      notification that the lock was allocated

As opposed to the existing model in which

   1. client A requests block byte range write lock on fid 1.2.3
   2. server replies EWOULDBLOCK
   3. another client drops the held lock
   4. server sends client A a callback indicating that the lock state on
      fid 1.2.3 has changed
   5. client decides whether or not to request the lock again.  if so,
      go to 1

The problem I see with your approach is that it imposes too much state 
on the file server.  The file server needs to be able to deal with the 
situations in which either client A is no longer reachable and is 
therefore holding the lock when others could be using it; and deal with 
the case where client A sends a lock request cancellation message racing 
with the lock allocation.

The existing model works today but suffers from the failure to know that 
the reason a callback was received was because the lock was dropped or 
because a Store occurred etc.
>
> 2. Jeff stated in IRC (in response to the discussion I initially
> started) that "when requesting FetchStatus or BulkStatus the client must
> be able to provide a callback filter for the events that it is
> interested in."  Assuming a polling interface, and also mandating
> fine-grained control over notifications on a per-object basis.  Is this
> mandate now retracted?  Moved to TellMeAboutYourself (not per object, 
> obviously, in that case)?
First improve the functionality of the existing implementation without 
breaking the existing model or adding additional state.  This includes 
sending more informative callback messages which permit the cache 
manager to make more intelligent decisions and minimize the need for 
reading unnecessary data from the file server.

Then we can work on a subsequent phase which permits the registration of 
multi-event callbacks requests.  These requests must permit the cache 
manager to specify which events are of interest.
>
> 3.  The term "call back" is too overloaded in AFS.  It confuses the
> meaning of a call back operation and the promise to make it.  Standard
> idiom, too, is about "breaking" call backs, but clearly, file change
> notifications are
>
> a. only logical if we have made the promise to call the client back, and
> b. they don't break the server's promise to call back the client again
jhutz hit this nail squarely on the head
>
> Jeffrey Hutzelman wrote:
> | --On Sunday, April 06, 2008 05:39:09 PM -0400 Jeffrey Altman
> | <jaltman@secure-endpoints.com> wrote:
>
> Yes, noted.
>
> |
> | Agree.  Clients should not have to do anything to receive extended
> | notifications other than advertise support for them.
> |
> | I'm inclined to agree, mostly.  I particularly like the idea of actually
> | distributing directory changes to clients holding callbacks on a
> | directory, as that could significantly improve performance in situations
> | where there is heavy contention for a directory, by eliminating the need
> | for clients to keep refetching that directory (if the client chooses to
> | apply the update locally).
> |
> | I agree with some of Matt's concerns about different triggers requiring
> | different extended data, but I'd really like to see a solution that does
> | not require a separate RPC for each trigger.
>
> Thanks, that is helpful.  Jeff's list of pseudo-prototypes suggest a
> small group of notification types and calls.  Would that be unreasonable.
>
> My sense is, people see it as more conservative, and therefore good, to
> have just one call back proc.  After reading my document, Jeff A. said
> he wanted to commit his own change, with just a "CallBackNew" to replace
> the old call back with _the_ new one.
> Obviously, no one wants to introduce unecessary RPCs, but frankly, a few
> clean calls should be more efficient than 1 hairy one with a switch and
> opaque buffers, at the client.  Faster too, I suspect.  Would you
> support that (Jeff H)?
Using multiple callback messages would make it hard to coalesce delayed 
callbacks of different types.

>
> |>>
> |> Why is there a range of byte range changes?  A StoreData operation
> |> does not take a list of independent commits.  Each StoreData will
> |> result in a BreakCallBack(). > Proc AccessChangeCallBack
> |
> | That's a sequence of byte ranges, to go with the sequence of FID's.  The
> | idea is to let you send several updates at once, possibly on different
> | FID's.  I suspect this is mostly useful on delayed callbacks.
> |
>
> As discussed in previous mail, it seems that there's a natural
> compression in batching notifications to one cache manager, especially
> to one file, grouped as Tom says, closely in time.  I assumed we would
> wish to support this.
This certainly is not something that would be considered in the near 
future by me with the one exception of delayed callbacks where the 
reason we are coalescing the messages is because the client is known to 
be unreachable.

Jeffrey Altman

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/x-pkcs7-signature
Size: 3355 bytes
Desc: S/MIME Cryptographic Signature
Url : http://michigan-openafs-lists.central.org/archives/afs3-standardization/attachments/20080407/8c09b41e/smime-0001.bin