Andrew File System
(AFS)
Craig Shih & Todor Avramov
University of Washington, Bothell
CSS534: Parallel Programming
December 8, 2016
Outline
>
>
>
>
>
>
>
>
Overview
Consistency
Architecture
Synchronization and Caching Issues
Scalability and Performance
AFS vs. NFS
References
Q&A
Overview
>
>
>
>
>
>
Introduced by researchers at Carnegie-Mellon University in the 1980’s
Distributed File System with Main goal of Scalability
Client-side (Venus), Server-side (Vice)
Basic idea of AFS – whole-file caching on local disk of client machine
2 version (AFSv1 and AFSv2)
4 major implementations
–
–
–
–
Transarc(IBM) (deprecated)
OpenAFS
ARLA
Linux Kernel Source Code
(early stages of development)
AFS Version 2
> Introduced Callbacks
– Reduces number of client/server interactions
– Is a promise from the server to the client that the server will inform client when a
file that the client has cached has been modified.
> Introduced notion of File Identifier (FID)
– Replaced whole pathnames as a way to specify file location to server
– FID consists of a Volume Identifier, File Identifier, and a “uniquifier”
> ”uniquifier” allows the reuse of Identifiers if a file is deleted.
Callback/
FID Example
Consistency Guarantees
- Update Visibility
- When will the server be updated with a new version of the file?
- Cache Staleness
- Once the server has a new version, how long before clients see the new version
instead of an older cached copy?
- Two cases to consider:
1. Consistency between processes on different machines
2. Consistency between processes on the same machine
- AFS Uses Weak (but practical) Consistency Model
- Guarantees that after the completion of an operation, the next operation performed
anywhere in the network will see the updated file system state
Consistency Mechanisms
Open/Close
Granularity
• New data written to a file is not stored back at the file server and visible to other clients until the
file is closed.
• Typical UNIX read/write semantics on the same machine – writes to a file are immediately visible to
other local processes
Whole-file caching
on local disk
• Client nodes that already have the data cached on their local disks will not have to retrieve from
the file server.
• Minimizes network communication and file server load
Write-through
Cache
• Every time a file is modified in the client side, in addition to storing the file in the local cache, a
copy is sent to the file server
• Minimizes the effects of a client failure
Callbacks
• When the server becomes aware that a particular file is modified by a client, the server breaks
callbacks (initiates invalidation callbacks) for any clients with local cached copies of that file.
• Subsequent opens on those clients require a re-fetch of the new version of the file from the server
Consistency Timeline
AFS Architecture
RPC
RPC
Vice
System Call Interception in AFS
Vice (File Server)
Synchronization and Caching Issues
> Callbacks depend on reliable delivery of messages from server to
clients Issues:
– Network is unreliable and may prevent delivery of callback; breaking messages
– Client may crash
– File server may crash
> Fixes
– After recovery, treat all cache contents as suspect
– Cache Timeout/Periodic polling (10 minutes)
> Synchronization: Update and Callback at the same time for the same
file?
– Locking the cache entries? (Too complicated and may result in deadlock)
– Add a callback sequence number at the file server? (Best solution)
– Discard the callback? (Ad hoc solution)
Scalability and Performance
Able to
support
about 50
clients per
server
Enterprise
deployments may
exceed 25,000
clients
Client-side
performance
came close
to local
performance
Files commonly
accessed locally.
File reads usually
from local disk
cache
AFS vs NFS
Performance
AFS vs NFS
Qualitative
AFS
NFS
Supported Clients
Requires Disks
Diskless
Network Traffic
As required (much less)
Periodic (high)
Server Load
As required (much less)
Periodic (high)
Client Response Times
Equal
Equal
OS Support
Minimal
More
Caching Protocol
Whole Files
Blocks of Files
Caching Mechanism
Client Disks
Client Memory
Security
Kerberos
Weaker more primitive
Admin Management
Simpler
Difficult
References
>
>
>
>
>
>
http://pages.cs.wisc.edu/~remzi/OSTEP/dist-afs.pdf
http://ra.adm.cs.cmu.edu/anon/usr/ftp/home/ftp/itc/CMU-ITC-063.pdf
http://ra.adm.cs.cmu.edu/anon/anon/usr0/ftp/itc/CMU-ITC-062.pdf
http://tele.informatik.uni-freiburg.de/lehre/ws01/dsys/Lectures/Lecture19-1.pdf
https://en.wikipedia.org/wiki/Andrew_File_System
Distributed Systems: Concepts and Design, Edition 4, Addison-Wesley
Copyright © George Coulouris, Jean Dollimore, Tim Kindberg, Pearson
Education 2005, Chapter 8.4 Andrew File System
Questions