CSMC 412
Operating Systems
Prof. Ashok K Agrawala
© 2006 Ashok Agrawala
Set 10
10.1
Objectives
„ To explain the function of file systems
„ To describe the interfaces to file systems
„ To discuss file-system design tradeoffs, including access methods,
file sharing, file locking, and directory structures
„ To explore file-system protection
10.2
1
File-System
„ General Concepts - Files and File Attributes
„ File Operations
„ Access Methods
„ Directory Structure
„ Linking File Names and File Content
„ File-System Mounting
„ File Sharing
„ File Protection
„ Network File Systems
10.3
UNIX File Systems
File:
„
Logical storage unit
„
Unit of abstraction:
„
Nonvolatile memory
„
UNIX files are unstructured text files
z
z
„
Physical properties of storage devices are abstracted away
Organized in a directory structure
Programs, commands: executable files
10.4
2
File Concept
„ Contiguous logical address space
„ Types:
z
Data
numeric
character
binary
z
Program
10.5
File Structure
„ None - sequence of words, bytes
„ Simple record structure
z
Lines
z
Fixed length
z
Variable length
„ Complex Structures
z
Formatted document
z
Relocatable load file
„ Can simulate last two with first method by inserting appropriate
control characters
„ Who decides:
z
Operating system
z
Program
10.6
3
File Attributes
„
„
„
Files are named
z For the convenience of the human user
z File names are case-sensitive
Attributes
z Name – only information kept in human-readable form
z Identifier – unique tag (number) identifies file within file system
z Type – needed for systems that support different types
Unix does not distinguish file types
z Location – pointer to file location on device
z Size – current file size, possibly maximum size
z Protection – controls who can do reading, writing, executing
z Time, date, and user identification – data for protection, security, and
usage monitoring
z Owner of a file
Information about files are kept in the directory structure, which is
maintained on the disk
10.7
File Operations
A file is an abstract data type
z Values + set of applicable operations
z A set of basic file operations has to be supported by the OS
„ Creating a file
z
Allocate space in the file system, generate directory entry
„ Writing a file
z Name of the file and information to be written has to be specified
z System hast to maintain a write pointer for the file
„ Reading a file
z File name and a pointer to memory to receive data has to be specified
z System maintains a read pointer for the file
10.8
4
File Operations (contd.)
„
„
„
Repositioning within a file (file seek)
z Directory is searched for the appropriate entry
z Read/write pointers for the file are set to a given value
Deleting a file
z Release all file space; erase directory entry
z Instead of deleting a file, UNIX allows to unlink() files – file is deleted by the OS if the last
link to a file disappears
Truncating a file
z File attributes remain unchanged
z File length is set to zero (or to some specified value)
Additional ops:
• Append
• Rename
• Set attributes
• ...
Read pointer
Write pointer
UNIX supports the „truncation-on-close“
flag to truncate files
10.9
Obtaining Access to a File
„
„
„
Open() system call
z Takes a file name, searches the directory, checks file protection
z Copies directory entry into open-file table
z Returns a pointer to the entry in open-file table for subsequent use
Close() system call
z Flushes cached file data back to the storage device
z Deletes entry from open-file table
z Frees system resources
Operation in a multiuser environment
z Per-process and global open-file tables
z System maintains reference counts for opened files
10.10
5
Open Files
„ Several pieces of data are needed to manage open files:
z
File pointer: pointer to last read/write location, per process that
has the file open
z
File-open count: counter of number of times a file is open – to
allow removal of data from open-file table when last processes
closes it
z
Disk location of the file: cache of data access information
z
Access rights: per-process access mode information
10.11
Open File Locking
„ Provided by some operating systems and file systems
„ Mediates access to a file
„ Mandatory or advisory:
z
Mandatory – access is denied depending on locks held and
requested
z
Advisory – processes can find status of locks and decide what
to do
10.12
6
File Locking Example – Java API
import java.io.*;
import java.nio.channels.*;
public class LockingExample {
public static final boolean EXCLUSIVE = false;
public static final boolean SHARED = true;
public static void main(String arsg[]) throws IOException {
FileLock sharedLock = null;
FileLock exclusiveLock = null;
try {
RandomAccessFile raf = new RandomAccessFile("file.txt", "rw");
// get the channel for the file
FileChannel ch = raf.getChannel();
// this locks the first half of the file - exclusive
exclusiveLock = ch.lock(0, raf.length()/2, EXCLUSIVE);
/** Now modify the data . . . */
// release the lock
exclusiveLock.release();
10.13
File Locking Example – Java API (cont)
// this locks the second half of the file - shared
sharedLock = ch.lock(raf.length()/2+1, raf.length(),
SHARED);
/** Now read the data . . . */
// release the lock
exclusiveLock.release();
} catch (java.io.IOException ioe) {
System.err.println(ioe);
}finally {
if (exclusiveLock != null)
exclusiveLock.release();
if (sharedLock != null)
sharedLock.release();
}
}
}
10.14
7
File Types – Name, Extension
10.15
Access Methods
„
Sequential Access
read next
write next
reset
no read after last write
(rewrite)
„
Direct Access
read n
write n
position to n
read next
write next
rewrite n
n = relative block number
10.16
8
Sequential-access File
10.17
Simulation of Sequential Access on a DirectDirect-access File
10.18
9
Example of Index and Relative Files
10.19
Directories
„
„
„
Record information about groups of files
Management of files
z Single-Level directory: most simple; all files in the same directory
z Two-Level directory: separate directory for each user
z Tree-Structured (hierarchical) directories: most common
Operations on directories:
z Search for a file
z Create a file (directory entry)
z Delete a file (directory entry)
z List a directory
z Rename a file
z Traverse the file system (recursive)
10.20
10
UNIX Directories
„
„
„
„
„
„
Fully hierarchical, tree-structured
Directories are represented as files
z Problem: Truncation
Processes have a current working directory
z pwd command
Each user has a home directory
z cd; echo $HOME – commands to obtain info about the home dir.
The file system has a single root directory
z cd / - command changes working directory to root directory
Special names identify neighbors in the directory tree
z ./ - the current directory
z ../ - the directory one level above the current directory
10.21
A Sample UNIX Directory Listing
10.22
11
Linking Names and File Content
„
z
z
„
Information contained in a UNIX i-node
UNIX separates file names and file content
file content may have multiple
(different) names
ln command associates new name with
existing file
File content identified by:
z
z
z
(Device, File system on device, i-node)
i-node contains references to all blocks
making up a file
a free-node list is maintained for each
file system
10.23
Directory Structure
„ A collection of nodes containing information about all files
Directory
Files
F1
F2
F3
F4
Fn
Both the directory structure and the files reside on disk
Backups of these two structures are kept on tapes
10.24
12
A Typical File-system Organization
10.25
Operations Performed on Directory
„ Search for a file
„ Create a file
„ Delete a file
„ List a directory
„ Rename a file
„ Traverse the file system
10.26
13
Organize the Directory (Logically) to Obtain
„ Efficiency – locating a file quickly
„ Naming – convenient to users
z
Two users can have same name for different files
z
The same file can have several different names
„ Grouping – logical grouping of files by properties, (e.g., all
Java programs, all games, …)
10.27
Single-Level Directory
„ A single directory for all users
Naming problem
Grouping problem
10.28
14
Two-Level Directory
„ Separate directory for each user
„ Path name
„ Can have the same file name for different user
„ Efficient searching
„ No grouping capability
10.29
Tree-Structured Directories
10.30
15
Tree-Structured Directories (Cont)
„ Efficient searching
„ Grouping Capability
„ Current directory (working directory)
z
cd /spell/mail/prog
z
type list
10.31
Tree-Structured Directories (Cont)
„ Absolute or relative path name
„ Creating a new file is done in current directory
„ Delete a file
rm <file-name>
„ Creating a new subdirectory is done in current directory
mkdir <dir-name>
Example: if in current directory /mail
mkdir count
mail
prog
copy prt exp count
Deleting “mail” ⇒ deleting the entire subtree rooted by “mail”
10.32
16
Acyclic-Graph Directories
„ Have shared subdirectories and files
10.33
Acyclic-Graph Directories (Cont.)
„ Two different names (aliasing)
„ If dict deletes list ⇒ dangling pointer
Solutions:
z
Backpointers, so we can delete all pointers
Variable size records a problem
z
Backpointers using a daisy chain organization
z
Entry-hold-count solution
„ New directory entry type
z
Link – another name (pointer) to an existing file
z
Resolve the link – follow pointer to locate the file
10.34
17
General Graph Directory
10.35
General Graph Directory (Cont.)
„ How do we guarantee no cycles?
z
Allow only links to file not subdirectories
z
Garbage collection
z
Every time a new link is added use a cycle detection
algorithm to determine whether it is OK
10.36
18
File System Mounting
„ A file system must be mounted before it can be
accessed
„ A unmounted file system (i.e. Fig. 11-11(b)) is
mounted at a mount point
10.37
(a) Existing. (b) Unmounted Partition
10.38
19
Mount Point
10.39
File Sharing
„ Sharing of files on multi-user systems is desirable
„ Sharing may be done through a protection scheme
„ On distributed systems, files may be shared across a network
„ Network File System (NFS) is a common distributed file-sharing
method
10.40
20
File Sharing – Multiple Users
„ User IDs identify users, allowing permissions and
protections to be per-user
„ Group IDs allow users to be in groups, permitting group
access rights
10.41
File Sharing – Remote File Systems
„ Uses networking to allow file system access between systems
„
z
Manually via programs like FTP
z
Automatically, seamlessly using distributed file systems
z
Semi automatically via the world wide web
Client-server model allows clients to mount remote file systems
from servers
z
Server can serve multiple clients
z
Client and user-on-client identification is insecure or
complicated
z
NFS is standard UNIX client-server file sharing protocol
z
CIFS is standard Windows protocol
Standard operating system file calls are translated into remote
calls
z
„ Distributed Information Systems (distributed naming services) such
as LDAP, DNS, NIS, Active Directory implement unified access to
information needed for remote computing
10.42
21
File Sharing – Failure Modes
„ Remote file systems add new failure modes, due to network
failure, server failure
„ Recovery from failure can involve state information about
status of each remote request
„ Stateless protocols such as NFS include all information in
each request, allowing easy recovery but less security
10.43
File Sharing – Consistency Semantics
„ Consistency semantics specify how multiple users are to access
a shared file simultaneously
z
Similar to Ch 7 process synchronization algorithms
Tend
to be less complex due to disk I/O and network
latency (for remote file systems
z
Andrew File System (AFS) implemented complex remote file
sharing semantics
z
Unix file system (UFS) implements:
Writes
to an open file visible immediately to other users of
the same open file
Sharing
file pointer to allow multiple users to read and write
concurrently
z
AFS has session semantics
Writes
only visible to sessions starting after the file is closed
10.44
22
Protection
„ File owner/creator should be able to control:
z
what can be done
z
by whom
„ Types of access
z
Read
z
Write
z
Execute
z
Append
z
Delete
z
List
10.45
File Protection
„
„
Access rights can be independently defined for:
z (u) user – Owner (creator) of a file
z (g) group
– Group
z (o) other – all other users of the UNIX system
Example:
luna test ( 48 )-% ls -lisa
total 2
421908 1 drwxr-xr-x
2
apolze
1024
116884 1 drwxr-xr-x
13
apolze
2048
116992 0 -rw------1
apolze
0
116991 0 -rw-rw-rw1
apolze
0
Jan
Jan
Jan
Jan
7 15:06 .
7 15:06 ..
7 15:05 Mail.txt
7 15:05 test.c
10.46
23
File Protection (contd.)
„
„
„
Access rights for a file:
z (r)
Read access right; List right for directorisy
z (w)
Write access right; includes delete/append rights
z (x)
Execute access right; Traverse right for directories
Binary representation:
z (x):
Bit 0 (+1)
z (w):
Bit 1 (+2)
z (r):
Bit 2 (+4)
Rights can be combined
z Read+Write access right: 6
z Read+Execute access right: 3
z Read-only: 2
10.47
Access Lists and Groups
„
Mode of access: read, write, execute
„
Three classes of users
a) owner access
7
⇒
b) group access
6
⇒
c) public access
1
⇒
RWX
111
RWX
110
RWX
001
„
Ask manager to create a group (unique name), say G, and add
some users to the group.
„
For a particular file (say game) or subdirectory, define an
appropriate access.
owner
chmod
group
public
761
game
Attach a group to a file
chgrp
G
game
10.48
24
Windows XP Access-control List Management
10.49
Distribution – Network File Systems
„
„
Various approaches towards distributed file systems:
z SUN Network File System (Standard)
z UNIX United
z Andrew File System
z Sprite
z Locus
SUN NFS
z Client/Server-System (based on remote procedure call (RPC))
z File system operations are forwarded from client to server
z Server executes actual file system operations, returns results
z Client has access to remote resources
z Stateless operation (Reliability !)
10.50
25
Operation of a Network File System
„
A set of operations is implemented as RPC-callable functions:
z
z
z
z
z
Searching for a file in a directory
Reading a set of directory entries
Manipulating links and directories
Accessing file attributes
Reading and writing files
„
Logical connection between client and server has to be established
„
NFS works in heterogeneous environments
„
Stateless protocol
z
z
z
mount protocol
Machine-independent protocol for data representation (XDR)
Network file system may tolerate client crashes (reboots)
10.51
Mounting a Remote File System
„
Computer "sun" exports the "/local"-file system to computer "moon"
Exported to moon
Exported to all computers on the net
10.52
26
Windows – UNIX File System
Interoperability
Windows uses per default the Server Message Block (SMB) protocol to implement distributed
file systems
SMB/CIFS (remote) resource sharing:
„
Samba – de.samba.org
POSIX libraries/tool chains for Windows API:
„
Cygwin – sources.redhat.com/cygwin/
„
U/WIN – www. research.att.com/sw/tools/uwin/
„
NuTCracker / MKS Toolkit – www.datafocus.com
„
Interix, SFU – www.microsoft.com/windowsserversystem/sfu/
10.53
What's Samba all
about?
„
Free SMB and CIFS client/server for UNIX & other OS
„
Functionality:
z
a SMB server, to provide Windows NT and LAN Manager-style file and print
services to SMB clients such as Windows, Warp Server, smbfs and others.
z
a NetBIOS (rfc1001/1002) nameserver, which amongst other things gives
browsing support. Samba can be the master browser on your LAN if you wish.
z
a ftp-like SMB client so you can access PC resources (disks and printers) from
UNIX, Netware and other operating systems
10.54
27