Advanced Operating Systems

Course Reading Material:



Lecture notes, class discussions will be the material
for examination purposes.
Recommend reading appropriate reference papers for
the different algorithms.
Other books discussing the algorithms will help too.
B. Prabhakaran
1
Advanced Operating Systems

Project Reference Text:

“Unix Network programming”, Richard Stevens,
Prentice Hall.
B. Prabhakaran
2
Contact Information
B. Prabhakaran
Department of Computer Science
University of Texas at Dallas
Mail Station EC 31, PO Box 830688
Richardson, TX 75083
Email: [email protected]
Fax: 972 883 2349
URL: http://www.utdallas.edu/~praba/cs6378.html
Phone: 972 883 4680
Office: ES 3.706
Office Hours: 12.45-1.15pm Saturdays
10.30-11am Wednesdays
Other times by appointments through email
Announcements: Made in class and on course web page.
TA: TBA.
B. Prabhakaran
3
Course Outline
Proposed Outline. Might be modified
based on time availability:
 Introduction to operating systems, inter-process
communication.
 Distributed Operating Systems






Architecture
Clock Synchronization, Ordering
Distributed Mutual Exclusion
Distributed Deadlock Detection
Agreement Protocols
Distributed Resource Management



Distributed File Systems
Distributed Shared Memory
Distributed Scheduling
B. Prabhakaran
4
Course Outline ...


Recovery & Fault Tolerance
Concurrency Control/ Security

Depending on time availability
Additional/modified topics might be introduced from other
texts and/or papers. References to those materials will be
given at appropriate time.
B. Prabhakaran
5
Evaluation





1 Mid-terms: in class. 75 minutes. Mix of MCQs
(Multiple Choice Questions) & Short Questions.
1 Final Exam: 75 minutes or 2 hours (Mix of MCQs and
Short Questions).
2 Quiz: in class. 5-6 MCQs or very short questions. 20 –
25 minutes.
Homeworks/assignments: at most 2-3
Programming Projects:


1 preparatory (to warm up on thread and socket programming) –
NOT Graded
2 projects based on algorithms discussed in class.
B. Prabhakaran
6
Grading



2 Home works: 5%
2 Quizzes: 15%
1 Mid-term + Final: 50%





Equally distributed 25% each
Intermediate & Final Projects (combined): 30%
Possible Letter Grades: A, A-, B+, B, B-, C+, C
Likely: B will be around class average
Letter grade trend will be informed after 2nd Quiz.
B. Prabhakaran
7
Schedule








Quiz 1: September 25, 2010
Quiz 2: November 20, 2010
Mid-term: October 9, 2010
Final Exam: As per UTD schedule (Dec. 11) or Last day of
class
Subject to minor changes
Homework schedules will be announced in class and
course web page, giving sufficient time for submission.
Tutorials may be given, if sufficient time available.
Likely Homework Due Dates: September 18, 2010 &
November 13, 2010.
Likely project deadlines:



Preparatory project: September 5, 2010 (NOT Graded)
Project 1: October 17, 2010
Project 2: December 5, 2010
B. Prabhakaran
8
Programming Projects







No copying/sharing of code/results will be tolerated. Any
instance of cheating in projects/homeworks/exams will be
reported to the University.
No copying code from the Internet.
2 individual students copying code from Internet
independently: still considered copying in the project !!
Individual projects.
Deadlines will be strictly followed for projects and
homeworks submissions.
Projects submissions through Web CT.
Demo will be needed
B. Prabhakaran
9
Two-track Course

Programming Project Discussions:




Announcements in class
Minimal Discussions in class
Design discussions during office hours based
on individual needs
Theory (Algorithms) Discussions:


Full discussion in class
Office hours for clarification if needed
B. Prabhakaran
10
eLearning


Go to:
https://elearning.utdallas.edu/webct/entryPageIns.dowebct
It has a discussion group that can be used for project and
other course discussions.
B. Prabhakaran
11
Cheating





Academic dishonesty will be taken seriously.
Cheating students will be handed over to
Head/Dean for further action.
Remember: home works/projects (exams too !)
are to be done individually.
Any kind of cheating in home works/ projects/
exams will be dealt with as per UTD guidelines.
Cheating in any stage of projects will result in 0
for the entire set of projects.
B. Prabhakaran
12
Projects





Involves exercises such as ordering, deadlock detection,
load balancing, message passing, and implementing
distributed algorithms (e.g., for scheduling, etc.).
Platform: Linux/Windows, C/C++/Java. Network
programming will be needed. Multiple systems will be
used.
Specific details and deadlines will be announced in class
and course webpage.
Suggestion: Learn network socket programming and
threads, if you do not know already. Try simple programs
for file transfer, talk, etc.
Sample programs and tutorials available at:
 http://www.utdallas.edu/~praba/projects.html
B. Prabhakaran
13
Homeworks


At most 2-3 home works, announced in class and course
web page.
Homeworks Submission:

Submit on eLearning (or) paper to TA/Instructor.
B. Prabhakaran
14
Basic Computer Organization






Input Unit
Output Unit
CPU
Memory
ALU (Arithmetic &
Logic Unit)
Secondary Storage
Memory
ALU
Disk
CPU
I/O
Display
Keyboard
B. Prabhakaran
15
Simplified View of OS
OS
Kernel
Physical
Memory
User i
Process j
OS Tools
User
Processes
Code
Data
Code
Code
Code
Data
User
Virtual
Memory Processes ..
Data
Data
Tools ++
Memory Space
B. Prabhakaran
16
Distributed View of the System
hardware
hardware
hardware
Process
hardware
hardware
B. Prabhakaran
17
Inter-Process Communication


Need for exchanging data/messages among processes
belonging to the same or different group.
IPC Mechanisms:

Shared Memory: Designate and use some data/memory as
shared. Use the shared memory to exchange data.


Message Passing: Use “higher” level primitives to “send” and
“receive” data.


Requires facilities to control access to shared data.
Requires system support for sending and receiving messages.
Operation oriented language constructs



Request-response action
Similar to message passing with mandatory response
Can be implemented using shared memory too.
B. Prabhakaran
18
IPC Examples

Parallel/distributed computation such as sorting: shared
memory is more apt.


Client-server type: message passing or RPC may suit
better.


Using message passing/RPC might need an array/data manager
of some sort.
Shared memory may be useful, but the program is more clear
with the other types of IPCs.
RPC vs. Message Passing: if response is not a must,
atleast immediately, simple message passing should
suffice.
B. Prabhakaran
19
Shared Memory
Writers/
Producers
Only one process
can write at any
point in time. No
access to readers.
Shared Memory
Multiple readers can
access. No access to
Writers.
Readers/
Consumers
B. Prabhakaran
20
Shared Memory: Possibilities





Locks (unlocks)
Semaphores
Monitors
Serializers
Path expressions
B. Prabhakaran
21
Message Passing





Blocked Send/Receive: Both sending and receiving
process get blocked till the message is completely
received. Synchronous.
Unblocked Send/Receive: Both sender and receiver are
not blocked. Asynchronous.
Unblocked Send/Blocked Receive: Sender is not blocked.
Receiver waits till message is received.
Blocked Send/Unblocked Receive: Useful ?
Can be implemented using shared memory. Message
passing: a language paradigm for human ease.
B. Prabhakaran
22
Un/blocked

Blocked message exchange



Easy to: understand, implement, verify correctness
Less powerful, may be inefficient as sender/receiver might
waste time waiting
Unblocked message exchange



More efficient, no waste on waiting
Needs queues, i.e., memory to store messages
Difficult to verify correctness of programs
B. Prabhakaran
23
Message Passing: Possibilities
receiver i
sender
receiver j
receiver i
sender
receiver k
receiver j
receiver k
B. Prabhakaran
24
Message Passing: Possibilities...
sender i
sender j
sender k
sender i
receiver
sender j
receiver
sender k
B. Prabhakaran
25
Naming




Direct Naming

Specify explicitly the receiver process-id.

Simple but less powerful as it needs the sender/receiver to know the
actual process-id to/from which a message is to be sent/received.

Not suitable for generic client-server models
Port Naming

receiver uses a single port for getting all messages, good for clientserver.

more complex in terms of language structure, verification
Global Naming (mailbox)

suitable for client-server, difficult to implement on a distributed
network.

complex for language structure and verification
Indirect Naming

Use a naming server, e.g., as in RPCs.
B. Prabhakaran
26
Communicating Sequential
Processes (CSP)
process reader-writer
OKtoread, OKtowrite: integer (initially = value);
busy: boolean (initially = 0);
*[ busy = 0; writer?request() ->
busy := 1; writer!OKtowrite;
•
busy = 0; reader?request() ->
busy := 1; reader!OKtoread;
•
busy = 1; reader?readfin() -> busy := 0;
•
busy = 1; writer?writefn() -> busy := 0;
]
B. Prabhakaran
27
CSP: Drawbacks


Requires explicit naming of processes in I/O commands.
No message buffering; input/output command gets
blocked (or the guards become false) -> Can introduce
delay and inefficiency.
B. Prabhakaran
28
Operation oriented constructs
Remote Procedure Call (RPC):
Task A
Service declarations
……
RPC: abc(xyz, ijk);
Task B
Service declarations
send xyz; wait for result
return ijk
…….
•
•
•
•
Service declaration: describes in and out parameters
Can be implemented using message passing
Caller: gets blocked when RPC is invoked.
Callee implementation possibilities:
• Can loop “accepting” calls
• Can get “interrupted” on getting a call
• Can fork a process/thread for calls
B. Prabhakaran
29
RPC: Issues


Pointer passing, global variables passing can be difficult.
If processes on different machines, data size (number of
bits for a data type) variations need to be addressed.





Abstract Data Types (ADTs) are generally used to take care of
these variations.
ADTs are language like structures that specify how many bits
are being used for integer, etc…
What does this imply?
Multiple processes can provide the same service? Naming
needs to be solved.
Synchronous/blocked message passing is equivalent to
RPC.
B. Prabhakaran
30
Ada
task [type] <name> is
entry specifications
end
Task body <name> is
Declaration of local
variables
begin
list of statements
…..
accept <entry id> (<formal parameters> do
body of the accept statement
end<entry id>
exceptions
Exception handlers
end;
B. Prabhakaran
task proc-buffer is
entry store(x:buffer);
remove(y:buffer);
end;
task body proc-buffer is
temp: buffer;
begin
loop
when flag accept store(x: bu
temp := x; flag :=0;
end store;
When !flag accept remove(y:
y := temp; flag :=1;
end remove;
end loop
end proc-buffer.
31
Ada Message Passing
Task A
Task B
entry store(...)
….
accept Store(.)
…...
xyz is sent from
….
Task B to A
result
Store(xyz);
…..
Somewhat similar to executing procedure call. Parameter value
for the entry procedure is supplied by the calling task. Value of
Result, if any, is returned to the caller.
B. Prabhakaran
32
RPC Design

Structure



Binding



Caller: local call + stub
Callee: stub + actual procedure
Where to execute? Name/address of the server that offers a
service
Name server with inputs from service specifications of a task.
Parameter & results


Packing: Convert to remote machine format
Unpacking: Convert to local machine format
B. Prabhakaran
33
RPC Execution
Binding
Server
Local Proc.
Local
call
Stub
Receive
Query
Query
binding
server
Return
Server Address
Params
packing
Register
Services
Stub
Remote Proc.
Unpack
Execute
procedure
Local
call
Wait
Return
Pack
results
Unpack
result
Caller
Return
Callee
B. Prabhakaran
34
RPC Semantics

At least once



Exactly once



A RPC results in zero or more invocation.
Partial call, i.e., unsuccessful call: zero, partial, one or more
executions.
Only one call maximum
Unsuccessful? : zero, partial, or one execution
At most once

Zero or one. No partial executions.
B. Prabhakaran
35
RPC Implementation

Sending/receiving parameters:



Use reliable communication? :
Use datagrams/unreliable?
Implies the choice of semantics: how many times a RPC may be
invoked.
B. Prabhakaran
36
RPC Disadvantage

Incremental results communication not possible: (e.g.,)
response from a database cannot return first few matches
immediately. Got to wait till all responses are decided.
B. Prabhakaran
37
Distributed Operating Systems
Issues :









Global Knowledge
Naming
Scalability
Compatibility
Process Synchronization
Resource Management
Security
Structuring
Client-Server Model
B. Prabhakaran
38
DOS: Issues ..

Global Knowledge



Lack of global shared memory, global clock, unpredictable
message delays
Lead to unpredictable global state, difficult to order events (A
sends to B, C sends to D: may be related)
Naming




Need for a name service: to identify objects (files, databases),
users, services (RPCs).
Replicated directories? : Updates may be a problem.
Need for name to (IP) address resolution.
Distributed directory: algorithms for update, search, ...
B. Prabhakaran
39
DOS: Issues ..

Scalability



System requirements should (ideally) increase linearly with the
number of computer systems
Includes: overheads for message exchange in algorithms used
for file system updates, directory management...
Compatibility



Binary level: Processor instruction level compatibility
Execution level: same source code can be compiled and
executed
Protocol level: Mechanisms for exchanging messages,
information (e.g., directories) understandable.
B. Prabhakaran
40
DOS: Issues ..

Process Synchronization


Resource Management



Distributed shared memory: difficult.
Data/object management: Handling migration of files, memory
values. To achieve a transparent view of the distributed system.
Main issues: consistency, minimization of delays, ..
Security

Authentication and authorization
B. Prabhakaran
41
DOS: Issues ..

Structuring


Monolithic Kernel: Not needed (e.g.,) file management not
needed fully on diskless workstations.
Collective kernel: distributed functionality on all systems.




Micro kernel + set of OS processes
Micro kernel: functionality for task, memory, processor
management. Runs on all systems.
OS processes: set of tools. Executed as needed.
Object-oriented system: services as objects.


Object types: process, directory, file, …
Operations on the objects: encapsulated data can be manipulated.
B. Prabhakaran
42
DOS: Communication
Computer
Switch
B. Prabhakaran
43
ISO-OSI Reference Model
Application
Application
Presentation
Presentation
Session
Communication Network
Transport
Session
Transport
Network
Network
Network
Network
Datalink
Datalink
Datalink
Datalink
Physical
Physical
Physical
Physical
B. Prabhakaran
44
Un/reliable Communication

Reliable Communication




Virtual circuit: one path between sender & receiver. All packets
sent through the path.
Data received in the same order as it is sent.
TCP (Transmission Control Protocol) provides reliable
communication.
Unreliable communication



Datagrams: Different packets are sent through different paths.
Data might be lost or out of sequence.
UDP (User datagram Protocol) provides unreliable
communication.
B. Prabhakaran
45