l l l l l l l l l l l l l l l l l l l l l giwl l l l lIl l l l l l lil l
US0051931
D
United States Patent [191
[11] Patent Number:
Strout, I] et a1.
{45]
[54]
FAST INTERRUPT MECHANISM FOR
4,745,545
5/1988 Schif?eger ........................ .1 364/200
INTERRUPTING PROCESSORS IN
4,754,398
6/1988
PARALLEL IN A MULTIPROCESSOR
4,796,176 1/1989 DAmlco et a1.
SYSTEM WHEREIN PROCESSORS ARE
"801116 2/1989 KWm" e‘ a"
ASSIGNED PROCESS ID NUMBERS
4,816,990
[75] lnvemors; Robe“ 1;_ shout’ 11, Livermore,
364/200
364/200
364/288
4,901,231 2/1990 815110}?! 21 31.
364/2
C- Miller’ Eau Claire.
4,920,485
,
,
4/1990 gauberi'"
Vahidsafa
eacoc
...............
'
.. 364/200
Cla1rle; Andrew E. Phelps, Eau
4,937,733 6/1990 Gillett. Jr. ct a1.
Claire, Wis; Brian D. Vanderwarn,
4,959,781 9/1990 Rubinstein et a1,
Eau Claire, Wis; Gregory G.
5,003,466
Gaertner, Eau Claire, Wis.
5,016,162 5/1991 Epstein et a1.
p
,
u
aire,
N
Nguyen et a1.
364/200
. . . . .. 364/200
364/200
.... 1. 364/200
Primary Examiner-Lawrence B. Anderson
Attorney, Agent, or Firm-Patterson & Keough
[5-1]
ABSTRACT
536199 J
fast interrupt mechanism is capahle of simultaneously
H 1990
b
interrupting a community of assoc1ated processors in a
doseg‘ua?ghois eg‘cogkinua'?oniintmén ‘of
313:‘
459 08'3 De‘; 29 ‘989
'
’
5/1991
Schan, Jr. et a1. . . . . .
364/200
5,062,040 10/1991 Bishop =1 :11. ............... ., 364/D1G. 1
is.
Related U_S_ Application Data
{S
3/1991
5,016,167
[21] Appl‘ No" “8387
[22] Filed:
Jun. 10, 1992
p
multiprocessor system. The fast interrupt mechanism
'
enab1es the more effective debugging of software exe
’
outing on a multiprocessor system by allowing all of the
Int- CLS .............................................. ..
processors in a community associatcd
US. C1. ........................... .. 395/650; 364/1316. 1;
process m be halted within a limited number of clock
8
F. m
[5 1
‘e
[56]
3 ’
3/1989 Williams ........... ..
4,839,800 6/1989 Barlow ct a1.
Wis‘;
artners ' ,
[52]
33€£
364/200
gupercongputgasyétlems Lamted
_
. . . .. 364/200
4,391,151 1/1990 can et a1.
_
Co t,
Pribnow . v . . . . . . _ .
4,845,722 7/1989 Kent et a1.
.
[63]
Mar. 9, 1993
Cali?; George A. Spix, Eau Claire,
c1a"e"'_v‘$~;M"-*"d"
Wis:
.’ Anthony
.
R ' Schooler
A- ’ 5'19"’
Eau
.
[73] Assigneez
Date of Patent:
5,193,187
a parallel
364/2814; 364/2817; 364/2302; 395/575
cycles following a hardware exception or processor
of Search """""""" " 395/650; 364/1310" ‘
breakpoint. The fast interrupt mechanism consists of a
References Cited
Us. PATENT DOCUMENTS
set of registers that are used to identify associations
among multiple processors, a comparison matrix that is
9/
used to select processors to be interrupted, a network of
Brooks 9‘ a1’ -
interconnections that transmit interrupt events to and
4‘543'63o 32982 321C112); ‘1 a1‘ """"""""""" "
from the processors, and elements in the processors that
4:636:942 1/1937 Chen et
364/200
4,718,006
1/1988
364/200
4,736,319
4/1988 Dasbuysta ......................... .. 364/200
Nishida ...... ..
create and respond to fast interrupt events.
19 cllims, 16 Drawing Sheets
['___’:—_——~—_—_,,—”
1
r
/
K
1 Ease“ raavliasmlgl
>_ 11
mm’, '_Z,__—1:
12
NOKEMEANS
|
ESTERS
'0
A,‘ mocsssons
I
1
'1
_,
{-2 1|
mm
m
1
‘
F__J______ “m”?
1
11o
2‘
l W s‘rMDAm-—ma; ooutzmmm/
i
HGH-SPEE
1
I
I
SECONDARY
112mm
$5751
F/
g’
1
1
1
L _____________ __1
K
‘a
US. Patent
Mar. 9, 1993
Sheet 1 of 16
5,193,187
Hg. 1
I_——
—
14
x
II
K 16
GLOBAL
MAIN
INTERRUPT
I
REGISTERS
MEMORY
MECHANISM
L
K
4
_
12/
___
I|
I
4
_____
ARBITRATION
NODE
MEANS
I
REMOTE
/ 78
2o
I
|
I
I
m/I/
r-
I
l
I
I
|
i
L-
(
22
EXTERNAL
PRocEssoRs
I
_
INTERFACE
PORTS
-_
J
I
iI
k 40
/
T
CLUSTERS
__
Ir
I
PERIPHERALS STANDARD —-—_-~
110
/24
CHANNELS __;__ coNcENTRAToR
A
02
/
j
‘ HIGH-SPEED
CHANNELS
34
30
T
SECONDARY
MEMoRY'
SYSTEM
28
E-/
I
US. Patent
Mar. 9, 1993
Sheet 2 of 16
5,193,187
FEE; 2a
F
I
I
I
SHARED RESOURCES
I
I
SHARED RESOURCES
I
I
I
|
I
I
I
I
I
/---20a
II
I
ARBITRATION
NoDE MEANS
I
I
ARBITRATION
NoDE MEANS
I
I
I
I
I
I
EXTERNAL
INTERFACE
PORTS
I
I
EXTERNAL
INTERFACE
PORTS
I
I
I
I
I
I
4
@
PROCESSORS '
_.____________.___L__________;_.___
24 i
y40b I
I
l/O
STANDARD
CHANNELS -
coNcENTRAToRS'
STANDARD:
CHANNELS '
I/O
CONCENTRATORS
K 24
HIGH-SPEED
CHANNELS
HIGH-SPEED
CHANNELS
SMS
/28
28 /
SMS
U.S. Patent
Mar. 9, 1993
Sheet 4 of 16
5,193,187
Pm
Mimi
5E9F.E
ZXOwFE<Cn=Kz| .5230 woEZ
CO.
.rwgOm
h
w
z
o
m
E.6352 5E3%.
mowo_umoE
Op
US. Patent
Mar. 9, 1993
Sheet 5 of 16
5,193,187
Fig. 4a
PROCESSOR SETN ADDRESS
SETNO
SETN1
SETN2
SETN3
SETN30
SETN31
SET]
CLUSTER 0 CLUSTER 1 CLUSTER 2 CLUSTER 3
8008
A000
C000
E000
8008
A008
C008
E008
8010
A010
C010
E010
8018
A018
C018
E018
80F0
80F8
8100
AOFO
AOF8
A100
COFO
COF8
C100
EOFO
EOF8
E100
Hg. 4b
IIO DEVICE SETN ADDRESSES
CLUSTERO CLUSTER1 CLUSTER2 CLUSTER 3
1
100002008
1 100002010
1 30000201
18 00002018 20000201
1
1000020F
1000020F8
1000021
2000021
US. Patent
Mar. 9, 1993
Sheet 6 of 16
5,193,187
Hg. 5
* FAST INTERRUPT
MECHANISM
__,
,
SETN REGISTER
coMPARISoN MATRIx
SETI REGISTER
GLOBAL
REGISTER
INPUT
cRoSS
BAR
10o
//
_____,
SIGNAL
MECHANISM
PROCESSOR
EXCEPTIONS
FAST
INTERRUPTS
sToRE
FAST
REFS
TO
INTERRUPT
SETN
REQUEST
SETI
TRAPS
SIGNAL
SlGNAL
"h"
Y
EVENT REQUEST MEANS
?
PRocESsoR
INSTRUSTIONS
“200
EVENT MASK MEANS
US. Patent
Mar. 9, 1993
Sheet 7 of 16
5,193,187
Fig. 6a
SM: SYSTEM MODE (16 BITS)
NOT USER WRITABLE.
BIT BIT BIT BIT BIT
15
14
13
12
11
BIT BI T
1O
9
BIT
8
DT3 DT2 DT1 DTO DFI
BIT BIT BIT BIT
7
6
DEX
5
4
BIT
BIT
3
2
BIT BIT
1
O
FIRM
THIS REGISTER IS SAVED IN OSM AND THEN SET TO ALL ONES
ON AN INTERRUPT, SYSTEM CALL. OR EXCEPTION. IT IS RESTORED
FROM OSM BY THE RTI' INSTRUCTION.
FIRM:
DEX:
1=FAST INTERRUPT REQUESTS MASKED. O=ENABLE.
THIS BIT PREVENTS AN EXCEPTION (AS REPORTED IN
EXCEPTION STATUS) OR A FAIR INSTRUCTION FROM
CAUSING A FAST INT ERRUPT REQUEST.
1=DISABLE CONTEXT SWITCH ON ES REGISTER vNON
ZERO, O=ENABLE
'
DFI:
1=DISABLE INCOMING FAST INTERRUPT, O=ENABLE.
DTO:
1=DISABLE TYPE 0 SIGNALS. O=ENABLE.
DT1 :
1=DISABLE TYPE ‘I SIGNALS. O=ENABLE.
DT2:
1=DISABLE TYPE 2 SIGNALS. O=ENABLE.
DT3:
1=DISABLE TYPE 3 SIGNALS. O=ENABLE.
US. Patent
Mar. 9, 1993
Sheet 8 of 16
5,193,187
Fig. 6b
Pl: PENDING INTERRUPTS (16 BITS)
NOT USER WRITABLE.
THIS REGISTER SHOWS THE STATUS OF INCOMING INTERRUPTS.
BIT
15
BIT
14
T3
T2
BIT
13
T1
BIT
12
TO
BIT
‘I 1
FI
BIT
1O
BIT
9
BIT
0
UNDEFINED
Fl:
1=FAST INTERRUPT SIGNAL HAS BEEN RECEIVED.
0=NO SIGNAL
TO:
1=TYPE O SIGNAL HAS BEEN RECEIVED. O=NO SIGNAL
T1:
1=TYPE 1 SIGNAL HAS BEEN RECEIVED, 0=NO SIGNAL
T2:
1=TYPE 2 SIGNAL HAS BEEN RECEIVED, 0=NO SIGNAL
T3:
1=TYPE 3 SIGNAL HAS BEEN RECENED. O=NO SIGNAL
WHEN READ, THE REGISTER SHOWS WHICH INTERRUPTS
ARE PENDING. ‘READING THE REGISTER CLEARS ALL
BITS TO ZERO. WHEN WRITTEN, WRITING A ZERO HAS NO
EFFECT ON THE STATE OF ANY BIT. WRITING A ONE
IN ANY BIT POSITION WILL SET THAT BIT TO ONE.
US. Patent
Mar. 9, 1993
Sheet 9 of 16
Hg. 60
UM: USER MODE (16 BITS)
USER WRITABLE.
BIT
15
BIT
14
BIT
BIT
12
BIT
11
BIT
1O
BIT
9
BIT
8
BIT
7
BIT
6
undf undf UI'Id'I UI’ICII EINV EFDN EDBZ EIOVF ESNAN EQNAN_
BIT
5
BIT
4
BIT
3
BIT
2
BIT
1
BIT
O
EINF EINX EUNF EFOVF EOME undf
EOME:
ENABLE TRAP ON OPERAND MAPPING ERROR. OR
MEMORY WRITE PROTECT ERROR. (WHEN SUCH AN
ERROR OCCURS. A WRITE CANNOT MODIFY MEMORY
AND A READ WILL NOT RETURN VALID DATA, REGARD
LESS OF THE STATE OF THIS BIT.)
EFOVF:
ENABLE TRAP ON FLOATING POINT OVERFLOW
ENUF:
ENABLE TRAP ON UNDERFLOW
EINX:
ENABLE TRAP ON INEXACT RESULT
EINF:
ENABLE TRAP WHEN AN OPERAND IS AN INFINITY
EQNAN:
ENABLE TRAP WHEN AN OPERAND IS A QUIET NaN
ESNAN:
ENABLE TRAP WHEN AN OPERAND IS A SIGNALING NaN
EIOVF:
ENABLE TRAP ON INTEGER OVERFLOW
EDBZ:
ENABLE TRAP ON DIVISION BY ZERO
EFDN:
ENABLE TRAP WHEN A DENORMALIZED NUMBER IS
FLUSHED TO ZERO
EINV:
-
ENABLE TRAP ON INVALID FLOATING-POINT OPERATION
US. Patent
Mar. 9, 1993
Sheet 10 of 16
5,193,187
Fig. 6d
ES: EXCEPTION STATUS (24 BITS)
NOT USER WRITABLE.
I
81'1
BBT
I
87.r
BIT
BIT BIT
BIT
6
5
4
BIT
BIT
BIT
2
1
O
3
-INVE FDNE DBZE IOVFE SNANE ONANE INFE INXE UNFE FOVFE OMEE undf
T
E213
BIT
21
BIT BIT BIT
2019
18
BIT
17
BIT
16
BIT
15
BIT
14
BIT
13
BIT
12
WAE IMEE OWPE IIE AE DJE IPE ISME UI'ICII undf undf' undI
A BIT WILL BE SET IN THIS REGISTER WHEN AN EXCEPTION HAS
OCCURED WITH ITS TRAP ENABLE BIT SET, OR AN EXCEPTION OCCURS
WHICH DOES NOT HAVE A TRAP ENABLE BIT. BITS ARE ONLY CLEARED
BY AN EXPLICIT MOVE TO THE CONTROL REGISTER. IF THIS REGISTER
IS NONZERO AT A TIME WHEN THE DEX (DISABLE EXCEPTIONS) BIT IN
SM IS ZERO, AN EXCEPTION CONTEXT SWITCH WILL OCCUR.
OMEE:
FOVFE:
UNFE:
INXE:
INFE'
QNANE:
QUIET NaN OPERAND OPERATION.
SNANE:
IOVFE:
DBZE:
FDNE:
INVE:
ISME:
SIGNALING NaN OPERAND EXCEPTION.
INTEGER OVERFLOW EXCEPTION.
DNIDE BY ZERO EXCEPTION.
DENORMALIZED NUMBER FLUSHED TO ZERO EXCEPTION.
INVALID FLOATING-POINT OPERATION EXCEPTION.
INVALID SM EXCEPTION.
IPE:
INSTRUCTION PRIVILEGE EXCEPTION.
DJE:
DELAYED-JUMP TRAP INSTRUCTION EXCEPTION.
AE:
OPERAND MAPPING ERROR EXCEPTION.
FLOATING-POINT OVERFLOW EXCEPTION.
UNDERFLOW EXCEPTION.
INEXACT RESULT EXCEPTION.
INFINITY OPERAND EXCEPTION.
.
ALIGNMENT (TWO-PARCEL INSTRUCTION NOT ON WORD
BOUNDARY) EXCEPTION.
IIE:
OWPE:
IMEE:
WAE:
ILLEGAL INSTRUCTION EXCEPTION.
OPERAND WRITE PROTECT VIOLATION EXCEPTION.
INSTRUCTION MAPPING ERROR'EXCEPTION.
WATCHPOINT ADDRESS EXCEPTION.
US. Patent
Mar. 9, 1993
Sheet 12 of 16
5,193,187
Fig.8
PROCESSOR MEANS LOGICAL ADDRESS
710
15
14
1312
H
T \
SErN CLUSTER
SELEcT SELECT
REGISTER
SELECT
a 2
o)
FILE
SELECT
l ‘
720
as
32 31
CLUSTER
SELEcT
_\
‘ ‘
RESERVED
14 1a
SETN
sELEcT
12
_L
REG'IS‘TER
SELECT
a 2
FILE
SELECT
PHYSICAL ADDRESS MAP AS USED AT THE NRCA AND AT THE l/O
'
CONCENTRATOR MEANS
0D
US. Patent
Mar. 9, 1993
Sheet 14 of 16
5,193,187
Fig. 10
\
O '—
j 450
J'\
SETN 1
"
= A COMPARE
'
SET
NUMBER <
SUN 31
OPERATION
J'\
v
H
\/
VALUES
CO Ft
(3
Q)
C)
C1 F‘
G <3
<3
FAST
INTERUPT
LOGIC
c2 Fl
\SEFI
(3
C)
O
O O
O
COMPARISON RESULTS
@m o
FIR 1
O
' FAST
HR 31
NTERUPT
REQUESTS 9
co FIR
COLLECT AND QUALIFY
COMPARISON RESULTS
C1FIR
c2 FIR
\SETIR
'Fl COMMANDS TO
PROCESSORS
/
US. Patent
Mar. 9, 1993
Sheet 15 of 16
5,193,187
Fig. 1 1
1
mmI2oODhEwIam».
o23
3
2:
FIR O
FIR 1
HR 2
FIR 3
OR
(FAST INTERRUPT
TO PROCESSOR'O)
(FAST
INTERRUPT
T0 PROcEssoR 1>
(FAST
INTERRUPT
T0 moCESSOR 2)
(FAST
INTERRUPT
TO PRO
cssson a)
US. Patent
Mar. 9, 1993
Sheet 16 of 16
5,193,187
7
TQ-OTHERCLUSTERS
E5
5
_ 5g
TRANSMISSlON _,oo
INTERFACE ‘_,C|
LOGIC
5
SUN a1________. 3
2
SETI
1
________>
FIR 0 --~
, C2
\l\52o
510
FIR 1 —>
TRANSMISSION
MULTIPLEXER
CONTROL
FIR 31 _>
530
SET"; __,
\_r\
50o
FROM OTHER cLusTERs/
H
—>
CO H
—> CO FIR
co---> RECEPTION
c1 ___, INTERFACE -’ C‘ F‘
C=CLUSTER )
1
5,193,187
2
many hundreds, or even thousands of clock cycles after
FAST INTERRUPT MECHANISM FOR
the failure is detected by the failing processor.
INTERRUPTING PROCESSORS IN PARALLEL IN
For example, most prior art massively parallel com
A MULTIPROCESSOR SYSTEM WHEREIN
puter processing systems utilize a wavefront technique
5 for interrupting the processors. The processor that en
PROCESSORS ARE ASSIGNED PROCESS ID
counters the exception or that generates the interrupt is
NUMBERS
treated as the center of the wave and the interrupt is
RELATED APPLICATION
This application is a continuation of Ser. No. 536199,
propagated out from that processor according to the
particular interconnection architecture for the system.
?led Jun. ll, 1990, now abandoned, which is a continua
the system will not be interrupted until the interrupt
wave reaches them, a time period that may be quite
variable.
tion-in-part of an application ?led in the United States
Patent and Trademark Office on Dec. 29, 1989, entitled
CLUSTER ARCHITECTURE FOR A HIGHLY
PARALLEL SCALAR/VECTOR MULTIPRO
CESSOR SYSTEM, Ser. No. 07/459,083, pending, and
assigned to the assignee of the present invention, which
is hereby incorporated by reference in the present appli
cation. This application is also related to copending
In this model, the processors on the outermost edge of
Another problem with many of the present interrupt
mechanisms for multiprocessor systems is that all of the
processors in the multiprocessor system are uncondi
tionally interrupted in the event of a fault, not just the
processors associated with a process group. The disad
vantage to this technique is that all programs executing
application ?led in the United States Patent and Trade 20 on the multiprocessor system are halted, not just the
mark Office concurrently herewith, entitled, DIS
processes associated with the program experiencing the
TRIBUTED INPUT/OUTPUT ARCHITECTURE
fault.
FOR A MULTIPROCESSOR SYSTEM, Ser. No.
Two associated problems arise out of the inability to
07/536,182, pending which is assigned to the assignee of
adequately direct signal requests to one or more proces
the present invention, and a copy of which is attached 25 sors. With many of the present interrupt mechanisms
and hereby incorporated in the present application.
for supercomputers, for example, the Cray-l and Cray
X-MP supercomputers available from Cray Research,
TECHNICAL FIELD
Inc., the interrupt mechanisms do not allow an intelli
gent peripheral to target a service request to a particular
This invention relates generally to the ?eld of signal
ing and interrupt mechanisms for computer and elec 30 processor that could best field the service request. A
second problem is that if a peripheral or processor re
tronic logic systems. More particularly, the present
invention relates to a method and apparatus for simulta
neously performing a fast interrupt on a community of
associated processors in a multiprocessor system.
BACKGROUND ART
The parent application identi?ed above describes a
new cluster architecture for high-speed computer pro
cessing systems, referred to as supercomputers. For
most supercomputer applications, the objective is to
provide a computer processing system with the fastest
processing speed and the greatest processing ?exibility,
i.e., the ability to process a large variety of traditional
application programs. In an effort to increase the pro
cessing speed and ?exibility of supercomputers, the
cluster architecture for highly parallel multiprocessors
described in the previously identi?ed parent application
provides an architecture for supercomputers wherein a
multiple number of processors and external interface
means can make multiple and simultaneous requests to a
common set of shared hardware resources, such as main
memory, secondary memory, global registers, interrupt
mechanisms, or other shared resources present in the
quires the attention of more than one other processor,
multiple sequential signals or interrupts must be sent,
one for each of the processors being requested.
35
Although the prior art interrupt mechanisms for mul
tiprocessor systems are acceptable under certain condi
tions, it would be desirable to provide a more effective
interrupt mechanism for a multiprocessor system that
was able to interrupt the execution of all processors in a
community of associated processors within a bounded
number of clock cycles from an interrupt. In addition, it
would be desirable to provide an interrupt mechanism
for the cluster architecture for the multiprocessor sys
tem described in the parent application that aids in pro
viding a fully distributed, multithreaded software envi
ronment capable of implementing parallelism by de
fault.
SUMMARY OF THE INVENTION
The present invention is a method and apparatus for
simultaneously interrupting a community of associated
processors in a multiprocessor system. The present
invention enables the more effective debugging of soft
ware executing on a multiprocessor system by allowing
system.
55 all of the processors in a community associated with a
One of the important problems in designing such
parallel process to be halted within a limited number of
shared resource, multiprocessor systems is providing an
clock cycles following a hardware exception or proces
effective mechanism for quickly interrupting processors
sor breakpoint.
in the event of a hardware exception or processor
breakpoint. Parallel processing software running on
present multiprocessor systems is sometimes very diffi—
The present invention consists of a set of registers
that are used to identify associations among multiple
processors, a comparison matrix that is used to select
processors to be interrupted, a network of interconnec
lack of an effective fast interrupt mechanism in present
tions that transmit interrupt events to and from the
supercomputers, it is dif?cult to stop all of the proces
processors, and elements in the processors that create
sors in the community associated with the parallel pro 65 and respond to Fast Interrupt events.
cult to debug in the event of a failure. Because of the
cess. The result is that one or more of the non-stopped
processors may destroy the information necessary to
identify the failure because they continue to operate for
The multiprocessor system of the preferred embodi
ment includes a number of interrupt-type actions that
will be referred to as events. Events include signals,
5,193,187
3
4
FIG. 12 is a block diagram of the inter-cluster Fast
traps, exceptions and interrupts. Because of the differ
Interrupt interface for the four cluster implementation
of the preferred embodiment.
DESCRIPTION OF THE PREFERRED
ences in each of the events, different software, and in
some cases, hardware is required to most efficiently
process an event. Unlike prior art multiprocessor sys
EMBODIMENT
tems, the preferred multiprocessor is capable of process
ing events both in the processors and in the peripheral
Referring to FIG. 1, the architecture of a single mul
controllers that are allowed to distributively access the
common data structures associated with the multith
tiprocessor cluster of the preferred embodiment of the
multiprocessor system for use with the present inven
tion will be described. The preferred cluster architec
ture for a highly parallel scalar/vector multiprocessor
readed operating system. Also, unlike prior art multi
processor systems, the present invention halts only
those processors executing processes associated with
system is capable of supporting a plurality of high-speed
the process group for a given program, instead of halt
processors 10 sharing a large set of shared resources 12
ing all of the processors in the multiprocessor system.
(e.g., main memory 14, global registers 16, and interrupt
An objective of the present invention is to provide a
method and apparatus for a fast interrupt mechanism for
a multiprocessor system that can quickly and effectively
interrupt all of the processors in a community associ
ated with a parallel process operating on a multiproces
mechanisms 18). The processors 10 are capable of both
vector and scalar parallel processing and are connected
sor system.
to the shared resources 12 through an arbitration node
means 20. Also connected through the arbitration node
means 20 are a plurality of external interface ports 22
20 and input/output concentrators (IOC) 24 which are
further connected to a variety of external data sources
26. The external data sources 26 may include a second
Another objective of the present invention is to pro
vide a fast interrupt mechanism that can resolve simul
taneous con?icting interrupts issued by a plurality of
processors.
A further objective of the present invention is to
provide a fast interrupt mechanism that halts only those
processors executing processes associated with the pro
ary memory system (SMS) 28 linked to the input/out~
put concentrator 24 via a high speed channel 30. The
25 external data sources 26 may also include a variety of
cess group for a given program, instead of halting all of
the processors in a multiprocessor system.
These and other objectives of the present invention
will become apparent with reference to the drawings,
other peripheral devices and interfaces 32 linked to the
input/output concentrator 24 via one or more standard
channels 34. The peripheral devices and interfaces 32
may include disk storage systems, tape storage system,
printers, external processors, and communication net
works. Together, the processors 10, shared resources
12, arbitration node 20 and external interface ports 22
comprise a single multiprocessor cluster 40 for a highly
the detailed description of the preferred embodiment
and the appended claims.
parallel multiprocessor system in accordance with the
DESCRIPTION OF THE DRAWINGS
35 preferred embodiment of the present invention.
The preferred embodiment of the multiprocessor
FIG. I is a block diagram of a single multiprocessor
clusters 40 overcomes the direct-connection interface
cluster of the preferred embodiment of the present in
problems of present shared-memory supercomputers by
vention.
physically organizing the processors 10, shared re
FIG. 2a and 2b is block diagram of a four cluster
implementation of the preferred embodiment of the
present invention.
FIG. 3 is a block diagram of the interconnections for
the Fast Interrupt Mechanism of the present invention.
FIGS. 40 and 4b represent the addresses for the SET]
and SETN registers in the four cluster implementation
of the preferred embodiment of the present invention.
FIG. 5 is a block diagram showing the flow of fast
interrupt events in a cluster in the preferred embodi
ment.
FIG. 6a is a diagram of the System Mode register.
FIG. 6b is a diagram of the Pending Interrupts regis
ter.
sources 12, arbitration node 20 and external interface
ports 22 into one or more clusters 40. In the preferred
embodiment shown in FIG. 2a and 2b, there are four
clusters: 40a, 40b, 40c and 40d. Each of the clusters 40a,
40b, 40c and 40d physically has its own set of processors
10a, 10b, 10c and 10d, shared resources 12a, 12b, 12c and
12d, and external interface ports 22a, 22b, 22c and 22d
that are associated with that cluster. The clusters 40a,
40b, 40c and 40d are interconnected through a remote
cluster adapter 42 that is a logical part of each arbitra
50 tion nodes means 200, 20c, 20c and 20d. Although the
clusters 40a, 40b, 40c and 40d are physically separated,
the logical organization of the clusters and the physical
interconnection through the remote cluster adapter 42
FIG. 6c is a diagram of the User Mode register.
FIG. 6a‘ is a diagram of the Exception Status register.
enables the desired symmetrical access to all of the
shared resources 12a, 12b, 12c and 12d across all of the
FIG. 7 is a block diagram of showing the implementa
tion of the Fast Interrupt mechanism as part of the
NRCA means of the preferred embodiment of the mul
clusters ‘00, 40b, 40c and 40d.
Referring to FIG. 3, the Fast Interrupt mechanism
100 will be described. The Fast Interrupt mechanism
tiprocessor system.
100 allows a processor 10 to simultaneously send an
physical address maps for the global registers.
same process. The Fast Interrupt mechanism 100 can
FIG. 8 is a schematic representation of the logical and 60 interrupt to all other processors 10 associated with the
simultaneously process all of the interrupt signals re
ceived at the interrupt mechanism 18 in a single cycle.
However, signal delays may cause both the issuance and
FIG. 10 is a schematic diagram of the Fast Interrupt
65 receipt of Fast Interrupts to be delayed for a plurality of
dispatch logic.
cycles before or after the interrupt logic. Processors 10
FIG. 11 is a detailed implementation of one embodi
are
mapped into logical sets for purpose of operating
ment of the Fast Interrupt dispatch logic shown in FIG.
FIG. 9 is a block diagram of the Fast Interrupt mech
anism SESTN and SETI register data paths.
10.
system control by the contents of a group of Set Num
5
5,193,187
her (SETN) registers that are part of each cluster 40. In
the preferred embodiment, there are 32 SETN registers
in the global register system for a single cluster 40, one
for each processor 10 in the multiprocessor system. A
processor 10 is assigned to a set (process group) by
6
FIRM is the Fast Interrupt Request Mask located in
the System Mode SM register (bit 4)
FIRM disables generation of a Fast Interrupt request
when any exception is encountered. Setting FIRM to
one disables Fast Interrupt request. If an individual
loading that processor’s SETN register with the value
exception is disabled, the Fast Interrupt can’t occur for
identi?ed with the desired set. The 7-bit set value per
that type of exception. Note that setting Disable Excep
mits identi?cation of 128 unique sets, one for each pro
tion (DEX) to one disables Fast Interrupt requests re
cessor in a full system. Both processors and peripheral
controllers can both read and write the SETN registers.
gardless of the state of FIRM.
Another register, called the SET Interrupt (SET)
A processor 10 can also generate a Fast Interrupt
request through the FAIR instruction. Issuing a FAIR
causes a Fast Interrupt to be sent to all processors
instruction will cause all processors 10 whose SETN
registers contain the same value as the SETN of the
issuing processor to receive a Fast Interrupt. It should
and the SETI registers written via the Group 1 MOVE
instruction. Attempting the other Group 1 instructions
occurs by writing into the SETI register. All processors
register, can be used to generate a Fast Interrupt to an
explicit set of processors. Writing a value to SETI
be noted that the processor 10 that executed the FAIR
whose SETN contents match the written value. The
instruction
will NOT receive a Fast Interrupt itself. In
7-bit value written to SETI is compared to the contents
addition, Fast Interrupt requests made by the FAIR
of all of the SETN registers when SETI is written.
instruction are masked by FIRM (System Mode SM
Interrupts are sent to all processors whose SETN regis
register, bit 4). Setting FIRM to one disables Fast Inter
ter contents match the value in SETI. Both processors
20 rupt request.
and peripheral controllers can write SETI, but SETI
Although both peripheral devices 32 and the SMS 28
cannot be read.
may initiate Fast Interrupts, only processors 10 can be
SETN and SETI registers are part of the set of global
interrupted by Fast Interrupt Operations. The input
registers 16 and are accessed in the same manner. From
/output subsystem allows a peripheral device 32 to
a processor, the SETN registers are read and written 25 directly write the number of the set to be interrupted to
the Fast Interrupt logic. As previously described, this
on the SETN registers causes no change to the regis
10 whose SETN registers contain the value written are
ters, and any data returned will contain unpredictable
then interrupted. Writing to SETI is the only mecha
results. The SETN and SETI registers occupy bits 6 30 nism that allows Fast Interrupts to be initiated from
through 0 of the highest global register addresses in a
peripheral devices 32.
cluster. They are addressed by setting a binary one in bit
In operation, the operating system software (includ
15 of the GOFFSET register. FIG. 4a shows the SETN
ing a user-side scheduler in the preferred embodiment)
and SETI address map. Input/output devices have a
assigns the processors 10 to a process group by writing
memory map that is different than the map used by the 35 the process number in the SETN register for that pro
processors. The SETN addresses as seen from an pe
ripheral controller for an input/output device are
shown in the FIG. 4b.
Referring now to FIG. 5, the method for generating
and receiving Fast Interrupt events in the processors 40
will be described. The event request means 200 creates
Fast Interrupt events in two basic ways. The ?rst
method is through assertion of the Fast Interrupt Re
quest Line. The Fast Interrupt Request Line can be
cessor 10. Two steps are necessary to include a proces
sor 10 in a set (process group). First, the SETN register
for that processor 10 must be written with the number
of the set it will be associated with. Second, the DFI bit
in that processor’s System Mode register must be set to
zero. Peripheral devices 32 cannot receive Fast Inter
rupts. For a more detailed description of how the oper
ating system assigns processors 10 to process groups,
reference is made to the appendices in the previously
asserted by: (1) an exception condition (captured in the 45 identi?ed copending application entitled DISTRIB
Exception Status register); or (2) issuing a Fast Associ
UTED INPUT/OUTPUT ARCHITECTURE FOR
ate Interrupt Request (FAIR) instruction to request an
A MULTIPROCESSOR SYSTEM.
interrupt in the set of associated processors. The second
Referring now to FIG. 7, the physical organization of
method for creating Fast Interrupt Events is through
the Fast Interrupt Mechanism in the four-cluster pre
writing a set number to the SETI register.
ferred embodiment of the present invention will be
The various processor control registers that are used
by the processor in handling events are shown and
described. The preferred embodiment provides address
ing for up 128 SETN registers located among the four
described in FIGS. 60, 6b, 6c and 6d. The Pending
clusters 40. There are 32 SETN registers per cluster 40.
Interrupt register (PI) contains an indicator that shows
In this embodiment, the SETN registers 16 for each
that a Fast Interrupt has been received (FI, bit 11). A 55 cluster 40 are physically located within the NRCA
System Mode register bit, Disable Fast Interrupt (DFI,
bit 11), disables incoming Fast Interrupt requests. A
processor cannot be interrupted by a Fast Interrupt
request while DFI is set to one.
The algorithm for generating a Fast Interrupt request
by processor exception is (ENBx and EXCEPTIONx)
and (not DEX) and (not FIRM) where;
ENBx is one of exception enables in the User Mode
UM register
means 46 of that cluster. For the purposes of addressing,
the SETI register is treated as one of the SETN regis
ters and is accessed in the same manner, except that the
SETI register cannot be read.
There are sixteen ports 47 to the global registers 16,
signal logic 3], and fast interrupt logic 33 from the
thirty-two processors 10 and thirty-two external inter
face ports 22 in a cluster 40. Each port 47 is shared by
two processors 10 and two external interface ports 22
EXCEPTIONx is the associated exception condition 65 and is accessed over the path 52. A similar port 49 ser
as shown in the Exception Status ES register
vices inter’cluster requests for the global registers 16,
fast interrupt logic 31, and signal logic 33 in this cluster
DEX is the disable exception bit in the System Mode
SM register (bit 6)
as received by the MRCA means 48 and accessed over