A Proposal to Extend the OpenMP Tasking Model
for Heterogeneous Architectures
E. Ayguade1,2 , R.M. Badia2,4 , D. Cabrera2 , A. Duran2 ,
M. Gonzalez1,2 , F. Igual3 , D. Jimenez1 , J. Labarta1,2 ,
X. Martorell1,2 , R. Mayo3 , J.M. Perez2 and E.S. Quintana-Ortí3
1 Universitat
Politècnica de Catalunya (UPC)
Supercomputing Center (BSC-CNS)
3 Universidad Jaume I, Castellon
4 Centro Superior de Investigaciones Científicas (CSIC)
2 Barcelona
June 3rd 2009
Outline
Outline
1
Motivation
2
Proposal
3
Runtime considerations
4
Conclusions
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
2 / 21
Motivation
Motivation
Architecture trends
Current trends in architecture point to heterogeneous systems with
multiple accelerators raising interest:
Cell processor (SPUs)
GPGPU computing (GPUs)
OpenCL
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
3 / 21
Motivation
Motivation
Heterogeneity problems
In these environments, the user needs to take care of a number of
problems
1
Identify parts of the problem that are suitable to offload to an
accelerator
2
Separate those parts into functions with specific code
3
Compile them with a separate tool-chain
4
Write wrap-up code to offload (and synchronize) the computation
Possibly, optimize the function using specific features of the
accelerator
5
Portability becomes an issue
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
4 / 21
Motivation
Simple example
Blocked Matrix multiply
In a SMP
void matmul ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C ) {
f o r ( i n t i =0; i < BS ; i ++)
f o r ( i n t j =0; j < BS ; j ++)
f o r ( i n t k =0; k < BS ; k ++)
C [ i ∗BS+ j ] += A [ i ∗BS+k ] ∗ B [ k∗BS+ j ] ;
}
f l o a t ∗A [NB ] [ NB] , ∗B [NB ] [ NB] , ∗C[NB ] [ NB ] ;
i n t main ( void ) {
int i , j , k ;
f o r ( i = 0 ; i < NB; i ++)
f o r ( j = 0 ; j < NB; j ++) {
A [ i ] [ j ] = ( f l o a t ∗ ) m a l l o c (BS∗BS∗s i z e o f ( f l o a t ) ) ;
B [ i ] [ j ] = ( f l o a t ∗ ) m a l l o c (BS∗BS∗s i z e o f ( f l o a t ) ) ;
C [ i ] [ j ] = ( f l o a t ∗ ) m a l l o c (BS∗BS∗s i z e o f ( f l o a t ) ) ;
}
f o r ( i = 0 ; i < NB; i ++)
f o r ( j = 0 ; j < NB; j ++)
f o r ( k = 0 ; k < NB; k ++)
matmul ( A [ i ] [ k ] , B [ k ] [ j ] , C[ i ] [ j ] ) ;
}
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
5 / 21
Motivation
Simple example
Blocked Matrix multiply
In CUDA
_ _ gl o b a l _ _ void matmul_kernel ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C ) ;
# define THREADS_PER_BLOCK 16
void matmul ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C ) {
...
/ / a l l o c a t e d e v i c e memory
f l o a t ∗d_A , ∗d_B , ∗d_C ;
cudaMalloc ( ( void ∗∗) &d_A , BS∗BS∗s i z e o f ( f l o a t ) ) ;
cudaMalloc ( ( void ∗∗) &d_B , BS∗BS∗s i z e o f ( f l o a t ) ) ;
cudaMalloc ( ( void ∗∗) &d_C , BS∗BS∗s i z e o f ( f l o a t ) ) ;
/ / copy h o s t memory t o d e v i c e
cudaMemcpy ( d_A , A , BS∗BS∗s i z e o f ( f l o a t ) , cudaMemcpyHostToDevice ) ;
cudaMemcpy ( d_B , B , BS∗BS∗s i z e o f ( f l o a t ) , cudaMemcpyHostToDevice ) ;
/ / setup e x e c u t i o n parameters
dim3 t h r e a d s (THREADS_PER_BLOCK, THREADS_PER_BLOCK ) ;
dim3 g r i d (BS / t h r e a d s . x , BS / t h r e a d s . y ) ;
/ / execute t h e k e r n e l
matmul_kernel <<< g r i d , t h r e a d s >>>(d_A , d_B , d_C ) ;
/ / copy r e s u l t from d e v i c e t o h o s t
cudaMemcpy (C, d_C , BS∗BS∗s i z e o f ( f l o a t ) , cudaMemcpyDeviceToHost ) ;
/ / c l e a n up memory
cudaFree ( d_A ) ; cudaFree ( d_B ) ; cudaFree ( d_C ) ;
}
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
5 / 21
Motivation
Simple example
Blocked Matrix multiply
In a Cell PPE
void matmul_spe ( f l o a t ∗A ,
f l o a t ∗B ,
f l o a t ∗C ) ;
void matmul ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C ) {
f o r ( i =0; i <num_spus ; i ++) {
/ / I n i t i a l i z e t h e t h r e a d s t r u c t u r e and i t s parameters
...
/ / Create c o n t e x t
t h r e a d s [ i ] . i d = s p e _ c o n t e x t _ c r e a t e (SPE_MAP_PS, NULL ) ;
/ / Load program
r c = spe_program_load ( t h r e a d s [ i ] . i d , &matmul_spe ) ) ! = 0 ;
/ / Create t h r e a d
r c = p t h r e a d _ c r e a t e (& t h r e a d s [ i ] . pthread , NULL ,
&p p u _ p t h r e a d _ f u n c t i o n , &t h r e a d s [ i ] . i d ) ;
/ / Get t h r e a d c o n t r o l
t h r e a d s [ i ] . c t l _ a r e a = ( s p e _ s p u _ c o n t r o l _ a r e a _ t ∗)
spe_ps_area_get ( t h r e a d s [ i ] . i d , SPE_CONTROL_AREA ) ;
}
/ / S t a r t SPUs
f o r ( i =0; i <spus ; i ++) send_mail ( i , 1 ) ;
/ / Wait f o r t h e SPUs t o complete
f o r ( i =0; i <spus ; i ++)
r c = p t h r e a d _ j o i n ( t h r e a d s [ i ] . pthread , NULL ) ;
}
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
5 / 21
Motivation
Simple example
Blocked Matrix multiply
In Cell SPE
void matmul_spe ( f l o a t ∗A ,
f l o a t ∗B ,
f l o a t ∗C )
{
...
while ( b l o c k s _ t o _ p r o c e s s ( ) ) {
next_block ( i , j , k ) ;
c a l c u l a t e _ a d d r e s s ( baseA , A , i , k ) ;
c a l c u l a t e _ a d d r e s s ( baseB , B , k , j ) ;
c a l c u l a t e _ a d d r e s s ( baseC , C, i , j ) ;
mfc_get ( l o c a l A , baseA , s i z e o f ( f l o a t )∗BS∗BS, i n _ t a g s , 0 , 0 ) ;
mfc_get ( l o c a l B , baseB , s i z e o f ( f l o a t )∗BS∗BS, i n _ t a g s , 0 , 0 ) ;
mfc_get ( l o c a l C , baseC , s i z e o f ( f l o a t )∗BS∗BS, i n _ t a g s , 0 , 0 ) ;
mfc_write_tag_mask ((1 < <( i n _ t a g s ) ) ) ;
mfc_read_tag_status_all ( ) ;
/∗ Wait f o r i n p u t data
f o r ( i i =0; i i < BS ; i i ++)
f o r ( j j =0; j j < BS ; j j ++)
f o r ( kk =0; kk < BS ; kk ++)
l o c a l C [ i ] [ j ]+= l o c a l A [ i ] [ k ]∗ l o c a l B [ k ] [ j ] ;
mfc_put ( l o c a l C , baseC , s i z e o f ( f l o a t )∗BS∗BS, out_tags , 0 , 0 ) ;
mfc_write_tag_mask ((1 < <( o u t _ t a g s ) ) ) ;
mfc_read_tag_status_all ( ) ;
/∗ Wait f o r o u t p u t data
}
...
}
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
5 / 21
Motivation
Our proposal
Extend OpenMP so it incorporates the concept of multiple
architectures so
it takes care of separating the different pieces
it takes care of compiling them adequately
it takes care of offloading them
The user is still responsible for identifying interesting parts to offload
and optimize for the target.
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
6 / 21
Motivation
Example
Blocked matrix multiply
Parallelization for SMP
void matmul ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C ) {
/ / o r i g i n a l s e q u e n t i a l matmul
}
f l o a t ∗A [NB ] [ NB] , ∗B [NB ] [ NB] , ∗C[NB ] [ NB ] ;
i n t main ( void ) {
f o r ( i n t i = 0 ; i < NB; i ++)
f o r ( i n t j = 0 ; j < NB; j ++)
f o r ( i n t k = 0 ; k < NB; k ++)
#pragma omp task inout ( [ BS ] [ BS ] C)
matmul ( A [ i ] [ k ] , B [ k ] [ j ] , C[ i ] [ j ] ) ;
}
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
7 / 21
Proposal
Target directive
#pragma omp target device ( devicename− l i s t ) [ c l a u s e s ]
omp task | f u n c t i o n −header | f u n c t i o n −d e f i n i t i o n
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
8 / 21
Proposal
Target directive
#pragma omp target device ( devicename− l i s t ) [ c l a u s e s ]
omp task | f u n c t i o n −header | f u n c t i o n −d e f i n i t i o n
Clauses
copy_in (data-reference-list)
copy_out (data-reference-list)
implements (function-name)
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
8 / 21
Proposal
The device clause
Specifies that a given task (or function) could be offloaded to any
device in the device-list
Appropriate wrapping code is generated
The appropriate frontend/backends are used to prepare the outlines
If not specified the device is assumed to be smp
Other devices can be: cell, cuda, opencl, ...
If a device is not supported the compiler can ignore it
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
9 / 21
Proposal
Moving data
A common problem is that data needs to be moved into the accelerator
memory at the beginning and out of it at the end
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
10 / 21
Proposal
Moving data
A common problem is that data needs to be moved into the accelerator
memory at the beginning and out of it at the end
copy_in and copy_out clauses allow to specify such movements
Both allow to specify object references (or subobjects) that will
copied to/from the accelarator
Subobjects can be:
Field members
a.b
Array elements
a[0], a[10]
Array sections
a[2:15], a[:N], a[0:][3:5]
Shaping expressions
[N] a, [N][M] a
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
10 / 21
Proposal
Example
Blocked matrix multiply
void matmul ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C ) {
/ / o r i g i n a l s e q u e n t i a l matmul
}
f l o a t ∗A [NB ] [ NB] , ∗B [NB ] [ NB] , ∗C[NB ] [ NB ] ;
i n t main ( void ) {
f o r ( i n t i = 0 ; i < NB; i ++)
f o r ( i n t j = 0 ; j < NB; j ++)
f o r ( i n t k = 0 ; k < NB; k ++)
#pragma omp target device ( smp , c e l l ) copy_in ( [ BS ] [ BS ] A , [ BS ] [ BS ] B , [ BS ] [ BS ] C)
copy_out ( [ BS ] [ BS ] C)
#pragma omp task inout ( [ BS ] [ BS ] C)
matmul ( A [ i ] [ k ] , B [ k ] [ j ] , C[ i ] [ j ] ) ;
}
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
11 / 21
Proposal
Device specific characteristics
Each device may define other clauses that will be ignored for other
devices
Each device may define additional restrictions
No additional OpenMP
No I/O
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
12 / 21
Proposal
Taskifying functions
Proposal
Extend the task construct so it can be applied to functions
a la Cilk
Each time the function is called a task is implicitely created
If preceded by a target directive offloaded to the appropriate device
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
13 / 21
Proposal
Implements clause
implements ( f u n c t i o n −name )
It denotes that a give function is an alternative to another one
It allows to implement specific device optimizations for a device
It uses the function name to relate to implementations
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
14 / 21
Proposal
Example
Blocked matrix multiply
#pragma omp task inout ( [ BS ] [ BS ] C)
void matmul ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C) {
/ / o r i g i n a l s e q u e n t i a l matmul
}
#pragma omp target device ( cuda ) implements ( matmul )
copy_in ( [ BS ] [ BS ] A , [ BS ] [ BS ] B , [ BS ] [ BS ] C) copy_out ( [ BS ] [ BS ] C)
void matmul_cuda ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C) {
/ / o p t i m i z e d k e r n e l f o r cuda
}
/ / library function
#pragma omp target device ( c e l l ) implements ( matmul )
copy_in ( [ BS ] [ BS ] A , [ BS ] [ BS ] B , [ BS ] [ BS ] C) copy_out ( [ BS ] [ BS ] C)
void matmul_spe ( f l o a t ∗A , f l o a t ∗B , f l o a t ∗C ) ;
f l o a t ∗A [NB ] [ NB] , ∗B [NB ] [ NB] , ∗C[NB ] [ NB ] ;
i n t main ( void ) {
f o r ( i n t i = 0 ; i < NB; i ++)
f o r ( i n t j = 0 ; j < NB; j ++)
f o r ( i n t k = 0 ; k < NB; k ++)
matmul (A [ i ] [ k ] , B [ k ] [ j ] , C[ i ] [ j ] ) ;
}
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
15 / 21
Runtime considerations
Runtime considerations
Scheduling
The runtime chooses among the different alternatives which one
to use
Ideally taking into account resources
If all the possible resources are complete, the task waits until one
is available
If all possible devices are unsupported, a runtime error is
generated
Optimize data transfers
The runtime can try to optimize data movement:
by using double buffering or pre-fetch mechanisms
by using that information for scheduling
Schedule tasks that use the same data on the same device
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
16 / 21
Runtime considerations
Current Status
We have separate prototype implementations
for SMP, Cell, GPU
They take care
task offloading
task synchronization
data movement
They use specific optimizations for each platform
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
17 / 21
Conclusions
Conclusions
Our proposal allows:
Tag tasks and functions to be executed in a device
It takes care of:
task offloading
task synchronization
data movement
It allows to write code that is portable across multiple
environments
User still can use (or develop) optimized code for the devices
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
18 / 21
Conclusions
Future work
Actually, fully implement it :-)
We have several speficic implementations
We lack one that is able to exploit multiple devices at the same time
Implement the “OpenCL” device
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
19 / 21
Conclusions
The End
Thanks for your attention!
A. Duran (BSC)
OpenMP for Heterogeneous Architectures
June 3rd 2009
20 / 21
Conclusions
Some results
Cholesky factorization
Cholesky factorization on 4 GPUs
400
Cholesky factorization on 8 SPUs
200
GPUSs
CUBLAS
350
150
250
GFLOPS
GFLOPS
300
Hand-coded (static scheduling)
CellSs
200
150
100
100
50
50
0
0
0
4000
A. Duran (BSC)
8000
12000
Matrix size
16000
20000
0
1000
OpenMP for Heterogeneous Architectures
2000
Matrix size
3000
June 3rd 2009
4000
21 / 21