High performance Computing
Applied to a Saltwater Intrusion
Numerical Model
E. Canot
J. Erhel
C. de Dieuleveult
IRISA/CNRS
IRISA/INRIA Rennes
IRISA/INRIA Rennes
1
Outline


Context
Model and Test Cases



Parallel performances



Coupled Model
Test Cases : Henry and Elder
Presentation
Results
Conclusion
2
Context
Context
Effect of pumping in coastal aquifers
Necessity to predict the evolution of the water supply.
3
Coupled
Model
Presentation of the software



Based on TVDV-2D software
developped at IMFS in Strasbourg
Simulation of density driven coupled
flow and transport in a porous media
Originally sequential
4
Coupled
Model
Model
 FLOW
 n 1  Density (C n 1 )

n
n
n 1
n 1
n 1
n 1
)
h
,
C
,
C
,

(
b

h
*
)

(
Α
flow
 flow

v n 1  Velocity (h n 1 )


 TRANSPORT C *  b (v n 1 , C n )
conv

D n 1  Dispersion _ tensor (v n 1 )

 A ( D n 1 ) * C n 1  b (C * , D n 1 )
dis
 dis
convection
dispersion
5
Coupled
Model
Strong Coupling
Time
n
Velocity
FLOW
TRANSPORT
Density
Concentration
Time
n+1
6
Test Cases
Henry
Stable test case
7
Test Cases
Elder
Unstable test case
8
Parallel
performances
First parallel version (v1)

ρ n 1  Density(C n 1 )

n 1
n 1
n 1
n 1
n
n
Α
(ρ
)*h

b
(ρ
,C
,C
,h
)
flow
 flow

v n 1  Velocity(h n 1 )

*
n 1
n
C

b
(v
,C
)
conv


D n 1  Dispersion _tensor(v n 1 )

n 1
n 1
*
n 1
A
(D
)*C

b
(C
,D
)
dis
 dis
Parallel
linear
solvers
9
Parallel
performances
Parallel sparse solver
Use of MUMPS (« MUltifrontal Massively
Parallel Solver »), a free package for
solving linear systems of equations
Ax=b
Adapted for sparse unsymmetric,
and symmetric definite positive
matrices.
10
Parallel
performances
Global parallelisation (v2)

Partitioning mesh thanks to METIS, a free
package for partitioning graphs, meshes and
for producing fill reducing orderings for sparse
matrices.
Partitioning example with 5 parts

Better data distribution.
11
Results
Sensitivity to convergence criterion
Elder (mesh 256x160)
Time
1 processor
2 processors
step
Iteration Time step Iteration Time step
number number (in day) number (in day)
1
10
1/4
10
1/4
2 to 7
12
1/4
12
1/4
8
26
1/4
11
1/4
(no convergence)
1/8
1/4
Due to the convergence criterion
12
Results
Sensitivity to convergence criterion
Elder (mesh 256x160), time step number 8
with 1 processor
with 2 processors
1,00E+02
1,00E+02
1,00E+00
1,00E+00
1,00E-02
1
4
7
10
13
16
19
22
25
1,00E-02
1,00E-04
1,00E-04
1,00E-06
1,00E-06
1,00E-08
1,00E-08
1,00E-10
1,00E-10
1,00E-12
1,00E-12
1,00E-14
1
4
7
10
13
16
19
22
25
1,00E-14
error in flow
error in transport
convergence criterion
13
Results
Results for the first version (v1)
Henry (mesh 510x254)
Wall clock
(in seconds)
700
600
500
Total wall
clock
MUMPS wall
clock
400
300
200
100
0
1
2
4
8
16
Number of processors
14
Results
Results for the second version (v2)
Henry (mesh 510x254)
Wall clock
(in seconds)
700
600
v1 Total wall
clock
v1 MUMPS
wall clock
Total wall
clock
MUMPS wall
clock
500
400
300
200
100
0
1
2
4
8
16
Number of processors
15
Conclusion
Conclusion - Perspectives



3D geometry
Adaptative mesh
Improved coupling
16