Public Release: Native Overset Mesh in
FOAM-Extend
Vuko Vukčević and Hrvoje Jasak
Faculty of Mechanical Engineering and Naval Architecture, Uni Zagreb, Croatia
Wikki Ltd. United Kingdom
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17 January 2017
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 1
Outline
Objective
• Announce public release of native Overset Mesh capability
• Present validation and verification and parallel scaling results for the Overset mesh
solver
Topics
• Introduction
• Native implementation of the Overset Mesh support in foam-extend
• Overset Mesh: Validation and parallel scaling results
• Pre-announcement of NUMAP Summer 2017 and OFW 12
• Summary
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 2
Introduction
Open Source Native Overset Mesh Capability in foam-extend
• We have been involved in several previous Overset projects
• . . . but none reached the maturity or provided flexibility required for a
general-purpose CFD platform
• Project Objective: Complete rewrite without limitations on release
◦ Native implementation of Overset Mesh technology in FOAM-Extend without
reference to external libraries
◦ Operator-based support for Finite Volume (FV) discretisation: implicit and
explicit operators (fvc and fvm)
◦ Setup via overset patch type, with minimal impact on existing code
components: full re-use of existing physical models
◦ Consistent support for strict mass conservation and second order accurate
discretisation in space and time
• Further requirements
◦ Overset mesh functionality must operate on unstructured polyhedral mesh
hierarchies
◦ Basic layout for massive parallelisation in domain decomposition mode
◦ Support for dynamic overset mesh simulations
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 3
Overset Mesh
Overset Mesh Capability in foam-extend: An Update
• Revisited native implementation of Overset Mesh: public release
• Rewrite of basic Overset functionality to improve performance and parallel scaling
Overview of New Overset Capability
• Explicit and Implicit fringe treatments: chosen per-field, depending on application
• Rewrite of parallelised Overset assembly
• New research item: Implicit conservative Overset
• Additional fringe reduction algorithms
• Rewrite of parallel communications to improve scaling on HPC machines
Public release of the Overset Solver
compatible with foam-extend-4.0
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 4
Overset Mesh
Explicit Fringe Treatment
• Interpolation is performed before the solution of each equation, setting values in
acceptor cells (careful with the pressure equation!)
• It is also performed in certain places in the code where the interpolation is
necessary (eg. 1/A update, etc.)
• Benefits
◦ Better parallel scaling than implicit fringe (linear solver)
◦ Region-wise mass conservative without special treatment
◦ It seems to be much more robust than implicit (especially for the pressure
equation), which is probably linked to its conservative properties
• Drawbacks
◦ At least one non-orthogonal corrector is needed for the pressure equation in
order to have regions behaving in a coupled way (otherwise, you can see that
regions are not coupled and the mean value of pressure is different)
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 5
Overset Mesh: New Development
Implicit Fringe Treatment
• Interpolation is performed within the matrix-vector multiplication in the linear
system solver
• Benefits
◦ No need for non-orthogonal correctors in order to update the Overset after the
pressure equation (regions will always be strongly coupled)
• Drawbacks
◦ Some reduction in parallel scaling compared to implicit fringe
◦ Without special treatment, implicit Overset interpolation is non-conservative!
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 6
Overset Mesh: New Development
New Parallelised Overset Assembly Algorithm
• Using new fringe detection and minimisation variants
• Improved performance of fringe detection and prioritisation of domains
Choice of Fringe Algorithms
1. manual fringe - user specified cellSet for acceptors
2. faceCells fringe - acceptors are face cells of a given patch
3. overlap fringe - automatically calculates overlap between regions, which can be
minimised using different Donor Suitability Functions
Choice of Donor Suitability Functions
1. noSuitability: full overlap (expensive interpolation!)
2. patchDistance: pushes the overlap at a given distance between specified
patches,
3. cellVolumes: cells with similar volumes constitute donor/acceptor pairs
4. faceArea: cells with similar minimum face areas constitute donor/acceptor pairs
5. cellBoundingBoxDiagonal - cells which have similar sizes of their bounding
box diagonals constitute donor/acceptor pairs
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 7
Overset Mesh: New Development
New Overset Interpolation Algorithms
• Transfer of data from donors to acceptors is carried out using run-time selectable
oversetInterpolation schemes
Choice of Overset Interpolation Schemes
1. injection: simply takes the donor value and injects it into the acceptor (first
order accurate, bounded)
2. averageValue: average value of all donors (first order accurate, bounded)
3. inverseDistance: weights for donor cells calculated using inverse distance
from a given donor to acceptor (first/second order accurate and bounded)
4. leastSquaresGradient: value in acceptor calculated using least squares fit
through its donors (second order accurate/unbounded)
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 8
Overset Mesh: Implicit Conservation
Implicit Conservative Overset
• If performed implicitly, Overset interpolation is not conservative because of
arbitrary interpolation
• From the continuity equation, the net flux through fringe layer must be zero: this
represents an additional constraint on the global pressure equation
Handling Conservation at Overset Interfaces
• The pressure equation consists of three parts
1. Pressure gradient flux (implicit, on LHS)
2. Pressure gradient in non-orthogonal correction (explicit, on RHS)
3. Divergence of the intermediate flux (explicit, on RHS)
• Last two parts should be adjusted across the fringe (since they are explicit) before
the insertion into the pressure equation
• How to handle the first, implicit part?
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 9
Overset Mesh: Implicit Conservation
Handling Conservation at Overset Interfaces
• We need to add an additional constraint in the pressure equation, stating that the
sum of the pressure gradient flux in fringe cells needs to be zero
• This constraint is enforced using the additive correction approach
• In each linear solver iteration, we calculate an additive correction to the
non-conservative Overset interpolation based on fringe mass imbalance and add
scatter this value across all acceptors
• This approach ensures mass conservation up to specified pressure equation
tolerance
Notes
• If explicit Overset is used (no interpolation during linear system solver), the
pressure gradient will take care of ensuring global mass conservation
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 10
Overset Mesh: Example
Handling Conservation at Overset Interfaces: Example
• Moving 2-D square with constant velocity through the stagnating fluid
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 11
Overset Mesh
Handling Conservation at Overset Interfaces: Example
• Pressure equation solved to the tolerance of 1e-7 in all cases
• Continuity error, fvc::div(phi) for implicit Overset:
◦ Order of magnitude 0.1 without conservative correction approach
◦ Order of magnitude 1e-5 with conservative correction approach
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 12
Overset Mesh: Example
Overset Mesh Validation: DTC Propeller Case
• Transient propeller simulation for a model scale propeller for DTC hull (Duisburg
Test Case, a container ship)
• Case details
◦ Model scale propeller for DTC hull, Diameter, D = 0.15 m
◦ n = 890 RPM (14.8343 RPS)
◦ Advance coefficient (J = v/(n*D) = 0.75)
• Simulation set-up
◦ deltaT = 2 degrees per time step
◦ endTime = 10 periods (10 full rotations)
◦ 1.85M cells (1.16M for propeller + 0.685M for background)
◦ Max Courant number during simulation is approximately 15
◦ Euler on ddt, convection/diffusion all standard second order
◦ k-omega SST turbulence model,
◦ Explicit Overset
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 13
Overset Mesh: Example
Overset Mesh Validation: DTC Propeller Case
• Simulation time: approximately 4 hours for a single rotation on a Desktop machine
• Errors compared to experiments:
◦ 4.3% under-prediction in thrust
◦ 6.6% over-prediction in torque
• Errors probably due to large time step and poor mesh
Thrust & torque coefficients; KT & 10KQ
0.4
KQ, EXP
0.3
0.25
0.2
0.15
0.1
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
KT, CFD (overset,transient)
KT, EXP
KQ, CFD (overset, transient)
0.35
0
0.05
0.1
0.15
0.2
0.25
0.3 0.35
Time; t, s
0.4
0.45
0.5
0.55
0.6
0.65
Public Release: Native Overset Mesh in FOAM-Extend – p. 14
Overset Mesh: Parallel Scaling
Parallel Efficiency of the Overset Mesh
• Implementation of Overset interpolation performed similar to GGI
◦ Interpolation performed in out-of-core multiplication with parallel comms
◦ Parallelised using mapDistribute tool
• Parallel scaling test case
◦ Scaling test performed on 20M cells submarine mesh
◦ Approximately 40K donor/acceptor cells (0.4% of total cell count)
◦ Performed 20 iterations with explicit and implicit Overset fringe
• Parallel speed-up on 64 cores: 41 (implicit) and 46 (explicit)
• Parallel efficiency on 64 cores: 64% (implicit) and 74% (explicit)
64
100
Explicit fringe
Implicit fringe
Linear scaling
80
Parallel efficiency, E, %
Parallel speed-up, S
32
16
8
4
40
Explicit fringe
Implicit fringe
20
2
1
60
1
2
4
8
Number of cores, n
16
32
64
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
0
1
2
4
8
Number of cores, n
16
32
64
Public Release: Native Overset Mesh in FOAM-Extend – p. 15
Overset Mesh: Parallel Scaling
Overset Mesh: Parallel Scaling Test
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 16
Overset Mesh: Future Work
Future Work
• Improved overlap fringe algorithm: the current algorithm is under-performing both
in serial and in parallel because we first find the full overlap and then minimise it. I
think this is not a small task - funding is required
• Need to improve the parallelisation of the donor search
• Validation and verification!
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 17
NUMAP Summer 2017
NUMAP Summer 2017: 10-day Summer School at University of Zagreb
• NUMAP Spring not planned for 2017
• At Uni Zagreb, we may do a new Programming course in OpenFOAM based on
new training materials
• Pre-announcement for NUMAP Summer 2017: 21/Aug – 1/Sep/2016. Applications
open from 1/Mar/2017
• ESI/OpenCFD sponsoring the 2017 NUMAP? (Always ask in public!)
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 18
Summary
UK and Eire OpenFOAM User Day
• Who’s next for the OpenFOAM UK and Eire User Group Meeting?
12th OpenFOAM Workshop
24–27 July 2017 Exeter, UK
German OpenFOAM Stammtisch
United 2017, 20-21/Feb/2017, Kassel
NUMAP-FOAM Summer School 2017
Zagreb Croatia, 21/Aug–1/Sep/2016
5th OpenFOAM UK and Eire User Group Meeting, Dublin, 16-17/Jan/2017
Public Release: Native Overset Mesh in FOAM-Extend – p. 19