The Methods of Numerical Relativity

Why numerical relativity?
The field equations
Code ecosystem and numerical methods
The Methods of Numerical Relativity
David Hilditch
Theoretisch-Physikalisches Institut,
Friedrich-Schiller-Universität Jena
Octopus Developers Meeting
January 29, 2015
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Outline
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
When not to try Numerical Relativity
We seek solutions to the gravitational field equations,
Gab = Rab − 12 gab R = 8πTab .
which describe how curvature of spacetime evolves and is related
to matter distribution. Non-linear PDEs. How can we find
solutions? Different physics:
• Is the desired solution time independent, or with a lot of
symmetry? There might be an exact solution. Read a book.
• Is the desired solution close to an exact solution that we
already know? Perturbation theory/asymptotic expansions.
Second-order?
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
When to try Numerical Relativity
On the other hand:
• Are the fields highly dynamical? Are non-linear terms
expected to dominate the behavior? Is there no symmetry?
Use Numerical Relativity; numerical methods to find
approximate solutions that converge to the continuum
solution in the limit of infinite resolution.
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Ingredients
Mathematics:
• PDEs theory.
• Differential geometry.
• Numerical analysis.
Physics:
• GR.
• Matter models.
Computation:
• HPC.
• Numerical methods.
Event horizon of merging blackholes. Thierfelder. 2009
• Visualization.
David Hilditch
The Methods of Numerical Relativity
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Example problems
• Critical collapse. Can we
make arbitrarily small
blackholes?
• Computational astrophysics:
supernova core collapse.
• Gravitational wave
astronomy.
Curvature in GW collapse evolution. DH, Weyhausen,
Brügmann. 2015
David Hilditch
The Methods of Numerical Relativity
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Example problems
• Critical collapse. Can we
make arbitrarily small
blackholes?
• Computational astrophysics:
supernova core collapse.
• Gravitational wave
astronomy.
LIGO Hanford
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Example problems
t/M = 428.264
−3
8
• Critical collapse. Can we
−4
make arbitrarily small
blackholes?
supernova core collapse.
4
−4.5
2
y/M
• Computational astrophysics:
−3.5
Z4c
6
−5
0
−5.5
−6
−2
−6.5
−4
−7
• Gravitational wave
−6
−7.5
−8
−8
astronomy.
−6
−4
−2
0
x/M
2
4
6
8
Contour plot of density of NS. DH+. 2013
David Hilditch
The Methods of Numerical Relativity
−8
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
A brief history of NR
From Wikipedia NR page and Smarr:
• Late 1950s: ADM decomposition.
• 1960s. First numerics, including Smarr [nice talk:
http://online.itp.ucsb.edu/online/numrel00/smarr/].
• 1970s. Axisymmetric spacetimes. York 3+1 decomposition.
• 1980s. Numerical Relativity becomes a field.
• 1990s. Critical Phenomena. 3d work becomes feasible. Grand
challenge.
• 2000s. Compact binaries.
David Hilditch
The Methods of Numerical Relativity
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Outline
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Evolution equations and constraints
Decomposition:
∂t γij = −2αKij + 2D(i βj) ,
k
∂t Kij = −Di Dj α + α[Rij − 2K i Kjk + KKij ] + Lβ Kij ,
H = R − Kij K
ij
2
+ K = 0,
ρ=4/ρ=8
ρ=8/ρ=16
ρ=16/ρ=32
ρ=32/ρ=64
ρ=64/ρ=128
ρ=128/ρ=256
6 ρ=256/ρ=512
ρ=512/ρ=1024
10
8
j
Mi = D (Kij − γij K ) = 0 .
4
• Evolution and constraint
equations.
2
0
• Constraint subsystem closed.
-2
-4
• Gauge freedom similar to
E&M / YM.
0
0.1
0.2
0.3
0.4
0.5
Cao and DH. 2011
Formulation crucial!
David Hilditch
The Methods of Numerical Relativity
0.6
0.7
0.8
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Evolution equations and constraints
Decomposition:
∂t γij = −2αKij + 2D(i βj) ,
k
∂t Kij = −Di Dj α + α[Rij − 2K i Kjk + KKij ] + Lβ Kij ,
H = R − Kij K
ij
2
+ K = 0,
ρ=4/ρ=8
ρ=8/ρ=16
ρ=16/ρ=32
ρ=32/ρ=64
ρ=64/ρ=128
ρ=128/ρ=256
6 ρ=256/ρ=512
ρ=512/ρ=1024
10
8
j
Mi = D (Kij − γij K ) = 0 .
4
• Evolution and constraint
equations.
2
0
• Constraint subsystem closed.
-2
-4
• Gauge freedom similar to
E&M / YM.
0
0.1
0.2
0.3
0.4
0.5
Cao and DH. 2011
Formulation crucial!
David Hilditch
The Methods of Numerical Relativity
0.6
0.7
0.8
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Outline
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Methods
Bare-bones summary:
Grids:
• Grid-up (physical).
• Time integration: MoL.
• Moving-box/adaptive MR.
• Multipatch.
Spatial derivatives:
• Corotation.
• Finite differences.
Popular codes:
• HRSC. (Fluids).
• The Einstein Toolkit
• Pseudospectral.
• Spectral Einstein Code.
Boundaries:
• BAM and bamps. (Jena)
• Elliptic for ID.
Various licences.
• CPBCs.
David Hilditch
The Methods of Numerical Relativity
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Parallelization
Implementation typically with:
• Automated generation.
Parallelization:
• MPI/OMP standard.
102
Time units per hour
• Mixture of
C++ /C/Fortran.
Ideal Scaling
SuperMUC - only MPI
SuperMUC - 2 OMP/MPI
SuperMUC - 4 OMP/MPI
quadler - only MPI
quadler - 2 OMP/MPI
quadler - 4 OMP/MPI
101
• GPU? Memory a problem.
Some applications.
100
• MR & balancing an issue.
96
192
384
768
1536
Number of cores
Strong scaling of BAM at LRZ.
David Hilditch
The Methods of Numerical Relativity
3072
6144
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Parallelization
Implementation typically with:
10
• Mixture of C++ /C/Fortran.
Parallelization:
• MPI/OMP standard.
• GPU? Memory a problem.
Some applications.
• MR & balancing an issue.
David Hilditch
Time units per hour
• Automated generation.
1.0
Ideal Scaling
SuperMUC
0.1
279
558
1115
Number of cores
2230
Strong scaling of bamps at LRZ.
The Methods of Numerical Relativity
4459
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Outline
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
David Hilditch
The Methods of Numerical Relativity
Summary
Why numerical relativity?
The field equations
Code ecosystem and numerical methods
Summary
Some parting thoughts...
Points for discussion:
• Solid mathematical foundation, although implementation of
principle solutions not always clear.
• Smooth solutions often expected. Meaningful error analysis
possible!
• Perfect parallelization hard. Libraries?
One expects a great deal of overlap with the rest of computational
physics. Sadly much develoment in isolation.
David Hilditch
The Methods of Numerical Relativity