IDR(𝑠) as a projection method
Marijn Bartel Schreuders
Supervisor:
Date:
Dr. Ir. M.B. Van Gijzen
Monday, 24 February 2014
IDR(𝑠) as a projection method
1
Overview of this presentation
• Iterative methods
• Projection methods
• Krylov subspace methods
• Eigenvalue problems
• Linear systems of equations
• The IDR(𝑠) method
• General idea behind the IDR(𝑠) method
• Numerical examples
• Ritz-IDR
• Research Goals
IDR(𝑠) as a projection method
2
Iterative methods
• Consider a linear system
𝐴𝑥 = 𝑏
(1)
with 𝐴 ∈ ℝ𝑛×𝑛 and 𝑏 ∈ ℝ𝑛
• Find an approximate solution 𝑥𝑚 to (1), with initial guess 𝑥0
• Residual 𝑟𝑚 = 𝑏 − 𝐴𝑥𝑚
IDR(𝑠) as a projection method
3
Projection methods
Subspaces
• Define 𝒦𝑚 ⊂ ℝ𝑛×𝑛 of dimension 𝑚 ≤ 𝑛
• ‘Subspace of candidate approximants’ or ‘Search subspace’
• Define ℒ𝑚 ⊂ ℝ𝑛×𝑛 of dimension 𝑚 ≤ 𝑛
• ‘Subspace of constraints’ or ‘Left subspace’
IDR(𝑠) as a projection method
4
Projection methods
Definition
Find 𝑥𝑚 ∈ 𝑥0 + 𝒦𝑚
such that
𝑟𝑚 ⊥ ℒ𝑚
• Find 𝑥𝑚 ∈ 𝑥0 + 𝒦𝑚
• Let 𝑉𝑚 = 𝑣1 , 𝑣2 , … , 𝑣𝑚
• Then 𝑥𝑚 = 𝑥0 +
𝑚
𝑗=1
form an orthonormal basis for 𝒦𝑚
𝑦𝑚,𝑗 𝑣𝑗
= 𝑥0 + 𝑉𝑚 𝑦𝑚
How to find this vector?
IDR(𝑠) as a projection method
5
Projection methods
How to find 𝒚𝒎
• Let Wm = 𝑤1 , 𝑤2 , … , 𝑤𝑚
form an orthonormal basis for ℒ𝑚
• 𝑟𝑚 = 𝑏 − 𝐴𝑥𝑚
= 𝑏 − 𝐴 𝑥0 + 𝑉𝑚 𝑦𝑚
= 𝑟0 − 𝐴𝑉𝑚 𝑦𝑚
• 𝑊𝑚𝑇 𝑟𝑚 = 0
• Hence:
𝑊𝑚𝑇 𝑟0
=
(𝑊𝑚𝑇 𝐴𝑉𝑚 )𝑦𝑚
→
ym =
−1 T
T
Wm AVm Wm r0
𝑥𝑚 = 𝑥0 + 𝑉𝑚 𝑦𝑚
IDR(𝑠) as a projection method
6
Projection methods
General algorithm
• How to choose the subspaces?
IDR(𝑠) as a projection method
7
Krylov subspace methods
General
• 𝒦𝑚 𝐴, 𝑣1 = 𝑠𝑝𝑎𝑛 𝑣1 , 𝐴𝑣1 , 𝐴2 𝑣1 , … , 𝐴𝑚−1 𝑣1
• Different methods for different choices of ℒ𝑚
• Can be used for
• eigenvalue problems
• linear systems of equations
IDR(𝑠) as a projection method
8
Krylov subspace methods
Overview
IDR(𝑠) as a projection method
9
Krylov subspace methods
Overview
IDR(𝑠) as a projection method
10
Krylov subspace methods
Eigenvalue problems
• Computing all eigenvalues can be costly
• A is a full matrix
• A is large
• Idea:
find smaller matrix for which it is easy to compute ‘Ritz values’
• Good approximations to some of the eigenvalues of A
IDR(𝑠) as a projection method
11
Krylov subspace methods
Overview
IDR(𝑠) as a projection method
12
Krylov subspace methods
Overview
IDR(𝑠) as a projection method
13
Krylov subspace methods
Symmetric matrices
• Conjugate Gradient method (CG)
• Optimality condition
• Uses short recurrences
• Minimises the residual
IDR(𝑠) as a projection method
14
Krylov subspace methods
Nonsymmetric matrices
• GMRES-type methods
• Long recurrences
• Minimisation of the residual
• Bi-CG – type methods
• Short recurrences
• No minimisation of the residual
• Two matrix-vector operations per iteration
• Are their any other possibilities?
IDR(𝑠) as a projection method
15
Induced Dimension Reduction (s)
• Residuals are forced to be in certain subspaces
• Compute 𝑠 residuals in each iteration
IDR(𝑠) as a projection method
16
Induced Dimension Reduction (s)
IDR theorem
Theorem 1 (IDR theorem):
Let 𝐴 ∈ ℝ𝑛×𝑛 and 𝑣0 ∈ ℝ𝑛
Let 𝒢0 = 𝒦𝑛 𝐴, 𝑣0
Let 𝒮 ⊂ ℝ𝑛 such that 𝒮 and 𝒢0 do not share a subspace of 𝐴
Define: 𝒢𝑗 = (𝐼 − 𝜔𝑗 𝐴)(𝒢𝑗−1 ∩ 𝒮)
Then the following holds:
(i)
(ii)
𝒢𝑗+1 ⊂ 𝒢𝑗 ∀𝑗 ≥ 0
𝒢𝑗 = {0} for some 𝑗 < 𝑛
IDR(𝑠) as a projection method
17
Induced Dimension Reduction (s)
Numerical experiments
• Convection diffusion equation:
𝐿 𝑢 = −Δ𝑢 + 𝛽𝛻𝑢
• Discretise using finite differences on unit cube;
Dirichlet boundary conditions
• 20 internal points → 8000 equations
• Stopping criterion:
𝑟𝑗 < 𝑏 ∙ 10−8
IDR(𝑠) as a projection method
18
Induced Dimension Reduction (s)
Numerical experiments
• This is an example of a slide
IDR(𝑠) as a projection method
19
Induced Dimension Reduction (s)
Numerical experiments
• Matrix Market: 𝑎𝑑𝑑20 matrix
• Real, nonsymmetric, sparse 2395 × 2395 matrix
http://math.nist.gov/MatrixMarket/data/misc/hamm/add20.html
IDR(𝑠) as a projection method
20
Induced Dimension Reduction (s)
Numerical experiments
• This is an example of a slide
IDR(𝑠) as a projection method
21
Induced Dimension Reduction (s)
Numerical experiments
• This is an example of a slide
IDR(𝑠) as a projection method
22
Induced Dimension Reduction (s)
How to choose 𝝎𝒋
• Recall: 𝒢𝑗 = (𝐼 − 𝜔𝑗 𝐴)(𝒢𝑗−1 ∩ 𝒮)
How to choose 𝜔𝑗 ?
• Minimisation of the residuals
• Random?
• …… ?
IDR(𝑠) as a projection method
23
Induced Dimension Reduction (s)
Ritz-IDR
• Valeria Simoncini & Daniel Szyld
• Ritz-IDR
• Calculates Ritz values 𝜃𝑗
• 𝜔𝑗 =
1
𝜃𝑗
IDR(𝑠) as a projection method
24
Research goals
• Research goals are twofold:
1. Make clear how we can see IDR(𝑠) in the framework of
projection methods
2. Use the IDR(s) algorithm for calculating the 𝜔𝑗′ 𝑠
IDR(𝑠) as a projection method
25
IDR(𝑠) as a projection method
Marijn Bartel Schreuders
Supervisor:
Date:
Dr. Ir. M.B. Van Gijzen
Monday, 24 February 2014
IDR(𝑠) as a projection method
26
IDR(𝑠) as a projection method
27
Research goals
Let 𝐴𝑣𝑛 = 𝑡𝑛
• 𝑟𝑚 = 𝑣𝑛 − 𝜔𝑗 𝐴𝑣𝑛
= 𝑣𝑛 − 𝜔𝑗 𝑡𝑛
= 𝑣𝑛 − 𝜔𝑗 𝑡𝑛
𝑇
𝑣𝑛 − 𝜔𝑗 𝑡𝑛
= 𝑣𝑛𝑇 𝑣𝑛 − 2𝜔𝑗 𝑡𝑛𝑇 𝑣𝑛 + 𝜔𝑗2 𝑡𝑛𝑇 𝑡𝑛
• This is a polynomial in 𝜔𝑗
• To minimise, take derivative w.r.t. 𝜔𝑗
IDR(𝑠) as a projection method
28
Krylov subspace methods
Eigenvalue problems
Arnoldi Method
Lanczos method
&
Bi-Lanczos method
IDR(𝑠) as a projection method
29