To bin or not to bin
Javier Casado Gómez
Still homeless but...
Universidad Autónoma de Madrid (UAM)
Leibniz institute for Astrophysics Potsdam (AIP)
Yago Ascasibar
Rubén García Benito
Enrica Bellocchi
Giovanni Guidi
Omar Choudhury
2nd SELGIFS Advanced School on
Integral-Field Spectroscopy Data Analysis
Outline
◦ Why binning? Are there no other
options?
◦ What is out there?
◦ New tools for new data
◦ Our approach and some maths
◦ What we can do and WHAT IS
DIFFERENT/NEW
◦ Some science! Finally!
Why binning?
Binning is not something new
◦ extended sources.
▫
▫
derive their properties
large extragalactic surveys
■ SExtractor (Bertin and Arnouts, 1996)
■ Automatization
■ Criteria: thresholds, signal-to-noise
Why binning?
Binning is not something new
◦ extended sources.
▫
▫
derive their properties
large extragalactic surveys
■ SExtractor (Bertin and Arnouts, 1996)
■ Automatization
■ Criteria: thresholds, signal-to-noise
However, nowadays it is mainly used to
INCREASE the SIGNAL-TO-NOISE
Why binning?
Binning is not something new
◦ extended sources.
▫
▫
derive their properties
large extragalactic surveys
■ SExtractor (Bertin and Arnouts, 1996)
■ Automatization
■ Criteria: thresholds, signal-to-noise
However, nowadays it is mainly used to
INCREASE the SIGNAL-TO-NOISE
Are there any other options?
◦ Smoothing
◦ S/N criteria (and neglect spaxels)
▫ biases… and who knows what else?
What is out there?
What is out there?
SExtractor - Bertin and Arnouts (1996)
What is out there?
SExtractor - Bertin and Arnouts (1996)
HIIExplorer - Sánchez et al. (2012)
What is out there?
SExtractor - Bertin and Arnouts (1996)
PORTO3D
HIIExplorer - Sánchez et al. (2012)
What is out there?
SExtractor - Bertin and Arnouts (1996)
PORTO3D
HIIExplorer - Sánchez et al. (2012)
VORONOI - Cappellari and Copin (2003)
What is out there?
SExtractor - Bertin and Arnouts (1996)
PORTO3D
HIIExplorer - Sánchez et al. (2012)
VORONOI - Cappellari and Copin (2003)
Designed to solve a problem
NEW TOOLS
NEW DATA
◦
◦
◦
resolved scales
IFU data
the overwhelming case of MUSE
NEW TOOLS
NEW DATA
◦
◦
◦
resolved scales
IFU data
the overwhelming case of MUSE
Sánchez et al. (2015)
NEW TOOLS
NEW DATA
◦
◦
◦
resolved scales
IFU data
the overwhelming case of MUSE
Sánchez et al. (2015)
Our initial concerns...
◦ What if we do not bin “properly”?
◦ Neglecting regions
◦ Low S/N, no smoothing
Characterize and “properly” treat our data.
Bayesian
Technique for
Multi-image
Analysis
Bayesian
Technique for
Multi-image
Analysis
Yes… BaTMAn
Bayesian
Technique for
Multi-image
Analysis
BaTMAn is going to look at the
data, information → D:{X, E}
A
Yes… BaTMAn
Bayesian
Technique for
Multi-image
Analysis
BaTMAn is going to look at the
data, information → D:{X, E}
A
Value A (units of A)
Yes… BaTMAn
Bayesian
Technique for
Multi-image
Analysis
Yes… BaTMAn
BaTMAn is going to look at the
data, information → D:{X, E}
3
A
2
Value A (units of A)
4
1
Bayesian
Technique for
Multi-image
Analysis
Yes… BaTMAn
BaTMAn is going to look at the
data, information → D:{X, E}
A
2
Value A (units of A)
4
1
Bayesian
Technique for
Multi-image
Analysis
Yes… BaTMAn
BaTMAn is going to look at the
data, information → D:{X, E}
6
7
A
1
2
4
Value A (units of A)
3
1
Bayesian
Technique for
Multi-image
Analysis
Yes… BaTMAn
BaTMAn is going to look at the
data, information → D:{X, E}
3
A
2
Value A (units of A)
4
1
Bayesian
Technique for
Multi-image
Analysis
Value B (units of B)
Yes… BaTMAn
3
1
2
A
4
Value A (units of A)
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
◦ considers the entire image 2D/3D
▫ (m x n) x λ
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
◦ considers the entire image 2D/3D
▫ (m x n) x λ
▫ (m x n) = Nreg regions
▫ Evaluates all possible coadditions
Nreg → Nreg-1
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
◦ considers the entire image 2D/3D
▫ (m x n) x λ
▫ (m x n) = Nreg regions
▫ Evaluates all possible coadditions
Nreg → Nreg-1
CRITERIUM - BAYESIAN PROBABILITY
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
◦ considers the entire image 2D/3D
▫ (m x n) x λ
▫ (m x n) = Nreg regions
▫ Evaluates all possible coadditions
Nreg → Nreg-1
CRITERIUM - BAYESIAN PROBABILITY
◦ Do while ( P(Nreg-1) > K x P(Nreg) )
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
◦ considers the entire image 2D/3D
▫ (m x n) x λ
▫ (m x n) = Nreg regions
▫ Evaluates all possible coadditions
Nreg → Nreg-1
CRITERIUM - BAYESIAN PROBABILITY
◦ Do while ( P(Nreg-1) > K x P(Nreg) )
and
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
◦ considers the entire image 2D/3D
▫ (m x n) x λ
▫ (m x n) = Nreg regions
▫ Evaluates all possible coadditions
Nreg → Nreg-1
CRITERIUM - BAYESIAN PROBABILITY
◦ Do while ( P(Nreg-1) > K x P(Nreg) )
and
What does BaTMAn really do?
SOME MATHS. BAYES (SORRY!)
◦ considers the entire image 2D/3D
▫ (m x n) x λ
▫ (m x n) = Nreg regions
▫ Evaluates all possible coadditions
Nreg → Nreg-1
CRITERIUM - BAYESIAN PROBABILITY
◦ Do while ( P(Nreg-1) > K x P(Nreg) )
◦ It implies one funny thing…
BaTMAn may not bin your multi-image
What can we do? Why is it different?
◦
It will only coadd spaxels if they are consistent with
having the same information D:{X, E}
What can we do? Why is it different?
◦
It will only coadd spaxels if they are consistent with
having the same information D:{X, E}
◦ Problem-independent, “universal”
What can we do? Why is it different?
◦
It will only coadd spaxels if they are consistent with
having the same information D:{X, E}
◦ Problem-independent, “universal”
What can we do? Why is it different?
◦
It will only coadd spaxels if they are consistent with
having the same information D:{X, E}
◦ Problem-independent, “universal”
What can we do? Why is it different?
◦
It will only coadd spaxels if they are consistent with
having the same information D:{X, E}
◦ Problem-independent, “universal”
◦ MULTI-IMAGE MODE
Some science… finally
Weak lines?
MAP
(WHAT CAN BE DONE?)
SHIFU (Rubén García-Benito)
Some science… finally
Weak lines?
(WHAT CAN BE DONE?)
SHIFU (Rubén García-Benito)
RGB provides the maps,
measures directly the spectra
We bin the maps
“monochromatically”
MAP
MAPS
Some science… finally
Weak lines?
(WHAT CAN BE DONE?)
SHIFU (Rubén García-Benito)
We bin in cutted datacubes
∓15Å around every Balmer line
MULTI - IMAGE
RGB measures with the same
pipeline but with
BaTMAn binning
SPECTRUM
Some science… finally
Weak lines?
(WHAT CAN BE DONE?)
SHIFU (Rubén García-Benito)
We bin in cutted datacubes
∓15Å around every Balmer line
MULTI - IMAGE
RGB measures with the same
pipeline but with
BaTMAn binning
SPECTRUM
Some science… finally
Weak lines?
(WHAT CAN BE DONE?)
SHIFU (Rubén García-Benito)
We bin in cutted datacubes
∓15Å around every Balmer line
MULTI - IMAGE
RGB measures with the same
pipeline but with
BaTMAn binning
SPECTRUM
Some science… finally
Weak lines?
MAP
(WHAT CAN BE DONE?)
SHIFU (Rubén García-Benito)
MAPS
SPECTRUM
Some science… finally
(WHAT CAN BE DONE?)
Combinations of them, multi-image-variable
CALIFA - NGC2906
BPT diagnostic diagram.
Binned with BaTMAn, multi image/MAP,
OIII, NII, Hα, Hβ
PINGS - SQ - NGC7319
Some science… finally
(WHAT CAN BE DONE?)
Combinations of them, multi-image-variable
CALIFA - NGC2906
BPT diagnostic diagram.
Binned with BaTMAn, multi image/MAP,
OIII, NII, Hα, Hβ
PINGS - SQ - NGC7319
Some science… finally
(WHAT CAN BE DONE?)
Attack the spectrum directly. What do we obtain?
Some science… finally
(WHAT CAN BE DONE?)
Attack the spectrum directly. What do we obtain?
What does BaTMAn see?
- BaTMAn “sees”
points and error,
no spectra
- BaTMAn bins
luminosity
- BaTMAn bins
velocity
Better questions are:
What do we want
BaTMAN to see?
What can we interpret
out of it?
“VELOCITY MAP”
Binned with BaTMAn, multi-image,
Cutted datacube, ∓15Å around Hα [6563]
Some science… finally
(WHAT CAN BE DONE?)
No binning! Image characterization.
NGC0842
“IS ANNULI BINNING A GOOD IDEA?” - searching for α-enhancements
Binned with BaTMAn, multi-image,
Cutted datacube with two features, H and Mgb [4850-4876, 5161-5198]
Some science… finally
(WHAT CAN BE DONE?)
No binning! Image characterization.
NGC0842
“IS ANNULI BINNING A GOOD IDEA?” - searching for α-enhancements
Binned with BaTMAn, multi-image,
Cutted datacube with two features, H and Mgb [4850-4876, 5161-5198]
Some science… finally
(WHAT CAN BE DONE?)
No binning! Image characterization.
NGC0842
“IS ANNULI BINNING A GOOD IDEA?” - searching for α-enhancements
Binned with BaTMAn, multi-image,
Cutted datacube with two features, H and Mgb [4850-4876, 5161-5198]
Some science… finally
(WHAT CAN BE DONE?)
No binning! Image characterization.
NGC0842
ZED
OR
LI
MA
XN
FLU
VE
CO LOCI
T
RR
EC Y
TED
“IS ANNULI BINNING A GOOD IDEA?” - searching for α-enhancements
Binned with BaTMAn, multi-image,
Cutted datacube with two features, H and Mgb [4850-4876, 5161-5198]
Some science… finally
No binning! Image characterization.
(WHAT CAN BE DONE?)
BO
TH
NGC0842
ZED
OR
LI
MA
XN
FLU
VE
CO LOCI
T
RR
EC Y
TED
What can we interpret here?
- What is dominant, for
sure.
- No much info left if we
get rid of flux and
velocity.
“IS ANNULI BINNING A GOOD IDEA?” - searching for α-enhancements
Binned with BaTMAn, multi-image,
Cutted datacube with two features, H and Mgb [4850-4876, 5161-5198]
Some philosophy
◦ Is it better?
▫ vs. Voronoi
▫ vs. S/N thresholds
Some philosophy
◦ Is it better?
▫ vs. Voronoi
▫ vs. S/N thresholds
◦ Problem-independent
▫ Cool, but it requires everyone to
think in advance
Some philosophy
◦ Is it better?
▫ vs. Voronoi
▫ vs. S/N thresholds
◦ Problem-independent
▫ Cool, but it requires everyone to
think in advance
◦ No binning, image characterization
Some philosophy
◦ Is it better?
▫ vs. Voronoi
▫ vs. S/N thresholds
◦ Problem-independent
▫ Cool, but it requires everyone to
think in advance
◦ No binning, image characterization
Maybe hard to sell
◦ S/N criteria
and
Conclusions
◦ Binning and image characterization
◦ Bins based on the INDEPENDENT
information available in your data.
▫ Do you trust your errors?
◦ Problem independent
◦ Flexible, problem dependent
◦ … and Multi-image
You have a problem, maybe BaTMAn has a
solution.
Thanks!
ONE POSSIBLE PRACTICAL CASE
◦ Binning for H⍺ in CALIFA/MUSE
(NGC2906)
▫ Cut the datacube around the line…
ONE POSSIBLE PRACTICAL CASE
ONE POSSIBLE PRACTICAL CASE
◦ Binning for H⍺ in CALIFA/MUSE
(NGC2906)
▫ Cut the datacube around the line…
▫ Feed BaTMAn with it!
ONE POSSIBLE PRACTICAL CASE
IMPORT BaTMAn
ONE POSSIBLE PRACTICAL CASE
IMPORT BaTMAn
Transpose the matrix… yep, sorry
ONE POSSIBLE PRACTICAL CASE
IMPORT BaTMAn
Transpose the matrix… yep, sorry
Feed the beast!
ONE POSSIBLE PRACTICAL CASE
INPUT AND OUTPUT OF BaTMAn
◦
INPUT
▫ Image: (m x n) x λ
▫ error: (m x n) x λ
▫ num=?
▫ ndim=λ
ONE POSSIBLE PRACTICAL CASE
INPUT AND OUTPUT OF BaTMAn
◦
◦
INPUT
▫ Image: (m x n) x λ
▫ error: (m x n) x λ
▫ num=?
▫ ndim=λ
OUTPUT
▫ Image: (m x n) x λ
▫ error: (m x n) x λ
▫ labels
ONE POSSIBLE PRACTICAL CASE
◦ Binning for H⍺ in CALIFA/MUSE
(NGC2906)
▫ Cut the datacube around the line…
▫ Feed BaTMAn with it!
▫ Plot the result
ONE POSSIBLE PRACTICAL CASE
◦ Binning for H⍺ in CALIFA/MUSE
(NGC2906)
▫ Cut the datacube around the line…
▫ Feed BaTMAn with it!
▫ Plot the result
ONE POSSIBLE PRACTICAL CASE
ONE POSSIBLE PRACTICAL CASE
◦ Binning for H⍺ in CALIFA/MUSE
(NGC2906)
▫
▫
▫
▫
Cut the datacube around the line…
Feed BaTMAn with it!
Plot the result
Interpret the result
ONE POSSIBLE PRACTICAL CASE
ONE POSSIBLE PRACTICAL CASE
◦ Binning for H⍺ in CALIFA/MUSE
(NGC2906)
▫
▫
▫
▫
▫
Cut the datacube around the line…
Feed BaTMAn with it!
Plot the result
Interpret the result
Think better!
Some science… finally
(WHAT CAN BE DONE?)
Reduce the scatter. Are the observed trends real?
107 yr
SL
Casado et al. (2015)
Ha(EW)
SDSS data, 3” fiber, 0.02 <z 0.07
ER
ST
FA
OW
ER
(u-r) color
108.5 yr
TWO PROXIES FOR THE SSFR
BUT WITH DIFFERENT TIME SCALES
“AGEING OF GALAXIES AND SOMETHING ODD”
Do we get rid of it if we reduce the scatter?
Binned with BaTMAn, multi-image,
Cutted datacube, ∓20Å around Hα [6563]
Some science… finally
(WHAT CAN BE DONE?)
Reduce the scatter. Are the observed trends real?
Casado et al. (in preparation)
107 yr
SL
Ha(EW)
BaTMAnized
CALIFA
ER
ST
FA
OW
ER
(g-r) color
10<8.5 yr
TWO PROXIES FOR THE SSFR
BUT WITH DIFFERENT TIME SCALES
“AGEING OF GALAXIES AND SOMETHING ODD”
Do we get rid of it if we reduce the scatter?
Binned with BaTMAn, multi-image,
Cutted datacube, ∓20Å around Hα [6563]
SOME MATHS.
BAYESIAN APPROACH
We use Bayes “TWICE”.
- Parameter estimation (pure bayesian)
-
Model selection (pure bayesians won’t like it!)
SOME MATHS.
BAYESIAN APPROACH
We use Bayes “TWICE”.
- Parameter estimation (pure bayesian)
- Model selection (pure bayesians won’t like it!)
Bayes needs:
▫ MODEL
SOME MATHS.
BAYESIAN APPROACH
We use Bayes “TWICE”.
- Parameter estimation (pure bayes)
- Model selection (pure bayesians won’t like it!)
Bayes needs:
▫ MODEL
▫ PRIOR
SOME MATHS.
BAYESIAN APPROACH
We use Bayes “TWICE”.
- Parameter estimation (pure bayes)
- Model selection (pure bayesians won’t like it!)
Bayes needs:
▫ MODEL
▫ PRIOR
▫ POSTERIOR
SOME MATHS.
BAYESIAN APPROACH
We use Bayes “TWICE”.
- Parameter estimation (pure bayes)
- Model selection (pure bayesians won’t like it!)
Bayes needs:
▫ MODEL
▫ PRIOR
▫ POSTERIOR
We have:
▫ information
▫ Initial tessellation
(m x n) = Nreg regions
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL
▫ PRIOR
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
Given the tessellation R and the data D
D:
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
Given the tessellation R and the data D
D:
▫ POSTERIOR
pure bayesian
μrλ , σrλ for every defined region
of the tessellation
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR → μrλ , σrλ for every defined region of the
tessellation
Model selection (pure bayesians won’t like it)
▫ MODEL
▫ PRIOR
▫ POSTERIOR
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR → μrλ , σrλ for every defined region of the
tessellation
Model selection (pure bayesians won’t like it)
▫ MODEL → we consider all possible tessellations.
▫ PRIOR
▫ POSTERIOR
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR → μrλ , σrλ for every defined region of the
tessellation
Model selection (pure bayesians won’t like it)
▫ MODEL → we consider all possible tessellations.
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR → μrλ , σrλ for every defined region of the
tessellation
Model selection (pure bayesians won’t like it)
▫ MODEL → we consider all possible tessellations.
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR
Pure bayes → smoothing, weighting
SOME MATHS.
BAYESIAN APPROACH
Parameter estimation (pure bayes)
▫ MODEL → gaussian probability
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR → μrλ , σrλ for every defined region of the
tessellation
Model selection (pure bayesians won’t like it)
▫ MODEL → we consider all possible tessellations.
▫ PRIOR → uniform, non informative, objective, proper
▫ POSTERIOR
Pure bayes → smoothing, weighting
Instead:
◦ P(Nreg-1) > K x P(Nreg) → P(A⋃N3) > K x P(A,N3)