COMPARATIVE MODEL PERFORMANCE EVALUATION
OF CMAQ-VISTAS, CMAQ-MADRID, AND CMAQMADRID-APT FOR A NITROGEN DEPOSITION
ASSESSMENT OF THE
ESCAMBIA BAY, FLORIDA WATERSHED
6th Annual CMAS Conference
Chapel Hill, NC
1-3 October 2007
Presented by Jay Haney
ICF International, San Rafael, CA
Co-Authors:


Sharon Douglas
Tom Myers

Justin Walters
John Jansen

Krish Vijayaraghavan

ICF
Southern Company
AER
Project sponsored by Southern Co.
Background/Objectives



Atmospheric deposition of nitrogen is a source of contamination in
Escambia Watershed
Air quality modeling performed to estimate change in nitrogen
deposition in watershed due to controls at a local EGU as part of larger
combined air/water quality modeling analysis
Objective for this part of study: Assess the ability of air quality models
to replicate observed gaseous and particulate concentrations and wet
and dry deposition
Air Quality Models Used:
Based on CMAQ 4.5.1



CMAQ-VISTAS: CB-IV, AERO4, modified SOA by VISTAS
CMAQ-MADRID: Sectional representation of particle size
distribution as opposed to modal for CMAQ
CMAQ-MADRID-APT: “Advanced plume treatment” based on
SCIPUFF with CHEMistry – SCICHEM
Air Quality Modeling Databases


Meteorological inputs: VISTAS 2002 inputs from RPO modeling
analysis
Emissions:




CMAQ-VISTAS: Base_G1
MADRID & APT: Base_F
Domain: 12-km ALGA, subset of VISTAS domain centered on Alabama
& Georgia
Annual simulations for 2002
CMAQ ALGA Subdomain/
Escambia Watershed
Escambia Watershed
Yorkville
N. Birmingham
Jefferson St.
Centreville
Oak Grove
OLF
Gulfport
Pensacola
CMAQ ALGA Subdomain
Plant Crist
Air Quality Data Used in Evaluation




SEARCH: Hourly gaseous and 3-day speciated PM2.5
concentrations
IMPROVE: 3-day speciated PM2.5 concentrations
CASTNET: Weekly particulate concentrations and derived dry
deposition based on concentration/ambient conditions
NADP: Weekly particulate concentrations and wet deposition
Model Performance Measures




Mean bias, normalized bias, fractional bias, mean error, normalized
gross error, and fractional gross error
Paired for appropriate time interval
Statistics calculated using daily averages, except for CASTNET and
NADP weekly measurements
Statistics calculated for all sites/species in ALGA domain with focus on
sites near Escambia watershed
Location of SEARCH and CASTNET Sites in CMAQ
Subdomain
Yorkville
N. Birmingham
Jefferson St.
Coffeeville
Sand
Mountain
Georgia
Station
Centreville
Oak Grove
OLF
Gulfport
Pensacola
SEARCH Sites
Sumatra
CASTNET Sites
Location of IMPROVE and NADP Sites in CMAQ
Subdomain
Mammoth Cave
Linville Gorge
Great Smoky Mtn
Cadiz
Shining Rock
Cohutta
Sipsey
Cape
Romain
Okefenokee
Baldwin Co.
Quincy
St. Marks
Chassahowitzka
Mobile Co.
Sumatra
Breton
IMPROVE Sites
NADP Sites
Results for Gaseous Species:
SO2 for SEARCH Sites
Mean Observed & Simulated SO2
8
ppb
6
OBS
4
CMAQ
MADRID
APT
2
0
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Results for Gaseous Species:
SO2 for SEARCH Sites
Percent (%)
Fractional Bias & Error: SO2
100
80
60
40
20
0
-20
-40
-60
-80
-100
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Results for Gaseous Species:
HNO3 for SEARCH Sites
Mean Observed & Simulated HNO3
2.0
ppb
1.5
OBS
1.0
CMAQ
0.5
MADRID
APT
0.0
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Results for Gaseous Species:
HNO3 for SEARCH Sites
Percent (%)
Fractional Bias & Error: HNO3
200
160
120
80
40
0
-40
-80
-120
-160
-200
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Gaseous Species Summary



For SO2, all models slightly underestimate concs nearby
(evidence of differences between MADRID and APT in Atlanta
area)
For HNO3, all models consistently overestimate at nearby sites
For NO2, all models do well and for NO, all models
underestimate, but these are typically not major contributors to
nitrogen deposition
Results for Particulate Species:
NO3 for SEARCH Sites
Mean Observed & Simulated NO3
2.0
ugm-3
1.6
1.2
OBS
0.8
CMAQ
MADRID
0.4
APT
0.0
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Results for Particulate Species:
NO3 for SEARCH Sites
Percent (%)
Fractional Bias & Error: NO3
200
160
120
80
40
0
-40
-80
-120
-160
-200
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Results for Particulate Species:
NH4 for SEARCH Sites
Mean Observed & Simulated NH4
4
ugm-3
3
OBS
2
CMAQ
MADRID
1
APT
0
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Results for Particulate Species:
NH4 for SEARCH Sites
Percent (%)
Fractional Bias & Error: NH4
100
80
60
40
20
0
-20
-40
-60
-80
-100
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
PNS
OLF
GFP
OAK
CTR
BHM
YRK
JST
Particulate Species Summary


For nitrate, CMAQ better simulates mean conc. but fractional
bias and error are lower for MADRID and APT at nearby sites
For ammonium, all models show consistent underestimation at
nearby sites, and overestimation at BHM and ATL
Results for Dry Deposition:
NO3 for CASTNET Sites
Percent (%)
Fractional Bias & Error: NO3 Dry Dep.
200
160
120
80
40
0
-40
-80
-120
-160
-200
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
Coffeeville, MS
Sand Mtn, AL
Georgia Station, GA
Sumatra, FL
Results for Dry Deposition:
NH4 for CASTNET Sites
Percent (%)
Fractional Bias & Error: NH4 Dry Dep.
200
160
120
80
40
0
-40
-80
-120
-160
-200
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
Coffeeville, MS
Sand Mtn, AL
Georgia Station, GA
Sumatra, FL
Results for Dry Deposition:
HNO3 for CASTNET Sites
Results for Wet Deposition:
NO3 for NADP Sites
Percent (%)
Fractional Bias & Error: NO3 Wet Deposition
100
80
60
40
20
0
-20
-40
-60
-80
-100
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
Baldwin Co., AL
Mobile Co., AL
Quincy, FL
Sumatra, FL
Results for Wet Deposition:
NH4 for NADP Sites
Percent (%)
Fractional Bias & Error: NH4 Wet Deposition
100
80
60
40
20
0
-20
-40
-60
-80
-100
MFB-CMAQ
MFE-CMAQ
MFB-MADRID
MFE-MADRID
MFB-APT
MFE-APT
Baldwin Co., AL
Mobile Co., AL
Quincy, FL
Sumatra, FL
Dry Deposition Summary




For nitrate and ammonium dry deposition, all models show consistent gross
underestimation
For HNO3 dry deposition, all models show consistent overestimation, with
MADRID and APT showing more overestimation than CMAQ
With HNO3 higher than NO3 (simulated and observed), net result is that all models
overestimate dry deposition of nitrates
Dry deposition estimates complicated by potential differences in meteorology used
for data vs. model
Wet Deposition Summary


Models do better in simulating wet deposition and are consistent in
underestimating wet deposition at nearby sites
Larger differences seen between models: effects of plume-in-grid
treatment for APT?
Summary and Key Findings



Results are mixed: none of the models stand out as better performing
Greatest contributor to nitrogen deposition is dry deposition of HNO3,
followed by wet deposition of nitrate (all forms)
Simulated net wet deposition of nitrogen is lower than observed while
simulated net dry deposition is higher, so total loading of nitrogen in
domain may be adequately simulated
Summary and Key Findings


Dry deposition monitoring not available in Escambia
watershed, so performance may not be representative
Deposition output from all three models was used in water
quality modeling assessment