Ambient Monitoring Program
PM 2.5 Data
Lean 6 Sigma
Air Director’s Meeting
May 2015
1
History
 PM2.5 is collected from 23 sites across Georgia
– Each site has an FRMs - Integrated filter-based samplers
– 2 Continuous FEMS for QA and 15 Continuous (non-FEM samplers)
 Due to weather conditions, personnel and equipment issues - not have enough data
(2011-2013) to demonstrate that we were meeting the 2012 standard. Therefore,
EPA could not make a determination of the air quality of Georgia -- Alabama and
South Carolina were affected as well for some areas
 As a result, we were asked to submit the 2014 data by February 27, 2015. This is a
33% reduction in time to submit the data
 Therefore, this project needed to address both quality and timeliness
2
Pilot Project
 It was decided to use Lean Six Sigma to create a common vocabulary and
set of tools along with a team approach to improve this process in a very
short period of time
 Lean reduces waste and bottlenecks to focus on the customer needs
 Six Sigma reduces variation so that each time, you can be certain of your
product
3
Brunswick Data Completeness by month
January 2011- August 2014
120%
100%
80%
75%
60%
50%
40%
20%
0%
4
Charter
 Reduce time needed to provide 2014 Ambient Monitoring Data to the EPA
by February 27, 2015 (a reduction from 3 months to 2 months, or 33%).
 Increase quality at each site to at least 95% per quarter.
 Soft Benefits of this effort include:
– Improved data quality
– Better teamwork and cooperation
– More satisfied employees and customers
5
Getting Started
 Created a flow chart of the process
 Identified five high level steps
 Further discussion revealed that the impact of the quality is multiplicative and reducing
the quality in one area would have detrimental effect on the total
 Therefore, at least 95% quality is needed to ensure for each of the three groups handling
filters (Lab, Field, Data validation)
6
Areas for Focus
Bottleneck Identificaiton
Looking at longest expected time, first
five bars will give you 80% of benefit in
improvement
• Reducing time for Lab export and
holding
• Sample pick up - look at 2x per
week pickup
• Reduce backlog in QA and Data
Validation
• Reduce Second QA Review
40
35
30
25
20
15
10
5
0
lab export
lab
sample Second
Lab
QA
OPs1
Filter to Sample
(44%)
holding picked up
QA
Weighing Backlog Backlog site (2%) loaded
time
(9%)
review
(8%)
(8%)
(7%)
(2%)
(11%)
(8%)
7
Teams formed
 Due to a large number of participants, we were able to form four subteams plus a
steering team to manage the Lean Six Sigma process
– Quality / data completeness
– Lab procedures
– Data validation procedures (QA review)
– Benchmarking (comparison with other states on how they handle PM2.5)
8
Data Completeness Team
 Focused on the reason for voids each year at each site
 Compared site voids
 Recommended Contingencies
– Check samplers remotely each day
– Supervisor verify operator’s installation, collection, shipping techniques
– Hand deliver samples
– Install continuous monitor for “problem sites”
– Investigate differences between sites
– Change shipping measures/procedures at problem sites
9
Lab Team
 Looked at reasons for high number of voids
 Identified ways to reduce the time to process filters while maintaining quality to increase
customer (Ambient Program) satisfaction
 Recommendations taken:
– Send AMP filter weights 2x per month (reduce process time by 2 weeks)
– Weigh filters with very small pinholes (increase valid samples by up to 10 per month)
– Standardize comment language to streamline data validation
 Delineate responsibility for samples that are questionable
– What is AMP’s call? What is Lab’s call?
– Update SOPs
10
Quality Assurance
 “Independent Quality Assurance Verification” Required by EPA
 Involves checking data for at least 20% of PM 2.5 integrated samples
– Amount of time taken and errors discovered were metrics
 Time at Start of Project:
– 2 weeks to process each month of data
– Process involved considerable manual manipulation of data in Excel (non-valueadded work)
– Some process steps may be redundant
11
Improvement Completed
 Excel automation macros of current data validation process
– Reduces time, likely more consistency and accuracy
– Likely less corrections needed by QA Verifier
– Easier to train new people
 Some process redundancies removed
– Documentation of what is done at each step
 QA Verifier only checks summary reports, no longer duplicates data validation process
 Estimated process time now 5 days (one week) per month of data
– Reduction from 10 days to 5 days
– Saves ~ 3 months of work for the year
12
Recommendations
 Reduce transcription errors
– Considering electronic field data sheets, new database
 Consider further analysis / improvement of QA Unit data validation process
– Additional redundancies among Lab, Data Reviewer, QA Unit?
– Focus on identifying and correcting errors that most impact quality
13
What have we done so far?
 Using Macros to eliminate manual work
 Bypassing outdated software that had issues
 Lab providing weights in two batches each month and much faster reporting of
weights
 Lab no longer voiding filters with very small pinholes
 Lab no longer voiding filters for field activities
 Standardized comment language on data entry
 Some process redundancies removed
 Implemented contingencies for 2015 winter weather
 Certified 2014 data on February 3 (24 days early)
 Developing contingencies for issues resulting in data loss - ongoing
14
Effect of LSS
January 2014 - January 2015 Total Voids
2014 storm
TS
DA
180
BJ
BI
160
BE
BD
140
BB
BA
120
AZ
AV
100
AT
AR
80
AQ
60
AO
Very small
pinholes
AN
AM
40
AL
LSS
AJ
20
AH
AG
0
January
February
March
April
May
June
July
August
September
October
November
December
January
AF
AC
15
Benefits of Project
 Communication!!!
 Automation of work allows time to focus on issues rather than tedious tasks
 Streamlined process
 Focus on preventative measures rather than reactionary measures
 Staff available for other projects
 More complete data
– Since October 2014 only one site was less than 75% of data completeness for the
month
– Most sites are meeting the 94% completeness established by the subteam
 Improvements made in PM2.5 process have been implemented on other pollutants as well
16
Questions?
?
DeAnna Oser
Ambient Program Manager
Georgia Environmental Protection – Air Protection Branch
[email protected]
(404) 363-7004
17