The Pacific Northwest
Water Quality Data Exchange
Oregon Department of Environmental Quality
Washington Department of Ecology
Idaho Department of Environmental Quality
Alaska Department of Environmental Conservation
Environmental Protection Agency, Region X
Pacific Northwest
Water Quality Data Exchange
Challenge Grant Objectives

Provide access to a comprehensive collection of
Water Quality data for the Pacific Northwest

Apply Exchange Network principles

Enable participation from a wide range of data
sources

Design data exchange flow to support partner
needs and eventual upload to EPA STORET
Pacific Northwest
Water Quality Data Exchange
Collaboration is Key to Success
Active Participants

Environmental Protection Agency Region 10

Nestucca-Neskowin Watersheds Council (Oregon)

Northwest Indian Fisheries Commission

State of Alaska Department of Environmental Conservation

State of Idaho Department of Environmental Quality

State of Idaho Soil Conservation Commission

State of Oregon Department of Environmental Quality

State of Washington Department of Ecology

University of Idaho - Water Resources Research Institute

Windsor Solutions, Inc.
Pacific Northwest
Water Quality Data Exchange
Collaboration is Key to Success
Affiliates

Coeur d’Alene Tribe

Confederated Tribes of the Siletz Indians

Idaho National Engineering and Environmental Laboratories

King County (Washington) Department of Natural Resources

Long Tom Watershed Council (Oregon)

National Park Service

Nez Perce Tribe

Pacific States Marine Fisheries Commission – StreamNet

Rogue Valley Council of Governments (Oregon)

South Coast Watersheds Council (Oregon)

State of Oregon Watershed Enhancement Board

Surfrider Foundation

Tanana Chiefs Council
Pacific Northwest
Water Quality Data Exchange
Collaboration is Key to Success
Related, Collaborative Project Contacts

Environmental Data Standards Council – Environmental
Sampling, Analysis, and Results Standard

National Water Quality Monitoring Council – Water Quality
Data Elements

USGS -NWIS / EPA –STORET Data Integration

Environmental Protection Agency – OEI / CDX

Environmental Protection Agency – OWOW / STORET
Pacific Northwest
Water Quality Data Exchange
Vision for the Exchange Data Flow
Data Access
Application
Washington
Data
Source
Idaho
Network
Node
.xml
.xml
Network
Node
Data
Source
INTERNET
.xml
Oregon
Data
Source
Alaska
.xml
Network
Node
Network
Node
Data
Source
.xml
Host
Database
Data
Catalog
EPA
CDX
STORET
Data
Warehouse
Pacific Northwest
Water Quality Data Exchange
Components of the Exchange Flow
DET
FCD
- Data Exchange Template
- data elements
- relationships and conditions
- derive XML schema
TPA
Pacific Northwest
Water Quality Data Exchange
Data Exchange Template
Pacific Northwest
Water Quality Data Exchange
XML Schema
Pacific Northwest
Water Quality Data Exchange
Components of the Exchange Flow
DET
FCD
- Flow Configuration Document
- method definition, parameters
- query flow: filtering, wildcards,
paging, ordering, errors
- load flow: process sequence, security
- Data Exchange Template
- data elements
- relationships and conditions
- derive XML schema
TPA
Pacific Northwest
Water Quality Data Exchange
Flow Configuration Document
Pacific Northwest
Water Quality Data Exchange
Components of the Exchange Flow
DET
FCD
- Flow Configuration Document
- method definition, parameters
- query flow: filtering, wildcards,
paging, ordering, errors
- load flow: process sequence, security
- Data Exchange Template
- data elements
- relationships and conditions
- derive XML schema
TPA
- Trading Partner Agreement
- roles and responsibilities
- data stewardship
- administration
Pacific Northwest
Water Quality Data Exchange
Trading Partner Agreement
Pacific Northwest
Water Quality Data Exchange
Accessing the Exchange
http://deq12.deq.state.or.us/pnwwqx/
Pacific Northwest
Water Quality Data Exchange
First step: query the data sources
1. Select Data
Sources
2. Go to Map
Pacific Northwest
Water Quality Data Exchange
First step: query the data sources
3. Select
a Region
Pacific Northwest
Water Quality Data Exchange
First step: query the data sources
4. Coordinates
Returned from map
5. Choose More
Parameters
6. Search
initiates realtime Query.
Pacific Northwest
Water Quality Data Exchange
Second step: browse the data and the details
7. Click on any field in
a row to see the details
Pacific Northwest
Water Quality Data Exchange
Second step: browse the data and the details
Pacific Northwest
Water Quality Data Exchange
Second step: browse the data and the details
8. Download initiates
off-line Solicit method
Pacific Northwest
Water Quality Data Exchange
Third step: download the data
9. Email used to return
Download Status page
link to user
10. Solicit is processed
later, maybe overnight
Pacific Northwest
Water Quality Data Exchange
Third step: download the data
11. All processed files
are zipped together with
a Readme file.
Pacific Northwest
Water Quality Data Exchange
Third step: download the data
12. Each file comes in a
common, tilde-delimited format
with a standardized header.
Pacific Northwest
Water Quality Data Exchange
Third step: download the data
13. These files readily
parse into spreadsheet
or database formats.
Pacific Northwest
Water Quality Data Exchange
Next Steps


Engage additional Exchange partners

Municipalities, other agencies

Watershed councils, volunteer groups
Pilot ambient WQ data flow with EPASTORET
Pacific Northwest
Water Quality Data Exchange
Challenges

Challenges:




Inventing a regional-scale model of the NEIEN
Communications with EPA; code lists
Allow for a wider range of (non-node) data sources
Long term funding for administration
Pacific Northwest
Water Quality Data Exchange
Lessons Learned
Here is your heads-up from our experience:

You are going to learn. (You will walk through a cloud)




New Technology
New Acronyms
New Ways of Doing Business
You will understand better what Data Quality means.



Everyone you let in the door will be able to see your data in its
current state.
Acceptable data o your organization may not be acceptable data
to your trading partners.
Old out-of-date data is seen by others as valid new data.
Pacific Northwest
Water Quality Data Exchange
Lessons Learned cont.

You will appreciate Data Standards and other
standards.





If your organization does not have some form of formal data
standards, develop them now.
Name things what they are. Later in mapping data elements to a
schema, this is gold.
Do not make your standards too hard to use or understand.
Avoid complex abbreviation schemes to ease readability and
understanding. Use names for elements like Person_Last_NM
instead of PLastName, Address_ID not AID, or Last_Update_DT
not LUpDay.
Collaborate with related data standards and network projects.
Pacific Northwest
Water Quality Data Exchange
Lessons Learned cont.

You will redevelop a new meaning for change control






Things change as you progress.
Things change rapidly at first and more rapidly later.
Many things may need to be changed at the same time.
Your critical path will become confused if you are not actively
managing it.
What appears to be an insignificant request may require a
significant amount of effort.
You will learn that this is not just a simple thing that
anyone can do.



This type of project requires dedicated staff to manage project.
The learning curve on these projects is not quick.
Ensure broad user participation.
Pacific Northwest
Water Quality Data Exchange
Lessons Learned cont.

You will see the importance of the 3 flow
components. (The DET and FCD are your
roadmap to success.)
–
–
–

Data Exchange Template - DET/Schema
Flow Configuration Document - FCD
Trading Partner Agreement - TPA
Use existing work if you can find it. (Borrowing
beats building any day).
–
–
Need to reconcile published schemas
Reuse available XML schemas (FRS, Beaches and others
will help in the future)
Pacific Northwest
Water Quality Data Exchange
Lessons Learned - Summary
We have learned:
 that the goals we set were achievable,
 that we built more then a flow or software,
 we built a functioning team and a deeper
understanding of our business,
 and we built an access point to data so that
others can use it to study and help us as a result
of that research, to provide a more
environmentally sound collection of states and a
cleaner, safer future.
Pacific Northwest
Water Quality Data Exchange
Thank you!
Questions?????
Contact Miles Neale for details
[email protected]
360-407-6592
Pacific Northwest
Water Quality Data Exchange