Developing the STELLA Model for a DSS for Mitigation Strategies for Transportation Infrastructure: Testing the Model Silvana V. Croope A working paper submitted to the University of Delaware University Transportation Center (UD‐UTC) January 29, 2010
1
DISCLAIMER: The contents of this working paper reflect the views of the authors, who are responsible for the facts and the accuracy of the information presented herein. This document is disseminated under the sponsorship of the Department of Transportation University Transportation Centers Program, in the interest of information exchange. The U.S. Government assumes no liability for the contents or use thereof. 2
Table of Contents Table of Contents ................................................................................................................ 3
List of Figures ...................................................................................................................... 4
List of Tables ....................................................................................................................... 5
Introduction ........................................................................................................................ 6
Background ..................................................................................................................... 6
Objective of this Working Paper ................................................................................... 10
Testing the Model ............................................................................................................. 10
Setting up the Testing ............................................................................................... 11
Interpreting the Results ............................................................................................ 14
Conclusion from Sensitivity Analysis ................................................................................ 15
References ........................................................................................................................ 16
3
List of Figures Figure 1 CIR‐DSS System Dynamics Diagram ...................................................................... 7
Figure 2 Setting up Sensitivity Analysis to Test the Model in STELLA .............................. 11
Figure 3 Defining the Graph for the Sensitivity Analysis of the Model ............................ 13
Figure 4 Result from Running the Sensitivity Analysis of the Model ................................ 14
Figure 5 Running the Model Without Sensitivity Analysis On .......................................... 15
4
List of Tables Table 1 GIS Analysis Results for Seaford Transportation Infrastructure ............................ 8
Table 2 HAZUS‐MH MR3 Analysis Results for Seaford Transportation Infrastructure ....... 8
5
Introduction Background The research for the PhD dissertation titled “Managing Critical Civil Infrastructure Systems for Disaster Resilience: A Challenge” includes developing a model, referred to as the Critical Infrastructure Resilience Decision Support System (CIR‐DSS). This working paper documents the testing of the model. The overall objective of this research is to develop a Decision Support System to improve the resilience of critical infrastructure. This involves the exploration of the potential impacts of natural disasters on infrastructure operation and management. This includes understanding the nature of operations and management, the data and tools to support decision making and an analysis of the consequences of failure or degraded operations and performance. This also includes the use of existing computational systems to develop a geographical context, civil infrastructure systems analysis, asset management systems, and insights into mitigation strategies to development the system. The model uses the concept of resilience to support infrastructure decision making using Systems Dynamics. The framework is shown in Figure 1. 6
Figure 1 CIR‐DSS System Dynamics Diagram To implement this framework, inputs to the system dynamics model are generated using Geographical Information System (GIS) tools and HAZUS‐MH, a tool to assess the impact of hazards. These inputs are used to describe the overall resilience of an infrastructure system. The system is then analyzed using systems dynamics. Graphically oriented modeling software, STELLA, is used to develop the systems dynamics models. The concepts and are illustrated using the June 25, 2006 flood event in Seaford, Delaware. This event also demonstrates how the complex system changes over time. The analysis developed in GIS and HAZUS‐MH is not repeated in STELLA. GIS and HAZUS‐
MH are used to generate maps for vulnerability assessment, and estimate exposure. The Level 2 analysis in HAZUS‐MH organizes and structures relevant data. The results from GIS are shown in Table 1. The maps originally developed are not readable in this table, but are included to demonstrate how to organize results. 7
Table 1 GIS Analysis Results for Seaford Transportation Infrastructure System Results Description GIS (ArcInfo) From the left to the right: • Detours Set Up during the Flood of June 25, 2006 (DelDOT’s paper map), • Seaford Study Area, • Seaford Area Elevation Profile in 3D Image, • Rainfall recorded in Seaford area, • Flooded Area and Bridges impacted in Seaford area, • Seaford Road Network and Detours Analysis, • Location of Damaged Infrastructure in the Seaford Flooded Area. Event information supplied and maps developed can help direct relief supplies to areas of critical need and give out‐of‐state teams knowledge of local terrain and access to places. The results from HAZUS‐MH are shown in Table 2, including maps, tables and reports, helping organize all existing outputs. Table 2 HAZUS‐MH MR3 Analysis Results for Seaford Transportation Infrastructure System Results
HAZUS‐MH MR3 Comments From the left to the right: • Base Map built in HAZUS‐MH for Seaford Area (include limited area around US13), • Seaford Area Annual Losses Map of Depth, • “What if” Levee Protection Scenario, • “What if” Flow Regulation Scenario, • Floodwater Velocity Estimation Scenario, • Damage related to US13 in Sussex County, • (There is an embedded mitigation measure for “warning” not reflected in the images). 8
Continue Table 2 HAZUS‐MH MR3 Organized information for helping interpret results (left to right) • Hazards Identification for Working with HAZUS‐MH, • Hazard Identification and Characterization, • Profile Hazard for Case Study, • Similar Federal Disasters and Damage between 1962 and 2006 in Sussex County, • Federal Disasters Damage Graph ‐ Sussex‐DE, Analyses Results
• Summarized Report for Transportation System Dollar Exposure, • Summarized Report of Estimation for Debris (require 112 truckloads), • Summarized Mitigation Measures based on HAZUS‐MH and History for Transportation Infrastructure – Roads, HAZUS‐MH gives no value for direct economic loss analysis for transportation. Transportation Inventory table is adjusted in excel for modeling. Once GIS and HAZUS‐MH have been used to generate some results important to the overall analysis of the resilience of an infrastructure system, the resilience of the system is better analyzed using systems dynamics. The software system STELLA (from i‐seeTM systems Inc), is used to do this. STELLA provides a graphical user interface to capture dynamic behavior of the system. The systems dynamics model of the transportation infrastructure describes the state of the system before and after the June 25, 2006 flood event, and how the complex system changes over time. Once this model is developed, the results are organized for presentation to the target audience. Systems Thinking skills include trade‐offs of time, and management possibilities, and forecasting factors that are included in the model. The items in italics in Table 2 are important for the model in STELLA. These items in italics include data used in STELLA and mitigation options according to the FEMA STAPLEE criteria for being a feasible mitigation measure. The mitigation options include enhancing the resilience of the system as opposed to a regular rebuilding or repair of the infrastructure system segments according to its original design. The Highway 9
inventory in HAZUS‐MH is not in a proper format to be an input in STELLA. This data exported to EXCEL is used in the modeling and simulation process imported into STELLA. Each named column in EXCEL must match the elements in the model in STELLA. Also, to simplify the demonstration of the model, a sample was identified ‐‐ US13. The data related to US13 was obtained by comparing the Highway inventory from HAZUS‐MH, and the road data from DataMIL clipping it to fit the study region in HAZUS‐
MH and then highlighting the HAZUS‐MH segment links to identify their given identification code. This process used the Select Feature tool, because when opening the inventory table from ArcMap or HAZUS‐MH interface, the available tables did not preserve the information for “name” of US13 segments and the value for “cost”. Also, to highlight US13 in GIS for a qualitative network assessment, the creation of this new layer helps set up the boundary for the analysis later on. The model in STELLA cannot handle these geographical spatial analyses, therefore the need for integrating the results from these different systems. The working paper “Working with HAZUS‐MH” (Croope 2009) describes in more detail how the results were obtained. The working paper “Developing the STELLA Model for a DSS for Mitigation Strategies for Transportation Infrastructure: Introduction to STELLA” (Croope 2010) provides background on how complex systems are represented in STELLA. The working paper “Developing the STELLA Model for a DSS for Mitigation Strategies for Transportation Infrastructure: Building the Model in STELLA” documents the model development process (Croope 2010). Objective of this Working Paper The purpose of this working paper is to document the testing of the model developed in STELLA. Testing the Model involves doing a sensitivity analysis to understand the model strengths and outcomes. The conclusions of the tests are summarized in a conclusion. Testing the Model To explore how the model reacts to changes in input variables and parameters, a sensitivity analysis was undertaken. STELLA provides some functionality to assist with this process (isee systems 2004). The sensitivity analysis included in STELLA is for one time analysis only. This means that if the sensitivity analysis is conducted using different variables than those originally chosen, the current analysis will change. Frequent changes of parameters for different sensitivity analysis of a big model uses up the computer memory and ends up generating results with errors. The way to deal with this problem is to close all open software in the 10
computer and restart it. This brings back the STELLA software to its normal performance. Setting up the Testing The sensitivity analysis opted for in this testing is “damaged IS”. Go to the “run menu” and choose “Sensi Specs” (double click). Choose the variable “damaged IS” from the allowable list of variables in the Sensitivity Specs dialog box. Maintain the “# of Runs” as 3 times, the default value. Use “Incremental” for “Variation Type”. Define the range of values starting with 50,000 and ending with 500,000 (value of damage for the case study). Push the “Set” button to confirm distribution values in the list under “Run# Value”; which will change the value of damage to test the model. Observe that the box for Sensitivity analysis is “On”. Figure 2 shows the dialog box for Sensitivity Specs. Figure 2 Setting up Sensitivity Analysis to Test the Model in STELLA Selecting different types of graphs (“Graph Type”) determines the number of variable that can be displayed (isee systems 2004): • Time Series or Bar Graph: up to 5 variables , • Scatter Plot: 2 variables , and • Comparative Graph: only 1 variable. 11
Click on the “graph button”, or the Table button depending on the output desired for the Sensitivity Analysis. Choosing the graph button, it opens another dialog box for the selection of the variables that will be shown in the graph. Choose the “Bar” option and selected variables: • recovery_NPV, • recovery_NPV_2, • recovery_NPV_3, • mitigation _NPV, and • mitigation_NPV_2. Because the “Bar Graph” limits the displayed variables to 5, the variable “mitigation_NVP_3” was not included. The “recovery_NPV” and “mitigation_NPV” variables relate to the Net Present Value of projects to fix the damaged infrastructure when the probability for events are 4% (or 4 events in 100 years). The “recovery_NPV_2” and “mitigation_NPV_2” variables has the event probability value changed to 1%, and the “recovery_NPV_3” is considering an 8% event probability value. Different probabilities and different infrastructure fix strategy (recovery or mitigation), allows for capturing changes on benefits and on the overall project cost. It helps finding under witch probability is probably best to chose a recovery strategy rather than a mitigation strategy. Define the Title for the graph – Sensitivity Analysis of Damaged Infrastructure for Recovery and Mitigation NPV. If one would like to change the scale of the variables, just click on one variable and click on the double arrow to the right of the variable and change the scale in the new space that opens in the current dialog box. There were no changes of scale for the variables. The box for “Mark for export” was checked, and finally > OK. Figure 3 shows the dialog box for “Define Graph”. 12
Figure 3 Defining the Graph for the Sensitivity Analysis of the Model After clicking OK the graph dialog box itself opens. Click on the magnifier glass “icon” to open the graph and maintain it in the computer screen. It opens the graph box. Click on S‐Run from the “run menu” to view the sensitivity analysis results. Figure 4 shows the bar graph with the chosen variables after running the model. 13
Figure 4 Result from Running the Sensitivity Analysis of the Model Running the model allows for the damage value to change the benefits and recovery and mitigation NPV values for each different probability scenario. The bars sizes change when compared to the running the model without sensitivity analysis shown in Figure 5, which is reflected on the project NPV values on the “y‐axis”. Variations on the bar size for recovery strategy variables happen according to the different probabilities, while mitigation strategy variables reach a certain NPV value and does not change with different probability values. “The bigger the damage to the infrastructure, the bigger the infrastructure maintenance project NPV cost”. Interpreting the Results To better understand what the sensitivity analysis did; go back to the beginning of the set up for the sensitivity analysis and un‐check the Sensitivity Analysis Button (turn off the Sensitivity Analysis). Run it. The results now show the values as calculated in the model prior to running the sensitivity analysis. Error! Reference source not found. shows original values of the model before running the sensitivity analysis. 14
Figure 5 Running the Model Without Sensitivity Analysis On It is possible to see that “recovery_NPV_3” (probability 8%) reaches the top (project NPV cost) either way, with or without sensitivity analysis on or off (changing infrastructure damage value). All the other bars show different results, suggesting that the initial assumption of the benefits of mitigation independent of the frequency of events is something that needs a little bit more of interpretation. Mitigation strategies after being determined and the its value estimated, even though tend to be the same across different event probabilities, it relates to the value of infrastructure damage. The “mitigation_NPV” when compared to the “recovery_NPV” and the different event probabilities plays a major role for determining which strategy (recovery or mitigation) a group of stakeholders should consider. It draws a threshold to which mitigation may be more beneficial than recovery. Conclusion from Sensitivity Analysis Mitigation for the case study is advantageous when the observed frequency of flood is above 4%. If damage value is bigger, mitigation most likely is advantageous when chances are of 7% or more of a 100‐year storm in a 100 year period. In reality the historical frequency of disaster events for the study area, not only for similar flood types, is greater than 4%. If considering just the similar types of events, it is important to recognize that the recorded events used in the simulation do not cover the period of 100 years, so there is time for more similar disasters to take place. In more precise terms, the period analyzed covers about 45 years, which is not even 50% of the 100‐year analysis period for flooding occurrences. This means the historical trend is definitely 15
important, and places with a high frequency of hazardous events are well‐advised to adopt mitigation from an economic perspective rather than locations with a small frequency of hazardous events. However, the expected impact and damage to infrastructure may suggest the strategy to be used raising the threshold value to when mitigation should be considered the best option. Nevertheless the trade‐off then becomes safety and security with less loss of lives versus the most beneficial use of money. Other social factors (e.g. death compensations) with financial impact can definitely make the difference and support pro mitigation. Current model results also shows it is important to analyze case‐by‐case individual disasters taking advantage of historical records to have a more accurate analysis result and account for historical impact and frequency. Also it is important to carefully choose and measure potential benefits of recovery and mitigation projects. References Croope, Silvana V. 2010. University of Delaware library institutional repository: Developing the STELLA model for a DSS for mitigation strategies for transportation infrastructure: Building the model in STELLA. University of Delaware . ———. 2010. University of Delaware library institutional repository: Developing the STELLA model for a DSS for mitigation strategies for transportation infrastructure: Introduction to STELLA. University of Delaware . ———. 2009. University of Delaware library institutional repository: Working with HAZUS‐MH. University of Delaware, 2009 (accessed 09/01/2009). isee systems, Inc. 2004. An introduction to systems thinking ‐ STELLA software isee systems, www.iseesystems.com (accessed 02/07/2008). ———. 2004. An introduction to systems thinking ‐ STELLA software ‐ help. Sensitivity analysis ‐ graph pad dialog operations, STELLA software help. isee systems, www.iseesystems.com (accessed 12/07/2008). 16