Broad Agency Announcement World Modelers HR001117S0017 March 27, 2017 Defense Advanced Research Projects Agency Information Innovation Office 675 North Randolph Street Arlington, VA 22203-2114 Table of Contents I. Funding Opportunity Description .................................................................................................... 5 A. Introduction .................................................................................................................................. 5 B. Technical Objectives .................................................................................................................... 6 C. The World Modelers Analytic Process: An Example .................................................................. 7 D. Relationship to Machine Reading Technology ........................................................................... 11 E. Relationship to Human-Machine Interaction Technology.......................................................... 12 F. Program Structure ....................................................................................................................... 12 G. Technical Areas (TAs)................................................................................................................ 13 H. Technology Evaluation ............................................................................................................... 18 I. Schedule...................................................................................................................................... 19 J. Deliverables ................................................................................................................................ 20 K. Government-furnished Property/Equipment/Information .......................................................... 20 L. Intellectual Property ................................................................................................................... 20 M. References .................................................................................................................................. 20 II. Award Information......................................................................................................................... 28 A. Awards ........................................................................................................................................ 28 B. Fundamental Research ................................................................................................................ 29 III. Eligibility Information ................................................................................................................... 30 A. Eligible Applicants ..................................................................................................................... 30 B. Organizational Conflicts of Interest............................................................................................ 31 C. Cost Sharing/Matching ............................................................................................................... 32 D. Other Eligibility Requirements ................................................................................................... 32 IV. Application and Submission Information ................................................................................... 33 A. Address to Request Application Package ................................................................................... 33 B. Content and Form of Application Submission ........................................................................... 33 C. Submission Date and Time ......................................................................................................... 43 D. Funding Restrictions ................................................................................................................... 43 E. Other Submission Requirements ................................................................................................ 43 V. Application Review Information ................................................................................................... 46 A. Evaluation Criteria ...................................................................................................................... 46 B. Review and Selection Process .................................................................................................... 46 VI. A. Award Administration Information ............................................................................................ 48 Selection Notices ........................................................................................................................ 48 HR001117S0017 WORLD MODELERS 2 B. Administrative and National Policy Requirements .................................................................... 48 C. Reporting .................................................................................................................................... 51 VII. Agency Contacts ......................................................................................................................... 53 VIII. Other Information ....................................................................................................................... 54 A. Frequently Asked Questions (FAQs).......................................................................................... 54 B. Proposers Day ............................................................................................................................. 54 C. Submission Checklist ................................................................................................................. 54 HR001117S0017 WORLD MODELERS 3 PART I: OVERVIEW INFORMATION Federal Agency Name: Defense Advanced Research Projects Agency (DARPA), Information Innovation Office (I2O) Funding Opportunity Title: World Modelers Announcement Type: Initial Announcement Funding Opportunity Number: HR001117S0017 Catalog of Federal Domestic Assistance Numbers (CFDA): 12.910 Research and Technology Development Dates o Posting Date: March 27, 2017 o Proposers Day: February 3, 2017 o Proposal Due Date: May 11, 2017, 12:00 noon (ET) o BAA Closing Date: May 11, 2017, 12:00 noon (ET) Anticipated Individual Awards: Multiple awards anticipated Technical Areas (TAs): There are five TAs under this BAA. Proposers should note that a proposal may address more than one TA. Please also note that there is technical proposal page limitation described in Section IV.B.a. Types of Instruments that May be Awarded: Procurement contracts, grants, cooperative agreements or Other Transactions Agency Contacts o Technical POC: Dr. Paul Cohen, Program Manager, DARPA/I2O o BAA Email: [email protected] o BAA Mailing Address: DARPA/I2O ATTN: HR001117S0017 675 North Randolph Street Arlington, VA 22203-2114 o I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities HR001117S0017 WORLD MODELERS 4 PART II: FULL TEXT OF ANNOUNCEMENT I. Funding Opportunity Description DARPA is soliciting innovative research proposals in the area of causal modeling, forecasting, and analysis techniques. Proposed research should investigate innovative approaches that enable revolutionary advances in science, mathematics, or technology. Specifically excluded is research that primarily results in evolutionary improvements to the existing state of practice. This Broad Agency Announcement (BAA) is being issued, and any resultant selection will be made, using procedures under Federal Acquisition Regulation (FAR) 6.102(d)(2) and 35.016. Any negotiations and/or awards will use procedures under FAR 15.4 (or 32 CFR § 200.203 for grants and cooperative agreements). Proposals received as a result of this BAA shall be evaluated in accordance with evaluation criteria specified herein through a scientific review process. DARPA BAAs are posted on the Federal Business Opportunities (FBO) website (https://www.fbo.gov/) and the Grants.gov website (http://www.grants.gov/). The following information is for those wishing to respond to this BAA. A. Introduction The purpose of the World Modelers program is to develop technology that will enable analysts to rapidly build models to analyze questions relevant to national and global security. Questions for analysis will typically be framed at subnational scales and look one to five years into the future, although the factors that influence outcomes of interest might operate on larger spatial and temporal scales. Food insecurity is a growing problem in many parts of the world where societies struggle to feed growing populations. Food insecurity has both humanitarian challenges and regional security ramifications, as food shortages can result in migration/displacement and conflict between peoples. Food insecurity will be the initial use case for the World Modelers program, and World Modelers technologies will enable researchers to efficiently and rigorously perform analytic tasks such as “Analyze food insecurity in each district of South Sudan two years into the future.” World Modelers technologies will be applied to additional (and increasingly varied) use cases as they mature through a sequence of program phases. The subnational focus of World Modelers reflects the changing nature of conflict and security, which, increasingly, plays out in cities and districts. Analysis at larger scales produces generalities (e.g., Sub-Saharan Africa is likely to go hungry) rather than targeted recommendations (e.g., increasing security in South Sudan will allow internally displaced farmers to return to their fields). As to temporal scale, World Modelers is committed to evaluating the accuracy of its forecasts, so use cases that forecast one to five years into the future are preferred. HR001117S0017 WORLD MODELERS 5 B. Technical Objectives World Modelers aims to develop technologies to facilitate analyses that are comprehensive, targeted, causal, quantitative, probabilistic, and timely enough to recommend specific actions that could avert crises. We expand on these attributes in the context of food insecurity in South Sudan: Comprehensive analysis: Many factors influence food insecurity, generally. A comprehensive analysis considers all relevant factors, in contrast with, as examples, an economic analysis that considers only food prices, or a security analysis that considers whether farmers can safely work their farms. Targeted analysis: Targeted analyses will parameterize models to reflect the facts of each particular situation. For example, to model food insecurity in South Sudan we would parameterize general crop-yield factors, such as weather and soil fertility, and also economic factors that (relative to food security) might initially appear of secondary importance. Such a targeted analysis would be more likely to reveal that when South Sudan devalued its currency in 2015, food prices (and food insecurity) increased despite favorable weather and good yields. Causal analysis graphs: World Modelers will develop formal representations of causal influences (such as weather and currency devaluation) called causal analysis graphs. Analysis graphs will be the skeletons on which machines and humans build both qualitative and quantitative analyses. Causal analyses contrast with those based on correlations between indicators and outcomes, such as correlations between food prices and the incidence of food riots (e.g., Lagi, Bertram and Bar-Yam, 2011). Quantitative models: Analyses can be comprehensive, targeted and causal, yet qualitative. Indeed, much analysis is qualitative (e.g., NIC2030, OECD/FAO2016, US-UK2015). One goal of World Modelers is to marry qualitative analysis with quantitative models whenever possible. For example, instead of saying that crop yields in South Sudan are expected to be “good,” it would be preferable to run crop yield models (e.g., Jones et al., 2003) to get quantitative estimates given local conditions. Probabilistic analysis: Uncertainty in an analysis of food insecurity in South Sudan arises because the true values of causal factors are unknown. For example, South Sudan depends on rainfall for its crops, but El Niño conditions, which recently favored South Sudan, are hard to predict. One goal of World Modelers is to derive probability distributions over outcomes. A related goal is to apply uncertainty-reducing methods (e.g., ensemble methods) wherever possible. World Modelers aims to develop automated techniques whereby machines perform these analyses with guidance from human analysts. Timely analysis: Humans already develop comprehensive, targeted, quantitative and probabilistic analyses, but it is largely a manual process and, as such, is costly and timeconsuming. An inspiring recent example of marrying qualitative and quantitative analysis comes from the Commonwealth Scientific and Industrial Research Organisation (CSIRO) in Australia (Hatfield-Dodds et al., 2015). The Australian National Outlook team built a workflow of quantitative models of climate, land use, electricity use, the global economy, and other systems, to explore qualitative scenarios about the future of water, energy and food in the context of climate change. Much as World Modelers intends, CSIRO integrated quantitative models HR001117S0017 WORLD MODELERS 6 involving natural resource use, macroeconomics, and energy concerns at the national and global scales. The quantitative models CSIRO used were well established with some in existence for decades. Their integration represents an advancement in the multi-disciplinary modeling of these issues, which were typically modeled separately in the past. This impressive effort took a team of domain experts and computer scientists more than two years to integrate nine quantitative models to evaluate twenty qualitative scenarios. As evidenced by CSIRO’s work, it is possible to do the type of analysis envisioned within World Modelers, but it is extremely time-consuming. It is also difficult to run more than a few scenarios when relying upon humans to parameterize and set them up. It is possible to merge qualitative and quantitative analyses in a single computational framework, but machine assistance might facilitate setting up these complicated analyses. The aim of World Modelers is to automate much of the work done by humans so as to conduct such analyses in roughly one month. C. The World Modelers Analytic Process: An Example A human-machine analysis of food security in South Sudan might proceed in seven steps as follows: o Step 1: Generate or retrieve a generic, qualitative analysis graph for food security. o Step 2: Modify the analysis graph for the specific analyses of South Sudan. o Step 3: Integrate expert, quantitative models into the analysis graph, where available. o Step 4: Parameterize quantitative models and the analysis graph. o Step 5: Configure scenarios and run analyses, producing quantitative results for factors of interest (e.g., food prices, calorie intake). o Step 6: Produce an “uncertainty report” that documents sources of uncertainty, run uncertainty-reduction procedures and sensitivity analyses. o Step 7: Identify possible actions to affect factors of interest (e.g., peacekeepers at markets). This seven-step procedure is meant to be illustrative, not prescriptive. Proposers might design different analytical workflows, but the seven steps discussed here will serve to illustrate the capabilities desired of World Modelers technology. Some of these steps would be revisited, iteratively, to improve the analysis, as depicted in the graphic that follows: Figure 1 illustrates the World Modelers analytic process in terms of the analysis graph, which lies at the center of a World Modelers analysis. Analysis graphs have a qualitative causal HR001117S0017 WORLD MODELERS 7 structure but, step by step, become more quantitative until they are, in effect, coupled simulations of specific geophysical, biophysical, and socio-economic processes. Figures 1A - 1F depict the sequential elaboration of the analysis graph. World Modelers intends that much of the work of analysis should be done by machines. (The remainder of this page has been left intentionally blank to accommodate Figure 1, which starts on page 9.) HR001117S0017 WORLD MODELERS 8 shortage/surplus shortage/surplus A utilization B population ethiopia currency devaluation utilization population sudan kenya access access uganda profits profits markets urban population security transport refugee population markets imports refugee population prices prices availability availability aid urban population security transport stock internal displacement production cross-border displacement aid imports stock internal displacement production cross-border displacement sudan shocks growth conditions floods disease climate rural population soil irrigation climate ethiopia currency devaluation drought rainfall inputs shortage/surplus C floods disease A generic model of food insecurity ENSO growth conditions uganda conflict drought rainfall shocks labor utilization conflict rural population soil irrigation inputs nodes in the generic model considered relevant by the machine/analyst new nodes added to the generic model by reading about Southern Sudan ENSO “devaluation of local currency by 84% ” D shortage/surplus ethiopia currency devaluation population sudan utilization access = f(market,price,production) kenya access labor population sudan kenya access uganda uganda profits profits markets prices imports stock refugee population markets individuals” cross-border displacement aid sudan imports growth conditions floods disease drought rainfall conflict soil inputs NOAA Global Tropical SST Forecasts E shortage/surplus ethiopia currency devaluation utilization urban population security prices refugee population climate imports stock internal displacement Sources for numeric parameters (in green) include text, tables, and functions that combine outputs of model components ethiopia utilization population sudan kenya access uganda profits markets security transport aid imports stock urban population Shortages are due primarily to prices FAO which and OECD AgLink-Cosimo access, is due to prices Model ofat Agriculture and security markets.Markets We could increase security at markets. We could also increase internal productionfood aid to internal refugee displacement populations. refugee population cross-border displacement sudan sudan shocks disease drought rainfall shocks growth conditions uganda labor DSSAT AgMIP crop yield model irrigation ENSO conflict rural population soil NOAA Global Tropical SST Forecasts climate conflict shortage/surplus currency devaluation availability cross-border displacement inputs F FAO and OECD AgLink-Cosimo Model of Agriculture Markets production cross-border displacement rural population soil ENSO JFMAMJJASOND JFMAMJJASOND aid labor DSSAT AgMIP crop yield model irrigation kenya uganda availability drought JFMAMJJASOND JFMAMJJASOND profits growth conditions NOAA Global Tropical SST Forecasts sudan transport floods rainfall population access markets shocks920,000 disease Blue boxes are extant software models, e.g., rainfall, crop, and market models. internal displacement production uganda rural population irrigation ENSO climate labor DSSAT AgMIP crop yield model stock “estimated at tons” sudan shocks uganda refugee population FAO and OECD AgLink-Cosimo “about 647000 Model of Agriculture Markets availability internal displacement production urban population security transport prices FAO and OECD AgLink-Cosimo Model of Agriculture Markets availability aid urban population security transport inputs Analysts can “drive” scenarios by having the machine sample parameter values from distributions, in blue (e.g. rainfall) growth conditions uganda disease floods rainfall drought labor DSSAT AgMIP crop yield model soil irrigation NOAA Global Tropical SST Forecasts climate ENSO conflict rural population inputs Green nodes are influences or effects. The green box represents one of perhaps many plans to influence food shortages Figure 1: An example of the sequential construction of a World Modelers analysis graph, including appropriate contributions from humans and machines at each step. HR001117S0017 WORLD MODELERS 9 For example, a generic, qualitative analysis graph for food insecurity (Step 1, Fig. 1A) might be built by machine reading (e.g., Allen et al. 2015, Hahn-Powell et al., 2016). The generic, qualitative analysis graph for food insecurity produced in this fashion will provide a good framework for later analysis steps, but it will not produce a targeted analysis of the kind described above in Section B. Technical Objectives. The initial qualitative analysis graph must be specialized to fit South Sudan (Step 2, Fig. 1B). Here, too, machine reading might help. For example, in the Food and Agriculture Organization country report on South Sudan a machine could read, “Prices of cereals started to soar in late 2015 on account of local currency devaluation, a general economic downturn and widespread insecurity.” And Al Jazeera reports, “According to the UN, neighboring Uganda hosts the largest number of South Sudanese refugees. Ethiopia, Kenya, Sudan, Democratic Republic of Congo and Central African Republic also have received tens of thousands of people fleeing.” One technical challenge is to have machines map texts like these to grounded entities in a causal model (shown as darker-colored nodes in Fig. 1B). Another is to have machines forage for this kind of information. Step 3 begins to transform this targeted but still qualitative analysis graph into a quantitative one. Many quantitative models have been developed by experts in fields such as climate, weather, hydrology, cropping, commodities analysis, and so on. The appropriate models must be incorporated into the analysis graph. Figure 1C suggests that nodes in the qualitative model, such as “growth conditions”, might somehow pass information to quantitative models, such as the Decision Support System for Agrotechnology Transfer (DSSAT) cropping model, which in turn pass quantitative information back to the node labeled “production.” Step 3 involves figuring out how to use quantitative models to estimate quantitative parameters in the analysis graph. Once it has been decided how quantitative models will be used in analyses, it is necessary to set the parameters of these models to represent local conditions (Step 4, Fig. 1D). Machines might help with this task, adept as they are at gathering information. For example, rainfall data (including forecasts) is available online. Remote sensing data and crowd-sourced information might also be used. One technical challenge is the semantic mismatch between what’s available online and what the quantitative models need. For example, DSSAT needs solar radiation data while online sources might instead provide day-length information, in which case a conversion is required. And while day-length might be quite easily transformed to solar radiation, the problem of finding suitable proxies for model parameters can require inference and domain knowledge. This problem is important because a good solution would make it possible to apply expert, quantitative models -many of which have been under development for decades and are superbly accurate -- much more widely than heretofore. A model like DSSAT could apply to any farm in the world, limited only by our ability to set its parameters to reflect local conditions. By setting up quantitative models at points in the analysis graph one begins to transform a qualitative analysis to a quantitative analysis. However, not every node in the analysis graph will take output from a quantitative model; for example, the node labeled “access” in Figure 1D is not associated with a quantitative model. What is the degree of access to food in the East Equatoria districts in South Sudan? Where quantitative models do not exist, it might be necessary for humans to write functions that gather up quantitative information in the analysis HR001117S0017 WORLD MODELERS 10 graph (e.g., production information from DSSAT) and combine it to produce quantitative estimates of parameters elsewhere in the graph. World Modelers will develop technology to make this specification of combining functions as easy as possible for humans. The next step in analysis is to configure some scenarios (Step 5, Fig. 1E). This is usually done by setting exogenous “driver” parameters, many of which will be time series. For example, one might use historical rainfall data to generate a synthetic series to represent two years of rainfall in the East Equatoria districts in South Sudan. Human analysts will generally be responsible for saying which scenarios to analyze, but machines might help set up scenarios. One important role for machines is to check the semantic consistency of analysis graphs. For example, a human who wants to consider drought conditions might set a rainfall distribution that is different from the one generated by a quantitative weather model, so the analysis has two conflicting sources of information. How the conflict is resolved may be the analyst's decision, but detecting such conflicts in very complicated analysis graphs is an appropriate task for machines. As scenarios run, numerical results “bubble up” through the analysis graph. Because scenarios might involve sampling from distributions, each run of a scenario will give slightly different results. Thus, an outcome variable such as shortage/surplus will have variance (Figure 1E). Step 6 is to produce an "uncertainty report" for each analysis. Uncertainty reports are required because without them, one cannot know whether to trust analyses or how to improve them. Uncertainty reports differ from standard measures such as classification accuracy and R2 that merely summarize uncertainty but do not analyze its causes. The precise contents of uncertainty reports are for proposers to specify, but these reports should identify where uncertainty enters an analysis and how it propagates through the analysis graph, and should document uncertaintyreducing operations. Although analysts are the principal users of uncertainty reports, one can imagine that machines might modify analysis graphs more or less automatically based on machine-readable uncertainty reports. Steps 2 - 6 might be repeated iteratively until the analyst is satisfied with the analysis. The uncertainty report is an important source of evidence of the quality of the analysis, and generally will guide the iterative refinement of the analysis graph. Step 7, identifying actions, is currently done by analysts who say what they think is, or will be, happening, and decision-makers who decide what to do. World Modelers anticipates a more integrated process in which decision-makers use analysis graphs to explore interventions. Here, too, machines can help. Humans are not good at considering combinations of actions or multiple consequences of actions, but machines are. One can imagine machines examining the analysis graph to find efficient combinations of actions that are expected to achieve beneficial effects (Fig. 1F). D. Relationship to Machine Reading Technology Machine reading is an application of natural language understanding (NLU) in artificial intelligence. Machine reading aims to endow machines with reading comprehension capabilities. World Modelers will extend machine reading technologies to deal with heterogeneity in subject (e.g., climate, food, economics, etc.); in model types (e.g., general equilibrium models, agentbased models, etc.); in use cases (e.g., scientific understanding, risk analysis, etc.); and in spatial and temporal scales. Note that the ontologies currently in use to facilitate model-building in HR001117S0017 WORLD MODELERS 11 machine reading will not be available in all World Modelers domains. Finally, World Modelers will require technology for integrating extant, quantitative models into qualitative analysis graphs, a problem that does not arise in other domains. E. Relationship to Human-Machine Interaction Technology Modern research in human-machine interaction aims to enable symmetric communication between people and computers in which machines are not merely receivers of instructions but collaborators, able to harness a full range of natural modes including language, gesture and facial or other expressions. A major challenge arises from the fact that language is ambiguous and depends in part on context, which can augment language and improve the specification of complex ideas. For World Modelers, human-machine interaction technology can enhance the communication of complex models between the human analyst and the machine. F. Program Structure A government evaluation team will develop World Modelers use cases for development and testing in each phase. The development plan for the World Modelers program is to build and refine complete World Modelers systems continuously, as opposed to waiting to integrate component technologies until they mature. There will not be an integration contractor, and it is not necessary for anyone to propose to build a complete World Modelers system. Instead, compatible groups of performers will be assembled after source selection to ensure that one or more integrated World Modelers systems is built. (Proposals should budget for integrative activities, such as travel, hackathons, and group code bases.) Phase 1: The use case for Phase 1 will address food insecurity (Barrett, 2010; FAO Food Insecurity). The causal relationships between food security and other kinds of national and global security are debated (e.g., Bellamare, 2015; Buhaug et al., 2015; Natalini, Wynne Jones and Bravo, 2015; Wischnath and Buhaug, 2014a, 2014b, de Waal, 2015; Schleussner et al. 2016) but there is little doubt that food insecurity is a consequence, if not a cause, of conflict. The general form of the Phase 1 test question is: For any subnational location (e.g., a district in South Sudan) generate food shortage scenarios two years out. Consider a comprehensive set of causes. Integrate current and historical data and human expert analysis. Model the scenarios quantitatively and probabilistically. Make forecasts, explain sources of uncertainty, and identify and explain specific potential solutions. Update models, forecasts and probabilities on receipt of new data. Various organizations already do this (e.g., USAID and FEWS-NET; Food and Agriculture Organization and FAO-GIEWS). Others address longer-term, food insecurity (e.g., OEDC-FAO, 2016; US-UK2015; von Grebmer et al. 2015), effects of climate on food insecurity (e.g., Lobell et al. 2008; Rosenzweig et al., 2014; Conway et al., 2015; White et al., 2011; Phalkey et al., 2015; IPCC5; ), trade and food insecurity (e.g., Marchand et al., 2016; D'Orico et al., 2014, Rutten, Shutes and Maijerink, 2013), food security and insurance (e.g., Lloyds2015) and many related topics. The fact that food insecurity is so well-analyzed makes it a good first domain for the World Modelers program, because many resources have been developed by and for analysts, and the program can compare World Modelers analyses with current practice. HR001117S0017 WORLD MODELERS 12 The evaluation team will provide analysis problems in the area of food security to aid development during Phase 1. The test at the end of Phase 1 will be similar to these development problems. For example, the test might ask for analysis of food insecurity in a new country or under different conditions. Phase 2: While development during Phase 2 will continue to focus on food insecurity, the test will involve a different domain, albeit one that shares big chunks of analysis graphs with food insecurity. For example, the test might involve analyzing migration or another phenomenon that shares many of the same causal links as food insecurity. Phase 3: The test domain in Phase 3 will be quite different from others encountered in the program and will only be revealed to World Modelers performers at a time as close to the evaluation as possible. In this way, the Phase 3 test will exercise the capability of World Modelers technologies “… to build comprehensive, targeted, causal, probabilistic models to analyze questions relevant to national and global security, rapidly enough to recommend specific actions that could avert crises.” The test problem for Phase 3 will be authentic in the sense that real analysts of national and global security questions care about the problem. Proposers are welcome to suggest use cases. However, to help ensure that World Modelers technology is general, these problems must be made available to all participants and all participants must solve the test problems set by the evaluation team. These requirements will help to avoid situations in which participants work in isolation and develop problem-specific technologies. G. Technical Areas (TAs) The five technical areas for the World Modelers solicitation are: o TA1: Build Qualitative Models from Online Sources. Machine-assisted construction of comprehensive, targeted, causal, qualitative, analysis graphs. "Which factors should be in a particular analysis and how are they related?" o TA2: Workflow Compiler (for Integration of Quantitative Models). Machine-assisted marriage of qualitative with quantitative analysis. "Which quantitative models should be in the analysis and how do they connect to each other and run in a common computational environment?" o TA3: Parameterize Models. Machine-assisted foraging for data from multiple sources, machine-assisted translation of available data into proxies for required parameters. "Where are the data to parameterize quantitative models, and what are the mappings between available data and required parameters?" o TA4: From Scenarios to Actions. Interfaces for humans to specify scenarios, machineassisted recommendation and testing of interventions. "What will be the effects of particular actions under particular conditions?" o TA5: Uncertainty Reports. Machine-assisted, thorough analysis of sources of uncertainty and efforts to reduce it. "Can the analysis be trusted, and what can be done to improve it?" HR001117S0017 WORLD MODELERS 13 In relation to the seven-step World Modelers food insecurity example presented earlier, TAs enable steps as follows: TA Description Supports Steps 1 Build Qualitative Models from Online Sources 1, 2, 4 2 Workflow Compiler 3 3 Parameterize Models 4 4 From Scenarios to Actions 5, 7 5 Uncertainty Reports 6 Technology Machine reading for causal fragments, grounding in ontologies, assembly algorithms Scientific workflow technology, grid computing, distributed simulation, collaborative human-machine dialog capability Automated machine table reading, remote sensing feeds, crowdsourcing, polling Mechanistic or machine processing technology for finding "pressure points" in causal networks, AI planning technology for multi-step plans Collaborative technology for humanmachine examination of models could help. However, human-machine uncertainty analysis for very complicated models requires new science Technical Area 1: Build Qualitative Models from Online Sources The objective of this Technical Area is to build qualitative, causal analysis graphs semiautomatically. The analysis graph is the central data structure in a World Modelers system. It has many functions: o It represents the causal and other relationships among domain entities; o It represents a common understanding between human and machine about what is being analyzed, and, as it is refined, it provides a record of how analysis proceeds; o It has a qualitative causal structure that is transformed into a quantitative simulation by incorporating quantitative models; o It represents one or more complicated causal narratives; o It is the basis for explanations of the results of analyses; o It is the basis for reasoning about interventions; o It is the basis for constructing uncertainty reports and for modifying analyses to reduce uncertainty; o It might contain provenance and other meta-data about information, and it might be "connected" to online information sources; and HR001117S0017 WORLD MODELERS 14 o It is machine-readable and, ideally, re-usable: One can imagine libraries of analysis graphs available to “copy-edit.” Several analysis communities have recognized the value of standardized causal “stories” or “pathways” (e.g., IPCC5; Van Vuren et al. 2014; O'Neill et al., 2014; Valdivia et al., 2015; Dellink, et al. 2015). What World Modelers adds is formal, machine-readable representations -analysis graphs -- of causal pathways. Proposers to TA1 must describe semi-automated techniques that enable analysis graphs to be built largely by machines, with limited involvement from human analysts. Techniques of this nature have been developed in DARPA's Big Mechanism program (Cohen, 2015), but the problem is far from solved. In addition to thorny reading issues (e.g., event coreference) there is the problem of grounding linguistic entities in some sort of domain theory, ontologies, controlled vocabularies and so on (e.g., AGRVOC; CUAHSI2016; Neubert, 2009; Peckham, 2014). In the Big Mechanism program, words and phrases could be grounded in domain ontologies such as GO and UniProt, but ontologies will not be available (or will be less well-developed) for all the domains addressed by World Modelers. Proposals to TA1 should focus on the structure, semantics, construction, grounding, maintenance and uses of analysis graphs; and particularly the complementary roles of machines and humans in constructing, maintaining and reasoning with analysis graphs. Because World Modelers technologies are intended to support analysis in any domain, proposers cannot assume that domain ontologies will be available. Proposers should explain how their technologies will work when domain ontologies are absent or incomplete. World Modelers test problems will introduce novel domains, but even in a single domain, targeted analysis will introduce novel factors; for example, food insecurity in most countries does not include currency devaluation as a factor, so when this concept is encountered for the first time (e.g., in the analysis of South Sudan), it must somehow be formalized and added to ontologies or other resources. Technical Area 2: Workflow Compiler (for Integration of Quantitative Models) World Modelers will gather a collection of open-source quantitative models that are relevant to selected use cases, such as food insecurity. Proposers are welcome to contribute models, as well. This Technical Area will entail integration of these quantitative models into analysis graphs. Generally, quantitative models are designed to answer one question or a few related questions; for example, the DSSAT cropping model predicts crop yields. Thus, one quantitative model will inform only a few nodes in an analysis graph, and several quantitative models will generally be needed for analyses (e.g., Fig. 1C includes models for forecasting El Niño conditions, for crop yield forecasts, and for forecasting crop prices). TA2 proposals should address problems encountered when integrating diverse quantitative models – models developed in different languages, with different variable names and ontologies, running on different clocks, the outputs of which may be upscaled or downscaled – within a common software environment (CSDMS2016; Glass et al., 2011; Deelman et al., 2015; Chen and Deelman, 2012; Elliot et al., 2014; Madduri et al., 2015; Epperly et al. 2011; Hill et al. 2004; OpenMI; Dokoohaki et al., 2014; Porter et al., 2014). HR001117S0017 WORLD MODELERS 15 Other technical challenges of interest in TA2 are reminiscent of automatic programming or planning issues: If a quantitative model is a function or a planning operator, how should it be used to achieve the goals of analysis; that is, where does it fit in an analysis plan or workflow (Bergmann and Gil, 2014; Garijo et al., 2014)? To illustrate, note that the unit of analysis for the DSSAT cropping model is the field, not an entire district, county or nation. How then might DSSAT estimate, say, maize yields for the East Equatoria districts in South Sudan? While it might seem obvious to humans that DSSAT should be run repeatedly to estimate yields for individual farms and the results aggregated over a region, this isn't necessarily obvious to machines. Other challenges of interest to TA2 concern machines doing what in humans would take inference and judgment. For example, parameter values might be estimated by running quantitative models, or might simply be read from (or extrapolated from) online feeds. Which to do? For the purposes of a particular analysis, is it best to run a quantitative model or simply grab a number from the Food and Agriculture Organization? Some proposals to TA2 might suggest methods by which machines can frame and answer questions like these. Technical Area 3: Parameterize Models Quantitative models have three kinds of parameters: endogenous parameters that describe the phenomena being modeled, initial conditions, and exogenous “driver” variables that may be the outputs of other models. Endogenous parameters remain constant during a run of a model, initial conditions describe the context surrounding a modeled process at the beginning of a run, and exogenous variables describe context that changes during a run. For example, in the DSSAT crop model (Jones et al., 2003), the parameters of a particular wheat cultivar are endogenous, while the composition of the soil in which the virtual plant will grow is an initial condition, and daily rainfall is an exogenous variable. Another way to classify parameters is to note that some are “given” and some are “derived” by executing quantitative models. In general, endogenous parameters are given, whereas exogenous parameters are derived; but there are exceptions, such as using historical (i.e., given) time series data to drive a model. TA3 is about having machines automate some of the task of setting given parameters. One technical challenge is the semantic mismatch between what is available online and what the quantitative models need. For example, DSSAT needs solar radiation data while online sources might instead provide day-length information. Some sort of translation is required. And while day-length might be quite easily transformed to solar radiation, the problem of finding suitable proxies for model parameters can require inference and domain knowledge. A related problem is syntactic mismatches between variable names in model merging or model intercomparison projects (e.g., Elliott et al. 2015; Porter et al., 2014; Janssen et al. 2009). These problems are well worth solving, because a good solution would make it possible to apply expert, quantitative models -- many of which have been under development for decades and are superbly accurate -much more widely than heretofore. Not every node in the analysis graph will take output from a quantitative model; for example, the node labeled “access” in Figure 1D is not associated with a quantitative model. What is the degree of access to food in the East Equatoria districts in South Sudan? Where quantitative models do not exist, it might be necessary for humans to write functions that gather up HR001117S0017 WORLD MODELERS 16 quantitative information in the analysis graph (e.g., production information from DSSAT) and combine it to produce quantitative estimates. World Modelers will develop technology to make this specification of combining functions as easy as possible for humans. Proposals to TA3 must describe automated techniques to convert data into values of parameters of quantitative models, aligned with specific analyses. Ideally, the techniques would enable machines to access and incorporate a wide variety of data feeds such as weather stations, remote sensing, field trials, crowdsourcing, and so on (e.g., Xie et al., 2015; Ines et al. 2013; Lobell et al., 2015; Sheffield et al., 2014; Morozov, Relikoff and Chubak, 2006; OGC2011; Sloate et al. 2015). For example, a TA3 problem might be to find all the farms in a district, all the crops grown on those farms, and all the growing conditions, and convert these into parameter settings for cropping models to forecast yields, farm by farm. This would be an absurd amount of work for humans, the question to be answered by proposers to TA3 is, “how can machines help?” Proposals to TA3 must give strongly technical accounts of how machines will find data and convert it into forms that correspond to model parameters. Technical Area 4: From Scenarios to Actions Scenarios are defined by initial conditions and “given” exogenous parameters. Initial conditions might be “as they are now” (e.g., the soil types on farms in South Sudan, today) or they might be specified by analysts to explore alternatives (e.g., imagine that half of the internally displaced farmers in South Sudan return to cultivate their land). Analysts might also give exogenous parameters, rather than letting them be derived from models. For example, a drought scenario might be specified by a given time series of rainfall. One reason for setting up analyses is to explore the effects of actions. Typically, this is done by specifying proximal effects of actions as model parameter values, then running models to calculate distal or downstream effects. For example, the proximal effect of increasing security in the countryside might be the return of 50% of displaced farmers to their lands, the distal effects are discovered by running the models in the analysis given this proximal effect. TA4 is about humans and machines sharing the work of specifying scenarios and analyzing the effects of actions. TA4 has many aspects or sub-problems, including the design of humanmachine interfaces, the configuration of scenarios, the summarization of results, and humanmachine planning to achieve specific goals. Proposers to TA4 should describe how the work of specifying scenarios and analyzing actions will be shared between machines and humans. Proposers should identify specific technical goals for human-machine interactions (e.g., identifying sets of exogenous variables that have the most impact on output variables, or efficiently searching spaces of possible actions) and explain how these goals will be achieved. Technical Area 5: Uncertainty Reports The purpose of uncertainty reports is to give analysts and their clients reasons to believe (or disbelieve) analyses. An uncertainty report might point out, for example, that an analysis of food security in South Sudan is based on the assumption that the security situation remains the same, but, as all sides are negotiating at the time of the analysis, this assumption is questionable. An uncertainty report might include the results of sensitivity analysis. It might describe results under best-case and worst-case conditions. As general guidance, uncertainty reports should: HR001117S0017 WORLD MODELERS 17 o Document sources of uncertainty; o Document which model parameters are affected by uncertainty; o Characterize the sensitivity of output variables to uncertainty; and o Document efforts to reduce uncertainty. The idea of uncertainty reports can be extended to various kinds of machine assistance in dealing with uncertainty; for example, machines might automatically build ensemble models, or might automatically learn lightweight approximations of heavyweight models to speed up sensitivity analysis, and so on. To the best of our knowledge, the construction of uncertainty reports is a new problem, so proposers to TA5 have considerable latitude in what they propose. For example, to some, an uncertainty report might track sources of uncertainty through analyses; to others, an uncertainty report serves primarily to guide human-machine revision of an analysis graph. Some proposers might focus on the technology of sensitivity analysis, which is then scaled up to analyses containing multiple quantitative models. Others might design algorithms for machines to reduce uncertainty at key points in the analysis graph by searching for ground truth data. Because the subject of uncertainty reports is so broad, proposers should say precisely what they mean by uncertainty reports, and should describe specific technologies that contribute to their conception of uncertainty reports. Proposers are encouraged to give strongly technical descriptions of these technologies. H. Technology Evaluation As previously noted, DARPA will use a Government evaluation team to develop test problems. Good test problems will have these attributes: they will be relevant to national and global security; span climate, biophysical, economic, and social systems; span spatial and temporal scales; and involve feedback loops. Where social phenomena are modeled, they will be “close” to their measurable causal antecedents (e.g., economic and physical security as antecedents of decision to migrate) so we can “back off” to forecasting antecedents if the social models are inadequate. Mechanistic, ideally quantitative, models will be either developed and/or included for integration from available sources. A mechanistic model is explanatory and provides relationships among complicated systems in which interactions have important causal effects. Data must be available for parameter estimation, for “forecasting the past,” and for testing accuracy. The problem should afford many opportunities to forecast, and, thus, test the accuracy of forecasts. The technology evaluation criteria for integrated World Modelers solutions to test problems include these: o Utility and Accuracy: Can real analysts use the technology, and do they want to keep on using it? Are the models’ forecasts accurate enough to be useful, i.e., to support recommendations? o Plausibility: Are analyses plausible, do they agree with concurrent, human-only, expert analyses (e.g., do we get roughly the same forecasts for South Sudan as experts, and if not, are they plausible enough to warrant analysts’ attention)? HR001117S0017 WORLD MODELERS 18 o Diagnosticity and Robustness: Do users understand the sources of uncertainty in their analyses? Can they make analyses more robust with the help of the machine? o Latency: How long does it take to set up and run analyses? Note that the accuracy of forecasts is grouped with utility, above, because models can be useful even when they are not very accurate. Nevertheless, the World Modelers program will work in domains that afford opportunities to continuously track the accuracy of forecasts. For example, migration is an ongoing process, so at any moment one can ask for a forecast of the number of people who move from one location to another during an interval of time, and at the end of the interval one can test the accuracy of the forecast. Migration is a good domain because it affords thousands of such tests, in contrast with analysis questions about one-off events (e.g., tsunami). Accuracy is important for a very practical reason: World Modelers aims to marry qualitative with quantitative analysis based on the thesis that qualitative analyses can be improved by including quantitative models. The difference between qualitative and quantitative analysis is precision: Where a qualitative analysis might forecast increasing numbers of refugees, a quantitative analysis is expected to be precise about the increase. But precision is worse than useless -- it is misleading -- unless it is accurate (Pindyk, 2015; Stanton, Ackerman and Kartha, 2009; Millner and McDermott, 2016). World Modelers does not insist that all forecasts be accurate, or that all forecasts be quantitative, but, rather, that analysts should know when they can trust quantitative forecasts. Without this assurance, World Modelers technology will not be adopted. This is why it is important to track the accuracy of World Modelers forecasts. I. Schedule Subject to the availability of funding, World Modelers is planned for 48 months (4 years), and is structured as two 18-month phases, and one 12-month phase. It is envisioned that there will be no formal down select. However, DARPA will regularly assess individual performer efforts and their overall progress toward World Modelers goals, and projects that do not make adequate progress will be discontinued. For budgeting purposes, a proposer should use September 2017 as an estimated start date. Principal Investigator (PI) meetings will be held approximately every 6 months, in various locations in the continental United States. The locations for meetings and other events will be specified by the Government. The goals of the PI meetings will be to: (a) review methods, architectures and general progress; (b) review the accomplishments of each performer; (c) demonstrate prototypes and other accomplishments; and (d) coordinate interactions between performers. For budget planning purposes, PI meetings may alternate between continental United States locations on the east coast and west coast by phase. Additionally, performers should anticipate one to two trips to the Washington DC metro area per phase for meetings with the Government. The Government envisions making periodic site visits to the performers’ facilities. Regular teleconference meetings are encouraged to enhance communications with the Government team. Should important issues arise between program reviews, the Government team will be available to support informal interim technical interchange meetings. HR001117S0017 WORLD MODELERS 19 Academic performers are highly encouraged to involve graduate and post-graduate students (in addition to faculty and research staff) in PI meetings, site visits, and teleconferences. J. Deliverables All performers shall be required to provide the following deliverables: Technical papers and reports. Initial reports shall be submitted within one month after the program kickoff meeting and after each principal investigator meeting (taking place twice per year, roughly six months apart). Monthly progress reports. A monthly progress report describing progress made, resources expended, and any issues requiring the attention of the Government team shall be provided by the 10th day of each month under contract. Monthly financial reporting in a provided format will be required for any non-fixed price awards. Any fixed price efforts will not be required to submit financial reports. Final report. The final report shall concisely summarize the effort conducted. Formal representations of results. Additionally, software should be made available via link on the DARPA Open Catalog (https://opencatalog.darpa.mil), along with links to publications and/or publication abstracts generated through this program K. Government-furnished Property/Equipment/Information The Government will provide a Phase 1 test question and use case to all selected performers. L. Intellectual Property The program will emphasize creating and leveraging open source technology and architecture. Intellectual property rights asserted by proposers are strongly encouraged to be aligned with open source regimes. It is desired that all noncommercial software (including source code), software documentation, hardware designs and documentation, and technical data generated by the program be provided as deliverables to the Government, with a minimum of Government Purpose Rights (GPR), as lesser rights may adversely impact the lifecycle costs of affected items, components, or processes. See Section VI.B.1 for more details on intellectual property. M. References AGROVOC: AGROVOC Multilingual agricultural thesaurus. http://aims.fao.org/agrovoc Allen, J., de Beaumont, W., Galescu, L., & Teng, C. M. (2015). "Complex Event Extraction using DRUM." ACL-IJCNLP 2015, 1. HR001117S0017 WORLD MODELERS 20 Jang Won Bae, Euihyun Paik, Kiho Kim, Karandeep Singh, and Mazhar Sajjad. "Combining Microsimulation and Agent-Based Model for Micro-Level Population Dynamics." Proceedings of the International Conference on Computational Science (ICCS), 2016. Barrett, Christopher B. "Measuring food insecurity." Science 327.5967 (2010): 825-828. Batchelor, Roy. "The IMF and OECD versus consensus forecasts." City University Business School, London, August (2000). Bellemare, Marc F. "Rising food prices, food price volatility, and social unrest." American Journal of Agricultural Economics 97.1 (2015): 1-21. Bergmann, Ralph, and Yolanda Gil. "Similarity assessment and efficient retrieval of semantic workflows." Information Systems 40 (2014): 115-127. Buhaug, Halvard, J. Nordkvelle, T. Bernauer, T. Böhmelt, M. Brzoska, J. W. Busby, A. Ciccone et al. "One effect to rule them all? A comment on climate and conflict." Climatic Change 127, no. 3-4 (2014): 391-397. Chen, Weiwei, and Ewa Deelman. "Workflowsim: A toolkit for simulating scientific workflows in distributed environments." In E-Science (e-Science), 2012 IEEE 8th International Conference on, pp. 1-8. IEEE, 2012. Cohen, Paul R. "DARPA's Big Mechanism program." Physical Biology 12, no. 4 (2015): 045008. Conway, D., van Garderen, E.A., Deryng, D., Dorling, S., Krueger, T., Landman, W., Lankford, B., Lebek, K., Osborn, T., Ringler, C. and Thurlow, J., 2015. "Climate and southern Africa's water-energy-food nexus." Nature Climate Change, 5(9), pp.837-846. CSDMS2016: "The Community Surface Dynamics Modeling System (CSDMS)." https://csdms.colorado.edu/wiki/Help:Tools_CSDMS_Handbook CUAHSI2016: "Master Controlled Vocabulary Registry. Consortium of Universities for the Advancement of Hydrologic Science (CUAHSI)", 2016. Available from http://his.cuahsi.org/mastercvreg/cv11.aspx Deelman, Ewa, Karan Vahi, Gideon Juve, Mats Rynge, Scott Callaghan, Philip J. Maechling, Rajiv Mayani et al. "Pegasus, a workflow management system for science automation." Future Generation Computer Systems 46 (2015): 17-35. Dellink, Rob, Jean Chateau, Elisa Lanzi, and Bertrand Magné. "Long-term economic growth projections in the Shared Socioeconomic Pathways."Global Environmental Change (2015). de Waal, Alex. "Armed Conflict and the Challenge of Hunger: Is an End in Sight?" Chapter 3 of von Grebmer, 2015. http://dx.doi.org/10.2499/9780896299641 HR001117S0017 WORLD MODELERS 21 D'Odorico, Paolo, Joel A. Carr, Francesco Laio, Luca Ridolfi, and Stefano Vandoni. "Feeding humanity through global food trade." Earth's Future 2, no. 9 (2014): 458-469. Dokoohaki, Hamze, Mahdi Gheysari, Sayed-Farhad Mousavi, Shahrokh Zand-Parsa, Fernando E. Miguez, Sotirios V. Archontoulis, and Gerrit Hoogenboom. "Coupling and testing a new soil water module in DSSAT CERES-Maize model for maize production under semi-arid condition." Agricultural Water Management 163 (2016): 90-99. Elliott, Joshua, David Kelly, James Chryssanthacopoulos, Michael Glotter, Kanika Jhunjhnuwala, Neil Best, Michael Wilde, and Ian Foster. "The parallel system for integrating impact models and sectors (pSIMS)." Environmental Modelling & Software 62 (2014): 509-516. Elliott, Joshua, and Christoph Müller. "The AgMIP GRIDded Crop Modeling Initiative (AgGRID) and the Global Gridded Crop Model Intercomparison (GGCMI)." Handbook of Climate Change and Agroecosystems: The Agricultural Model Intercomparison and Improvement Project (AgMIP) Integrated Crop and Economic Assessments—Joint Publication with American Society of Agronomy, Crop Science Society of America, and Soil Science Society of America (In 2 Parts) 3 (2015): 175. Thomas G. W. Epperly, Gary Kumfert, Tamara Dahlgren, Dietmar Ebner, Jim Leek, Adrian Prantl, Scott Kohn, "High-performance language interoperability for scientific computing through Babel", 2011, International Journal of High-performance Computing Applications (IJHPCA), LLNL-JRNL-465223, DOI: http://dx.doi.org/10.1177/1094342011414036. ESMF2016: "The Earth System Modeling Framework (ESMF)." Available from https://www.earthsystemcog.org/projects/esmf/. Stephen Eubank, Hasan Guclu, V. S. Anil Kumar, Madhav V. Marathe, Aravind Srinivasan, Zoltán Toroczkai, and Nan Wang. "Modelling disease outbreaks in realistic urban social networks." Nature 429, 180-184, 13 May 2004, doi:10.1038/nature02541. FAOSTAT: Food and Agricultural Organization of the United Nations, Statistics Division: http://faostat3.fao.org/browse/Q/*/E FAO Food Outlook Biannual Report on Global Food Markets. http://www.fao.org/3/ai6198e.pdf FAOFoodInsecurity: http://www.fao.org/docrep/013/al936e/al936e00.pdf FAO-GIEWS Country Briefs: http://www.fao.org/giews/countrybrief/index.jsp FEWS-NET: Famine Early Warning System Network. http://www.fews.net/ FEWSManual: "FEWS Technical Manual 2.0." 2012. http://www.fews.net/sites/default/files/uploads/IPC-Manual-2-Interactive.pdf HR001117S0017 WORLD MODELERS 22 Garijo, Daniel, Pinar Alper, Khalid Belhajjame, Oscar Corcho, Yolanda Gil, and Carole Goble. "Common motifs in scientific workflows: An empirical analysis." Future Generation Computer Systems 36 (2014): 338-351. Robert J. Glass, Arlo L. Ames, Theresa J. Brown, S. Louise Maffitt, Walter E. Beyeler, Patrick D. Finley, Thomas W. Moore, John M. Linebarger, Nancy S. Brodsky, Stephen J. Verzi, Alexander V. Outkin, and Aldo A. Zagone. "Complex Adaptive Systems of Systems (CASoS) Engineering: Mapping Aspirations to Problem Solutions." Sandia National Laboratories Report SAND 2011-3354, 2011. Goldstone, Jack A., Robert H. Bates, David L. Epstein, Ted Robert Gurr, Michael B. Lustik, Monty G. Marshall, Jay Ulfelder, and Mark Woodward. "A global model for forecasting political instability." American Journal of Political Science 54, no. 1 (2010): 190-208. von Grebmer, Klaus; Bernstein, Jill; de Waal, Alex; Prasai, Nilam; Yin, Sandra; Yohannes, Yisehac. 2015. 2015 Global hunger index: "Armed conflict and the challenge of hunger." Bonn, Germany; Washington, D.C. and Dublin, Ireland: Welthungerhilfe; International Food Policy Research Institute (IFPRI) and Concern Worldwide. http://dx.doi.org/10.2499/9780896299641 Hahn-Powell, G., Bell, D., Valenzuela-Escárcega, M. A., & Surdeanu, M. (2016). "This before That: Causal Precedence in the Biomedical Domain." arXiv preprint arXiv:1606.08089. Hatfield-Dodds, S., P. D. Adams, T. S. Brinsmead, B. A. Bryan, F. Chiew, J. J. Finnigan, P. Graham et al. "Australian National Outlook 2015: Economic activity, resource use, environmental performance and living standards, 1970-2050." (2015). Hill, C., C. DeLuca, V. Balaji, M. Suarez, and A. da Silva. "Architecture of the Earth System Modeling Framework." Computing in Science and Engineering, Volume 6, Number 1, 2004. doi: http://dx.doi.org/10.1109/MCISE.2004.1255817. ICASA2013: ICASA Master Variable List. http://research.agmip.org/display/dev/ICASA+Master+Variable+List Ines, Amor VM, Narendra N. Das, James W. Hansen, and Eni G. Njoku. "Assimilation of remotely sensed soil moisture and vegetation with a crop simulation model for maize yield prediction." Remote Sensing of Environment 138 (2013): 149-164. http://www.sciencedirect.com/science/article/pii/S0034425713002307 IPCC5: "International Panel on Climate Change Fifth Assessment Report." https://www.ipcc.ch/report/ar5/ Janssen, Sander, Erling Andersen, Ioannis N. Athanasiadis, and Martin K. van Ittersum. "A database for integrated assessment of European agricultural systems." Environmental Science & Policy 12, no. 5 (2009): 573-587. Jones, James W., Gerrit Hoogenboom, Cheryl H. Porter, Ken J. Boote, William D. Batchelor, L. A. Hunt, Paul W. Wilkens, Upendra Singh, Arjan J. Gijsman, and Joe T. Ritchie. "The HR001117S0017 WORLD MODELERS 23 DSSAT cropping system model." European journal of agronomy 18, no. 3 (2003): 235265. Karra, Mahesh, David Canning, and Joshua Wilde. "A Simulation Model of the Effect of Fertility Reduction on Economic Growth in Africa." No. 0315. 2015. Lagi, Marco, Karla Z. Bertrand, and Yaneer Bar-Yam. "The food crises and political instability in North Africa and the Middle East." Available at SSRN 1910031 (2011). Lloyds2015. Global Food Insecurity: "The insurance impacts of acute disruption to global food supply." http://bit.ly/232U8Wj Lobell, David B., David Thau, Christopher Seifert, Eric Engle, and Bertis Little. "A scalable satellite-based crop yield mapper." Remote Sensing of Environment 164 (2015): 324-333 http://www.sciencedirect.com/science/article/pii/S0034425715001637 David B. Lobell, Marshall B. Burke, Claudia Tebaldi, Michael D. Mastrandrea, Walter P. Falcon, Rosamond L. Naylor. 2008. "Prioritizing Climate Change Adaptation Needs for Food Security in 2030." Science.01 Feb 2008: 607-610 Madduri, Ravi, Kyle Chard, Ryan Chard, Lukasz Lacinski, Alex Rodriguez, Dinanath Sulakhe, David Kelly, Utpal Dave, and Ian Foster. "The Globus Galaxies platform: delivering science gateways as a service." Concurrency and Computation: Practice and Experience 27, no. 16 (2015): 4344-4360. Philippe Marchand, Joel A Carr, Jampel Dell’Angelo, Marianela Fader, Jessica A Gephart, Matti Kummu, Nicholas R Magliocca, Miina Porkka, Michael J Puma, Zak Ratajczak, Maria Cristina Rulli, David A Seekell, Samir Suweis, Alessandro Tavoni, Paolo D’Odorico. "Reserves and trade jointly determine exposure to food supply shocks." Environmental Research Letters, 11, 9, 2016. Porkka, Miina, Matti Kummu, Stefan Siebert, and Olli Varis. "From food insufficiency towards trade dependency: a historical analysis of global food availability." PloS one 8, no. 12 (2013): e82714. Davide Natalini, Aled Wynne Jones and Giangiacomo Bravo. "Quantitative Assessment of Political Fragility Indices and Food Prices as Indicators of Food Riots in Countries." Sustainability 2015, 7, 4360-4385; doi:10.3390/su7044360 NetCDF 2016: "The Network Common Data Form (NetCDF). Unidata." Available from http://www.unidata.ucar.edu/software/netcdf/ NIC2030: "Global Trends 2030: Alternative Worlds." National Intelligence Council. https://www.dni.gov/index.php/about/organization/global-trends-2030 Millner, Antony, and Thomas KJ McDermott. "Model confirmation in climate economics." Proceedings of the National Academy of Sciences 113, no. 31 (2016): 86758680. HR001117S0017 WORLD MODELERS 24 Igor Morozov, Brian Reilkoff, and Glenn Chubak. "A generalized web service model for geophysical data processing and modeling". Computers & Geosciences 32 (9): 1403, 2006. doi: http://dx.doi.org/10.1016/j.cageo.2005.12.010 OECD/FAO (2016), OECD-FAO "Agricultural Outlook 2016-2025," OECD Publishing, Paris. DOI: http://dx.doi.org/10.1787/agr_outlook-2016-en OGC2011: “GeoSPARQL: A Geographic Query Language for RDF Data.” Open Geospatial Consortium (OGC), 2011. Available from http://www.opengeospatial.org/standards/geosparql O’Neill, Brian C., Elmar Kriegler, Keywan Riahi, Kristie L. Ebi, Stephane Hallegatte, Timothy R. Carter, Ritu Mathur, and Detlef P. van Vuuren. "A new scenario framework for climate change research: the concept of shared socioeconomic pathways." Climatic Change 122, no. 3 (2014): 387-400. http://link.springer.com/article/10.1007/s10584-0130905-2/fulltext.html OpenMI. The Open Modeling Interface (OpenMI). The OpenMI Association, 2011. http://www.openmi.org/learning-more/OpenMIScopev2.pdf?attredirects=0 Neubert, Joachim. "Bringing the Thesaurus for Economics on to the Web of Linked Data." LDOW 25964 (2009). Peckham, Scott D. "The CSDMS standard names: Cross-domain naming conventions for describing process models, data sets and their associated variables." (2014). http://www.iemss.org/sites/iemss2014/papers/iemss2014_submission_263.pdf Phalkey, Revati K., Clara Aranda-Jan, Sabrina Marx, Bernhard Höfle, and Rainer Sauerborn. "Systematic review of current efforts to quantify the impacts of climate change on undernutrition." Proceedings of the National Academy of Sciences 112, no. 33 (2015): E4522-E4529. Pindyck, Robert S. "The use and misuse of models for climate policy." No. w21097. National Bureau of Economic Research, 2015. Porter, Cheryl H., Chris Villalobos, Dean Holzworth, Roger Nelson, Jeffrey W. White, Ioannis N. Athanasiadis, Sander Janssen et al. "Harmonization and translation of crop modeling data to ensure interoperability." Environmental Modelling & Software 62 (2014): 495508. http://bit.ly/2amTmRU Robinson, Sherman, Daniel Mason-D'Croz, Timothy Sulser, Shahnila Islam, Ricky Robertson, Tingju Zhu, Arthur Gueneau, Gauthier Pitois, and Mark W. Rosegrant. "The international model for policy analysis of agricultural commodities and trade (IMPACT): Model description for version 3." (2015). Rosenzweig, Cynthia, Joshua Elliott, Delphine Deryng, Alex C. Ruane, Christoph Müller, Almut Arneth, Kenneth J. Boote et al. "Assessing agricultural risks of climate change in the 21st century in a global gridded crop model intercomparison." Proceedings of the National Academy of Sciences 111, no. 9 (2014): 3268-3273. HR001117S0017 WORLD MODELERS 25 Rutten, Martine, Lindsay Shutes, and Gerdien Meijerink. "Sit down at the ball game: How trade barriers make the world less food secure." Food Policy 38 (2013): 1-10. Salmeron, Montserrat, José Cavero, Ramón Isla, Cheryl H. Porter, James W. Jones, and Kenneth J. Boote. "Dssat nitrogen cycle simulation of cover crop–maize rotations under irrigated mediterranean conditions." Agronomy Journal106, no. 4 (2014): 1283-1296. Schleussner, Carl-Friedrich, Jonathan F. Donges, Reik V. Donner, and Hans Joachim Schellnhuber. "Armed-conflict risks enhanced by climate-related disasters in ethnically fractionalized countries." Proceedings of the National Academy of Sciences 113, no. 33 (2016): 9216-9221. Justin Sheffield, Eric F. Wood, Nathaniel Chaney, Kaiyu Guan, Sara Sadri, Xing Yuan, Luke Olang, Abou Amani, Abdou Ali, Siegfried Demuth, and Laban Ogallo, 2014: A Drought Monitoring and Forecasting System for Sub-Sahara African Water Resources and Food Security. Bull. Amer. Meteor. Soc., 95, 861–882, doi: 10.1175/BAMS-D-12-00124.1. Sloate, Sam, Vincent Hsiao, Nina Charness, Ethan Lowman, Chris Maxey, Guannan Ren, Nathan Fields, and Leora Morgenstern. "Extracting Protein-Reaction Information from Tables of Unpredictable Format and Content in the Molecular Biology Literature." 2015. https://pdfs.semanticscholar.org/c7ae/b216ccd2feefa9f65a6a3b98fd9d1cd9abdf.pdf US-UK2015. Extreme weather and resilience of the global food system (2015). Final Project Report from the UK-US Taskforce on Extreme Weather and Global Food System Resilience, The Global Food Security programme, UK http://www.foodsecurity.ac.uk/assets/pdfs/extreme-weather-resilience-of-global-foodsystem.pdf Valdivia, Roberto O., John M. Antle, Cynthia Rosenzweig, Alexander C. Ruane, Joost Vervoort, Muhammad Ashfaq, Ibrahima Hathie et al. "Representative agricultural pathways and scenarios for regional integrated assessment of climate change impact, vulnerability and adaptation."Handbook of Climate Change and Agroecosystems (2015): 101-145. Van Vuuren DP, Edmonds J, Thomson A, Riahi K, Kainuma M, Matsui T, Hurtt GC, Lamarque J-F, Meinshausen M, Smith S, Granier C, Rose SK, Hibbard KA (2011) "Representative concentration pathways: an overview." Climatic Change 109:5–31 White, Jeffrey W., Gerrit Hoogenboom, Bruce A. Kimball, and Gerard W. Wall. "Methodologies for simulating impacts of climate change on crop production." Field Crops Research 124, no. 3 (2011): 357-368. Wischnath, G. & Buhaug, H. On climate variability and civil war in Asia. Climatic Change (2014a) 122: 709. doi:10.1007/s10584-013-1004-0 Gerdis Wischnath, Halvard Buhaug. "Rice or riots: On food production and conflict severity". Political Geography Volume 43, Pages 6-15 (2014b) HR001117S0017 WORLD MODELERS 26 Xie, Michael, Neal Jean, Marshall Burke, David Lobell, and Stefano Ermon. "Transfer learning from deep features for remote sensing and poverty mapping." arXiv preprint arXiv:1510.00098 (2015). HR001117S0017 WORLD MODELERS 27 II. Award Information A. Awards Multiple awards are anticipated. The level of funding for individual awards made under this solicitation has not been predetermined and will depend on the quality of the proposals received and the availability of funds. Awards will be made to proposers whose proposals are determined to be the most advantageous and provide the best value to the Government, all factors considered, including the potential contributions of the proposed work, overall funding strategy, and availability of funding. See Section V for further information. The Government reserves the right to: select for negotiation all, some, one, or none of the proposals received in response to this solicitation; make awards without discussions with proposers; conduct discussions with proposers if it is later determined to be necessary; segregate portions of resulting awards into pre-priced options; accept proposals in their entirety or to select only portions of proposals for award; fund proposals in increments and/or with options for continued work at the end of one or more phases; request additional documentation once the award instrument has been determined (e.g., representations and certifications); and remove proposers from award consideration should the parties fail to reach agreement on award terms within a reasonable time or the proposer fails to provide requested additional information in a timely manner. Proposals selected for award negotiation may result in a procurement contract, grant, cooperative agreement, or Other Transaction (OT) depending upon the nature of the work proposed, the required degree of interaction between parties, and other factors. Proposers looking for innovative, commercial-like contractual arrangements are encouraged to consider requesting Other Transactions. To understand the flexibility and options associated with Other Transactions, consult www.darpa.mil/work-with-us/contractmanagement#OtherTransactions. In all cases, the Government contracting officer shall have sole discretion to select award instrument type, regardless of instrument type proposed, and to negotiate all instrument terms and conditions with selectees. DARPA will apply publication or other restrictions, as necessary, if it determines that the research resulting from the proposed effort will present a high likelihood of disclosing performance characteristics of military systems or manufacturing technologies that are unique and critical to defense. Any award resulting from such a determination will include a requirement for DARPA permission before publishing any information or results on the program. For more information on publication restrictions, see the section below on Fundamental Research. HR001117S0017 WORLD MODELERS 28 B. Fundamental Research It is DoD policy that the publication of products of fundamental research will remain unrestricted to the maximum extent possible. National Security Decision Directive (NSDD) 189 defines fundamental research as follows: ‘Fundamental research’ means basic and applied research in science and engineering, the results of which ordinarily are published and shared broadly within the scientific community, as distinguished from proprietary research and from industrial development, design, production, and product utilization, the results of which ordinarily are restricted for proprietary or national security reasons. As of the date of publication of this BAA, the Government expects that program goals as described herein may be met by proposers intending to perform fundamental research and does not anticipate applying publication restrictions of any kind to individual awards for fundamental research that may result from this BAA. Notwithstanding this statement of expectation, the Government is not prohibited from considering and selecting research proposals that, while perhaps not qualifying as fundamental research under the foregoing definition, still meet the BAA criteria for submissions. If proposals are selected for award that offer other than a fundamental research solution, the Government will either work with the proposer to modify the proposed statement of work to bring the research back into line with fundamental research or else the proposer will agree to restrictions in order to receive an award. Proposers should indicate in their proposal whether they believe the scope of the research included in their proposal is fundamental or not. While proposers should clearly explain the intended results of their research, the Government shall have sole discretion to select award instrument type and to negotiate all instrument terms and conditions with selectees. Appropriate clauses will be included in resultant awards for non-fundamental research to prescribe publication requirements and other restrictions, as appropriate. This clause can be found at www.darpa.mil/work-with-us/additional-baa. For certain research projects, it may be possible that although the research being performed by the awardee is restricted research, a subawardee may be conducting fundamental research. In those cases, it is the awardee’s responsibility to explain in their proposal why its subawardee’s effort is fundamental research HR001117S0017 WORLD MODELERS 29 III. Eligibility Information A. Eligible Applicants DARPA welcomes engagement from all responsible sources capable of satisfying the Government's needs, including academia (colleges and universities); businesses (large, small, small disadvantaged, etc.); other organizations (including non-profit); entities (foreign, domestic, and government); FFRDCs; minority institutions; and others. DARPA welcomes engagement from non-traditional sources in addition to current DARPA performers. 1. Federally Funded Research and Development Centers (FFRDCs) and Government Entities FFRDCs FFRDCs are subject to applicable direct competition limitations and cannot propose to this BAA in any capacity unless they meet the following conditions: (1) FFRDCs must clearly demonstrate that the proposed work is not otherwise available from the private sector. (2) FFRDCs must provide a letter on official letterhead from their sponsoring organization citing the specific authority establishing their eligibility to propose to Government solicitations and compete with industry, and their compliance with the associated FFRDC sponsor agreement’s terms and conditions. This information is required for FFRDCs proposing to be awardees or subawardees. Government Entities Government Entities (e.g., Government/National laboratories, military educational institutions, etc.) are subject to applicable direct competition limitations. Government entities must clearly demonstrate that the work is not otherwise available from the private sector and provide written documentation citing the specific statutory authority and contractual authority, if relevant, establishing their ability to propose to Government solicitations. Authority and Eligibility At the present time, DARPA does not consider 15 U.S.C. § 3710a to be sufficient legal authority to show eligibility. While 10 U.S.C.§ 2539b may be the appropriate statutory starting point for some entities, specific supporting regulatory guidance, together with evidence of agency approval, will still be required to fully establish eligibility. DARPA will consider FFRDC and Government entity eligibility submissions on a case-by-case basis; however, the burden to prove eligibility for all team members rests solely with the proposer. 2. Foreign Participation Non-U.S. organizations and/or individuals may participate to the extent that such participants comply with any necessary nondisclosure agreements, security regulations, export control laws, and other governing statutes applicable under the circumstances. HR001117S0017 WORLD MODELERS 30 B. Organizational Conflicts of Interest FAR 9.5 Requirements In accordance with FAR 9.5, proposers are required to identify and disclose all facts relevant to potential OCIs involving the proposer’s organization and any proposed team member (subawardee, consultant). Under this Section, the proposer is responsible for providing this disclosure with each proposal submitted to the BAA. The disclosure must include the proposer’s, and as applicable, proposed team member’s OCI mitigation plan. The OCI mitigation plan must include a description of the actions the proposer has taken, or intends to take, to prevent the existence of conflicting roles that might bias the proposer’s judgment and to prevent the proposer from having unfair competitive advantage. The OCI mitigation plan will specifically discuss the disclosed OCI in the context of each of the OCI limitations outlined in FAR 9.505-1 through FAR 9.505-4. Agency Supplemental OCI Policy In addition, DARPA has a supplemental OCI policy that prohibits contractors/performers from concurrently providing Scientific Engineering Technical Assistance (SETA), Advisory and Assistance Services (A&AS) or similar support services and being a technical performer. Therefore, as part of the FAR 9.5 disclosure requirement above, a proposer must affirm whether the proposer or any proposed team member (subawardee, consultant) is providing SETA, A&AS, or similar support to any DARPA office(s) under: (a) a current award or subaward; or (b) a past award or subaward that ended within one calendar year prior to the proposal’s submission date. If SETA, A&AS, or similar support is being or was provided to any DARPA office(s), the proposal must include: The name of the DARPA office receiving the support; The prime contract number; Identification of proposed team member (subawardee, consultant) providing the support; and An OCI mitigation plan in accordance with FAR 9.5. Government Procedures In accordance with FAR 9.503, 9.504 and 9.506, the Government will evaluate OCI mitigation plans to avoid, neutralize or mitigate potential OCI issues before award and to determine whether it is in the Government’s interest to grant a waiver. The Government will only evaluate OCI mitigation plans for proposals that are determined selectable under the BAA evaluation criteria and funding availability. The Government may require proposers to provide additional information to assist the Government in evaluating the proposer’s OCI mitigation plan. If the Government determines that a proposer failed to fully disclose an OCI; or failed to provide the affirmation of DARPA support as described above; or failed to reasonably provide additional information requested by the Government to assist in evaluating the proposer’s OCI mitigation plan, the Government may reject the proposal and withdraw it from consideration for award. HR001117S0017 WORLD MODELERS 31 C. Cost Sharing/Matching Cost sharing is not required; however, it will be carefully considered where there is an applicable statutory condition relating to the selected funding instrument (e.g., OTs under the authority of 10 U.S.C. § 2371). D. Other Eligibility Requirements 1. Ability to Receive Awards in Multiple Technical Areas - Conflicts of Interest Proposers may submit proposals for up to all five technical areas. The decision as to which proposal to consider for award is at the discretion of the Government; however, there are no conflicts between Technical Areas. 2. Ability to Support Classified Development No classified development is anticipated, as World Modelers is a fundamental research program. This is an unclassified research program. HR001117S0017 WORLD MODELERS 32 IV. Application and Submission Information A. Address to Request Application Package This document contains all information required to submit a response to this solicitation. No additional forms, kits, or other materials are needed except as referenced herein. No request for proposal (RFP) or additional solicitation regarding this opportunity will be issued, nor is additional information available except as provided at the Federal Business Opportunities website (https://www.fbo.gov), the Grants.gov website (http://www.grants.gov/), or referenced herein. B. Content and Form of Application Submission 1. Proposals Proposals consist of Volume 1: Technical and Management Proposal (including mandatory Appendix A and optional Appendix B); Volume 2: Cost Proposal; the Level of Effort Summary by Task Excel™ spreadsheet. All pages shall be formatted for printing on 8-1/2 by 11-inch paper with 1-inch margins, single-line spacing, and a font size not smaller than 12 point. Font sizes of 8 or 10 point may be used for figures, tables, and charts. Document files must be in .pdf, .odx, .doc, .docx, .xls, or .xlsx formats. Submissions must be written in English. Reminder – Each proposal submitted in response to this BAA may address more than one TA. Organizations may submit multiple proposals to any one TA, or they may propose to multiple TAs. Proposals not meeting the format prescribed herein may not be reviewed. a. Volume 1: Technical and Management Proposal The maximum page count for Volume 1 is dependent upon the total number of task areas to which the proposer is proposing. The maximum page count for Volume 1 for a proposer addressing 1 TA is 15 pages, including all figures, tables, and charts. For each additional technical area addressed in the “Technical Plan” section of the proposal, the proposer is allowed one extra page, up to a possible maximum total of 19 pages (if a proposer is addressing all five TAs), including all figures, tables, and charts. To be clear, proposals addressing one TA have a limit of 15 pages for the entire Volume 1: Technical and Management Proposal; proposals addressing two TAs have a limit of 16 pages; proposals addressing three TAs have a limit of 17 pages; proposals addressing four TAs have a limit of 18 pages; and proposals addressing all five TAs have a limit of 19 pages. All of these page limits include all figures, tables, and charts, but do not include the cover sheet, table of contents or appendices. A submission letter is optional and is not included in the page count. Appendix A does not count against the page limit and is mandatory. Appendix B does not count against the page limit and is optional. Additional information not explicitly called for here must not be submitted with the proposal, but may be included in the bibliography in Appendix B. Such materials will be considered for the reviewers’ convenience only and not evaluated as part of the proposal. HR001117S0017 WORLD MODELERS 33 Volume 1 must include the following components: i. Cover Sheet: Include the following information. Label: “Proposal: Volume 1” BAA number (HR001117S0017) Technical Area Proposal title Lead organization (prime contractor) name Type of organization, selected from the following categories: Large Business, Small Disadvantaged Business, Other Small Business, HBCU, MI, Other Educational, or Other Nonprofit Technical point of contact (POC) including name, mailing address, telephone, and email Administrative POC including name, mailing address, telephone number, and email address Award instrument requested: procurement contract (specify type), grant, cooperative agreement or OT.1 Total amount of the proposed effort Place(s) and period(s) of performance Other team member (subcontractors and consultants) information (for each, include Technical POC name, organization, type of organization, mailing address, telephone number, and email address) Proposal validity period (minimum 120 days) Data Universal Numbering System (DUNS) number2 Taxpayer identification number3 Commercial and Government Entity (CAGE) code4 Proposer’s reference number (if any) ii. Table of Contents iii. Executive Summary: Provide a synopsis of the proposed project, including answers to the following questions: What is the proposed work attempting to accomplish or do? How is it done today, and what are the limitations? How much will it cost, and how long will it take? 1 Information on award instruments can be found at http://www.darpa.mil/work-with-us/contract-management. The DUNS number is used as the Government's contractor identification code for all procurement-related activities. Go to http://fedgov.dnb.com/webform/index.jsp to request a DUNS number (may take at least one business day). For further information regarding this subject, please see www.darpa.mil/work-with-us/additionalbaa for further information. 3 See http://www.irs.gov/businesses/small/international/article/0,,id=96696,00.html for information on requesting a TIN. Note, requests may take from 1 business day to 1 month depending on the method (online, fax, mail). 4 A CAGE Code identifies companies doing or wishing to do business with the Federal Government. For further information regarding this subject, please see www.darpa.mil/work-with-us/additional-baa. 2 HR001117S0017 WORLD MODELERS 34 The executive summary should include a description of the key technical challenges, a concise review of the technologies proposed to overcome these challenges and achieve the project’s goal, and a clear statement of the novelty and uniqueness of the proposed work. iv. Innovative Claims and Deliverables: Describe the innovative aspects of the project in the context of existing capabilities and approaches, clearly delineating the uniqueness and benefits of this project in the context of the state of the art, alternative approaches, and other projects from the past and present. Describe how the proposed project is revolutionary and how it significantly rises above the current state of the art. Describe the deliverables associated with the proposed project and any plans to commercialize the technology, transition it to a customer, or further the work. Discuss the mitigation of any issues related to sustainment of the technology over its entire lifecycle, assuming the technology transition plan is successful. v. Technical Plan (plus one additional page for each additional TA addressed, up to a maximum of plus four additional pages if proposing against all five TAs): Outline and address technical challenges inherent in the approach and possible solutions for overcoming potential problems. Demonstrate a deep understanding of the technical challenges and present a credible (even if risky) plan to achieve the project’s goal. Discuss mitigation of technical risk. Provide appropriate measurable milestones (quantitative if possible) at intermediate stages of the project to demonstrate progress, and a plan for achieving the milestones. vi. Management Plan: Provide a summary of expertise of the proposed team, including any subcontractors/consultants and key personnel who will be executing the work. Resumes count against the proposal page limit so proposers may wish to include them in Appendix B below. Identify a principal investigator (PI) for the project. Provide a clear description of the team’s organization including an organization chart that includes, as applicable, the relationship of team members; unique capabilities of team members; task responsibilities of team members; teaming strategy among the team members; and key personnel with the amount of effort to be expended by each person during the project. Provide a detailed plan for coordination including explicit guidelines for interaction among collaborators/subcontractors of the proposed project. Include risk management approaches. Describe any formal teaming agreements that are required to execute this project. List Government-furnished materials or data assumed to be available. vii. Personnel, Qualifications, and Commitments: List key personnel, showing a concise summary of their qualifications, discussion of previous accomplishments, and work in this or closely related research areas. Indicate the level of effort in terms of hours to be expended by each person during each contract year and other (current and proposed) major sources of support for them and/or commitments of their efforts. DARPA expects all key personnel associated with a proposal to make a substantial time commitment to the proposed activity and the proposal will be evaluated accordingly. It is DARPA’s intention to put key personnel conditions into the awards, so proposers should not propose personnel that are not anticipated to execute the award. Include a table of key individual time commitments as follows: HR001117S0017 WORLD MODELERS 35 Key Individual Name 1 Name 2 Project World Modelers Project Name 1 Project Name 2 Status (Current, Pending, Proposed) Proposed Current Pending Hours on Project Phase 1 x x n/a Phase 2 x x x Phase 3 x n/a x World Modelers Project Name 3 Proposed Proposed x x x x x x viii. Capabilities: Describe organizational experience in relevant subject area(s), existing intellectual property, or specialized facilities. Discuss any work in closely related research areas and previous accomplishments. ix. Statement of Work (SOW): The SOW must provide a detailed task breakdown, citing specific tasks and their connection to the interim milestones and metrics, as applicable. Each phase of the project should be separately defined. The SOW must not include proprietary information. For each defined task/subtask, provide: A general description of the objective; A detailed description of the approach to be taken to accomplish each defined task/subtask; Identification of the primary organization responsible for task execution (prime contractor, subcontractor(s), consultant(s)), by name; A measurable milestone, (e.g., a deliverable, demonstration, or other event/activity that marks task completion); A definition of all deliverables (e.g., data, reports, software) to be provided to the Government in support of the proposed tasks/subtasks; and Identify any tasks/subtasks (by the prime or subcontractor) that will be accomplished at a university and believed to be fundamental research. x. Schedule and Milestones: Provide a detailed schedule showing tasks (task name, duration, work breakdown structure element as applicable, performing organization), milestones, and the interrelationships among tasks. The task structure must be consistent with that in the SOW. Measurable milestones should be clearly articulated and defined in time relative to the start of the project. xi. Level of Effort Summary by Task: Provide a one-page table summarizing estimated level of effort per task (in hours) broken out by senior, mid-level and junior personnel, in the format shown below in Figure 2. Also include dollar-denominated estimates of travel, materials and equipment. For this table, consider materials to include the cost of any data sets or software licenses proposed. For convenience, an Excel™ template is available for download along with the BAA. Submit the Level of Effort Summary Excel™ file (do not convert the Excel™ file to pdf format) in addition to Volume 1 and Volume 2 of your full proposal. This Excel™ file does not count towards the total page count. HR001117S0017 WORLD MODELERS 36 Duration SOW Task (months) Labor Hours Senior Mid Junior Total Conslt Total 240 680 24 944 - 200 944 4 80 280 360 - 200 360 3 160 400 24 584 - - 584 1.2.0 <Phase 1 Task 2 name> 6 108 400 1,800 2,308 1,400 - 3,708 1.2.1 <Subtask 1.2.1 name> 3 48 320 1,600 1,968 600 - 2,568 1.2.2 <Subtask 1.2.2 name> 3 60 80 200 340 800 - 1,140 348 1,080 1,824 1.1.0 <Phase 1 Task 1 name> 7 1.1.1 <Subtask 1.1.1 name> 1.1.2 <Subtask 1.1.2 name> : : : Phase 1 Total Hours Phase 1 Costs : 16 : - : : : 3,252 Travel $ 44,000 Materials & Equipment $ SubC 8,000 1,400 : : 200 4,652 $12,000 $2,000 $ 58,000 $ $ $ - - 8,000 2.1.0 <Phase 2 Task 1 name> 8 176 560 64 800 100 100 900 2.1.1 <Subtask 2.1.1 name> 7 96 240 24 360 100 100 460 2.1.2 <Subtask 2.1.2 name> 4 80 320 40 440 - - 440 2.2.0 <Phase 2 Task 2 name> 6 180 520 1,800 2,500 1,240 - 3,740 2.2.1 <Subtask 2.2.1 name> 4 140 400 1,200 1,740 400 - 2,140 2.2.2 <Subtask 2.2.2 name> 4 40 120 600 760 - 1,600 : : : : : : Phase 2 Total Hours Phase 2 Costs : 16 356 1,080 1,864 3,300 Travel $ 48,000 Materials & Equipment $ 840 : - : : 1,340 100 4,640 $13,000 $2,400 $ 63,400 $ $ $ - - - 3.1.0 <Phase 3 Task 1 name> 9 120 400 120 640 100 100 740 3.1.1 <Subtask 3.1.1 name> 3 40 200 40 280 100 100 380 3.1.2 <Subtask 3.1.2 name> 6 80 200 80 360 - - 360 3.2.0 <Phase 3 Task 2 name> 6 160 800 1,800 2,760 1,200 - 3,960 3.2.1 <Subtask 3.2.1 name> 4 80 400 1,000 1,480 600 - 2,080 3.2.2 <Subtask 3.2.2 name> 3 80 400 800 1,280 600 - 1,880 280 1,200 : : : Phase 3 Total Hours Phase 3 Costs : : 1,920 : Project Total Hours 48 Travel $141,000 Materials & Equipment $ 8,000 984 3,360 5,608 : 3,400 Travel $ 49,000 Materials & Equipment $ Project Costs : 16 1,300 : : 100 4,700 $12,000 $2,000 $ 63,000 $ $ $ 9,952 4,040 400 13,992 $37,000 $6,400 $ 184,400 $ $ - - $ 8,000 Figure 2: Example level-of-effort summary table. Numbers illustrate roll-ups and subtotals. The SubC column captures all subcontractor hours and the Conslt column captures all consultant hours. xii. Appendix A: This section is mandatory and must include all of the following components. If a particular subsection is not applicable, state “NONE”. (1). Team Member Identification: Provide a list of all team members including the prime, subcontractor(s), and consultant(s), as applicable. Identify specifically whether any are a non-US organization or individual, FFRDC and/or Government entity. Use the following format for this list: HR001117S0017 WORLD MODELERS 37 Individual Name Role (Prime, Subcontractor or Consultant) Non-US? Organization Org Ind. FFRDC or Govt? (2). Government or FFRDC Team Member Proof of Eligibility to Propose: If none of the team member organizations (prime or subcontractor) are a Government entity or FFRDC, state “NONE”. If any of the team member organizations are a Government entity or FFRDC, provide documentation (per Section III.A.1) citing the specific authority that establishes the applicable team member’s eligibility to propose to Government solicitations to include: 1) statutory authority; 2) contractual authority; 3) supporting regulatory guidance; and 4) evidence of agency approval for applicable team member participation. (3). Government or FFRDC Team Member Statement of Unique Capability: If none of the team member organizations (prime or subcontractor) are a Government entity or FFRDC, state “NONE”. If any of the team member organizations are a Government entity or FFRDC, provide a statement (per Section III.A.1) that demonstrates the work to be performed by the Government entity or FFRDC team member is not otherwise available from the private sector. (4). Organizational Conflict of Interest Affirmations and Disclosure: If none of the proposed team members is currently providing, or has provided within the last year, SETA or similar support as described in Section III.B, state “NONE”. If any of the proposed team members (individual or organization) is currently performing SETA or similar support, furnish the following information: Prime Contract Number DARPA Technical Office supported A description of the action the proposer has taken or proposes to take to avoid, neutralize, or mitigate the conflict (5). Intellectual Property (IP): If no IP restrictions are intended, state “NONE”. The Government will assume unlimited rights to all IP not explicitly identified as having less than unlimited rights in the proposal. For all technical data or computer software that will be furnished to the Government with other than unlimited rights, provide (per Section VI.B.1) a list HR001117S0017 WORLD MODELERS 38 describing all proprietary claims to results, prototypes, deliverables or systems supporting and/or necessary for the use of the research, results, prototypes and/or deliverables. Provide documentation proving ownership or possession of appropriate licensing rights to all patented inventions (or inventions for which a patent application has been filed) to be used for the proposed project. Use the following format for these lists: NONCOMMERCIAL Technical Data and/or Summary of Basis for Computer Software To Intended Use in Assertion be Furnished With the Conduct of Restrictions the Research (List) (Narrative) (List) (List) (Narrative) (List) Asserted Rights Category Name of Person Asserting Restrictions (List) (List) (List) (List) Asserted Rights Category Name of Person Asserting Restrictions (List) (List) (List) (List) COMMERCIAL Technical Data and/or Summary of Basis for Computer Software To Intended Use in Assertion be Furnished With the Conduct of Restrictions the Research (List) (Narrative) (List) (List) (Narrative) (List) (6). Human Subjects Research (HSR): If HSR is not a factor in the proposal, state “NONE”. If the proposed work will involve human subjects, provide evidence of or a plan for review by an institutional review board (IRB). For further information on this subject, see Section VI.B.2. (7). Animal Use: If animal use is not a factor in the proposal, state “NONE”. If the proposed research will involve animal use, provide a brief description of the plan for Institutional Animal Care and Use Committee (IACUC) review and approval. For further information on this subject, see Section VI.B.2. (8). Representations Regarding Unpaid Delinquent Tax Liability or a Felony Conviction under Any Federal Law: For further information regarding this subject, please see www.darpa.mil/work-with-us/additional-baa. Please also complete the following statements. (1) The proposer is [ ] is not [ ] a corporation that has any unpaid Federal tax liability that has been assessed, for which all judicial and administrative remedies have been exhausted or have lapsed, and that is not being paid in a timely manner pursuant to an agreement with the authority responsible for collecting the tax liability, (2) The proposer is [ ] is not [ ] a corporation that was convicted of a felony criminal violation under a Federal law within the preceding 24 months. HR001117S0017 WORLD MODELERS 39 (9). Cost Accounting Standards (CAS) Notices and Certification: For any proposer who submits a proposal which, if accepted, will result in a CAScompliant contract, must include a Disclosure Statement as required by 48 CFR 9903.202. The disclosure forms may be found at http://www.whitehouse.gov/omb/procurement_casb. If this section is not applicable, state “NONE”. For further information regarding this subject, please see www.darpa.mil/work-with-us/additional-baa. (10). Grant Abstract: Proposals for the award of a grant must include an abstract as described in Section VI.B.5. xiii. Appendix B: If desired, include a brief bibliography to relevant papers, reports, or resumes. Do not include technical papers. This section is optional, and the materials will not be evaluated as part of the proposal review. b. Volume 2 - Cost Proposal This volume is mandatory and must include all the listed components. No page limit is specified for this volume. The cost proposal should include a working spreadsheet file (.xls or equivalent format) that provides formula traceability among all components of the cost proposal. The spreadsheet file should be included as a separate component of the full proposal package. Costs must be traceable between the prime and subcontractors/consultants, as well as between the cost proposal and the SOW. Pre-award costs will not be reimbursed unless a pre-award cost agreement is negotiated prior to award. i. Cover Sheet: Include the same information as the cover sheet for Volume 1, but with the label “Proposal: Volume 2.” ii. Cost Summary Tables: Provide a single-page summary table broken down by government fiscal year (October 1 through September 30) listing cost totals for labor, materials, other direct charges (ODCs), indirect costs (overhead, fringe, general and administrative (G&A)), and any proposed fee for the project. Include costs for each task in each fiscal year of the project by prime and major subcontractors, total cost and proposed cost share, if applicable. Provide a second table containing the same information broken down by project phase. iii. Cost Details: For each task, provide the following cost details by month. Include supporting documentation describing the method used to estimate costs. Identify any cost sharing. (1) Direct Labor: Provide labor categories, rates and hours. Justify rates by providing examples of equivalent rates for equivalent talent, past commercial or HR001117S0017 WORLD MODELERS 40 Government rates from a Government audit agency such as the Defense Contract Audit Agency (DCAA), the Office of Naval Research (ONR), the Department of Health and Human Services (DHHS), etc. (2) Indirect Costs: Identify all indirect cost rates (such as fringe benefits, labor overhead, material overhead, G&A, or F&A, etc.) and the basis for each. (3) Materials: Provide an itemized list of all proposed materials, equipment, and supplies for each year including quantities, unit prices, proposed vendors (if known), and the basis of estimate (e.g., quotes, prior purchases, catalog price lists, etc.). For proposed equipment/information technology (as defined in FAR 2.101) purchases equal to or greater than $50,000, include a letter justifying the purchase. Include any requests for Government-furnished equipment or information with cost estimates (if applicable) and delivery dates. (4) Travel: Provide a breakout of travel costs including the purpose and number of trips, origin and destination(s), duration, and travelers per trip. (5) Subcontractor/Consultant Costs: Provide above information for each proposed subcontractor/consultant. Subcontractor cost proposals must include interdivisional work transfer agreements or similar arrangements. If the proposer has conducted a cost or price analysis to determine reasonableness, submit a copy of this along with the subcontractor proposal. The proposer is responsible for the compilation and submission of all subcontractor/consultant cost proposals. At a minimum, the submitted cost volume must contain a copy of each subcontractor or consultant non-proprietary cost proposal (i.e., cost proposals that do not contain proprietary pricing information such as rates, factors, etc.) Proprietary subcontractor/consultant cost proposals may be included as part of Volume 2. Proposal submissions will not be considered complete unless the Government has received all subcontractor/consultant cost proposals. If proprietary subcontractor/consultant cost proposals are not included as part of Volume 2, they may be emailed separately to [email protected]. Email messages must include “Subcontractor Cost Proposal” in the subject line and identify the principal investigator, prime proposer organization and proposal title in the body of the message. Any proprietary subcontractor or consultant proposal documentation which is not uploaded to BAAT as part of the proposer’s submission or provided by separate email shall be made immediately available to the Government, upon request, under separate cover (i.e., mail, electronic/email, etc.), either by the proposer or by the subcontractor/consultant organization. Please note that a ROM or similar budgetary estimate is not considered a fully qualified subcontract cost proposal submission. Inclusion of a ROM or similar budgetary estimate, or failure to provide a subcontract proposal, will result in the full proposal being deemed non-compliant. HR001117S0017 WORLD MODELERS 41 (6) ODCs: Provide an itemized breakout and explanation of all anticipated other direct costs. iv. Proposals Requesting a Procurement Contract: Provide the following information where applicable. (1) Proposals for $750,000 or more: Provide “certified cost or pricing data” (as defined in FAR 2.101) or a request for exception in accordance with FAR 15.403. (2) Proposals for $700,000 or more: Pursuant to Section 8(d) of the Small Business Act (15 U.S.C. § 637(d)), it is Government policy to enable small business and small disadvantaged business concerns to be considered fairly as subcontractors to organizations performing work as prime contractors or subcontractors under Government contracts, and to ensure that prime contractors and subcontractors carry out this policy. In accordance with FAR 19.702(a)(1) and 19.702(b), prepare a subcontractor plan, if applicable. The plan format is outlined in FAR 19.704. (3) Proposers without an adequate cost accounting system: If requesting a cost-type contract, provide the DCAA Pre-award Accounting System Adequacy Checklist to facilitate DCAA’s completion of an SF 1408. Proposers without an accounting system considered adequate for determining accurate costs must complete an SF 1408 if a cost type contract is to be negotiated. To facilitate this process, proposers should complete the SF 1408 found at http://www.gsa.gov/portal/forms/download/115778 and submit the completed form with the proposal. To complete the form, check the boxes on the second page, then provide a narrative explanation of your accounting system to supplement the checklist on page one. v. Proposals Requesting an Other Transaction (OT) for Prototypes Agreement: Proposers must indicate whether they qualify as a nontraditional Defense contractor5 have teamed with a nontraditional Defense contractor, or are providing a one-third cost share for this effort. Provide information to support the claims. Provide a detailed list of milestones including: description, completion criteria, due date, and payment/funding schedule (to include, if cost share is proposed, contractor and Government share amounts). Milestones must relate directly to accomplishment of technical metrics as defined in the solicitation and/or the proposal. While agreement type (fixed price or expenditure based) will be subject to negotiation, the use of fixed price milestones with a payment/funding schedule is preferred. Proprietary information must not be included as part of the milestones. 2. Proprietary and Classified Information DARPA policy is to treat all submissions as source selection information (see FAR 2.101 and 3.104) and to disclose the contents only for the purpose of evaluation. Restrictive notices 5 For definitions and information on OT agreements see http://www.darpa.mil/work-with-us/contract-management. HR001117S0017 WORLD MODELERS 42 notwithstanding, during the evaluation process, submissions may be handled by support contractors for administrative purposes and/or to assist with technical evaluations. All DARPA support contractors performing this role are expressly prohibited from performing DARPA-sponsored technical research and are bound by appropriate nondisclosure agreements. a. Proprietary Information Proposers are responsible for clearly identifying proprietary information. Submissions containing proprietary information must have the cover page and each page containing such information clearly marked. b. Classified Information Classified submissions (classified technical proposals or classified appendices to unclassified proposals) will not be accepted under this solicitation. World Modelers is an unclassified research program. C. Submission Date and Time Proposers are warned that submission deadlines as outlined herein are strictly enforced. Note: some proposal requirements may take from 1 business day to 1 month to complete. See the proposal checklist in Section VIII.C for further information. When utilizing the DARPA BAA Submission Website, as described below in Section IV.E. below, a control number will be provided at the conclusion of the submission process. This control number should be used in all further correspondence regarding your abstract/proposal submission. For proposal submissions requesting cooperative agreements, Section IV.E.1.b, you may request a control number for your submission via email at [email protected]. Please note that the control number will not be issued until after the proposal due date and time. Failure to comply with the submission procedures outlined herein may result in the submission not being evaluated. Proposals The proposal package -- full proposal (Volume 1 and 2) and, as applicable, proprietary subcontractor cost proposals -- must be submitted per the instructions outlined herein and received by DARPA no later than May 11, 2017, at 12:00 noon (ET). Submissions received after this date and time will not be reviewed. D. Funding Restrictions Not applicable. E. Other Submission Requirements 1. Unclassified Submission Instructions HR001117S0017 WORLD MODELERS 43 Proposers must submit all parts of their submission package using the same method; submissions cannot be sent in part by one method and in part by another method nor should duplicate submissions be sent by multiple methods. Email submissions will not be accepted. a. Proposals Requesting a Procurement Contract or Other Transaction DARPA/I2O will employ an electronic upload submission system (https://baa.darpa.mil/) for UNCLASSIFIED proposals requesting award of a procurement contract or Other Transaction under this solicitation. First time users of the DARPA BAA Submission Website must complete a two-step account creation process at https://baa.darpa.mil/. The first step consists of registering for an Extranet account by going to the above URL and selecting the “Account Request” link. Upon completion of the online form, proposers will receive two separate emails; one will contain a user name and the second will provide a temporary password. Once both emails have been received, proposers must go back to the submission website and log in using that user name and password. After accessing the Extranet, proposers must create a user account for the DARPA BAA Submission Website by selecting the “Register Your Organization” link at the top of the page. The DARPA BAA Submission Website will display a list of solicitations open for submissions. Once a proposer’s user account is created, they may view instructions on uploading their proposal. Proposers who already have an account on the DARPA BAA Submission Website may simply log in at https://baa.darpa.mil/, select this solicitation from the list of open DARPA solicitations and proceed with their proposal submission. Note: Proposers who have created a DARPA BAA Submission Website account to submit to another DARPA technical office’s solicitations do not need to create a new account to submit to this solicitation. All submissions submitted electronically through DARPA's BAA website must be uploaded as zip files (.zip or .zipx extension). The final zip file should contain only the files requested herein and must not exceed 50 MB in size. Only one zip file will be accepted per submission. Note: Submissions not uploaded as zip files will be rejected by DARPA. Please note that all submissions MUST be finalized, meaning that no further editing will be possible, when submitting through the DARPA BAA Submission Website in order for DARPA to be able to review your submission. If a submission is not finalized, the submission will not be deemed acceptable and will not be reviewed. Website technical support may be reached at [email protected] and is typically available during regular business hours (9:00 AM – 5:00 PM ET, Monday-Friday). Questions regarding submission contents, format, deadlines, etc. should be emailed to [email protected]. Since proposers may encounter heavy traffic on the web server, they should not wait until the day proposals are due to request an account and/or upload the submission. Full proposals should not be submitted via Email. Any full proposals submitted by Email will not be accepted or evaluated. HR001117S0017 WORLD MODELERS 44 b. Proposals Requesting a Grant or Cooperative Agreement Proposers requesting grants or cooperative agreements may submit proposals through one of the following methods: (1) hard copy mailed directly to DARPA; or (2) electronic upload per the instructions at http://www.grants.gov/applicants/apply-for-grants.html. Grant or cooperative agreement proposals may not be submitted through any other means. If proposers intend to use Grants.gov as their means of submission, then they must submit their entire proposal through Grants.gov; applications cannot be submitted in part to Grants.gov and in part as a hard-copy. Proposers using the Grants.gov do not submit paper proposals in addition to the Grants.gov electronic submission. Grants.gov requires proposers to complete a one-time registration process before a proposal can be electronically submitted. If proposers have not previously registered, this process can take between three business days and four weeks if all steps are not completed in a timely manner. See the Grants.gov user guides and checklists at http://www.grants.gov/web/grants/applicants/applicant-resources.html for further information. Once Grants.gov has received an uploaded proposal submission, Grants.gov will send two email messages to notify proposers that: (1) their submission has been received by Grants.gov; and (2) the submission has been either validated or rejected by the system. It may take up to two business days to receive these emails. If the proposal is rejected by Grants.gov, it must be corrected and re-submitted before DARPA can retrieve it (assuming the solicitation has not expired). If the proposal is validated, then the proposer has successfully submitted their proposal and Grants.gov will notify DARPA. Once the proposal is retrieved by DARPA, Grants.gov will send a third email to notify the proposer. The proposer will then receive an email from DARPA acknowledging receipt and providing a control number. To avoid missing deadlines, proposers should submit their proposals to Grants.gov in advance of the proposal due date, with sufficient time to complete the registration and submission processes, receive email notifications and correct errors, as applicable. For more information on submitting proposals to Grants.gov, visit the Grants.gov submissions page at: http://www.grants.gov/web/grants/applicants/apply-for-grants.html. Proposers electing to submit a grant or cooperative agreement proposal as a hard copy must complete the SF 424 R&R form (Application for Federal Assistance, Research and Related) available on the Grants.gov website http://apply07.grants.gov/apply/forms/sample/RR_SF424_2_0-V2.0.pdf. Proposers choosing to mail hard copy proposals to DARPA must include one paper copy and one electronic copy (e.g., CD/DVD) of the full proposal package. Technical support for the Grants.gov website may be reached at 1-800-518-4726 and [email protected]. Questions regarding submission contents, format, deadlines, etc. should be emailed to [email protected]. HR001117S0017 WORLD MODELERS 45 V. Application Review Information A. Evaluation Criteria Proposals will be evaluated using the following criteria listed in descending order of importance: Overall Scientific and Technical Merit; Potential Contribution and Relevance to the DARPA Mission; and Cost Realism. Overall Scientific and Technical Merit: The proposed technical approach is innovative, feasible, achievable, and complete. The task descriptions and associated technical elements are complete and in a logical sequence, with all proposed deliverables clearly defined such that a viable attempt to achieve project goals is likely as a result of award. The proposal identifies major technical risks and clearly defines feasible mitigation efforts. Proposers should also take note to the information provided in Section I, as DARPA will also look at how a proposer addresses the technical challenges relevant to each TA, as well as view how key personnel will work on those challenges. Potential Contribution and Relevance to the DARPA Mission: The potential contributions of the proposed effort are relevant to the national technology base. Specifically, DARPA’s mission is to make pivotal early technology investments that create or prevent strategic surprise for U.S. National Security. This includes considering the extent to which any proposed intellectual property restrictions will potentially impact the Government’s ability to transition the technology. Cost Realism: The proposed costs are realistic for the technical and management approach and accurately reflect the technical goals and objectives of the solicitation. The proposed costs are consistent with the proposer's Statement of Work and reflect a sufficient understanding of the costs and level of effort needed to successfully accomplish the proposed technical approach. The costs for the prime proposer and proposed subawardees are substantiated by the details provided in the proposal (e.g., the type and number of labor hours proposed per task, the types and quantities of materials, equipment and fabrication costs, travel and any other applicable costs and the basis for the estimates). B. Review and Selection Process The review process identifies proposals that meet the evaluation criteria described above and are, therefore, selectable for negotiation of awards by the Government. DARPA policy is to ensure impartial, equitable, comprehensive proposal evaluations and to select proposals that meet DARPA technical, policy, and programmatic goals. If necessary, panels of experts in the appropriate areas will be convened. As described in Section IV, proposals must be deemed conforming to the solicitation to receive a full technical review against the evaluation criteria; proposals deemed non-conforming will be removed from consideration. HR001117S0017 WORLD MODELERS 46 DARPA will conduct a scientific/technical review of each conforming proposal. Conforming proposals comply with all requirements detailed in this BAA; proposals that fail to do so may be deemed non-conforming and may be removed from consideration. Proposals will not be evaluated against each other since they are not submitted in accordance with a common work statement. DARPA’s intent is to review proposals as soon as possible after they arrive; however, proposals may be reviewed periodically for administrative reasons Selections may be made at any time during the period of solicitation. Pursuant to FAR 35.016, the primary basis for selecting proposals for award negotiation shall be technical, importance to agency programs, and fund availability. Conforming proposals based on a previously submitted abstract will be reviewed without regard to feedback resulting from review of that abstract. Furthermore, a favorable response to an abstract is not a guarantee that a proposal based on the abstract will ultimately be selected for award negotiation. Proposals that are determined selectable will not necessarily receive awards. For evaluation purposes, a proposal is defined to be the document and supporting materials as described in Section IV.B.2. Subject to the restrictions set forth in FAR 37.203(d), input on technical aspects of the proposals may be solicited by DARPA from non-Government consultants/experts who are strictly bound by the appropriate non-disclosure requirements. No submissions will be returned. HR001117S0017 WORLD MODELERS 47 VI. Award Administration Information A. Selection Notices After proposal evaluations are complete, proposers will be notified as to whether their proposal was selected for award negotiation as a result of the review process. Notification will be sent by email to the technical and administrative POCs identified on the proposal cover sheet. If a proposal has been selected for award negotiation, the Government will initiate those negotiations following the notification. B. Administrative and National Policy Requirements 1. Intellectual Property Proposers should note that the Government does not own the intellectual property of technical data/computer software developed under Government contracts; it acquires the right to use the technical data/computer software. Regardless of the scope of the Government’s rights, performers may freely use their same data/software for their own commercial purposes (unless restricted by U.S. export control laws or security classification). Therefore, technical data and computer software developed under this solicitation will remain the property of the performers, though DARPA desires to have a minimum of Government Purpose Rights (GPR) to technical data/computer software developed through DARPA sponsorship. If proposers desire to use proprietary software or technical data or both as the basis of their proposed approach, in whole or in part, they should: (1) clearly identify such software/data and its proposed particular use(s); (2) explain how the Government will be able to reach its program goals (including transition) within the proprietary model offered; and (3) provide possible nonproprietary alternatives in any area that might present transition difficulties or increased risk or cost to the Government under the proposed proprietary solution. Proposers expecting to use, but not to deliver, commercial open source tools or other materials in implementing their approach may be required to indemnify the Government against legal liability arising from such use. All references to "Unlimited Rights" or "Government Purpose Rights" are intended to refer to the definitions of those terms as set forth in the Defense Federal Acquisition Regulation Supplement (DFARS) Part 227. a. Intellectual Property Representations All proposers must provide a good faith representation of either ownership or possession of appropriate licensing rights to all other intellectual property to be used for the proposed project. Proposers must provide a short summary for each item asserted with less than unlimited rights that describes the nature of the restriction and the intended use of the intellectual property in the conduct of the proposed research. If proposers desire to use proprietary software or technical data or both as the basis of their proposed approach, in whole or in part, they should: (1) clearly identify such software/data and its proposed particular use(s); (2) explain how the Government will be able to reach its program goals (including transition) within the proprietary model offered; and (3) provide possible nonproprietary alternatives in any area that might present transition difficulties or increased HR001117S0017 WORLD MODELERS 48 risk or cost to the Government under the proposed proprietary solution. b. Patents All proposers must include documentation proving ownership or possession of appropriate licensing rights to all patented inventions to be used for the proposed project. If a patent application has been filed for an invention, but it includes proprietary information and is not publicly available, a proposer must provide documentation that includes: the patent number, inventor name(s), assignee names (if any), filing date, filing date of any related provisional application, and summary of the patent title, with either: (1) a representation of invention ownership, or (2) proof of possession of appropriate licensing rights in the invention (i.e., an agreement from the owner of the patent granting license to the proposer). c. Procurement Contracts Noncommercial Items (Technical Data and Computer Software): Proposers requesting a procurement contract must list all noncommercial technical data and computer software that it plans to generate, develop, and/or deliver, in which the Government will acquire less than unlimited rights and to assert specific restrictions on those deliverables. In the event a proposer does not submit the list, the Government will assume that it has unlimited rights to all noncommercial technical data and computer software generated, developed, and/or delivered, unless it is substantiated that development of the noncommercial technical data and computer software occurred with mixed funding. If mixed funding is anticipated in the development of noncommercial technical data and computer software generated, developed, and/or delivered, proposers should identify the data and software in question as subject to GPR. In accordance with DFARS 252.227-7013, “Rights in Technical Data - Noncommercial Items,” and DFARS 252.227-7014, “Rights in Noncommercial Computer Software and Noncommercial Computer Software Documentation,” the Government will automatically assume that any such GPR restriction is limited to a period of 5 years, at which time the Government will acquire unlimited rights unless the parties agree otherwise. The Government may use the list during the evaluation process to evaluate the impact of any identified restrictions and may request additional information from the proposer, as may be necessary, to evaluate the proposer’s assertions. Failure to provide full information may result in a determination that the proposal is not compliant with the solicitation. A template for complying with this request is provided in Section IV.B.2.a.xii.(5). Commercial Items (Technical Data and Computer Software): Proposers requesting a procurement contract must list all commercial technical data and commercial computer software that may be included in any deliverables contemplated under the research project, and assert any applicable restrictions on the Government’s use of such commercial technical data and/or computer software. In the event a proposer does not submit the list, the Government will assume there are no restrictions on the Government’s use of such commercial items. The Government may use the list during the evaluation process to evaluate the impact of any identified restrictions and may request additional information from the proposer to evaluate the proposer’s assertions. Failure to provide full information may result in a determination that the proposal is not compliant with HR001117S0017 WORLD MODELERS 49 the solicitation. A template for complying with this request is provided in Section IV.B.2.a.xii.(5). d. Other Types of Awards Proposers responding to this solicitation requesting an award instrument other than a procurement contract shall follow the applicable rules and regulations governing those award instruments, but in all cases should appropriately identify any potential restrictions on the Government’s use of any intellectual property contemplated under those award instruments in question. This includes both noncommercial items and commercial items. The Government may use the list as part of the evaluation process to assess the impact of any identified restrictions, and may request additional information from the proposer, to evaluate the proposer’s assertions. Failure to provide full information may result in a determination that the proposal is not compliant with the solicitation. A template for complying with this request is provided in Section IV.B.2.a.xii.(5). 2. Human Research Subjects/Animal Use Proposers that anticipate involving Human Research Subjects or Animal Use must comply with the approval procedures detailed at www.darpa.mil/work-with-us/additional-baa. 3. Electronic and Information Technology All electronic and information technology acquired through this solicitation must satisfy the accessibility requirements of Section 508 of the Rehabilitation Act (29 U.S.C. § 794d) and FAR 39.2. Each project involving the creation or inclusion of electronic and information technology must ensure that: (1) Federal employees with disabilities will have access to and use of information that is comparable to the access and use by Federal employees who are not individuals with disabilities; and (2) members of the public with disabilities seeking information or services from DARPA will have access to and use of information and data that is comparable to the access and use of information and data by members of the public who are not individuals with disabilities. 4. System for Award Management (SAM) and Universal Identifier Requirements All proposers must be registered in SAM unless exempt per FAR 4.1102. FAR 52.204-7, “System for Award Management” and FAR 52.204-13, “System for Award Management Maintenance” are incorporated into this BAA. See www.darpa.mil/work-with-us/additionalbaa for further information. Note that new registrations can take an average of 7-10 business days to process in SAM. SAM registration requires the following information: DUNS number TIN CAGE Code - If a proposer does not already have a CAGE code, one will be assigned during SAM registration. Electronic Funds Transfer information (e.g., proposer’s bank account number, routing number, and bank phone or fax number). HR001117S0017 WORLD MODELERS 50 5. Publication of Grant Awards Per Section 8123 of the Department of Defense Appropriations Act, 2015 (Pub. L. 113-235), all grant awards must be posted on a public website in a searchable format. To comply with this requirement, proposers requesting grant awards must submit a maximum one (1) page abstract that may be publicly posted and explains the program or project to the public. The proposer should sign the bottom of the abstract confirming the information in the abstract is approved for public release. Proposers are advised to provide both a signed PDF copy, as well as an editable (e.g., Microsoft word) copy. Abstracts contained in grant proposals that are not selected for award will not be publicly posted. C. Reporting 1. Technical and Financial Reports The number and types of technical and financial reports required under the contracted project will be specified in the award document, and will include, as a minimum, monthly financial and technical status reports (unless it is a firm-fixed price award) and a yearly status summary. A final report that summarizes the project and tasks will be required at the conclusion of the performance period for the award. The reports shall be prepared and submitted in accordance with the procedures contained in the award document. 2. Representations and Certifications If a procurement contract is contemplated, prospective awardees will need to be registered in the SAM database prior to award and complete electronic annual representations and certifications consistent with FAR guidance at 4.1102 and 4.1201; the representations and certifications can be found at www.sam.gov. Supplementary representations and certifications can be found at www.darpa.mil/work-with-us/additional-baa. . 3. Wide Area Work Flow (WAWF) Unless using another means of invoicing, performers will be required to submit invoices for payment directly at https://wawf.eb.mil. If applicable, WAWF registration is required prior to any award under this solicitation. 4. Terms and Conditions A link to the DoD General Research Terms and Conditions for Grants and Cooperative Agreements and supplemental agency terms and conditions can be found at www.darpa.mil/work-with-us/contract-management#GrantsCooperativeAgreements.. 5. FAR and DFARS Clauses Solicitation clauses in the FAR and DFARS relevant to procurement contracts and FAR and DFARS clauses that may be included in any resultant procurement contracts are incorporated herein and can be found at www.darpa.mil/work-with-us/additional-baa. See also Section II.C regarding the disclosure of information and compliance with safeguarding covered defense information controls (for FAR-based procurement contracts only). HR001117S0017 WORLD MODELERS 51 6. i-Edison Award documents will contain a requirement for patent reports and notifications to be submitted electronically through the i-Edison Federal patent reporting system at http://sedison.info.nih.gov/iEdison. 7. Controlled Unclassified Information (CUI) on Non-DoD Information Systems Further information on Controlled Unclassified Information on Non-DoD Information Systems is incorporated herein can be found at www.darpa.mil/work-with-us/additional-baa. HR001117S0017 WORLD MODELERS 52 VII. Agency Contacts DARPA will use email for all technical and administrative correspondence regarding this solicitation. Technical POC: Dr. Paul Cohen, Program Manager, DARPA/I2O Email: [email protected] Mailing address: DARPA/I2O ATTN: HR001117S0017 675 North Randolph Street Arlington, VA 22203-2114 I2O Solicitation Website: http://www.darpa.mil/work-with-us/opportunities HR001117S0017 WORLD MODELERS 53 VIII. Other Information A. Frequently Asked Questions (FAQs) Administrative, technical, and contractual questions should be sent via email to [email protected]. All questions must be in English and must include the name, email address, and the telephone number of a point of contact. DARPA will attempt to answer questions in a timely manner; however, questions submitted within 7 days of closing may not be answered. If applicable, DARPA will post FAQs to http://www.darpa.mil/work-with-us/opportunities. B. Proposers Day The Proposers Day was held on February 3, 2017 in Arlington, VA. The special notice regarding the World Modelers Proposers Day, DARPA-SN-17-18, can be found at https://www.fbo.gov/index?s=opportunity&mode=form&id=422492442b9df2673a0dd37cf7d63 1af&tab=core&_cview=0. For further information regarding the World Modelers Proposers Day, including slides from the event, please see http://www.darpa.mil/work-with-us/opportunities under HR001117S0017. C. Submission Checklist The following items apply prior to proposal submission. Note: some items may take up to 1 month to complete. Item Obtain DUNS number Obtain Taxpayer Identification Number (TIN) Register in the System for Award Management (SAM) HR001117S0017 BAA Section IV.B.2.a.i IV.B.2.a.i VI.B.4 WORLD MODELERS Applicability Required of all proposers Required of all proposers Required of all proposers Comment The DUNS Number is the Federal Government's contractor identification code for all procurementrelated activities. See http://fedgov.dnb.com/webform/index.jsp to request a DUNS number. Note: requests may take at least one business day. A TIN is used by the Internal Revenue Service in the administration of tax laws. See http://www.irs.gov/businesses/small/international/ article/0,,id=96696,00.html for information on requesting a TIN. Note: requests may take from 1 business day to 1 month depending on the method (online, fax, mail). The SAM combines Federal procurement systems and the Catalog of Federal Domestic Assistance into one system. See www.sam.gov for information and registration. Note: new registrations can take an average of 7-10 business days. SAM registration requires the following information: -DUNS number -TIN -CAGE Code. A CAGE Code identifies companies doing or wishing to do business with the Federal Government. If a proposer does not already have a CAGE code, one will be assigned during SAM registration. 54 Ensure representations and certifications are up to date VI.C.2 Required of all proposers Ensure eligibility of all team members III Required of all proposers Register at Grants.gov IV.E.1.b Required for proposers requesting grants or cooperative agreements -Electronic Funds Transfer information (e.g., proposer’s bank account number, routing number, and bank phone or fax number). Federal provisions require entities to represent/certify to a variety of statements ranging from environmental rules compliance to entity size representation. See http://www.sam.gov for information. Verify eligibility, as applicable, for in accordance with requirements outlined in Section 3. Grants.gov requires proposers to complete a onetime registration process before a proposal can be electronically submitted. If proposers have not previously registered, this process can take between three business days and four weeks if all steps are not completed in a timely manner. See the Grants.gov user guides and checklists at http://www.grants.gov/web/grants/applicants/appli cant-resources.html for further information. The following items apply as part of the submission package: Item BAA Section Applicability Volume 1 (Technical and Management Proposal) IV.B.2 Required of all proposers Appendix A IV.B.2.a.xii Required of all proposers Volume 2 (Cost Proposal) IV.B.2.b Required of all proposers Level of Effort Summary by Task Excel™ spreadsheet IV.B.2.a.xi Required of all proposers Comment Conform to stated page limits and formatting requirements. Include all requested information. -Team member identification - Government/FFRDC team member proof of eligibility - Organizational conflict of interest affirmations - Intellectual property assertions - Human subjects research - Animal use - Unpaid delinquent tax liability/felony conviction representations -CASB disclosure, if applicable - Cover Sheet - Cost summary - Detailed cost information including justifications for direct labor, indirect costs/rates, materials/equipment, subcontractors/consultants, travel, ODCs - Cost spreadsheet file (.xls or equivalent format) - If applicable, list of milestones for OTs - Subcontractor plan, if applicable Subcontractor cost proposals - Itemized list of material and equipment items to be purchased with vendor quotes or engineering estimates for material and equipment more than $50,000 - Travel purpose, departure/arrival destinations, and sample airfare A template LoE Excel™ file will be provided on the FedBizOpps website as an attachment. Submit the LoE Excel™ file (do not convert Excel™ file to pdf format). For information concerning agency level protests see http://www.darpa.mil/work-withus/additional-baa#NPRPAC. HR001117S0017 WORLD MODELERS 55
© Copyright 2026 Paperzz