ITA White Paper for the 2013 BPP
Fact Extraction and Reasoning using Controlled Natural
Language (v10)
This is a Continuation of Project 4, Task 2, with the addition of new research topics.
Research Issue/Technical Approach
This section should be up to 2000 words in length, describing the research issues to be
addressed and the technical approach to solve them. This should include:
 Problem background, the associated technical challenges in coalition environments, and
why it is important to solve this problem
 The technical approach used to address the problem
 Approach/intention to validate/verify the output of the research
 Key deliverables and planned research outputs
Identify the fundamental research issues to be addressed and how the research may advance the
state-of-the-art, develop fundamental knowledge, and provide generalizable results.
If this is a continuation of an existing project, identify what additional value will be obtained by
continuing the project.
Problem Statement
We propose to extend current P4T2 research on using a Controlled Natural Language,
specifically Controlled English (CE) [1,2,3], for fact extraction from unstructured text and
reasoning from these facts to provide high-value information [4,5]. By focusing on the
challenging problem of Natural Language (NL) processing and reasoning we address key issues
to improve our CE capabilities and facilitate greater adoption of CE, as recommended in the peer
review. Improving fact extraction is also a requirement in the whitepapers call (TA6-3a), and
improving reasoning capabilities will enhance information gathering for decision making.
We diagram a “motivating scenario”, a class of coalition problems combining reasoning to
obtain high-value information with interpretation of unstructured text, and exchange, explanation
and understanding of results with other humans and systems. Problems include collaborative
planning, decision making, and intelligence analysis, e.g. by a Battalion in a coalition (shown in
dark below):
ITA White Paper for the 2013 BPP
The context of the UK Battalion [BN] operation is given by a US Brigade [BDE], as plans,
intelligence and background information. The Battalion must work and communicate with a US
Battalion and an (unspecified) host nation unit. The UK Battalion intelligence officer must
provide high-value information and intelligence to the Brigade, based upon information fusion
from different sources and information types: from the US Battalion, host nation, Company
patrols, specialist reports, sensors, intelligence.
Information processing is summarized below, where analysts’ information requirements are
determined by decision-making, driving them to construct high-value information and rationale.
High-value facts are constructed by extracting basic facts from unstructured and other sources,
and reasoning to infer high-value information, patterns or significant events. The need to extract
facts may drive the search for other sources. In the coalition, basic information may be provided
by a partner, and high-value information may be provided back to other partners.
ITA White Paper for the 2013 BPP
CE fact extraction and reasoning has benefits:
1. CE as the target for fact extraction can integrate analysis and representation of
unstructured and structured information, providing a common human- and machinereadable form of English. This is also useful when the amount of unstructured
information is large, and human users need machine assistance in directing their attention
to relevant information.
2. CE as a means of configuring NL processing (via human-readable linguistic rules and
concepts) can help analysts to understand application of NL processing to their domain
specific concepts and ways of expressing those concepts in unstructured text. Analysts
can understand the effects of processing on quality of information and can configure the
system to process new types of information.
3. CE can provide analysts with a human-friendly unambiguous language and a reasoning
system for defining concepts and rules that infer high-value information, patterns or
significant events from basic facts extracted and allow users to explore hypotheses about
the interpretation of data.
4. CE can facilitate analysis under rapidly changing information needs by supporting
dynamic creation and modification of analytic concepts and rules, allowing analysts to
explore new ways of analyzing data and incorporate new sources of unstructured
information.
5. CE can facilitate assessment of the quality of high-value information and explanation of
conclusions via rationale including assumptions and uncertainty factors.
6. CE can facilitate information communication between coalition partners by providing
formal unambiguous concepts with means for expressing them that help reduce
misunderstanding.
ITA White Paper for the 2013 BPP
P4T2 research has produced results in these areas [4,5,6,7], but significant challenges remain to
achieve wider acceptance of CE. The first challenge (C1) is to improve quality of facts extracted
from a greater range of unstructured sources:


extending capabilities of NL processing configured in CE
extending CE-based reasoning capabilities
The second challenge (C2) is to extend capabilities of CE itself:


extending CE to allow expression of facts in a more readable way
providing explanatory capability for conceptual models
These challenges are related as fact extraction provides a powerful example of complex
reasoning which will drive extensions to CE and reasoning capabilities, benefiting future
acceptance of CE.
Proposed Approach
To meet the challenges, we propose research into interpretation of unstructured text leading to
CE facts and reasoning on extracted facts to provide high-value information, ( “Fact Extraction”
, “Analyst Reasoning” in the diagram). We build upon the P4T2 framework [4,5] where fact
extraction from unstructured text was achieved and configured using CE in the CE store. This
included a multilayered conceptual model of linguistics and semantics, based upon scientific
principles and permitting users to rapidly configure domain-specific NL processing and
reasoning, and providing rationale for conclusions, as demonstrated at ACITA12. We retain CE
at the core of this framework as it is a strong enabling factor in the six benefits noted above.
We propose to extend this framework to address the challenges:




(C1) by extending NL processing techniques in CE to cover more aspects of English and
to handle ambiguities, taking advantage of available linguistic resources and theories, to
make scientific advances in linguistics more accessible to non-specialists
(C1) by extending the reasoning component, allowing users to define assumptions,
hypotheses, uncertainties and constraints, and rules for inferring information within
different hypotheses
(C2) by extending CE syntax and semantics, using insights from our NL processing
research, to improve CE expressibility and logical power whilst avoiding ambiguities.
(C2) by extending rationale to provide better explanations of conceptual models
Enhancing NL processing is beneficial in its own right, but also demonstrates CE in a complex
reasoning problem, motivating extensions to the reasoning component and helping the ITA to
use CE in other areas. However it offers significantly more; understanding NL processing can
provide insights into selection of expressive but non-ambiguous extensions to CE and can inform
work on cognition and language. We will apply fact extraction to larger samples of unstructured
data (recommended in the peer review) whilst enhancing support to users for constructing and
understanding the processing.
ITA White Paper for the 2013 BPP
We believe that the CE-based approach to NL processing continues to be novel due to the
pervasive use of human-readable CE throughout the interpretation and reasoning as well as
extending CE to better support extraction of rich information and its exchange across
collaborating partners.
Enhancing Natural Language Capabilities
External Linguistic Resources
We propose to leverage external linguistic resources (lexicons and linguistic processing rules)
into our CE-based framework, providing greater fact extraction capability whilst making them
easier to understand and useable by reasoning components. This requires development of CEbased representations for linguistic concepts defined by these resources, such as lexical
categories, word forms and semantic constraints. The nature of these key structures is still a
matter of linguistic research, and the development of a unified CE model will be a major task
[8,9]. We will also research into how analysts may be involved in tailoring linguistic resources to
domain-specific conceptual models.
Potential resources include Wordnet [10,11], Verbnet [12] (lexical/semantic relations between
words) the English Resource Grammar (ERG) [13] (large English Grammar) and the Grammar
Matrix [14] (starter kit for developing grammars and language processing). The latter builds
upon the Linguistic Knowledge Base [15], developed by team member Prof Ann Copestake,
within the DELPH-IN consortium [16], and thus we will contribute to the NL community.
Deeper Semantics
Fundamental to NL processing for fact extraction is representation and interpretation of complex
relations between words and structures in a sentence and the underlying semantics/meaning of
the sentence components, and how these are composed into the propositions expressed in the
sentence, so that sentence meaning may be expressed unambiguously, e.g. CE facts based on the
conceptual model. More complex sentences require more sophisticated semantics, and in
addition to the proposed enhancements provided by leveraging external linguistic resources, such
“compositional semantics” remains a fundamental research area [17,33] for capabilities to
interpret and represent more complex sentences in CE.
We propose in-depth research on how compositional semantic representations used in broadcoverage parsers can both be extended and represented in the context of a CE-based information
framework. Focusing on Minimal Recursion Semantics (MRS) [18] used by the ERG parser, we
cover:

Relating MRS design decisions to CE modeling and reasoning capabilities (e.g.
assumptions, constraints)

Extending representation of specific linguistic aspects (e.g. tense and aspect) in MRS to
take advantage of the rich semantics of a CE model and designing how CE may be
extended to represent these aspects
ITA White Paper for the 2013 BPP

Integrating compositional semantics with constraints deriving from users’ CE conceptual
models, drawing on findings from external projects
This has the potential to improve fact extraction by incorporating more advanced techniques to
map syntax to semantics, allowing a greater range of sentences to be handled correctly; to guide
improvements to the semantics of the CE reasoning components, based upon parallel experiences
of using MRS in complex NL projects, providing feedback on their robustness and validity; to
provide insight into how CE reasoning capabilities of assumptions, constraints, uncertainty can
be used in a greater range of situations.
Handling Ambiguities
We propose research into ambiguities in NL by representing linguistic information (e.g.
selectional restrictions) in CE and applying it to disambiguation, using reasoning with
assumptions, constraints and uncertainties. [19]
Detecting Uncertainty in Sentences
We propose to extract uncertainty inherent in unstructured text, allowing these uncertainties to
contribute to quality of information of high-value facts generated by reasoning, using:




explicit linguistic cues in modal words such as “might”, “believe”, “possibly”
context of sentence container (e.g. confidence level from the enclosing report)
indirect clues in the language, e.g. hedges, question tags
domain knowledge to validate consistency of information
Extending Reasoning Capabilities
We propose research into extending the CE reasoning component to handle assumptions,
hypotheses, uncertainty and constraints based in part on requirements from the Pathfinder UK
transition projects. [20] These extensions allow users to express more complex reasoning in CE
as well as directly supporting NL processing.
Assumption-based reasoning [21] allows exploration of alternative “hypotheses” about facts
extracted and inferred high-value information. This will be extended to include numerical
uncertainty based upon degree of belief of assumptions, as in Pathfinder, where rationale graphs
show assumptions and uncertainties supporting the facts, allowing the user to assess the quality
of high-valued information. Generic mechanisms [22] will be used to propagate assumptions and
uncertainties, offering a CE-based framework for work on trust and subjective logic where a
specific calculus can be provided by the user.
Constraint-based reasoning [23] allows users to represent constraints and apply these in the
reasoning. Such techniques are used in problem solving e.g. planning [24, 25] and can also
guide NL processing in handling ambiguities.
Linguistically Motivated CE Enhancements
We propose to define CE syntax and semantic extensions to enhance naturalness (e.g.
prepositional phrases for expressing relationships between entities), and to increase logical
ITA White Paper for the 2013 BPP
expressivity (e.g. sentences involving “some”). These have significant benefit, but also a cost:
semantic processing becomes more complex and ambiguities may be introduced.
Key to our proposal is that studying NL processing gives insight into costs and benefits of CE
extensions. Since CE is a form of natural language there are strong parallels in how they may be
processed, and understanding NL structures and difficulties in their interpretation is beneficial in
determining which language structures should be used to extend CE. We believe our approach is
novel: as we extend CE with more complex structures we meet “ambiguity-barriers”, where
extension causes potential ambiguities, and we can determine suitable “controls” on the CE
specification to avoid them. We will also review expressiveness of other CNLs [31].
CE Model Explanation
CE models can become complex, and it is useful to provide users with explanations of the
models and rules, so they can more easily extend them to construct new concepts and reasoning.
We propose to explore the provision of structured views of the model in a more “tutorial” style
by extending rationale mechanisms [26] to analyse logical connections between concepts and
rules.
Key Deliverables and Planned Research Outputs
These are:







Contextual scenario
Extended fact extraction capabilities from unstructured text based on leverage of external
linguistic resources, ambiguity handling and research on deeper semantics
Enhanced reasoning techniques based on assumptions, hypotheses, uncertainty and
constraints, and CE syntax for expressing this knowledge
Model explanation
CE extensions
Enhanced fact extraction in the Experimental Framework
Informal evaluation of benefits of CE for fact extraction
Experimentation
This section should be up to 300 words in length. Describe how experimentation will be
employed as an integral part of the research and how the research will be validated in relevant
environments and what experiments will be designed/conducted that can validate the research
questions posed by the ITA research. Explain how the ITA Experimental Framework will be
used.
Clearly link (a) experimentation to the key deliverables described in the “Research
Issue/Technical Approach” section; and (b) contributor(s) responsible for leading the
experimental work in the “Contributors and their Roles” section.
We will contribute to the Experimental Framework CE store [27] with agents, models and
reasoning from our fundamental fact extraction and reasoning research, together with research
tools e.g. Analysts Helper [28] and review stations [5]. We will broaden our unstructured text
ITA White Paper for the 2013 BPP
sources from SYNCOIN [32] to include “Hursley reports” from UK Pathfinder projects, and
potentially open sources e.g. newsfeed, social media.
We will develop a scenario from the problems described above, including intelligence analysis
and humanitarian emergency response, incorporated into the ITA-wide scenario. We will use the
CollaborativePlanningModel [25,29] and associated tools [29,30] to model military context (e.g.
commander’s intent), supporting coalition decision making, and the “Decision-Making to
Support Mission Command” theme.
IBM UK will lead experiments to assess how CE assists users in the scenario, creating an
informal evaluation using personnel representing analysts and coalition partners. Analysts will
have information requirements and operational context, e.g. higher-level plan. Unstructured text
will be provided from which analysts attempt to extract relevant information. Possible stages
include testing the construction of simple reasoning, dynamic creation of NL processing for new
source types, explanation of conclusions to partners, utility of structured explanations of the CE
model. This will inform our research into how CE might be extended, how more advanced
reasoning processes can be understood and how the model can be best presented to users.
We will measure success:



pragmatically, via experimentation, as the quality of CE facts extracted from a range of
sources, the ability for users to understand, judge and communicate results and the
facility by which users can construct new domain specific concepts and reasoning.
theoretically, via the researching of sound linguistic and semantic models and reasoning,
as measured by the acceptance of the work in the linguistic community
by readiness of the technology for transition
Military and BPP13 Relevance
This section should be up to 300 words in length. Describe the military relevance of the
research, how it will tackle research challenges associated with Coalition Operations, and how
it is relevant to the coalition needs/research challenges for the ITA Technical Areas.
We aim to provide theoretical and technical underpinnings to solve the class of military problems
described above, applying human knowledge to interpretation of unstructured text (TA6 3a1),
reasoning to obtain high-value information and intelligence; exchange, explanation and
understanding of the results with other humans and systems within the coalition context. Our
research will support intelligence analysis, planning, logistics, resource allocation and has a key
role in supporting “Multi-Int” fusion. The MIPS UK ITA transition project applied CE fact
extraction techniques to such problems. Addressing a wider range of unstructured sources
provides opportunity to apply these techniques in new areas.
1
These (and others like it in this section) are references to the specifically relevant items in the BPP13 call for
whitepapers document.
ITA White Paper for the 2013 BPP
Using a Controlled Natural Language (e.g. CE) to configure NL processing and reasoning
facilitates users in applying specialist knowledge to extraction of high-value information thus
allowing them to adapt more readily to changing circumstances. This, when coupled with an
explanation capability, offers potential for greater understanding between coalition partners
about the meaning of the information and how that information is obtained, especially when CE
is extended with greater linguistic expressivity and logical power. By situating our work within
a contextual CPM-based scenario, we can integrate to BPP13 research involving decision
making, as well as external projects such as that involving the NATO tool TOPFAS.
There is additional relevance to BPP13: the work provides a comprehensive example of CE in a
complex task, motivating and guiding extensions to CE, encouraging wider uptake; the work
supports:





shared understanding within the coalition, via shared language and models
more complex reasoning capabilities via hypotheses (TA6 1d) uncertainty, constraints,
rationale, explanation, thus assisting decision making (TA6 1c)
research into argumentation using CE to express arguments
research into subjective logic and trust using CE to express support for conclusions
research into collective cognition using CE to communicate between agents
Transition Opportunities
This section should be up to 300 words in length describing potential transition opportunities,
exploitation possibilities and potential routes to market that may be created from the output of
the research.
The current P4T2 work, upon which we are building, has already been the basis for successful
transition. The execution of these transition projects has shown us many challenges that remain
and these have informed our proposals for further research. We anticipate the proposed research
will be capable of transition, following a similar path, to more areas characterized in the class of
problems described above, including:

reasoning and uncertainty handling for intelligence analysis, e.g. Pathfinder transition
projects,

fact extraction for intelligence analysis, e.g. MIPS transition project;

collaborative planning, e.g. NATO CPM-TOPFAS project;

intergroup sharing of information on socio-cultural infrastructure, e.g. the CWP proposal
New opportunities arise from extend our NL processing capabilities: using CE for NL processing
can feed into the research community such as the DELPH-IN consortium, as well as being
incorporated into existing NL systems such as LanguageWare and the Watson DeepQA project.
ITA White Paper for the 2013 BPP
There is potential collaboration with NS-CTA in areas such as text mining and ontologies. We
will continue to brief external agencies such as DRDC, Canada.
Further transition opportunities include:




support for multiple large scale intelligence analysis programs (Boeing)
support for knowledge based construction and as an advanced user interface front end for
the Integration Visualization Tool (Boeing)
support for Knowledge-Base Engineering for knowledge base construction and as an
advanced user interface front end (Boeing)
transitioning the CE store, incrementing its capabilities and hardening it into a more
production-strength asset. (IBM)
Transition could lead to new assistive applications and services that support collaborating users
in tasks such as intelligence analysis, Multi-Int fusion, collaborative planning, information
exchange, in military and non-military coalitions, based upon knowledge and data communicated
in structured and unstructured formats, and providing a user interface that more closely aligns the
users’ and the systems’ conceptual processing.
Dependencies (to be reviewed)
This section should be up to 100 words in length identifying the Technical Area (TA) that the
proposed research applies and its dependencies on previous and proposed research efforts.
Highlight whether this research is a continuation of current ITA research (identify the
project/task) or a new effort. Describe the dependencies that this research has with:
1) Current ITA research projects,
2) Other proposed research tasks responding to this Call for White Papers,
3) Any other technical efforts or data collection not included in this white paper.
This should not only highlight the linkages but also summarize how this program leverages these
other efforts or how it would be affected if other research tasks were not selected.
The research is in TA 6, based upon P4T2 research, including the CEStore, with links into the
following known proposals, though not critically dependent upon them:
“Agile decision-making through diversity
models and semantic enrichment”
“In-time Assembly and Presentation of
Relevant Information in Support of Decision
Making”
“Decision-Aware Assistance for Analysts at
the Network Edge”
Inter-agent communication using CE
“Agile (Re)Configuration of Services at the
Edge with Humans-in-the-Loop”
A Framework for Coalition Collective
Expressing trust and uncertainty in CE
Presentation of arguments using CE
Using CE and NL processing for
conversational queries
Using CE to express trust and to provide
ITA White Paper for the 2013 BPP
Intelligence
rationale for conclusions
There is further opportunity to collaborate in areas of distributed reasoning, language and
cognition.
Contributors and their Roles
This section should include a list of the key ITA researchers contributing to this task and should
describe their roles and contributions in the proposed research. Clearly show the synergy to be
gained
from
existing
or
proposed
collaborations,
including
US/UK,
academia/industry/government, cross-project, and cross-TA collaborations. You must make sure
you have some cross UK/US collaboration, i.e., UK industry and US academia or vice versa in
the proposal.
Each task should not exceed $400k a year. We don’t release the rates of the ITA partners, but
once you’ve submitted the proposal, we will cost it for you and provide feedback on how much
the proposal will cost.
As a guide for when you are initially putting the proposal together, it should be of the order of 1
full time student, plus not more than 1 person’s effort per year from industry; alternatively a
student plus a combination of effort from industry, a post graduate researcher or a professor
totalling not more than one person’s effort for a year. Other combinations are permitted of
course but tasks that are too big will need to be reworked to come in under the $400k cost.
The research team comprises a mix of US and UK, academic and industrial collaborators. The
work proposes to extend the current P4T2 research, and so the core IBM UK and Boeing
collaborators will continue to be involved in the same roles. IBM UK will continue to lead the
project and perform a significant portion of the research on fact extraction and reasoning using
CE, the extension of CE syntax and semantics, the building of experimental software and its
integration to the ITA Experimental Framework, and the design of the informal evaluation. The
Boeing team will continue to provide input on the use of linguistic theory and reasoning and
conceptual models for fact extraction as well as focusing on the tasks of detecting uncertainty in
sentences and developing the lexicons.
We propose to introduce a new partner, Prof Ann Copestake, a leading authority on linguistics
from Cambridge University, who researched and developed the language system on which the
enhanced linguistic resources from ERG and the Grammar Matrix (as described above) are
based. She will provide consultancy and guidance on the use of these resources and will perform
fundamental research into the deeper links between syntax and semantics. She is also a key
member of the DELPH-IN NL processing consortium, giving us the opportunity to feed back our
work on CE to the wider NL community. Prof Copestake will also offer projects based on this
work for two MPhil post graduate students per year, to run from November to June, supervised
by herself and informally advised (or “uncled”) by members of the team. This will augment the
research with relatively short term and focused projects on aspects of CE for fact extraction
relevant to the milestones of the proposal, as well as helping to provide experience to the
academic community in the techniques being researched.
ITA White Paper for the 2013 BPP
Resources
Please complete the following resources table together with the name and email address of the
task leader. The table will be used to calculate the cost of your proposal. You should identify the
name and affiliated organisation of each contributor to the task together with the number of days
per year they will devote to the research.
In the type column, please identify their role in the organisation. Where possible, use the
following abbreviations:
Academic Principal Investigator
PI
Academic Post-Doctoral Researcher
PDR
Academic Post-Graduate Researcher
PGR
Industrial Principal Researcher
PR
Industrial Senior Researcher
SR
Industrial Researcher
R
Task Leader:
Email address:
Dr David Mott
[email protected]
Assume 220 days = 1 Person Year
Organisation
IBM
Boeing
University of
Cambridge
Name
David Mott, Simon Laws
Stephen Poteet, Ping Xue, Anne Kao
Prof Ann Copestake
Type
PR
PR
PI
Days per Year
177
102
40
Plan for two MPhil postgraduate projects
per year supervised by Prof Ann
Copestake (expenses coming from the
University
of
Cambridge
budget
allocation)
In the following table, list the contact details for other ITA partners that have expressed interest
in collaborating with this proposal but are not to be funded. Note that any Government
collaborators (including TALs) MUST be identified.
Organisation
IBM
University of
Southampton
ARL
CMU
University of
Name
Dave Braines, Stephen Pipes, Saritha Arunkumar
Paul Smart
Cheryl Giammanco
YuQing Tang
Alun Preece
ITA White Paper for the 2013 BPP
Cardiff
Milestones and Deliverables
Please complete the following table with your initial set of research milestones and deliverables
for each quarter of the BPP.
Quarter
BPP13- Q1
BPP13- Q2
BPP13- Q3
BPP13- Q4
BPP13- Q5
Milestone
 Define motivating scenario,
including data sources, overall
flow of extraction and reasoning,
initial specification of evaluation
 Establish principles for mapping
between the Grammar Matrix
(and other resources) and CE
 Incorporate initial set of
linguistic resources (IBM) and
lexicon (Boeing) into CE
framework
 Understand initial relations
between semantics and the
reasoning component
 Establish MPhil project
proposals
 Define mechanisms for using
assumptions for ambiguities,
hypotheses and uncertainties
 Establish mechanisms for
extracting uncertainty from
sentences
 Define set of enhancements to
CE syntax and semantics
 Initial update of Experimental
Framework


BPP13- Q6

Walkthrough of informal
evaluation
Incorporate advanced set of
linguistic resources (IBM) and
lexicon (Boeing) into CE
framework
Incorporate advanced semantic
processing into CE framework
Deliverable
 Report: Motivating Scenario
(IBM, Boeing)
 Report: Initial principles for
mapping external resources into
CE framework (IBM, Boeing)









Demonstration:
ACITA13,
using initial enhancements of
linguistic resources and lexicon
(All, led by IBM)
Report: “Initial review of MRS
and its relationship to CE-based
reasoning”,
(University
of
Cambridge)
Paper: Extracting uncertainty
from
English sentences
(Boeing)
Report: CE-based mechanisms
for handling ambiguity in NL
(IBM)
Paper: “Specific representation
topics in MRS and CE”.
(University of Cambridge)
Paper: Extensions to CE (IBM,
Boeing)
Report: initial feedback on
walkthrough of evaluation
(IBM)
Paper: representing linguistic
resources in the CE framework
(Boeing, IBM)
Demonstration: ACITA14 using
rich explanation, more advanced
ITA White Paper for the 2013 BPP


BPP13- Q7
BPP13- Q8

Techniques for rich explanation
of conceptual model
Establish MPhil project
proposals



Further Update of Experimental
Framework
Preparation for evaluation

Undertake informal evaluation



reasoning and NL capabilities
(All, led by IBM)
Report: rich explanation of
conceptual models(IBM)
Paper: Fact Extraction and
reasoning using CE (IBM,
Boeing)
Paper:
“Integration
of
compositional semantics and
constraints
deriving
from
conceptual models (University
of Cambridge)
Paper: Evaluation results (All,
led by IBM)
Report: Final report (All)
References
[1] Mott, D. (2010). Summary of Controlled English, ITACS,
https://www.usukita.org/papers/5658/details.html.
[2] Sowa, J., Common Logic Controlled English, http://www.jfsowa.com/clce/clce07.htm
[3] Mott, d., Braines, D, Poteet, S., Kao, A., Ping, X., Slidesets of Controlled English, June 2012,
ITACS, https://www.usukitacs.com/node/2071
[4] Mott, D., Braines, D., Poteet, S., Kao, A., Controlled Natural Language to facilitate
information extraction, ACITA 2012, https://www.usukitacs.com/node/2226
[5] Xue, P., Mott, D., Braines, D., Poteet, S., Kao, A., Giammanco, C., Pham, T., McGowan, R.
(2012). Information Extraction using Controlled English to support Knowledge-Sharing and
Decision-Making. In 17th ICCRTS “Operationalizing C2 Agility.”, Fairfax VA, USA, June 2012
[6] Mott, D., Laws, S., Poteet, S., Demonstration of Fact Extraction using Controlled English at
ACITA12, Sept 2012 https://www.usukitacs.com/node/2231
[7] Mott, D., Summary of Communications processing, 08/14/2012,
https://www.usukitacs.com/node/2121
[8] Mott, D., Braines, D., Laws, S., Xue, P. Exploring Controlled English for representing
knowledge in the Linguistic Knowledge Builder, Sept 2012,
https://www.usukitacs.com/node/2231
ITA White Paper for the 2013 BPP
[9] Mott, D., Representing Typed feature Structures in Controlled English, 06/01/2012,
https://www.usukitacs.com/node/2066
[10] Wordnet, a lexical database for English, http://wordnet.princeton.edu/
[11] George A. Miller (1995). WordNet: A Lexical Database for English. Communications of the
ACM Vol. 38, No. 11: 39-41.
[12] Palmer, M., http://verbs.colorado.edu/~mpalmer/projects/verbnet.html
[13] Ann Copestake and Dan Flickinger (2000) An open-source grammar development
environment and broad-coverage English grammar using HPSG In Proceedings of the Second
conference on Language Resources and Evaluation (LREC-2000), Athens, Greece.
[14] LinGO Grammar Matrix http://www.delph-in.net/matrix/
[15] Ann Copestake, Implementing Typed Feature Structure Grammars, CSLI Publications, 2002
[16] DELPH-IN, http://www.delph-in.net/
[17] Cann, R., Formal Semantics: An Introduction, Cambridge University Press, Feb 6, 1993
[18] Ann Copestake, Dan Flickinger, Ivan A. Sag and Carl J. Pollard Minimal Recursion
Semantics: An Introduction, Research on Language and Computation (2005) 3:281-332,
Springer 2006
[19] Mott, D., Poteet, S, Ping, X, Hypothesis Support in Information Extraction, Nov 2012,
https://www.usukitacs.com/node/2235
[20] SIG day - CNL and Pathfinder
https://www.usukitacs.com/sites/default/files/SIG%20day%20%20CNL%20and%20Pathfinder%20v1.1.pdf
[21] de Kleer, J. (1986). An assumption-based TMS. Artificial Intelligence, 28:127-162.
[22] Gabbay, Labelled Deductive Systems: A Position Paper,
http://projecteuclid.org/DPubS/Repository/1.0/Disseminate?view=body&id=pdf_1&handle=eucl
id.lnl/1235423708
[23] Constraint-based Reasoning, Freuder, E., Mackworth, A, MIT Press 1994.
[24] Tate, A. Representing Plans as a set of constraints – the <I-N-O-V-A> model. In the
proceedings of the 3rd International Conference on Artificial Intelligence Planning Systems,
May 1996
ITA White Paper for the 2013 BPP
[25] Jitu Patel, Michael Dorneich, David Mott, Ali Bahrami, Cheryl Giammanco, Improving
Coalition Planning by Making Plans Alive, IEEE Intelligent Systems, 08/14/2012,
https://www.usukitacs.com/node/2126.
[26] Mott, D., Giammanco, C., Braines, D., Dorneich, M., and Patel, D., . Hybrid Rationale and
Controlled Natural Language for Shared Understanding. Proc 4th ACITA London, UK, Sep
2010.
[27] Braines, D., CE Store - Alpha Version 2 (software), ITA, Feb 2012,
https://www.usukitacs.com/node/1670.
[28] Mott, D., The Prolog Analyst's Helper, ITA, Feb 2012,
https://www.usukitacs.com/node/1909.
[29] Dorneich, M.C., Mott, D., Bahrami, A., Patel, J., & Giammanco, C.,. (2011). Evaluation of
a Shared Representation to Support Collaborative, Distributed, Coalition, Multilevel Planning,
ACITA11
[30] Mott, D., The representation of logic within semantic web languages, ITACS
https://www.usukitacs.com/?q=node/4986 August 2009
[31] P. R. Smart, “Controlled natural languages and the semantic web,” School of Electronics
and Computer Science, University of Southampton, Technical Report ITA/P12/SemWebCNL,
2008
[32] Graham, Rimland, & Hall, A COIN-inspired Synthetic Dataset for Qualitative Evaluation of
Hard and Soft Fusion Systems: Proc, 14th international conference on information
fusion. Chicago, IL, 2011
[33] Sag, Ivan A., Thomas Wasow, and Emily M. Bender.. Syntactic Theory: A formal
introduction, Second Edition. Stanford: CSLI Publications [distributed by University of Chicago
Press], 2003