KNOW-HOW MAPPING WITH ME-MAP Eric Yu, Arnon Sturm Daniel Gross, Soroosh Nalchigar, Jian Wang, Sadra Abrishamkar ¨ Knowledge Mapping, Goal-Oriented Requirement Engineering, Modeling, Visualization, Literature Review Outline 2 1. Introduction ¤ What is know-how? ¤ Why it is important to map-out know-how? ¤ Usage of know-how mapping 2. The landscape of knowledge mapping ¤ Desired properties from a knowledge mapping approach ¤ Review of potential approach for mapping out know-how 3. The ME-map approach for know-how mapping ¤ Introduction of ME-map concepts ¤ Guidelines for developing ME-maps 4. 5. 6. 7. Applying the ME-MAP approach Hands-on with ME-map Evaluation Summary and Concluding remarks 1. INTRODUCTION What is know-how? 1. Introduction 4 Know-how: knowledge of how to do something well [Merriam-Webster.com] We are aiming at making know-how reusable : a) more accessible b) of improved quality c) easier to evolve d) more cross-discipline Why it is important to map know-how? 1. Introduction 5 With innovation occurring globally at a fast pace, researchers and practitioners who are pushing the boundaries to better deal with new problems and needs, expend significant efforts to keep up with the current state of the art. ¨ knowledge needs to be managed and maintained to better understand tradeoffs among solutions and identify knowledge ¨ Who will use Know-how Maps? 1. Introduction 6 Researchers (e.g., the AOSE case) ¨ ¤ What research problems/solutions exist in a domain (interdisciplinary domains)? ¤ How good are existing solutions? ¤ What are currently outstanding problems? Practitioners (e.g., the sorting and the big data cases) ¨ ¤ What solutions exist for my current problems? ¤ Better utilization of knowledge resources Examples of Know-how Fragments 1. Introduction 7 ¨ Sorting Algorithms: Know-how for programmers ¤ A list can be sorted using Bubble sort, with complexity O(n2) and constant space. ¤ A list can be sorted using Quick sort, with complexity O(n*log(n)) and (possibly) changing space. (Cormen et al., Introduction to Algorithms, 2009) ¨ Adopting Big Data Tools: Know-how for organizations ¤ Hadoop reduces the cost of ownership and is scalable yet its maturity is questionable. ¤ Google MapReduce is mature and scalable, but the cost of ownership is enormous and its support for online queries is limited. (Eckerson, Big Data and its Impact on Data Warehousing ,2012) ¨ Agent-Oriented Software Methodologies: Know-how for researchers ¤ Tropos supports the development lifecycle and provide means for specifying the social and intentional aspects of MAS applications. ¤ MaSE provides means for specifying social aspects and reactivity, but neglects the intentional aspect. (Sturm and Shehory, A Comparative Evaluation of Agent-Oriented Methodologies, 2004) 2. THE LANDSCAPE OF KNOWLEDGE MAPPING Desired Properties of Know-how Mapping 2. The landscape of knowledge mapping 9 ¨ Ease of Use (EOU): the degree to which the approach is easy to apply, and to the degree to which it is easy to understand the resulting artifacts. ¨ Expressiveness (EXP): the degree to which the approach represents and captures all information relevant for mapping out the knowledge in a domain. ¨ Evolution (EVO): the degree to which the approach supports extending an existing mapping of a domain while adding new relevant knowledge . ¨ Reasoning (RES): the degree to which a resulting mapping of knowledge can be analyzed. This property requires a degree of formality of the representation used to mapping the knowledge. ¨ Process Support (PRS): the degree to which the approach guides constructing a knowledge map. Knowledge Mapping Techniques 10 2. The landscape of knowledge mapping Literature Review – Usually, a narrative description of the domain. ¨ Tabular (TAB) - Tables present selected features in some table dimensions. ¨ Classification (CLS) - is similar to the tabular approach in mapping out knowledge in a domain, but is less detailed. ¨ Hierarchical Value Maps (HVM) - HVMs link through means-ends links product or service attributes, to perceived benefits or costs, and to higher level values that represent customer believes. ¨ Cause maps (CM) - CMs are directed graphs which most often take some form of cause and effect graph. ¨ Conceptual graphs (CG) - CGs are composed of concepts and conceptual links which were formalized to support inference in a knowledge base. ¨ Formal Approaches(F) - are approaches that offer sophisticated logical machinery to represent and analyze knowledge. ¨ Scientific Research Ontology 2. The landscape of knowledge mapping 11 ScholOnto ¨ Cito ¨ Annotation for Understanding Research Papers ¨ ScholOnto 12 2. The landscape of knowledge mapping Shum, S.B., Motta, E., Domingue, J.: ScholOnto: An ontology-based digital library server for research documents and discourse. Int. J. Digit. Libr. 3(3), 237–248 (2000) CiTO 13 2. The landscape of knowledge mapping Shotton, D. (2010). CiTO, the Citation Typing Ontology. Journal of Biomedical Semantics, 1(Suppl 1), S6. Relation Annotation for Understanding Research Papers 14 2. The landscape of knowledge mapping Tateisi, Y., Shidahara, Y., Miyao, Y., and Aizawa, A.(2013). Relation annotation for understanding research papers. In Proceedings of the 7th Linguistic Annotation Workshop and Interoperability with Discourse, 140–148, Sofia, Bulgaria 3. THE ME-MAP APPROACH FOR KNOW-HOW MAPPING The ME-Map Approach - Principles 3. The ME-map approach 16 ¨ Following our observation that in technical domains, the relationship between a problem and a solution – the “means-ends” relationship – is a crucial aspect of knowledge structure of such domains, we focus on: ¤ Explicit and cumulative knowledge in order to make advancements ¤ Reusable know-how fragments rather than a complete situated problem-solving knowledge ¤ Means-end relationships, which, we assert, are at the center of know-how mapping within and across disciplines The ME-Map Approach - Principles 17 3. The ME-map approach Note that a ME-map is an index and not the complete actual knowledge. We tried to address the required properties of: Ease of Use, Expressiveness, Evolution, Reasoning, and Process Support. The Illustrative Domain – Business Process Variability Modeling 3. The ME-map approach 18 The domain deals with ways of modeling variability within business process models. ¨ Model variability can be achieved in two ways: jointly and separately. ¨ When addressing modeling variability jointly this can be done by restriction or by extension. ¨ Additionally, there is a tradeoff among the use of the various approaches. ¨ La Rosa, M., W.M.P. van der Aalst, M. Dumas, F.P. Milani. (2013) “Business Process Variability Modeling: A Survey”, QUT ePrints Reinhartz-Berger, I., P. Soffer, A., Sturm (2010) “Extending the Adaptability of Reference Models”, IEEE Transactions on Systems, Man and Cybernetics - part A (40) 5, pp. 1045-1056. La Rosa, M., W.M.P. van der Aalst, M. Dumas , T. Hofstrede. (2009). Questionnaire-based Variability Modeling for System Configuration. Software and Systems Modeling 8, 2, 251–274. Modeling Business Process Variability Domain The ME-map Approach – Meta Model 20 3. The ME-map approach 1..* Model Element * Context 1..* Reference specification 1..* title link authors date Link validity Named Model Element xor name * 1 Task-‐based Link source * * Acheived By Link ¨ ¨ Consists Of Link 1 target Quality Task 1 1 source source 1 * * * Association Link Contribution Link target 1 source target 1 * * Extended By Link The approach is inspired by Goal-Oriented Requirements Engineering. The ME-map builds upon a subset of i*. Task 21 3. The ME-map approach - Concepts Definition: A task can be interpreted either as a problem or a solution. Graphically, it is indicated as a rectangle with rounded corners. ¨ Formulation: Task should be defined as a simple verb phrase and avoid conjunction. ¨ Task 22 ¨ 3. The ME-map approach - Concepts Mapping and Action Guidelines: ¤ Any explicated goals, objectives, solutions should be mapped into tasks. ¤ Artifacts can also be transformed into active tasks. ¤ When identifying a solution consider what the problem it addresses and vice versa. Achieved by link 3. The ME-map approach - Concepts 23 ¨ ¨ Definition: An achieved by link represents a means-end relationship. It indicates that the target task is an alternative to achieve the source task. Formulation: Note that all means to achieve an end are alternatives which means that each of the alternative fully addresses the functional task. Thus, this implies “or” relationships among the alternatives. Achieved by link 3. The ME-map approach - Concepts 24 ¨ Mapping and Action Guidelines: ¤ Look for alternatives ¤ Look for sentences consisting of the terms like: Means end, Solves, Addresses, …. Consists Of Link 25 ¨ 3. The ME-map approach - Concepts Definition: A consists of link indicates that the target task is part of the source task, and that all connected target tasks should be accomplished in order to fully satisfy the source task. That means, that it is actually a problem or solution decomposition and all parts are required to fulfil the source task. Consists Of Link 3. The ME-map approach - Concepts 26 Formulation: Note that all parts of the whole task are required to be achieved. Thus, this implies “and” relationships among the parts. ¨ Mapping and Action Guidelines: ¤ Decompose the tasks ¤ Look for phrases consisting of the following: Part of, Consists of, Comprises, Include(s/ing) ¨ Quality 27 Definition: A quality is a characterization of a task. Graphically, it is indicated as an ellipse. ¨ Formulation: ¤ A quality should be defined as a simple adverbial phrase. Can use both adverb (-ility) and adjective (e.g. scalable) depend on what is common in the domain. ¨ Mapping and Action Guidelines: ¤ Look for criteria for assessing a task. ¨ 3. The ME-map approach - Concepts Association Link 28 The ME-map approach - Concepts Definition: An association links is used to connect qualities with tasks. It is specified by an unlabeled and non-directional line between a task and a quality. Its semantics implies that the qualities associated should be taken into account when evaluating alternatives for that task. ¨ Formulation: Note that qualities associated with a task affect also other tasks that further refined it via the achieved by links chain. ¨ Contribution Link 3. The ME-map approach - Concepts 29 Definition: A contribution link indicates that the source task (or quality) has a contribution to a quality. ¨ ¤ The contribution could be either positive or negative. Yet, in case quantitative metrics are provided, this can be incorporated as well. ¤ Optionally, a contribution link may hold its validity in terms of how the contribution was determined. These include various levels such as self-subjective evaluation, self-claim based evaluation, empirical evaluation, and external evaluation. Contribution Link 30 3. The ME-map approach - Concepts Extended by Link 3. The ME-map approach - Concepts 31 Definition: An extended by link indicates that the target task is an extension of the source task. ¨ Formulation: ¨ ¤ Note that the qualities associated to the extended task apply also to its extensions. n The default of the contribution links from the extended task are the same for its extensions. n Yet, these could be overwritten by specifying new contribution links from the extensions to the related qualities. ¤ Essentially, an extended by link is a shortcut to have another alternative with repeating multiple contribution links. ¨ Mapping and Action Guidelines: ¤ When introducing a task check whether it uses as a base another task. Extended by Link 32 3. The ME-map approach - Concepts References and Contexts 33 3. The ME-map approach - Concepts Reference ¨ Definition: A reference is an information source that provides justification for the elements within the map. ¨ Formulation: Each element within the map should be associated with at least one reference. Context ¨ Definition: A context is a descriptive information which states the conditions and settings in which the element was determined. A context may be an experimental setting, a data set, assumption etc. ¨ Formulation: Should be concise. Evaluation: Procedures 3. The ME-map approach 34 We conducted evaluations in various forms ¨ ¤ Apply the approach to various domains. ¤ Iteratively revisit the approach in light of the desired properties. ¤ Check the way people understand ME-map. ¤ Check the way people develop such maps and their perception over such maps. Evaluation: Results 3. The ME-map approach 35 Comprehension of a domain seems to be better than “regular” means it term of increased quality and decreased time. ¨ While constructing ME-maps, it is easy to follow the approach concepts and it was found to be useful in terms of understanding domains. ¨ ¤ Nevertheless, more (concrete) guidelines are required. 4. APPLYING THE ME-MAP APPROACH Domains Explored To-Date 4. Applying the ME-Map approach 37 ¨ Explored know-how mapping in several domains and at different levels of abstraction ¤ Geo-Engineering ¤ Web Data Mining ¤ Goal-Oriented Software Architecture ¤ Big Data ¤ Agent-Oriented Software Engineering ¤ Knowledge Mapping ¤ Data Mining ¤ Architecture Description Languages ¤ Variability in Business Processes ¨ Minimal set of concepts was so-far sufficient ¤ to identify problems and associated qualities ¤ solutions to these problems and their evaluation ¤ opportunities for innovation The Domain of ADL 38 4. Applying the ME-Map approach Connecting the Maps 4. Applying the ME-Map approach 39 The data mining domain The marketing domain 40 40 Tools 4. Applying the ME-Map approach 41 ¨ At this stage we are using a concept map tools called CMAPTools: http://cmap.ihmc.us/ ¨ We are currently developing a mapping framework that would better allow us to accomplish the goals of using ME-maps KNOW-HOW EVOLUTION Solution Domain-Classification in DM 43 43 Top Data mining application domain is CRM 44 Source: http://www.kdnuggets.com/polls/2012/where-applied-analytics-data-mining.html 44 Problem Domain- Customer Relation Management 45 linked to the previous map 45 5. HANDS-ON An Example…. 5. Hands-On 47 ¨ A Model-Driven Approach to Enterprise Data Migration Raghavendra Reddy Yeddula, Prasenjit Das, and Sreedhar Reddy In a typical data migration project, analysts identify the mappings between source and target data models at a conceptual level using informal textual descriptions. An implementation team translates these mappings into programs that migrate the data. While doing so, the programmers have to understand how the conceptual models and business rules map to physical databases. We propose a modeling mechanism where we can specify conceptual models, physical models and mappings between them in a formal manner. We can also specify rules on conceptual models. From these models and mappings, we can automatically generate a program to migrate data from source to target. We can also generate a program to migrate data access queries from source to target. The overall approach results in a significant improvement in productivity and also a significant reduction in migration errors. 48 5. Hands-On And now…Briefing 5. Hands-On 49 You got 3 abstracts from the conference proceedings. ¨ Map out these abstracts using the concepts of the ME-map. ¨ One can start with allocating the important sentences or adopt a more holistic approach and abstract the main concepts to be mapped. ¨ Start with the first abstract and will monitor the progress as we go. ¨ We will then initiate a reflection session over your experience. ¨ At this stage for developing the maps we are using concept maps – and their associated tool – CMAPTOOL: http://cmap.ihmc.us/ ¨ In the exercise we will develop the map by hand. ¨ Reflection on the ME-map Construction 5. Hands-On 50 The ME-map is easy to use when developing a new map. ¨ The ME-map is useful and helpful for mapping out research outcomes. ¨ Using the ME-map it is easy to construct a map of a single paper. ¨ Using the ME-map it is easy to construct a map of multiple papers. ¨ The ME-map facilitates better organization of the knowledge. ¨ The ME-map explicates differences among studies. ¨ The ME-map facilitates grouping or clustering studies. ¨ The ME-map helps in identifying important properties and features. ¨ The ME-map encourages critical thinking towards a research. ¨ 6. EVALUATION Evaluation 6. Evaluation 52 We conducted evaluations in various forms ¨ ¤ Apply the approach to various domains. ¤ Iteratively revisit the approach in light of the desired properties. ¤ Check the way people understand ME-map. ¤ Check the way people develop such maps and their perception over such maps. Comprehension Evaluation - Objectives 6. Evaluation 53 ¤ Evaluate the effectiveness of understanding a knowledge domain (maps vs. written review) ¤ Compare the two approaches in terms of level of understanding of the domain, and relative time to find answers. ¤ Evaluate the effects of participant’s familiarity with goal-oriented concepts and modeling. ¤ Analyze the extent to which participants understand a map generated by others. Comprehension Evaluation – Hypotheses 6. Evaluation 54 ¤ ME-Map is an effective way of knowledge transfer in terms of: n Domain comprehension n Time efficiency ¤ ME-Map is able to capture and transfer knowledge in aspects of: n Finding problems, finding alternatives, evaluating alternative solutions, evaluating solutions’ effects, and finding knowledge gaps. ¤ Participant’s familiarity with goal-oriented concepts can have positive affects on the performance over time. Comprehension Evaluation – Method 6. Evaluation 55 Questionnaire ¨ Subjects ¨ Participant Scholarly Search Goal Oriented Modelling Concepts Literature Reviews Web Mining Domain Knowledge Mapping G0 1 Usually Unfamiliar N/A Unfamiliar Somewhat familiar G0 2 Usually Unfamiliar N/A Unfamiliar Unfamiliar G0 3 Usually Unfamiliar N/A Unfamiliar Unfamiliar G0 4 Every day Unfamiliar N/A Unfamiliar Unfamiliar G1 1 Usually Very familiar N/A Very familiar Somewhat familiar G1 2 Usually Very familiar N/A Unfamiliar Unfamiliar G1 3 Usually Very familiar N/A Very familiar Very familiar G1 4 Usually Very familiar N/A Very familiar Very familiar W 1 Occasionally N/A Unfamiliar Unfamiliar Unfamiliar W 2 Occasionally N/A Very familiar Somewhat familiar Very familiar W 3 Usually N/A Very familiar Somewhat familiar Somewhat familiar W 4 Occasionally N/A Somewhat familiar Somewhat familiar Unfamiliar Comprehension Evaluation – Results 6. Evaluation 56 00:36:00 100% 90% 00:28:48 80% 70% 00:21:36 60% 50% 00:14:24 40% 30% 00:07:12 20% 10% 00:00:00 0% Average Time Average mark G0 G1 W Average performance score across the groups G0 G1 W Average time spent across the groups 56 Comprehension Evaluation – Results 6. Evaluation 57 100% Overall feature performance results 90% 80% 70% 60% G0 50% 40% G1 30% W 20% 10% 0% Finding Alterna8ves Finding Problems Evalua8ng Alterna8ve Evalua8ng Solu8on Effects Finding Knowledge Gap Solu8ons 00:04:54 00:04:11 Overall feature time results 00:03:27 00:02:44 G0 00:02:01 G1 W 00:01:18 00:00:35 Finding Alterna8ves Finding Problems Evalua8ng Alterna8ve Evalua8ng Solu8on Effects Finding Knowledge Gap Solu8ons 57 Comprehension Evaluation – Back to the Hypotheses 6. Evaluation 58 ¨ Domain comprehension and scores: ¤ ME-Map approach has some advantage on domain understandability and comprehension. ¨ Time: ¤ The ME-Map approach facilitates and speeds up information finding and domain understanding and learnability. 58 Comprehension Evaluation – Threats to Validity 6. Evaluation 59 Limited number of participants ¨ A single domain ¨ Construction Evaluation – Objectives 60 6. Evaluation (G1) Validate whether ME-map provide sufficient expressiveness for mapping a literature review (i.e., a domain). (G2) Examine the easiness of mapping a literature review using ME-map. (G3) Check whether literature review mapping using MEmap provides additional insights for the understanding and analysis of the domain. (G4) Explore whether ME-map facilitates the positioning of a research agenda. Construction Evaluation – Procedure 6. Evaluation 61 Participants: 4 graduate students Duration: a whole day Preparation: The students were asked to gather literature regarding their own research domain. Training: 1hr Tasks: n n n Mapping: n map out the literature they had gathered. n position their own research on that map. n search for additional alternatives that were triggered by examining the previous map. Questionnaire Discussion Construction Evaluation - Results 6. Evaluation 62 ¨ The quality of the maps increases as the evaluation progress. ¤ In ¨ the beginning the participants neglect qualities. The characteristics of the domain also affected the way the map were constructed: ¤ Software Ecosystems – not clear what the qualities are, as the domain is in its incubation stage. ¤ Organizational Flexibility – the focus here was on qualities as the meansend chain was less visible. ¤ Data Mining for Business Application – the means-end chain was much clearer. Construction Evaluation – Results 63 Issue/Participant The ME-map is useful and helpful for mapping out literature review in terms of research objectives. The ME-map is useful and helpful in facilitating gap identification within the literature review. The ME-map is useful and helpful in facilitating identification for future research related to the literature review. The ME-map is useful and helpful for identifying other domains that can potentially contribute to addressing problems in the domain that you are focusing on. Using the ME-map it is easy to construct a map of the literature review. The ME-map is easy to use when developing a new map. The ME-map is easy to use when maintaining or modifying an existing map The ME-map facilitates new insights regarding the literature review. The ME-map facilitates better organization of the knowledge of the literature review. The ME-map explicates differences among studies within the literature review. The ME-map facilitates grouping or clustering studies within the literature review. The ME-map helps in identifying important properties and features within the literature review. The map helps in better positioning your own work. The ME-map encourages critical thinking regarding the literature review. 6. Evaluation 1 2 3 4 1 6 5 6 5 3 5 5 1 5 5 5 3 4 6 4 6 7 1 4 6 6 4 6 7 7 5 6 7 5 7 6 7 7 7 5 5 7 7 2 5 5 4 4 5 7 5 3 6 7 7 6 6 5 7 2 Construction Evaluation – Results 6. Evaluation 64 ¨ ¨ ¨ ¨ ¨ ¨ Modeling notation vs. modeling process: although the approach facilitates the critical thinking and the identification of important aspect, it lacks the proper notations to indicate some other aspects, such as: marking your research contribution, indicating important references, and differentiating between problems and solutions. Usability: all participants stated that the approach is usable and easy to learn as it consists of a minimal set of mapping constructs. Usefulness: the approach provides a clear visualization of the state-of-the-art and the model can facilitate various analysis. In addition, it further helps in explicating the contribution of your own research. Respondents felt that the maps could foster collaboration among researchers. Expressiveness: with respect to expressiveness a concern was raised as to how to specify dependencies among qualities and tasks. Sometimes, different combination of qualities values can affect differently on other qualities. Another issue was the ability to determine the progression over time. Mapping: sometimes literature does not follow the means-ends structure thus making the mapping challenging. Another issue is how to determine the degree of contribution, since it may be subjective. It was suggested that a binary distinction between positive or negative contribution may be sufficient for the purpose of distinguishing between solution contribution to problem qualities Scalability: it is not clear, what are the available means for managing complex maps. Construction Evaluation – Back to the research objectives 6. Evaluation 65 It seems that although the domains were mapped correctly, some of the ME-map expressiveness features (G1) require further exploration (e.g., complex contribution and contribution level). ¨ The ME-map approach was found to be easy to use (G2). ¨ The ME-map approach offers additional insights for the understanding and analysis of existing literature reviews (G3). ¨ The ME-map approach facilitates the positioning of a research agenda (G4). ¨ Construction Evaluation – Threats to Validity 66 6. Evaluation Although the evaluation provided us with important insights, the results should be taken with caution due to the following reasons: ¨ the number of participants was low, nevertheless generated a large number of comments. ¨ the participants (being students) are somewhat depended on the researchers, though they were encouraged to provide critical remarks to help improve the approach, which they did. ¨ the selection of the domains and sources affects the results; we therefore referred to the characterizations of the domains and sources in our evaluation. SUMMARY AND CONCLUDING REMARKS The Way Forward 68 Provide additional guidelines for map developers. ¨ Improve scaleability of managing such maps. ¨ Implementing a framework to support the ME-map approach. ¨ Examine the notion of social know-how modeling ¨ ¤ What is the required expressiveness for crowd modeling of know-how? ¤ Trust management ¨ Develop tools for knowledge acquisition ¤ E.g. semi-automated extraction and integration of know-how structures from knowledge sources Devise reasoning mechanisms ¤ Identify gaps ¤ Tailor context-specific solutions ¨ Evaluate, Evaluate, Evaluate ¨ References 69 Arnon Sturm, Daniel Gross, Jian Wang, Soroosh Nalchigar, Eric S. K. Yu: Mapping and Usage of Know-How Contributions, CAiSE Forum 2014 post proceedings. ¨ Arnon Sturm, Daniel Gross, Jian Wang, Eric S. K. Yu: Analyzing Engineering Contributions using a Specialized Concept Map. CAiSE (Forum/Doctoral Consortium) 2014: 89-96 ¨ Jian Wang, Arnon Sturm, Daniel Gross, Eric S. K. Yu: Know-How Mapping: From i* to ME-maps. iStar 2014 ¨ Daniel Gross, Arnon Sturm, Eric S. K. Yu: Towards Know-how Mapping Using Goal Modeling. iStar 2013: 115-120 ¨ Questions??? 70 Eric Yu, Faculty of Information, University of Toronto, Canada [email protected] Arnon Sturm - Information Systems Engineering, Ben-Gurion University of the Negev, Israel [email protected] Daniel Gross, Faculty of Information, University of Toronto, Canada [email protected] Soroosh Nalchigar, Computer Science Department, University of Toronto, Canada [email protected] Jian Wang, State Key Lab of Software Engineering, Wuhan University, China [email protected]
© Copyright 2026 Paperzz