ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT Artifact#4a: Research Knowledge Assessment Roxanne Sylvester Walden University Applied Research in Education EDUC 8102-12 Dr. Jenelle Braun-Monegan December 12, 2010 ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT Abstract This paper provides an overview of philosophical framework of research, core research concepts, and information on various research designs and approaches. The philosophical framework explains the concept of scientific realism, social constructivism, pragmatism, and advocacy/liberatory utilizing real life situations that frame philosophical ideas and trends. The differences and similarities of theoretical and conceptual frameworks are outlined to explain phenomena or variables. Since the quality of a research study hinges critically on the rationale and purpose of the qualitative or quantitative sampling approach, the topic of sampling and population was addressed. More importantly, the information provided will be useful for the beginning researcher to understanding the elements that frame research study, as well as, the importance of research in the improvement of knowledge and the enhancement of practices. 2 ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 3 Research Knowledge Assessment Philosophy of Research Research is a systematic process of collecting and analyzing data for a specific purpose or measured phenomenon. Educational research has affected ideas about education and the practices used to achieve objectives in education. For instance, the No Child Left Behind (NCLB) enactment in 2001 has improved education for underprivileged students by holding national schools accountable for “monitoring and reporting student progress based on test scores”(Lodico, Spaulding, & Voegtle, 2010, p.2). The philosophical framework of educational research ideas is “scientific and disciplined inquiry (social science) using quantitative and qualitative approaches” (Mills, 2002, p.7). Researchers use basic, applied, and evaluation research to develop and change theories and exhibit the value of theories in research practices. Research provides valid information and knowledge about education to make informed decisions. According to Mills (2002), educational research has become a valuable source of information to a number of diverse groups. For example, educators are constantly trying to understand educational processes and must make professional decisions to influence students, teachers, parents, and the community. Researchers make decisions by relying on experience, expert opinion, tradition, feeling, and beliefs. Non-educational policy groups, such as state and federal legislatures and courts, have increasingly mandated changes in educational policies. Reviews of prior research have interpreted empirical evidence. For example, studies on retention indicate that retaining a child in a grade serves few educational purposes. (p. 5). ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 4 As a budding researcher, I questioned how empiricism relates to quantitative research methodology. An explanation was revealed in Long’s (2010) lecture on the development of philosophical frameworks and approaches that inform research methods. He indicated that empiricism is learned through observation and experience. According to Hergenhahn (2001), empiricism evolved during the modern period and gained influence and privilege as an epistemology (knowing through the experience), which was associated with sensory experience (knowing through observation). This form of empiricism was popularized before modern science emerged. In the modern period, logic and empiricism dominated the epistemological position previously held by religion. Empiricism and reason formed the new paradigm (modernism) and scientific empiricism resulted from the merger. In applying the principles of logic to the empirical process, a scientific method emerged and cultivated the scientific revolution of the modern period. This modernist approach to knowing maintained control until the postmodern period emerged. Postmodernism questioned any singular epistemology having a privileged position. (Hoffman, 2005). Scientific realism is connected to positivism (also connected to empiricism) and the assumption that “external things, the natural world or objects around us, influence our perceptions and understanding of the world” (Long, 2010). Mills (2002) explained that quantitative research is usually based on some form of “logical positivism,” which assumes there are stable, social facts with a single reality, separated from the feelings and beliefs of individuals. Ultimately, realists are inclined to engage in quantitative techniques when performing research. ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 5 Educational research on schools seeking to improve educational practices can be obtained through “systemic methods and techniques” (Lodico et al., 2010, p.4) such as scientific method, positivism, and post-positivism research. Using scientific method, researchers apply the following scientific process: 1) asks a question inquiry, 2) collects new data, 3) analyze the data, 4) interprets and reports findings, 5) generates a new question to investigate next. (Lodico et al., 2010, p.4). The concept of positivism is important to research because it is not based on assumptions; instead, it relies on positive facts or sensory objects. For instance in social science, positivism promotes testing of theory, recording and measuring observations, and making conclusion to determine if the observations or measurements support the theory. (Long, 2010). Post positivism on the other hand, emphasizes that knowledge and observation is not absolutely trustworthy, but still focuses on achieving objectivity utilizing “multiple measures and observations and triangulates those measures and observations across multiple sources” (Long, 2010) Objectivity refers to data collection and analysis procedures from which a reasonable interpretation can be made; and the quality of the data produced by procedures, which either control for bias or take into account subjectivity. For example the objective of No Child Left Behind legislation, is to make schools “accountable for monitoring and reporting student progress based on test scores…to increase achievements for America’s students” (lodico et al., 2010, p. 2). The importance of objectivity in research is critical when conducting and presenting research for the following purposes: 1) to identify the facts based on credible information and scientific method, and 2) to interpret the results of data analysis based on the observable facts of ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 6 the findings, presented factually, and not on our own subjective opinions. The challenges of objectivity are undertaking a detached role to maintain a clear objective and reliable conclusion. Grasping the philosophical frameworks for educational research is pivotal for a research novice. Speaking from experience, learning the different approaches and considering which approach influences my assumption of real world situations is testing. However, reviewing Lodico et al. (2010) and Long (2010) information on the philosophical developments of scientific realism, social constructivism, advocacy/liberatory, and pragmatism facilitated my understanding of knowledge-oriented approaches of scientific realism and social constructivism, and actionoriented approaches of advocacy-liberatory and pragmatism. The scientific realism of quantitative approach and the social constructivism of qualitative research capture assumptions about real life situations. Scientific realism researchers “answer research questions by producing numerical data that represent various constructs and variables… to establish cause-and-effect (Lodico et al., 2010, p. 6). For example in NCLB, researchers use annual testing, academic progress, report cards, and teaching quality to produce quantitative data. Social constructivism researchers dispute the findings of scientific realist and “use data collection methods that bring them closer to the participants using techniques such as in depth observations, life histories, interviews, videos, and pictures” (Lodico et al., 2010, p. 8). Paulo Freire (1921-1997) (as cited in Lodico et al. 2010) based his philosophical studies on the assumptions of advocacy or liberatory frameworks, which states “there are multiple possible realities that are dependent on social, political, and economic contexts” (Lodico et al., 2010, p.8). He also believed that research should provide “freedom from oppression and ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 7 debilitating living environment” Lodico et al., 2010, p. 9).This approach utilized mixed-methods (qualitative and quantitative) to collaborate with participants and collect data. Researchers focus on educating to produce knowledge and empowering people to use their voices to accomplish social change. Case in point, Athanases and Larrabee’s (2003) (as cited in Macgillivray & Jennings, 2008) investigation of the methods of instruction in teacher preparation coursework and the students’ response to instruction regarding gay issues in schools, promoted an advocacy/liberatory stance toward gay and lesbian students. (p. 176). The pragmatism approach also uses a mixed- method approach to identify what works in a given situation. In other words, find the problem, evaluate the alternatives, and come up with a solution. According to Lodico et al. (2010) “it does not matter if there is a single reality or multiple realities as long as we discover answers that help us do things that we want to do”(p.9). As a researcher, I am most concerned with understanding what works, and why or if something does not work. In research, theoretical and conceptual frameworks are key points in social and behavioral science. Long (2010) explained that a framework comprises of theories and concepts that explain phenomena or variables. In the social sciences, conceptual frameworks are created or designed to bring together a set of loose ideas or concepts, which helps to establish possible relationships. Using examples of data collection and analysis to assess student learning, Long (2010) explained that the concept includes cognitive, skills, attitudes, and values outcomes to develop causal relationship and research process. Conceptual framework is associated with qualitative research, and contributes to theory and theoretical frameworks by establishing possible relationship. In contrast, theoretical frameworks and theory is associated with ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 8 quantitative methods and attempts to establish causal relationship between ideas or concepts. For example, X causes Y, or Y is part of process caused or explained by X. (Long, 2010). Both theoretical and conceptual frameworks relate to research studies to develop research questions, test and develop hypotheses, and create or reinforce change. Core Concept for Research Design The core concept of research is to advance knowledge and improve practices. Reviewing the structure of research study will help beginning researchers to understand the structure, components, and variations of research articles. So let us take a journey to learn the core concepts of research and understand the research process. Before discussing the reason for identifying a research question, we must identify the important characteristics of quantitative and qualitative research and site these characteristics within the research process. “In this way, you will identify the differences between the two approaches in a meaningful way rather than simply learning the characteristics in an abstract sense” (Mills, 2002, p. 55). Understanding the characteristics allows beginning researchers to make the right decisions about how to proceed (will you write research questions or hypotheses) and how to design the research study. Knowing the characteristics allow you to recognize quantitative or qualitative research studies. Normally, research studies are developed from a research question that can be answered through data collection. “Researchable questions should also clearly define the variables (quantitative research) and identify themes, processes, and meaning (qualitative research) being investigated” (Lodico et al., 2010, p.27). In educational research, Lodico et al (2010) explained that hypothesis provides “a tentative explanation that can be tested by collecting data” (p.5) in ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 9 qualitative research. For instance, Arlin can hypothesize that when students participate in a 3-day training workshop on diversity education, there is no significant increase in their acceptance of diversity on a college campus. This hypothesis will be supported by the data Arlin collects. In quantitative research, Mills (2002) explained that researchers study problems to identify, describe, or explain trends that need to be developed for relationships among variables. Describing a trend means that the research problem can be answered by a study where the researcher seeks to establish the overall tendency of responses from individuals and to note how this tendency varies among people. For instance, a researcher “may seek to learn how voters describe their attitude toward a bond issue. The results from this study informed the researcher about how a larger population views an issue and how diverse their views are about the issue” (Mills, 2002, p. 57). In contrast, Mills (2002) continued to explain that qualitative research examines a research problem in which the researcher explores and seeks to understand a central phenomenon. An exploration means that little is known in the literature about the phenomenon of study and the researcher needs to learn more from participants. Case in point, researchers may not know much about how deaf children think when using sign language, and qualitative research can explore this phenomenon from the perspective of the children. To understand the concept, the researcher needs to learn the complexity of the phenomenon. The process of thinking when using sign language is a complex idea and aspects need to be studied qualitatively. “The research problem of learning how best to teach children who are deaf requires both an exploration and an understanding of qualitative research about how the children think” (Mills, 2002, p. 57). ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 10 Once the problem is identified, the researcher justifies the research problem by reviewing the literature (reading topic related articles). With the information gathered from reading primary or secondary sources, the researcher can think about specifying the “purpose” for the research study. In quantitative research, the researcher asks specific, narrow questions to obtain measurable and observable data on variables. “The major statements and questions of direction in a study—the purpose statement, the research questions, and the hypotheses—are specific and narrow because the researcher isolates a few variables to study” (Mills, 2002, p.59). In qualitative research, the purpose is much more open-ended than in quantitative research. Researchers ask general, broad questions to learn from participants. The purpose statement expresses the direction for researchers identify a single phenomenon of interest. For example, in a qualitative study examining the “professionalism” of teachers, the investigator may ask high school teachers, “What does it mean to be a professional?” The question focuses on understanding the ideas of being a professional; the responses yield the data for qualitative research. After identifying the research topic and questions, the researcher should proceed to conduct a literature review for the proposed study. Literature review summarizes and synthesizes relevant literature for a particular research problem and provides insight on the study. The purpose of the literature review is to gain knowledge on the subject matter, which is used to define the problem, develop the research design, relate the results of the study to historical data, propose further research, and develop a hypothesis. Literature for a review includes many types ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 11 of sources: articles in professional journals, reports, scholarly books and monographs, government documents, dissertations, and electronic resources. According to Lodico et al. (2010), effective researchers know how to use the source to conduct quality literature review. The authors stated, “Sources for a research proposal are considered strongest if they are peer reviewed…and rated the quality of the methods used in them” (p. 32). The majority of the articles reviewed should be selected from empirical research or primary sources (original research done by the author or authors), preferably in the last 5 years. Information gathered from secondary sources (articles written by someone describing original research) such as books, “describes empirical research, but it usually provides less information on the research methods and so is usually considered to be a weaker type of source” (Lodico, et al., 2010, p. 32). The quality of the information receive from non-empirical articles, expressing opinion or commentaries, must be scrutinized for credibility. Sampling and population techniques relate to quantitative and qualitative methods and examine the role random and non-random techniques play in creating a sample. In his lecture on “Sampling Rationale” Long (2010) implied that sampling allows the investigator to explore a particular group of individuals or organizations…“to curtail the size of the population so that time, resources, and money are not barriers to complete the research” (chap. 2.3). The population represents the entire group that is being considered for a research study. The sample is the portion of the population that was selected for the study. Sampling differs in both quantitative and qualitative methods. In terms of quantitative research, sampling methods can be useful to generalize a larger population and is attained ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 12 through random sampling. Let us consider this scenario: John decides to test second-year university students at 15 different colleges, who were required to take a 10-week diversity training course, will improve their sensitivity to diverse cultures, races, and ethnicities. In order to have a strong possibility at generalizing to a larger population, John needs to randomly select students when considering his sampling methods. Randomization provides a snapshot of the larger group or population. Concerning qualitative sampling, the method is useful to purposefully selecting relevant participants. For example, Amanda decided to conduct a study investigating the educational practices of Australian aboriginal families. Amanda uses purposeful sampling to collect data to support her hypothesis. Researchers must be familiar with the ideal and realistic population. In research study, the term variable refers to a “characteristic or attribute of an individual…that can be measured or observed and that varies among the people…being studied” (Creswell, 2009, p.49). Independent variable is the cause variable or the variable you are examining to identify causal differences. Dependent variable, on the other hand, is the effect variable. For instance, Armanda decides to investigate whether or not giving a math tutorial will improve students’ performance on the Math Assessment Inventory overall by a 10% increase in baseline scores. Clearly, a math tutorial can be used as an example of using an independent variable to identify probably cause for improvement. Through observations, individual interviews, and other field methods, she plans to collect and then analyze the data and report her findings. ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 13 The concept of validity in experimental research is shared by internal and external validity. Internal validity refers to the influence of experimental manipulation on the differences in dependent variable. However, external validity is the degree to which the results are generalized. According to Mills (2002), validity focuses on “ensuring that the data collected accurately gauges what the researcher is trying to measure” (p.175). For example, when using standardized instruments, teachers will ask, “Are the results of the California Achievement Test really valid…or will open ended assessment strategies really measure their students’ ability” (Mills, 2002, p.161). These questions focus on identifying how test scores accurately gauge what the teachers are trying to measure. Experimental research and designs include in-depth discussions about internal and external validity to employ techniques to maximize the validity and reliability of the results through objective means. Non-experimental approaches employ causal-comparative research to identify if independent variables may cause a change in a dependent variable. Reliability, on the other hand, examines the repeatability of the research measures. Focusing on consistency, researchers hope that the data measures what is attempted to measure with the same outcome repeated. For example, John measured his intelligence and scored a 120; if John repeated the same test, a reliable IQ test would generate the same scored. Additionally, grading rubrics can evaluate student’s performance and to support “the reliability of practitionermade measure” (Lodico et al., 2002. P. 134). The concepts of validity and reliability directly impact awareness of the quality and viability of data collection. ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 14 Subsequently, assumptions, limitations, and delimitations are considered in research for ethical and legal considerations for human beings, the public nature of education, the complexity of educational practices, and methodological limitations. Ethically, researchers should recognize their own personal biases and develop an ethical perspective that ensures they will do the right thing when confronted with a difficult ethical dilemma. The Institutional Review Board (IRB) was designed to ensure that all researchers adhere to key ethical and legal principles when conducting research. Both quantitative and qualitative research adheres to ethical consideration surrounding informed consent, protection from harm, and confidentiality. Finder (1992) (as cited in Mills, 2002) suggested “utilitarian, deontological, relational, and ecological ethics” (Mills, 2002, p. 173) as a useful framework for channeling ethical conduct in qualitative research. The American Educational Research Association and the American Psychological Association have established ethical standards and requirements for action researchers. The topics include The AERA ethical standards contain 45 standards that are organized into six topics. The topics include (a) responsibility to the field; (b) research populations, educational institutions, and the public; (c) intellectual ownership; (d) editing, reviewing, and appraising research; (e) sponsors, policy makers, and other users of research; and (f) students and student researchers. Individuals who are interested in conducting action research should read and understand the AERA standards. As with the AERA standards, those who are interested in conducting action research should review the APA standards in depth. The APA also includes six general principles ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 15 that should guide one who is considering conducting research. These principles include (a) competence, (b) integrity, (c) professional and scientific responsibility, (d) respect for people’s rights and dignity, (e) concern for other’s welfare, and (f) social responsibility. (Mills, 2002, p. 455-456). Other Approaches to Research Design Evaluation occurs in daily life; in schools teachers assess students’ skills in mathematics, reading, and social science to meet performance standards; at work employees’ performance are evaluated to determine if they need to make improvements. Social Service and education programs use evaluations to improve services, and document the effectiveness of the provided services. In other words, evaluation is a process of determining if something is effective. Therefore, program evaluation “examines programs to determine their worth and to make recommendations for programmatic refinement and success” (Lodico et al., 2010, p. 317). The goal is to provide feedback, design new programs, and make changes in existing approaches. Subsequently, program evaluation is categorized in two types of data collection: formative or summative, depending on the goal. According to Lodico et al., (2010) the goal of formative evaluations is to implement new programs or to make changes in an existing one. On the other hand, the goal of summative evaluations is to describe the implementation of a program and its effects on program participants. Formative and summative evaluation can be conducted in both quantitative and qualitative studies to collect data based on the rationale and the audience of the evaluation. Summative data is collected from standardized test scores (FCAT), surveys, and interviews t; the ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 16 findings are reported to the client at the end of the project or project agreement. In contrast to summative data collection, formative data is collected and reported to the client throughout the duration of the evaluation. The data for this evaluation is collected from memos or presentations. Overall, program evaluation id designed to determine the effectiveness of the program and decide if the program should be continued or discontinued. (Long, 2010). Lodico et al. (2010) discussed four types of program evaluation, which include objectivebased approach, goal-free evaluation, expertise-oriented evaluation, and participant-oriented evaluation. The advantages and disadvantages between program evaluation approaches are as follows: Objective-based approach is written statements describing the purpose of the evaluation and clearly defining the type of information to be collected. Although this approach will assist in determining the evaluation, evaluators may concentrate on the objectives and ignore other benefits of the findings. Goal-free evaluation is guided by the perception that the findings do not meet the established goals and objectives. These evaluators utilizing this concept think that the unexpected result is more important than the established outcome. The disadvantage of conducting this form of evaluation is that the funded projects are required to show result based on objectivity. Expertise-oriented evaluation is mainly used in project evaluation. Evaluator judges the program or service based on established criteria and their expertise in the area. ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 17 Participant-oriented evaluation differs from the other approaches because the evaluator involves participants in the evaluation of the program. “The participants develop instruments, collect data, analyze data, and report findings. (Lodico, 20010, p. 326). According to Mills (2000) (as cited in Lodico, Spaulding, Voegtle, 2010) action research is “any systemic inquiry conducted by teacher/learning environment, to gather information about the ways that their particular schools operate, how they teach, and how well students learn” (p.288). The framework of action research is based on emancipatory- liberatory (social advocacy) or pragmatic (identifying what works) ideas. Based on the philosophy, action research is defined as critical action research or critical pedagogy and practical action research to create prospects for all involved to improve the lives of children and learn teaching skills. Mills (2002) stated that “all action researchers, regardless of their particular school of thought or theoretical position, are committed to looking critically at what we do in our classrooms and the effects our actions have on the children in our care” (p. 117). The advantages and disadvantages of action research with other research designs include the following: action research is persuasive and authoritative; action research is relevant; with action research, teachers have access to research findings; action research challenges the intractability of reform of the educational system; and action research is not a fad. Action research done by teachers for teachers involves collection of persuasive data. They are persuasive because teachers have identified data sources that provide persuasive insights into the impact of an intervention on student outcomes. In doing action research, teacher researchers have developed solutions to their own problems. ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 18 The relevance of research published in journals to the real world of teachers is perhaps the most common concern raised by teachers when asked about the practical applications of educational research. Either the problems investigated by researchers are not the problems teachers really have or the schools or classrooms in which the research was conducted are not even close to their own school environment. The lack of connection between research and practice is due to teachers’ poor access to research findings. This apparent lack of impact of research on teaching is partially credited to teachers’ prior beliefs and values and the realization that teachers’ practices cannot simply be changed by informing them of the results of a study. The lack of connection between research and practice can be attributed to the education system itself, not the research. Despite the beliefs of many teachers, action research is decidedly not a fad because good teachers have always systematically looked at the effects of their teaching on student learning. They may not have called this practice action research, and they may not have thought their reflection was formal enough to be labeled research, but action research it was! (Mills, 2002, p.119-121). Conclusion ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 19 Recapturing the aforementioned designs and approaches in educational research would educate beginning researchers on the philosophy, framework, core concepts in research, and the influence of research in schools and policy-making decisions. Research influences the roles of educators, who are frequently unaware that they used the results of educational research or performed some part of a research process in their professional lives. For instance, utilized research results to aid in policy-making and decision-making; adopt programs identified from a research literature review; or administer standardized tests to increase the validity and reliability of the data. The impact of educational research is published in educational research journals, which can be used as primary and secondary sources in literature review. Equipped with a conceptual understanding of research, practitioners can read, conduct research studies, and take an active role in research projects. This newfound knowledge will be helpful in the completion of my thesis and continued research studies. With additional training and experience, scholar practitioners can design and conduct studies to improve knowledge and advance practices. ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 20 References Creswell, J. (2009). The selection of a research design. In Research design: qualitative, quantitative, and mixed methods approaches (pp. 1-49). Thousand Oaks, CA: Sage Publications. Hergenhahn, B. R. (2001). An introduction to the history of psychology (4th ed.). Belmont, CA: Wadsworth/Thompson Learning. Hoffman, L. (2005). Depth psychology and the empirically supported movement: Critical issues. Retrieved from the Depth Psychotherapy Network: http://www.depth-psychotherapynetwork.com /Professional_Section/Emprically_Supported_Treatment_Debate/Hoffman_ EST_1.htm Lodico, M., Spaulding, D. T., & Voegtle, K. H. (2010). Methods in educational research: From theory to practice (Laureate Education, Inc., custom ed.). San Francisco: John Wiley & Sons. Long, N. (Speaker). (2010). A personal connection to research: Survey Research: Additional Considerations [Media]. Walden University: http://sylvan.live.ecollege.com/ec/courses/53028/CRS-EDUC80004449580/presentation2/2-2/index.htm. Mills, G. E., (2002). Action Research: A guide for the teacher researcher. Upper Saddle River, NJ: Prentice Hill. ARTIFACT #4A: RESEARCH KNOWLEDGE ASSESSMENT 21 Macgillivray, I. K., Jennings, T. (2008). A content analysis exploring lesbian, gay, bisexual, and transgender topics in foundations of education textbooks. Journal of Teacher Education, 59(2), 170-188. doi:10.1177/0022487107313160.
© Copyright 2026 Paperzz