From the Conceptual Framework to the Empirical Model Gustavo Angeles MEASURE Evaluation University of North Carolina at Chapel Hill Workshop on Impact Evaluation of Population, Health and Nutrition Programs Accra, Ghana July 18-29, 2016 Importance of Conceptual Frameworks • A conceptual framework explicitly identifies the main relevant factors that influence the health result of interest • It also makes explicit the causal relationships between the factors, and, the pathways through which the program has an influence on the outcome • It provides the reference framework for the empirical model • It allows you to better identify endogenous explanatory factors • It also provides the reference framework for including or excluding variables in the empirical model Simple Conceptual Framework Individual / Household Characteristics Community Characteristics / Health Service supply Use of FP Methods Simple Conceptual Framework Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Simple Conceptual Framework What do we observe? Observed? YES YES YES NO NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Simple Conceptual Framework What do we observe? Observed? YES YES YES NO NO Observed? YES NO YES NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Simple Conceptual Framework What do we observe? Observed? YES YES YES NO NO Observed? YES NO YES NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Simple Conceptual Framework Building an empirical model Observed? YES YES YES NO NO Observed? YES NO YES NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use =f(Age , Edu , SES , Rur , Prog , Use of FP Methods Simple Conceptual Framework Building an empirical model Observed? YES YES YES NO NO Observed? YES NO YES NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Useij=f(Ageij, Eduij, SESij, Rurj, Progj, Use of FP Methods Simple Conceptual Framework Building an empirical model Observed? YES YES YES NO NO Observed? YES NO YES NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply Use of FP Methods - Rurality - Health facilities (HF) - Program - Sanitation Useij=f(Ageij, Eduij, SESij, Rurj, Progj, Riskij, Genij,HFj,Sanj) Simple Conceptual Framework Building an empirical model Observed? YES YES YES NO NO Observed? YES NO YES NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply Use of FP Methods - Rurality - Health facilities (HF) - Program - Sanitation Useij=f(Ageij, Eduij, SESij, Rurj, Progj, Riskij, Genij,HFj,Sanj) Observed Unobserved Simple Conceptual Framework Building an empirical model Useij=f(Ageij, Eduij, SESij, Rurj, Progj, Riskij, Genij,HFj,Sanj) ? What is the functional form of this function? Simple Conceptual Framework Building an empirical model Useij=f(Ageij, Eduij, SESij, Rurj, Progj, Riskij, Genij,HFj,Sanj) ? What is the functional form of this function? We don’t know. But, we need a functional form to estimate the relationship between “Use” and its determinants, in particular, the Program. So, what can we do? Simple Conceptual Framework Building an empirical model – An useful result Look at the statistics toolkit. An interesting result: If you have Y= f( X1, X2) with f(.) unknown, you can approximate it by: Simple Conceptual Framework Building an empirical model – An useful result Look at the statistics toolkit. An interesting result: If you have Y= f( X1, X2) with f(.) unknown, you can approximate it by: Y≈ α0 + α1X1 + α2X2 + α3X1X2 + α4X12 + α5X22 + α6X12 X22 + α 7X 13 + α 8X 23 + α 9X 13 X 23 + α10X14 + α11X24 + α12X14 X24 + … (polynomial of order n continues) Simple Conceptual Framework Building an empirical model – An useful result Look at the statistics toolkit. An interesting result: If you have Y= f( X1, X2) with f(.) unknown, you can approximate it by: Y≈ α0 + α1X1 + α2X2 + α3X1X2 + α4X12 + α5X22 + α6X12 X22 + α 7X 13 + α 8X 23 + α 9X 13 X 23 + α10X14 + α11X24 + α12X14 X24 + … (polynomial of order n continues) This is the Taylor Series approximation (Taylor polynomial; 1715) In practice, we use the linear part of the approximation: Y= α0 + α1X1 + α2X2 and sometimes you add interactions and squares. Simple Conceptual Framework Going back to our model So, we have: Useij=f(Ageij, Eduij, SESij, Rurj, Progj, Riskij, Genij,HFj,Sanj) Which can be approximated by a linear relationship: Useij = α0+ α1Ageij + α2Eduij+α3SESij+α4Rurj +α5Progj + (α6Riskij+ α7 Genij + α8HFj + α9Sanj) Simple Conceptual Framework Examining the structure of our model Useij = α0+ α1Ageij + α2Eduij+α3SESij+α4Rurj +α5Progj + (α6Riskij+ α7 Genij + α8HFj + α9Sanj) It has a basic structure: Useij = Observed factors + Unobserved factors Usually, all unobserved factors are summarized in the term εij Useij = α0 + α1Ageij + α2Eduij + α3SESij + α4Rurj + α5Progj + εij Simple Conceptual Framework Examining the structure of our model Useij = α0+ α1Ageij + α2Eduij+α3SESij+α4Rurj +α5Progj + (α6Riskij+ α7 Genij + α8HFj + α9Sanj) It has a basic structure: Useij = Observed factors + Unobserved factors Usually, all unobserved factors are summarized in the term εij Useij = α0 + α1Ageij + α2Eduij + α3SESij + α4Rurj + α5Progj + εij But, it is important to know what those unobserved factors are and how they relate to the other determinants. Simple Conceptual Framework Examining the structure of our model Notice that there are unobserved factors at the individual and community level. In particular, Unobserved = α6Riskij+ α7 Genij + α8HFj + α9Sanj Individual-level Unobservables Community-level Unobservables Now, what can we say about the relationship between the explanatory factors and Use? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Useij = α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj +α6Riskij+α7Genij+α8HFj+α9Sanj What if? ↑Edu Now, what can we say about the relationship between the explanatory factors and Use? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Useij = α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj +α6Riskij+α7Genij+α8HFj+α9Sanj What if? ↑Edu → ∆ Use Now, what can we say about the relationship between the explanatory factors and Use? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Useij = α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj +α6Riskij+α7Genij+α8HFj+α9Sanj What if? ↑Edu → ∆ Use ↑HF Now, what can we say about the relationship between the explanatory factors and Use? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Useij = α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj +α6Riskij+α7Genij+α8HFj+α9Sanj What if? ↑Edu → ∆ Use ↑HF → ∆ Use Now, what can we say about the relationship between the explanatory factors and Use? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Useij = α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj +α6Riskij+α7Genij+α8HFj+α9Sanj What if? ↑Edu → ∆ Use ↑HF → ∆ Use ↑Prog Now, what can we say about the relationship between the explanatory factors and Use? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Useij = α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj +α6Riskij+α7Genij+α8HFj+α9Sanj What if? ↑Edu → ∆ Use ↑HF → ∆ Use ↑Prog → ∆ Use Now, what can we say about the relationship between the explanatory factors and Use? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Characteristics / Health Service supply - Rurality - Health facilities (HF) - Program - Sanitation Use of FP Methods Useij = α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj +α6Riskij+α7Genij+α8HFj+α9Sanj What if? ↑Edu → ∆ Use ↑HF → ∆ Use ↑Prog → ∆ Use Then, we have direct and unique relationships between each determinant and “Use”. Then, we can estimate the effect of Prog on Use. Now, a simple modification: Conceptual Framework 2 Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Use of FP Methods Community Charact. - Rurality - Health facilities (HF) - Sanitation Program A simple modification: Conceptual Framework 2 What do we observe? Observed? Individual / Household Characteristics YES YES YES NO NO Observed? YES NO NO - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Use of FP Methods Community Charact. - Rurality - Health facilities(HF) - Sanitation Program A simple modification: Conceptual Framework 2 What is the associated empirical model? Observed? Individual / Household Characteristics YES YES YES NO NO Observed? YES NO NO - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Use of FP Methods Community Charact. - Rurality - Health facilities (HF) - Sanitation Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj A simple modification: Conceptual Framework 2 What is the associated empirical model? Observed? Individual / Household Characteristics YES YES YES NO NO Observed? YES NO NO - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Use of FP Methods Community Charact. - Rurality - Health facilities (HF) - Sanitation Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj A simple modification: Conceptual Framework 2 What is the associated empirical model? Observed? YES YES YES NO NO Observed? YES NO NO Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Use of FP Methods Community Charact. - Rurality - Health facilities (HF) - Sanitation Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj Now the empirical model has two equations. Notice that equation (1) is exactly as in the previous slides for the first conceptual framework. A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Charact. - Rurality - Health facilities (HF) - Sanitation Use of FP Methods Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Charact. - Rurality - Health facilities (HF) - Sanitation Use of FP Methods Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu → ∆ Use A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Charact. - Rurality - Health facilities (HF) - Sanitation Use of FP Methods Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu → ∆ Use ↑Risk A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Charact. - Rurality - Health facilities (HF) - Sanitation Use of FP Methods Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu → ∆ Use ↑Risk → ∆ Use A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Charact. - Rurality - Health facilities (HF) - Sanitation Use of FP Methods Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu → ∆ Use ↑Risk → ∆ Use ↑HF A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Individual / Household Characteristics - Age - Education - Household wealth/SES - Risk aversion - Biological endowments Community Charact. - Rurality - Health facilities (HF) - Sanitation Use of FP Methods Program Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu → ∆ Use ↑Risk → ∆ Use ↑HF → ∆ Prog → ∆ Use ∆ Use ∆ Use We don’t have a unique relationship between “Health Facilities (HF)” and “Use”!! A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu → ∆ Use ↑Risk → ∆ Use ↑HF → ∆ Prog → ∆ Use ∆ Use ∆ Use A change in HF generates two effects: 1. Change “Prog”, which changes “Use”. 2. Change “Use”, directly. Notice that we only observe the net effect on Use (∆ Use). A simple analysis will relate the observed change on “Use” to the observed change in “Prog”. Clearly, it estimates incorrectly the impact of Prog on Use. A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? ↑Edu → ∆ Use ↑Risk → ∆ Use ↑HF → ∆ Prog → Program Impact: α5 ∆ Use ∆ Use ∆ Use A change in HF generates two effects: 1. Change “Prog”, which changes “Use”. 2. Change “Use”, directly. Notice that we only observe the net effect on Use (∆ Use). A simple analysis will relate the observed change on “Use” to the observed change in “Prog”. Clearly, it estimates incorrectly the impact of Prog on Use. This is the problem of endogeneity. Cannot estimate α5 correctly. A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Endogenetity Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? Program Impact: α5 ↑Edu → ∆ Use ↑Risk → ∆ Use ↑HF → ∆ Prog → ∆ Use ∆ Use ∆ Use This problem is caused by unobserved factors, such as “HF”, that influence both the Outcome (“Use”) and the Program (“Prog”). Special methods are needed to disentangle the effects and separate out the effect of Prog on Use. Methods: - Instrumental Variables - Longitudinal data models - Some other “clever” methods. A simple modification: Conceptual Framework 2 What can we say about the relationships in the model? Endogenetity Useij=α0+α1Ageij+α2Eduij+α3SESij+α4Rurj+α5Progj+α6Riskij+α7Genij+α8HFj+α9Sanj Progj = β0+ β1Rurj + β2HFj + β3Sanj What if? Program Impact: α5 ↑Edu → ∆ Use ↑Risk → ∆ Use ↑HF → ∆ Prog → ∆ Use ∆ Use ∆ Use Question: Which kind of factors (or determinants) can be considered as suspects of being endogenous? Thanks. This presentation was produced with the support of the United States Agency for International Development (USAID) under the terms of MEASURE Evaluation cooperative agreement AID-OAA-L-14-00004. MEASURE Evaluation is implemented by the Carolina Population Center, University of North Carolina at Chapel Hill in partnership with ICF International; John Snow, Inc.; Management Sciences for Health; Palladium; and Tulane University. Views expressed are not necessarily those of USAID or the United States government. www.measureevaluation.org
© Copyright 2024 Paperzz
