Reference no: EM132643415 , Length: word count:1000

ECOM30003 Applied Microeconometric Modelling - University of Melbourne

Assignment 1

The government of a neighbouring country is replacing its paper based procurement program with an electronic based program, referred to as e-procurement. The implemen- tation involves rolling out the program across states gradually over time. The responsible government department is seeking advice on designing an the evaluation of the impact of the e-procurement program. It has contracted the organisation that you work for to provide this advice and you have been asked to assist. Should your company do a good job with designing the evaluation, there is the potential to also conduct the evaluation. This is high value contract for the organisation that employs you.

The starting point for designing the evaluation of the new e-procurement program is to read up on evaluations of similar programs. You have been asked to read "Can Electronic Procurement Improve Infrastructure Provision? Evidence from Public Works in India and Indonesia" by Lewis-Faupel, Neggers, Olken and Pande. Use the information in this paper as the basis of your response to the following questions.

Question 1. In advising on the design of the evaluation, your first task is to propose outcomes that could be impacted by the introduction of e-procurement for project works (such as road building) that can be used to evaluate the program, and that can be measured. What project related outcomes are used by Lewis-Faupel et al. Provide a justification for why these outcomes are relevant for assessing the impact of e-procurement. Ex-ante, what impact do you expect e-procurement to have on each of these outcomes.

Question 2. The responsible government Department is planning on evaluating the program by comparing the average value of outcomes before and after implemen- tation occurs. Explain why this may be problematic if the aim is to identify the causal impact of the e-procurement program.

Question 3. The Department next suggests an alternative estimator for the impact of the program that compares states that have introduced e-procurement with states that have not. Why might this not provide causal estimates of the impact of the program?

Question 4. Use Lewis-Faupel et al. to propose an evaluation framework that may be appropriate for identifying the causal impact of e-procurement, given that the pro- gram is rolled out across states at different points in time. Write out the econometric model (equation you would estimate) in implementing this framework. Include vari- ables that you might wish to control for if you were to replicate Lewis-Faupel et al. Be sure you define each variable in the model you write down and explain why it is included. eg what does each variable account for or control for?

Question 5. In the framework you have just described, what is the parameter of interest? What does this parameter estimate? Given the impact you expect e- procurement to have on each of the outcomes listed above, what sign do you think the parameter of interest will have for each outcome.

Question 6. What are the key conditions required in order for this methodology to deliver estimates of the causal impact of the program? Can you investigate any of these identifying assumptions? If so, how?

Question 7. Discuss any issues related to standard errors that should be taken into account. How do you propose doing this in practice?

Assignment 2

''Can Electronic Procurement Improve Infrastructure Provision? Evidence from Public Works in India and Indonesia" by Lewis-Faupel, Neggers, Olken and Pande conducts an evaluation of the impact of e- procurement in India and Indonesia on outcomes related to the process, as well as outcomes related to cost and quality of projects undertaken under government procurement. The following questions use an extract of the data used by the authors of this paper and asks you to conduct analysis related to results found in Table 4 of the paper.
Your submission should include your written responses to all questions, and then an appendix including all requested tables, and your stata code. Tables should be numbered sequentially. The table containing results for Questions 4-10 should have column headings indicating the question they correspond to. Please keep your written answers brief and to the point, referring to the tables by table number and column heading where appropriate.

Questions 1 and 2 asks you to look at variation in the data that will be used to identify the policy impact.

Question 1. In order to demonstrate the variation the authors use to identify their model, code the year that e-procurement is introduced as zero if it occurs after the years for which there is information on packets (tab package_year to see what years there is information on procurement packets). Collapse the data set to create a table that crosstabulates the year that e-procurement is introduced and state. Show year e-procurement was introduced in columns and states in the rows of the table and label the table Table 1. How many states are there in the data? How many states introduced e-procurement during the years for which we have data on procurement? Describe the timing and number of states introducing e-procurement. [Hint: use the stata command collapse.]

Question 2. Using the original data set on packets (of contracts) and the sample used in Table 4 column 6 of Lewis-Faupel et al., create a table cross tabulating e-procurement status (ie whether the policy is in effect) and state. Show e-procurement status in columns and states in the rows of the table and label the table Table 2. Comment on the percent of observations for which the policy is in effect. Are all states that you identified as introducing e-procurement in the years for which packet data are available in Table 1 represented in Table 2? If not, why is this the case? [Hint: use stata's help to learn about the post-estimation command e(sample) to define a sample used in estimating the model contained on Table 4 column 6.]

Question 3. What is the empirical challenge in identifying the causal impact of the e-procurement program. Briefly explain the strategy that the authors employ to overcome the identification problem and the empirical specification used to implement this strategy. What parameter measures the causal impact of e-procurement? Be explicit about the key identifying assumptions. How should the standard errors be modelled?

Questions 4-10 asks you to estimate various models. Please report (all) results in a single table labelled Table 3, and use the same format as Table 4 of Lewis-Faupel et al. Also, to rule out the potential for different sample sizes driving different estimates, make sure you use the sample used in Table 4, panel A, column 6 of Lewis-Faupel et al. The only specification for which the sample size should be different is the

specification estimated in Question 9. The standard errors reported for all specifications should account for any issues identified in Question 3. [Hint: See outreg2 or esttab to output stata results into excel.]

Question 4. Regress the outcome reported in Table 4 panel A column 6 on the indicator for e- procurement (without any other controls). What is the interpretation (sign, size and significance)? of the estimated coefficients?

Question 5. Add indicators for year of package to the above specification. What happens to the estimated coefficient on e-procurement (size and significance)? What does this tell you about whether the estimates have a causal interpretation?

Question 6. Add state fixed effects to the specification estimated in Question 5. What happens to the estimated coefficient on e-procurement (sign, size and significance)? What does this tell you about whether the estimates have a causal interpretation?

Question 7. Add log road length and log estimated costs to Question 6's specification. What happens to the estimated coefficient on e-procurement (sign, size and significance)? What does this tell you about whether the estimates have a causal interpretation?

Question 8. The specification reported in Table 4, panel A column 6 includes the year of first inspection and monitor fixed effects. Explain why they are included. What happens to the estimated coefficient on e-procurement (sign, size and significance) when you add them to the model estimated in Question 7?

Question 9. Re-estimate the specification estimated in Question 8, but exclude observations from the state Andhra Pradesh. What happens to the estimated coefficient on e-procurement (sign, size and significance) when you drop these observations from the estimation sample?

Question 10. Although not reported in the paper, the common trend assumption is typically investigated by allowing for different time paths for treated and non-treated units in the pre- treatment period. For the purpose of this assignment, we will take a simplified approach to this by asking whether there is a significant difference in the outcome variable between treatment and non- treatment states one period before treatment is introduced. This will involve including an additional explanatory variable to the specification reported in Table 4, panel A column 6. This additional variable is an indicator variable equal to one in the year before e-procurement is introduced in a state and zero otherwise. Here is the code to create this variable:

gen eprocL=(package_year==(eproc_start_year-1))

Create this additional variable and add it to the model reported Table 4, panel A column 6. What do the results from estimating this model suggest about the common trends assumption? What does this tell you about whether the estimates have a causal interpretation?