capture log close
set more off
set mem 2000m
log using yy1_yy1x_have_same_newid_barring_redesign.log, replace
** Purpose:  Demonstrate that, barring redesigns, YY1 and YY1x files have the same NEWIDs 
** There was a redesign in 2005. 
** by Jean Roth, jroth@nber.org , 2009-10-15 
program loop 
   local year1=`1'
   local yr1=substr(`"`1'"',3,2)
   local subdir1="`2'" 
   local year2=`3'
   local yr2=substr(`"`year2'"',3,2)
   local subdir2="`4'" 
   display "year1=`1' year2=`year2' yr1=`yr1' yr2=`yr2' subdir1=`subdir1' subdir2=`subdir2'"
   ! ls -l /homes/data/ces/`year1'/`subdir1'/fmli`yr2'1.dta 
   ! ls -l /homes/data/ces/`year2'/`subdir2'/fmli`yr2'1x.dta 
   use /homes/data/ces/`year1'/`subdir1'/fmli`yr2'1.dta 
   sort newid
   saveold ~/bulk/ces/fmli`yr2'1.dta, replace
   use /homes/data/ces/`year2'/`subdir2'/fmli`yr2'1x.dta 
   sort newid
   ** Merging quietly to suppress messages about recasting of variables from, for example, int to long
   quietly merge newid using ~/bulk/ces/fmli`yr2'1
   tab _merge
   ** See note below about Sample Redesign in 2005 OR page 6 of Intdoc05.pdf
   assert _merge==3 if `year2' != 2005
clear
end loop
loop 2001 intrvw01     2002 intrvw02 
loop 2002 intrvw02     2003 Intrvw03 
loop 2003 Intrvw03     2004 Intrvw04 
loop 2004 Intrvw04     2005 sas/Intrvw05 
loop 2005 sas/Intrvw05  2006 sas/Intrvw06 
loop 2006 sas/Intrvw06 2007 "2007_STATA/Intrvw07"
******loop 2007 "2007_STATA/Intrvw07"

*********************************************************************************************
** From page6 of Intdoc05.pdf which is the 
** 2005 CONSUMER EXPENDITURE INTERVIEW SURVEY PUBLIC USE MICRODATA DOCUMENTATION, February 16, 2007
**There was a sample redesign in 2005. 
**The sampling frame is now generated from the 2000 Population Census 100-percent-detail file. 
**As a result, users will not be able to link all consumer units whose participation spanned 2004 and 2005 sample designs. 
**It is suggested that users use the 2003 and 2004 CDs or the 2005 and 2006 (when available) CDs for any micro-level longitudinal analysis.
**2005 CD users should be aware that the 2005 first quarter (q051) data on the 2004 CD are from the old sample design, while the q051 data on the 2005 CD are from the new sample design. 
**
**
**Consequently, the data are not the same. 
**Due to the sample design change, there are no January interview data on the q051 files of the 2005 CD. 
**Please be aware that collection period estimates (see section V.A.1.b.) will be affected because of this. 
**Aggregate collection period estimates will require inflation by factors of 3/2 (for q051 estimates) to 12/11 (for 2005 annual collection period estimates). 
**This adjustment will roughly compensate for the missing January interviews. 
**For calculations of averages, January interview data are missing from both the numerator and the denominator and these estimates are not as severely affected. 
**Calendar period estimates are not affected because all expenditures from January interviews fall into the q044 time period.
*********************************************************************************************