Replication model:
We will first run the unrestricted replication model as defined by the original team. We created an indicator variable for the treatment called mcdad that was coded as 0 if participants saw the Prudential ad first and 1 if they saw the McDonalds ad first.
This replication model will be run using the following code [all analyses will be performed in MPlus]:
usevariables are
mcdad
npmcd npfries npfood
female femalemiss
age agemiss
hispanic hispanicmiss
black other racemiss
lesshs hsgrad somecollege edumiss
less50k less75k less100k more100k incomemiss
northeast midwest west regionmiss;
missing are all (9999);
Analysis:
ESTIMATOR = MLR;
ITERATIONS= 10000;
Model:
Lnp BY npmcd npfries npfood;
Lnp ON mcdad
female femalemiss
age agemiss
hispanic hispanicmiss
black other racemiss
lesshs hsgrad somecollege edumiss
less50k less75k less100k more100k incomemiss
northeast midwest west regionmiss;
Output:
Samp stdYX Residual Cinterval Tech4 tech1 tech3;
MLR was chosen as the response dependent variables (the net promotor scores) are ordered categoricla, Likert-type variables and we have sufficient sample size to analyze as a continuous quantity under robust maximum likelihood (Li et al., 2016).
After running the replication model, we will run a couple of confirmatory models testing the underlying assumptions of the replication mode.
First, we will verify that the latent 'net promotor' variable indeed is jsutified as a unitary factor. This will be done with the following code
usevariables are
npmcd npfries npfood;
missing are all (9999);
Analysis:
ESTIMATOR = MLR;
BOOTSTRAP = 10000;
ITERATIONS= 10000;
Model:
Lnp BY npmcd npfries npfood;
Output:
Samp stdYX Residual Cinterval Tech4 tech1 tech3;
We will set the target CFA at .95 and the target RMSEA at .08 for justification whether a single factor opposed to correlated individual variables is justified.
If this is so, then we will run the baseline test first without covariates to see if the advertisement increased the net promotor latent variable
usevariables are
mcdad
npmcd npfries npfood;
missing are all (9999);
Analysis:
ESTIMATOR = MLR;
BOOTSTRAP = 10000;
ITERATIONS= 10000;
Model:
Lnp BY npmcd npfries npfood;
Lnp ON mcdad;
Output:
Samp stdYX Residual Cinterval Tech4 tech1 tech3;
Finally, we will add in the demographic covariates. This will be done differently than in the replicaiton model, however. First, we will not include the dummy variables for missingness, instead allowing for FIML.
Second, We will construct an interaction of the treatment with all of the covariates. This is the recommended procedure when incorporating covariates into an exogeneous treatment model (e.g. Freedman, 2008; Lin, 2013; Wager et al., 2016).
This will be done using the following code:
usevariables are
female femalemiss
age agemcd
hispanic hispanicmcd
black blackmcd other othermcd
lesshs lesshsmcd hsgrad hsgradmcd somecollege somecmcd
less50k less50kmcd less75k less75kmcd less100k less100mcd more100k more100mcd
northeast nemcd midwest midwestmcd west westmcd;
missing are all (9999);
DEFINE:
Analysis:
ESTIMATOR = MLR;
ITERATIONS= 10000;
Model:
Lnp BY npmcd npfries npfood;
Lnp ON mcdad female femalemiss
age agemcd
hispanic hispanicmcd
black blackmcd other othermcd
lesshs lesshsmcd hsgrad hsgradmcd somecollege somecmcd
less50k less50kmcd less75k less75kmcd less100k less100mcd more100k more100mcd
northeast nemcd midwest midwestmcd west westmcd;
Output:
Samp stdYX Residual Cinterval Tech4 tech1 tech3;
Overall model fit statistics will be collected for both the treatment alone and the treatment+covariates models. We will only accept the results of the best fitting model of the two.
*Note* if the original CFA does not justify the three net protomot indictaors as belonging to a single underlying factor, the two above analyses will eb run but the Lnp will be cut and replaced with
npmcd npfries npfood ON ...
and added will be
npmcd WITH npfries npfood;
npfries WITH npfood;
**References**
Freedman, D. A. (2008). On regression adjustments to experimental data. *Advances in Applied Mathematics*, 40(2), 180-193.
Li, C. H. (2016). The performance of ML, DWLS, and ULS estimation with robust corrections in structural equation models with ordinal variables. *Psychological methods*, 21(3), 369.
Lin, W. (2013). Agnostic notes on regression adjustments to experimental data: Reexamining Freedman’s critique. *The Annals of Applied Statistics*, 7(1), 295-318.
Wager, S., Du, W., Taylor, J., & Tibshirani, R. J. (2016). High-dimensional regression adjustments in randomized experiments. *Proceedings of the National Academy of Sciences*, 113(45), 12673-12678.
