@[toc] ## Study Information ## **Carleton Transitions Study 1: Objectives 1a and 1b** **Principal Investigator** Andrea L. Howard **Research Questions** *Objective 1a:* Do students experience distinct types of parenting environments that can be classified as relatively adaptive (e.g., high autonomy support, age-appropriate parent involvement, intrinsic academic motivation) and maladaptive (e.g., helicopter parentin combined with extrinsic academic motivation) at the transition to university? **Analysis summary**: A latent class analysis of students' self-reports on measures of parents' involvement, autonomy support, warmth, 'helicopter' behaviours, and their own intrinsic or extrinsic academic motivation or amotivation will be performed to establish whether students' responses to these measures form two or more distinct clusters. *Objective 1b:* Are adaptive classes of parenting environments more strongly linked to responses to end-of-semester measures of success and well-being? **Analysis Summary**: Latent classes will be linked to end-of-semester self-reports of depressive symptoms, anxiety symptoms, academic burnout, social and academic adjustment to university, anticipated final grades, and dropout. ## Data description ## **Is this data open or publically available?** Not at this time **Data Source** - Own Lab Collection - Data were collected by one of the analysts’ lab **Sampling and data collection procedures** The data collection procedure is documented in a project wiki page, [located here][1] ## Knowledge of data ## **Prior work based on the dataset** The analyses described here are the first to come out of the Carleton Transitions Study (2018, 'Study 1') **Prior Research Activity** - I have never analysed these data before **Prior Knowledge current dataset** At the time of documenting this analysis plan, I have no substantial knowledge of the contents of the data. With the assistance of my students, I have begun to compile and label data, and to selectively discard cases, but no summary statistics have yet been calculated and no correlations or other associations have been examined. **Moment of preregistration** - Registration prior to any researcher on this team handling or analysis of the data ## Current study: Variables ## *INTAKE SURVEY* Between September 10 and September 29, 2018, participants completed self-report measures of the following (relevant to the present analysis plan). ***Measured variables: Multi-item scales*** **Perceptions of Parenting Scale**: 21 questions for each of mother and father, on a scale from 1=not at all true to 7=very true: - *Involvement*: e.g., "My mother finds time to talk with me" - *Autonomy support*: e.g., "My mother, whenever possible, allows me to choose what to do" - *Warmth*: e.g., "My mother accepts me and likes me as I am" **Helicopter Parenting**: 5 questions for each of mother and father, on a scale from 1=not at all like him/her to 5=a lot like him/her (e.g., "My mother intervenes in settling disputes with my roommates or friends") **Academic Motivation Scale**: 28 responses to the prompt "Why do you go to university?", on a scale from 1=does not correspond at all to 7=corresponds exactly: - *Intrinsic motivation: to know*: e.g., "Because I experience pleasure and satisfaction while learning new things" - *Intrinsic motivation: toward accomplishment*: e.g., "For the pleasure I experience while surpassing myself in my studies" - *Intrinsic motivation: to experience stimulation*: e.g., "For the intense feelings I experience when I am communicating my own ideas to others" - *Extrinsic motivation: identified*: e.g., "Because I think that a university education will help me better prepare for the career I have chosen" - *Extrinsic motivation: introjected*: e.g., "Because of the fact that when I succeed in university, I feel important" - *Extrinsic motivation: external regulation*: e.g., "In order to obtain a more prestigious job later on" - *Amotivation*: e.g., "Honestly, I don't know; I really feel that I am wasting my time in school" **How will item responses be aggregated?** The **parenting scales** (involvement, autonomy support, warmth, helicopter behaviours) will be separately subjected to an exploratory factor analysis using an oblique rotation method (i.e., permitting correlated factors) to verify that scale items are reasonably unidimensional and to isolate any problematic items (e.g., unusually low factor loadings; very low correlation with other items). EFA loadings will be used to compute Omega total reliability scores, and mean scores will be computed for later analysis. The **academic motivation scale** has seven subscales and these will be consolidated to fewer subscales (minimally, *intrinsic*, *extrinsic*, and *amotivation* scales). I will search for published examples of alternative factor structures for this scale and use that prior information to guide decision-making. I will also perform confirmatory factor analyses to compare alternate factor structures and use information about model fit to inform a final decision. Mean scores from the final selection of subscales will be computed for later analysis. ***Measured Variables: End-of-semester well-being outcomes*** End-of-semester well-being outcomes include: (1) the 10-item short version of the **Center for Epidemiologic Studies - Depression** scale, (2) a 7-item measure of **generalized anxiety disorder** symptoms, (3) a college student version of the Maslach **burnout** inventory, and (4) the **social** and **academic** subscales of the Student Adaptation to College Questionnaire. Summary measures for each outcome will be computed by taking mean scores. Omega total reliability will be reported. End-of-semester academic outcomes include a single item asking participants *Are you still a student?* with response options *yes*, *yes, but at a different university or college*, *no, with plans to return*, and *no, with no plans to return*. Students are coded 0 for any 'yes' response and 1 for any 'no' response to measure **dropout**. Another single item asks participants their **anticipated grades** for the semester, e.g., "mostly As (A-, A, A+)", "a mix of A and B grades", "mostly Bs (B-, B, B+), etc. ***Measured Variables: Sociodemographic and academic covariates*** Analyses will also incorporate the following covariates that may show differences across classes: - **Gender identity** (0=Female, 1=Male) - **Living situation** (Code 1: With parents vs. Away from parents; Code 2: In university residence vs. Own apartment) - **First-generation student** (Coded 1 if the student endorses being the first person in their family to attend university) - **Age** - **Race/ethnicity** (Effects codes for major racial/ethnic categories endorsed) - **Degree program** (Dummy code comparing Arts & Humanities programs to "STEM" programs) - **Family structure** (Single/no-parent vs. Two-parent family) - **Past-year parent income** - **Estimated hours working for pay per week** **Transformations** I anticipate mean-centering continuously-distributed covariates and using coded variables (dummy, effect, or contrast codes) for nominal variables. Parent and academic motivation variables serving as latent class indicators will not be transformed. If any of these variables are heavily skewed, I will either use a robust estimator to compute standard errors that account for non-normality, or a bootstrapping method, or I will model the outcomes as non-normal by imposing a different response distribution (e.g., Gamma, Poisson). **Data Inclusion/exclusion** All participants who provided responses to at least one item on the parent or academic motivation scales will be retained in the latent class analysis (a cursory look at some individual item responses suggests this is around 95% of the sample). **Outliers and Influential Cases** Data will be screened for outliers and influential cases by computing Cook's D values and likelihood displacement (i.e., to determine the influence of each case on the fit function being optimized). I will sort cases in order of highest to lowest influence and flag any cases with values that visibly exceed others in the sample (e.g., a Cook's D value that is 2x or 3x higher than the next highest on the list) for closer scrutiny. Any such cases will be examined closely to determine whether there are any errors in their data, evidence that they did not understand or did not read the survey instructions, or any other patterns of responding that suggest they are extreme in some way. Unless there are clear reasons to discard data, all cases will be retained. Analyses will be repeated with and without cases flagged as outliers to determine the stability of the latent class analysis when influential cases are omitted. **Weights** No sampling weights are used in this study. **Sample size** Sample size is n=460 students who completed the intake survey. The initial recruitment goal for this study was n=400 on the basis that LCAs with samples ranging from 200 to 600 people can [meaningfully distinguish][2] between 2- to 4-class solutions. For example, Given power=.80 and composite effect size w=.30 ("medium"), an LCA with 7 indicator variables requires n=244. **Missing data** Data have not been thoroughly examined but all participants in the final n=460 provided demographic information and responded to basic descriptive questions about their academic program. Around 95% of participants appear to have completed all study measures. All participants who provided responses to at least one item on the parent or academic motivation scales will be retained in the latent class analysis. Full information maximum likelihood estimation will by default retain cases with incomplete data. ## Current study: Analyses ## **Statistical models** To test the hypothesis that students report distinct patterns of (relatively adaptive vs. maladaptive) parent environments combined with academic motivation, I will use a three-step approach such as the 'BCH' method of [Bakk and Vermunt][3]. (note: LCA generally refers to analyses involving only categorical indicators but I use the term more generally to refer to any procedure in which a categorical latent variable is modeled from a set of continuous and/or categorical indicators). Non-technically, this procedure involves: (1) estimating class membership from an unconditional model containing indicators but no covariates or distal outcomes; (2) testing equality of means across latent classes (for continous covariates/outcomes) or testing differences in categorical covariates/outcomes across classes in multinomial logistic regression. For **objective 1a**, differences across classes in sociodemographic and academic covariates listed above will be tested. No interactions are proposed but any considered at a later time will be considered exploratory tests. For **objective 1b**, differences across classes will be assessed for end-of-semester well-being outcomes listed above. No interactions are proposed but any considered at a later time will be considered exploratory tests. **Follow-up analyses** None are planned at this time. **Inference criteria** The initial latent class solution will be decided on the basis of comparisons between a model with *k* classes versus *k-1* classes. The best-fitting model is chosen by examining several metrics: - Lo-Mendell-Rubin Likelihood Ratio Test (**LMR-LRT**), where statistically significant LR improvements suggest a better solution - Bayesian Information Criterion (**BIC**) values, where smaller values are preferred - Average **posterior probabilities**, where values closer to 1 suggest better classification precision - **Entropy**, an index of how discriminating the classes are (values must generally be above .80 and closer to 1 is better) It is not unusual for metrics of fit to provide contradictory information about which solution is best. Therefore, other subjective criteria will be considered, such as: - Are any of the classes in a particular solution very small (e.g., fewer than 10% of the sample classified)? - Are any of the classes in a particular solution only trivially different from each other (e.g., significant mean differences on just one or two indicators and/or differences that are small in magnitude)? All else being equal, a solution with *fewer* classes will be preferred over a solution with *more* classes unless there is strong objective and subjective evidence to support the solution with more classes. Inferences regarding *differences across classes* on covariates and end-of-semester outcomes will be evaluated by examining p-values (p<.05) using the [Benjamini-Hochberg][4] False Discovery Rate procedure. All mean and proportion differences will be interpreted in terms of meaningful effect size. For continuous measures, effect sizes include the size of a mean difference relative to the standard deviation of the measure, or the size of a mean difference relative to the units of the original scale. For categorical measures, effect sizes will be interpreted in terms of probability and risk ratio (e.g., students living away from parents have a 2x higher risk of being in class 2). **Sensitivity Analyses** None planned at this time. **Statistical Analysis Backup Plan** The proposed analyses are sufficiently complicated that it is difficult to anticipate any specific backup procedures. If model non-convergence is encountered, next steps will depend entirely on the reasons for model non-convergence. [1]: https://osf.io/dcx3z/wiki/Sampling%20and%20Data%20Collection/ [2]: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4196274/ [3]: https://www.tandfonline.com/doi/abs/10.1080/10705511.2014.955104 [4]: https://en.wikipedia.org/wiki/False_discovery_rate