General Information
Project description
To support building the pipeline of skilled workers in Ontario, the Ministry of Labour, Immigration, Training and Skills Development (MLITSD) introduced the Skilled Trades Strategy (STS) in 2020 which aims to modernize the skilled trades and apprenticeship system by breaking the stigma and attracting more young people to the skilled trades, simplifying the system, and encouraging employer participation. In collaboration with MLITSD, the Ontario Behavioural Insights Unit (BIU) plans to run a randomized controlled trial to test whether behaviourally informed notices reminding employers of their eligibility to apply for financial incentives can increase application rates and uptake of the program.
Analysis Plan
Pre-analysis plan: Is there a pre-analysis plan associated with this registration?
Hypothesis
Behaviourally informed reminders will increase application rates for the financial incentive relative to baseline (0%).
The effect of reminders on application rates and email engagement will vary as a function of reminder condition.
How hypothesis will be tested
Four conditions in a randomized controlled field experiment. Subjects will be randomly assigned to receive one of four reminder versions:
(1) behaviourally informed base design that features simple, personalized messaging and easy-to-navigate format;
(2) base design + loss aversion statement;
(3) base design + reciprocity statement;
(4) base design + social norm statement.
Reminders will be sent out as emails. Data will be collected on an individual level for four weeks and then analyzed to quantify the effect size of reminders and identify any statistically significant differences between reminder versions.
Dependent variables
Primary outcome
Submission of incentive application (yes/no) as created based on administrative data recording the date of application (the presence of an application date will be coded as submission; the absence of an application date will be coded as no submission).
Secondary outcome
Email engagement (i.e., email open and click-through rates) as recorded by the email platform.
Analyses
The effect of behaviourally informed reminders will be analyzed by means of a multilevel logistic regression model that assesses employers’ probability of applying for the financial incentive/engage with the email reminder. The model will specify a simple fixed effect for the reminder variant employers received (base, loss aversion, reciprocity, or social norm). To account for any potential nesting within the data, the model will further specify random effects for sector and/or region as determined by model fit. The random-effects structure (i.e., random intercepts vs. random slopes) will be selected based on the maximal complexity supported by the data.
Sample Size. How many observations will be collected or what will determine sample size?
Approximately 10,000 subjects as determined by the number of employers who are eligible for the financial incentive but have not applied for it at the time of the trial (~2,500 per condition).
Data Exclusion
N/A
Treatment of Missing Data
N/A
Who is behind the project?
Project status:
Pre-registration
Methods
What is the project about?
Date published:
26 March 2024