1. Email open rates will vary by subject line framing.

2. Link click rates will vary by framing.

3. People who click the links will be more likely to get a heat pump.

4. The guide will increase heat pump adoption.

5. At least one intervention will increase heat pump adoption compared to the control group.

A large utility will send emails to 100,000 households that were randomly assigned to one of 10 conditions:

1. Control (n = 20,000): No email

2. Basic framing (n = 10,000)

3. Environmental framing (n = 8,750)

4. Environmental framing + guide (n = 8,750)

5. Air quality framing (n = 8,750)

6. Air quality framing + guide (n = 8,750)

7. Social norm framing (n = 8,750)

8. Social norm framing + guide (n = 8,750)

9. Combined framing (n = 8,750)

10. Combined framing + guide (n = 8,750)

Each email will contain several paragraphs describing the benefits of heat pumps using the frame determined by the experimental condition. The emails will link to a page describing heat pumps and, depending on the experimental condition, may contain a link to a step-by-step guide to acquiring one.

1. Email open rates, as measured by a tracker within the email. This will range from 0 to the total sample size per condition and will be converted to a proportion.

2. Presence of a click per participant, defined as at least one click on any content-related link in the email (e.g., to learn more about heat pumps or to access the guide) within the subsample with a tracked email open. This will range from 0 to the size of the subsample that had a tracked email open and will also be converted to a proportion.

3. Heat pump acquisition one year later. This will be inferred by an exploratory model looking at the correlation between outdoor temperature and power consumption that would suggest electric heating. It may be combined with other measures (e.g., self-report) to verify accuracy.

All outcome variables are primary.

Order corresponds to hypotheses:

1. We will use Tukey’s test for non-directional comparisons of all pairs of the 5 email framing conditions (e.g., basic framing compared to combined environmental conditions). Family-wise type I error rate: .10.

2. Using link click rates only for participants who open the email, we will use the same 10 tests as in the previous hypothesis. Family-wise type I error rate: .10.

3. We will use a directional test of proportions to compare heat pump adoption among the sample of people who clicked the link (after opening the email) compared to those who did not click the link (including those who did not open the email). Type I error rate: .05.

4. We will use a directional test of proportions to compare heat pump adoption in the guide conditions and the associated non-guide conditions, in the subsample of participants who clicked the link (after opening the email). Type I error rate: .05.

5. We will use logistic regression with directional tests to compare heat pump adoption in each of the 5 framing conditions to the control (reference) group. Type I error rate: .05 with no family-wise error control.

100,000 households.

For hypotheses 3 and 4, we will exclude data from households that have electricity usage patterns that prevent us from inferring electric heating (e.g., relocating to a new residence during the study period or perhaps irregular electric vehicle charging).

Missing data will be removed, for each separate test.