General Information
Project description
Smart energy services and demand flexibility are increasingly important topics. Levels of awareness in an Irish context are uncertain, but engagement is low. We test different ways of framing information on engagement and intention to be flexible. We record flexibility preferences by activity and flexibility mechanisms.
Analysis Plan
Hypothesis
H1: When the topic of smart energy is framed in terms of collective environmental & energy security benefits or individual control & financial benefits, compared to when it is not, people will be more likely to choose it sooner from a list of topics.
H1b: The type of framing of smart energy will affect the order in which it is chosen [non-directional].
When smart energy information is framed in terms of collective environmental & energy security benefits, compared to when the same information is framed in terms of individual control & financial benefits:
H2: the level of engagement with the information will be different.
H3: people will have differing intentions to be flexible with the timing of electricity use and engage with smart services/products.
H4: When specific actions that people can take are presented, general intentions to be flexible with the timing of electricity use will be greater than when more general instructions are presented.
H5: Presenting information about substantial electricity demand of data centres will be negatively associated with intention to be flexible with the timing of electricity use.
H5b: The effect of perceived fairness on intention to be flexible will interact the types of benefits emphasised in the information.
How hypothesis will be tested
This is a multi-stage experiment. In the first stage, we use a three condition between-subject design. Each participant is presented with three energy-related topics on a page, from which they select topics in order of interest. Participants will be randomly assigned to see the topic of smart energy framed in terms of its environment benefits (treatment group 1), its control/monitoring benefits (treatment group 2), or not framed in terms of benefits at all (control group).
In the second stage, participants in the treatment groups will be presented with a page of information about smart energy. There will be two optional boxes on the page that they can open to read additional material not automatically presented. Here a 2x2x2 between-subjects design will be used. The framing manipulation from Stage 1 will be retained. In addition, we will manipulate ‘fairness’ by showing half the sample information about electricity use of data centres, and we will manipulate the specificity of possible smart energy actions included in the information: one half of the sample will see a list of specific actions they can take, the other will see general recommendations to change electricity consumption patterns.
Control group participants will not be shown the page of information i.e., they will skip stage 2.
Participants will then be asked to complete the following series of tasks:
- Participants in the treatment groups will be asked to indicate their impressions of the information in terms of how engaging it was, how likely they would be to seek more, and how likely they would be to tell others.
All participants do all of the remaining tasks
- Rate the relevance of smart energy to themselves personally, their perception of the importance of reducing electricity use at peak times and their interest in knowing how much electricity their household uses and when.
- Rate their likelihood to consider doing fewer activities, shift timing of use of appliances, and use more efficient appliances.
- Indicate whether they would switch to a time-of-use tariff or sign up to a ‘direct load control’ contract.
- List the main benefits they see of smart energy services.
- Answer multiple-choice questions about smart energy services.
- Indicate the frequency with which they use various electrical appliances at peak times and the extent to which they might change the timing of each.
- Participants who use an oven at peak will indicate the proportion of times they would substitute it with an air fryer. Participants who cook at peak will indicate the extent to which they might batch cook. Participants who use a tumble dryer at peak indicate the extent to which they might use alternative clothes drying methods.
- Rate the effort involved in changing the timing of each activity on 7-point scales.
- Rate the importance of motivations for using less at peak on 7-point scales.
- Indicate how much money they think they could save by using less at peak and indicate how much money they would need to save to consider switching tariff type.
- Rate their own understanding of how to save energy in day-to-day life.
- Indicate how much spare time they typically have.
- Rate how worried they are about climate change, cost of living, and energy security, and rate their level of trust in various energy-related bodies.
- Rate how fair it is that people are asked to change their consumption patterns, and how aware they were of the topic before the study.
- Complete personal and socio-demographic information including relevant technologies owned, meter and tariff types.
The experiment will be run online with a nationally representative sample of people living in Ireland.
Balanced block randomisation will be used to randomly split the sample into three groups. Participants will be randomised to control, treatment 1, and treatment 2 in a ratio of 1:2:2. Participants in the treatment groups will be further randomised into either unfair or fair conditions, and specific or general action conditions, both in 1:1 ratio.
At the start of the survey, participants will see three topics. One is smart energy. The framing of the smart energy topic is manipulated: no framing (control group), environment benefits framing (treatment 1), control & insight framing (treatment 2). This is the frame variable.
In the next stage, participants will be shown a page of information. The framing manipulation is retained, but participants for whom the topic was not framed in stage 1 skip this section i.e., they will not see the information (control group).
Within the treatment groups, we will additionally manipulate ‘fairness’ by showing half of participants information about (substantial) electricity use of data centres. We will also manipulate ‘action type’, the specificity of smart energy actions included in the information: one half of the sample will see a list of specific actions they can take, the other will see general recommendations to change electricity consumption patterns.
Dependent variables
Initial engagement: we will measure order of selection of the smart energy topic (as well as the other distractor topics).
Continued engagement with the information will be measured in multiple ways. We will measure the time spent reading the information. We will measure whether participants opened two optional boxes of additional information on the information page. We will measure retention of the information with a series of multiple-choice questions. We will measure subjective engagement on 7-point ratings scales of how engaging it was perceived to be, how likely participants say they would be to seek more, and how likely they would be to tell others about what they read.
We will also measure general engagement with smart energy in the full sample by asking participants to rate how relevant it is to them, how important they think it to reduce use at peak times, and how interested they are in knowing how much electricity their household uses and when.
General intention to be flexible will be measured by participants rating their likelihood to consider doing fewer activities, shifting use of appliances, and using more efficient appliances at peak times on 7-point scales.
We will measure likelihood to consider switching to a time-of-use tariff or sign up to a ‘direct load control’ contract.
Analyses
General intention to be flexible with timing of electricity use – we will calculate the average rating for the following three 7-point rating scales:
Would you consider using less electricity at peak times (5pm to 7 pm) by doing any of the following?
(a) doing fewer activities that use electricity in general
(b) shifting use of large electrical appliances to other times
(c) using more energy efficient appliances
[For each: Definitely not 1-7 Definitely would]
Subjective impressions of information – we will calculate the average rating for the following three 7-point rating scales:
“Thinking about the information about smart energy that you just read:
Q. How engaging did you find the information? [Not at all engaging 1-7 Very engaging]
Q. How likely are you to look for more information on smart energy? [Very unlikely 1-7 Very likely]
Q. How likely are you to tell someone about what you read today? [Very unlikely 1-7 Very likely]
Engagement with smart energy – we will calculate the average rating for the following three 7-point rating scales:
Q. How relevant do you think the topic of smart energy is to you personally? [Not at all relevant 1-7 Very relevant]
Q. How important do you think it is for people to reduce the amount of electricity they use at peak times?
(i.e. at times when a lot of electricity is being used across the country) [Not at all 1-7 Very important]
Q. How interested are you in knowing how much electricity your household uses and when? [Not at all 1 – 7 Very interested]
We will create a composite ‘comprehension’ score by summing the correct answers to the 5 multiple-choice questions.
We will check distributions of continuous variables and, if necessary for linear models, transform as appropriate. We will carry out and record any transformations before analyses are undertaken. We will do the same for ordinal variables (7-point rating scales responses).
We will check distribution of tariff type variable. If some categories have few cases we will combine them in standard, night-saver, and other time-of-use/smart.
We will create a categorical variable coded as follows: 0 = control (no framing), 1= environment frame condition, 2 = control/monitor frame condition.
Action type (0 = general; 1 = specific)
Fairness (0 = unfair; 1 = fair (no mention of data centre electricity use)
We will code categorical variables with multiple levels as follows:
Highly educated (degree or higher)
Employment status (employed full-time, part-time or self-employed)
Social grade (ABC1 = 1; C2DEF = 2)
Age (18-34 = 1; 35-54 = 2; 55+ = 3)
This analysis section focusses on the experimental structure of the study which will be analysed using statistical models but part of the purpose of the study is also to report absolute measures. In addition to the variables that will be used in the models that are described below, we will also describe the following:
- The level of intention to be flexible with electricity demand in different ways and for different activities
- The perceived relevance of the topic and interest in knowing about own use, and the perceived importance of reducing peak use
- The responses to questions about switching to time-of-use tariffs and signing up to direct load control contracts
- The listed benefits of smart energy services and the rankings of motivations to engage with them
- The proportion of participants that answer each multiple choice question correctly
- The perceived savings associated with reducing use at peak times if on a time-of-use tariff and the required amount to consider switching and the difference between the two
- Level of time people perceive themselves to have availability during weekdays and weekends and how flexible their daily routines are
- Levels of worry about relevant issues and how well people understand how to save energy (self-report), and level of trust in different bodies.
- Level of awareness of the topic prior to the study
- The time spent reading information in each condition, and the number of optional boxes of additional information opened.
- The openness to flexibility between different activities that use electricity.
H1 and H1b: To test whether framing smart energy in terms of its benefits, and the type of benefits used, changed the order in which the topic is selected from a list, we will run an ordinal logistic regression with choice order of smart energy as the dependent variable and experimental condition as the independent variable. We will add relevant sociodemographic and household variables (age, gender, education, social grade, household composition, energy poverty, tariff type), and prior awareness of the topic as covariates. We will control for order effects of the presentation order of the topics.
H2: To test whether presentation of information about smart energy increases engagement about the topic, we will run two separate regressions, one with composite engagement score as the DV and one with the composite comprehension score as the DV and experimental frame condition as the IV. We will add relevant sociodemographic and household variables (age, gender, education, social grade, household composition, energy poverty, tariff type), and prior awareness of the topic as covariates.
H3: To test whether presentation of information about smart energy increases intentions to flexible, we will run three separate regressions, one with the composite general intention to be flexible score as the DV, one with intention to switch to a time-of-use tariff as the DV, and one with likelihood to switch to direct load control contract as the DV, and experimental frame condition as the IV. We will add relevant sociodemographic and household variables (age, gender, education, social grade, household composition, energy poverty, tariff type), as covariates. We will control for prior awareness of the topic. We will also add time availability and perceived fairness.
H4, H5, H5b: To test whether the framing type of information, the specificity of the actions in the information, and the presence of information about substantial electricity demand of data centres change the intentions to be flexible with timing of electricity use, we will run a regression with the composite general intention score as the DV and experimental conditions (frame, fairness, actions type) as the IVs. We will add age, gender, education, social grade, household composition, energy poverty, technology owned, tariff type, and time availability as covariates. We will include an interaction term for the frame and fairness conditions.
Sample Size. How many observations will be collected or what will determine sample size?
We will collect data on 1,500 participants, 300 of which will be allocated to a control group, which means we will have 600 participants in each condition branch for stage 2, and 300 in each sub-condition. We rationalise this is enough to detect the single interaction we will test for, as well as interactions between manipulations and sociodemographic variables of interest.
Data Exclusion
We have included an attention check measure. Participants who fail are ejected from the study. We will check for repetitive, unreasonable responding. If found, we run analysis with such data excluded to check for differences in results.
Treatment of Missing Data
We don’t expect missing data as we have forced choice for questions.
Who is behind the project?
Project status:
Pre-registration
Methods
What is the project about?
Date published:
27 September 2024