WADA - Development of Brief Assessment Packages of Psychosocial Constructs Related to Doping

Development of Brief Assessment Packages of Psychosocial Constructs Related to Doping

Prepared for: World Anti-Doping Agency

Grant Recipients:

Professor Nikos Ntoumanis, University of Southern Denmark, Denmark Professor Ian Boardley, University of Birmingham, UK Professor Vassilis Barkoukis, Aristotle University of Thessaloniki, Greece Professor Anne Marte Pensgaard, Norwegian School of Sport Sciences, Norway Professor Andreas Ivarsson, Halmstad University, Sweden

Research assistant: Julie Thunbo Rivold, University of Southern

Denmark, DenmarkBrief Assessments of Psychosocial Constructs - 1

1. Executive summary 2 1.1 Introduction 2 1.2 Methodology 2 1.3 Findings 2 1.4 Conclusion 2

2. Introduction 3 2.1 Background Literature 3 2.2 Developing Short Versions of Questionnaires 5 2.3 The Aims of the Current Project 6

3. Method 6 3.1 Procedure 6 3.1.1 Phase 1-Selection of Constructs 6 3.1.2 Phase 2Survey of Experts 11 3.1.3 Phase 3-Data Collection with Athletes and Athlete Support Personnel 16 3.1.4 Phase 4 - Creation of the Final Tools 16

4. Results 17 4.1 Adolescent Athletes 17 4.2 Adult Athletes 17

4.3 Athlete Support Personnel 21

5. Conclusions 25

6. Appendix 27

Appendix 1-Results of Abbreviated Delphi Poll 27 Appendix 2Questionnaires for Adolescent Athletes 46 Appendix 3-Questionnaires for Adult Athletes 57 Appendix 4-Questionnaires for Athlete Support Personnel 67 Appendix 5-Changes in Rating Scales of Questionnaires 79 Appendix 6-Questionnaires for Adult Athletes (Final Version; Short) 80

Appendix 3-Questionnaires for Adult Athletes 57 Appendix 4-Questionnaires for Athlete Support Personnel 67 Appendix 5-Changes in Rating Scales of Questionnaires 79 Appendix 6-Questionnaires for Adult Athletes (Final Version; Short) 80 Appendix 7-Questionnaires for Adult Athletes (Final Version; Long) 86 Appendix 8-Questionnaires for Athlete Support Personnel (Final Version; Short) 96 Appendix 9-Questionnaires for Athlete Support Personnel (Final Version; Long) 105 Appendix 10-Information Sheets 115 Appendix 11-Consent Forms 121

7. References 125Brief Assessments of Psychosocial Constructs - 2

1. Executive summary 1.1 Introduction The World Anti-Doping Agency (WADA) promotes, coordinates and monitors the global fight against doping in sport. The overall aim of this project was to develop a set of assessment tools that can be used by anti-doping organizations (ADOs) to monitor and evaluate the effectiveness of their education programs with regard to their effects on psychosocial variables related to doping. To this end, this report presents the results of a series of phases that aimed to develop and validate three brief questionnaires, designed for adult athletes, athlete support personnel, and adolescent athletes, respectively.

1.2 Methodology The project was divided into four phases. In phase 1, the research team reviewed the literature and selected psychosocial constructs perceived to be amenable to anti-doping education. In phase 2, a survey with the selected constructs was sent to anti-doping experts, consisting of researchers and representatives of ADOs, who were asked to rate the importance of and rank order these constructs. The results of this abbreviated Delphi poll informed the constructs chosen for phase 3. In phase 3, the questionnaires were distributed by collaborating ADOs to adolescent athletes, and by us to adult athletes and athlete support personnel (ASP) in Denmark, Norway, and Sweden. All questionnaires were in English. In

informed the constructs chosen for phase 3. In phase 3, the questionnaires were distributed by collaborating ADOs to adolescent athletes, and by us to adult athletes and athlete support personnel (ASP) in Denmark, Norway, and Sweden. All questionnaires were in English. In phase 4, final evaluation tools were created (using data from 307 adult athletes and 296 ASP), by selecting the best items for each construct, primarily using the statistical program R (OASIS package). An insufficient number of adolescent athletes (n=31) were recruited and, hence, a tool for this group was not developed. 1.3 Findings Two questionnaires with 24 (adult athletes) and 28 (ASP) items, respectively, were formed. The questionnaires include 11 and 13 different constructs, respectively, and capture diverse aspects such as morality, motivation, perceived benefits, and perceived deterrents relevant to doping. The final questionnaires can be found in Appendix 6 and 8. In Appendices 7 and 9 we present alternative longer versions of these questionnaires (same constructs but with more items) for researchers who would like a minimum of three items per construct in order to calculate omega reliability coefficients.

1.4 Conclusion The developed questionnaires should be suitable for evaluation of anti-doping education worldwide, when the interest is on the effects of education on malleable psychosocial constructs. Future research should also investigate the psychometric measures of the preliminary questionnaire for adolescent athletes (see Appendix 2) with the purpose of creating a similar validated brief tool for this population group.Brief Assessments of Psychosocial Constructs - 3

2. Introduction In order to ensure high quality and consistent anti-doping and clean sport education, the World Anti-Doping Agency (WADA) has developed an international standard on how to plan, implement, monitor and evaluate education (International Standard for Education; ISE, 2021). The ISE defines the terminology in the field of education, clarifies the roles and responsibilities for all signatories responsible for planning, implementing, monitoring and

plan, implement, monitor and evaluate education (International Standard for Education; ISE, 2021). The ISE defines the terminology in the field of education, clarifies the roles and responsibilities for all signatories responsible for planning, implementing, monitoring and evaluating education programs, and helps signatories maximize the use of their resources. The ISE outlines four components of education delivery:

● Values-based education ● Awareness raising ● Information provision ● Anti-doping education

Furthermore, the ISE suggests that education programs should be evidence-based, informed by education theory, support the spirit of sport values and - if possible - informed by social science research. The programs should be monitored to aid reporting and evaluation and foster continuous improvement. They are also required to be evaluated annually with the purpose of updating the following year’s education plan. However, the instruments currently used for ADOs for monitoring and evaluation of education programs are not currently of practical value. With regard to the assessment of psychosocial variables that could be affected by participation in such programs, there are many different instruments that vary in quality and the psychosocial constructs they measure. Further, these instruments, when combined, can be very long which makes them impractical for assessment purposes. To address this challenge, a collection of good quality instruments that measure succinctly the most salient psychosocial measures for anti-doping education is needed. Such a succinct assessment guide could be of use to ADOs when they want to evaluate the effects of antidoping education on psychosocial variables related to doping, both from the perspective of athletes and from the perspective of athlete support personnel (ASP).

2.1 Background Literature This project aimed to develop brief measurement instruments that can be used to evaluate possible psychosocial outcomes of anti-doping education. Such brief instruments do not currently exist, and this is a problem for evaluation of education programs conducted by ADOs . Therefore, this project fills a significant methodological void in anti-doping research. In terms of assessing psychosocial constructs at the athlete level, the only available guide

currently exist, and this is a problem for evaluation of education programs conducted by ADOs . Therefore, this project fills a significant methodological void in anti-doping research. In terms of assessing psychosocial constructs at the athlete level, the only available guide for ADOs is that developed by Donovan et al. (2015). The aim of that guide was to develop resources for ADOs and other interested parties for their research and evaluation efforts. Specifically, the intention was that ADOs would be able to compare their athlete populations over time, before and after education training, and directly with other agencies’ athlete populations over time. Donovan et al. (2015) identified a number of constructs and questionnaires using reviews of anti-doping literature, as well as questionnaires related to the Sport Drug Control Model. The model considers both the sport socio-economic environment as well as the broad socio-cultural context. The sport socio-economicBrief Assessments of Psychosocial Constructs - 4 environment includes (but is not limited to) affordability and availability of doping, as well as reference group appraisal, legitimacy and personal morality. The broader socio-cultural context includes the cultural differences between countries and sub-populations within countries (e.g., individualism vs collectivism), the medicalization of society in general and in sport, cosmetic and cognitive enhancement in general, the use of illicit recreational drugs, globalization, the increasing commercialization of sport, and the intensification of sporting schedules (Donovan et al. (2015). The questionnaire package developed by Donovan et al (2015) includes the following 14 sections:

1. Morality and cheating.

2. Legitimacy perceptions.

3. Beliefs about the benefits of doping.

4. Beliefs about the harms of doping and the consequences, if caught.

5. Personality/psychological factors.

6. Beliefs about reference groups’ endorsement of doping methods/substances

7. Beliefs about the availability of, and relevant authorities’ control over trafficking of doping methods/substances.

8. Beliefs about the affordability of doping methods/substances.

5. Personality/psychological factors.

6. Beliefs about reference groups’ endorsement of doping methods/substances

7. Beliefs about the availability of, and relevant authorities’ control over trafficking of doping methods/substances.

8. Beliefs about the affordability of doping methods/substances.

9. Beliefs about other athletes’ attitudes towards and use of doping methods/substances.

10. Beliefs about societal influences on doping.

11. Performance-Enhancing Drug Use.

12. Use of Nutritional Supplements and Other Permitted Technologies.

13. Demographics and Sporting Background.

14. Overall Susceptibility to Doping; Intention to Dope in the Near Future; Overall Attitude to Doping.

Whilst comprehensive, the guide is over 120 pages long. This is likely to make it impractical for many practitioners, and different organizations are likely to choose different instruments for assessment purposes. Such diversity in instrument choices can hamper efforts for standardization of ADO's education assessments of psychosocial constructs across countries. Also, it is unlikely that all constructs captured in the guide will be considered equally important by the ADOs for the purpose of assessing the effectiveness of their education. Further, the instruments contained within the guide are primarily capturing constructs included in the Sport Drug Control Model. Over the years, other theoretical approaches and models have been introduced to the anti-doping literature, with concomitant new constructs and measurement instruments to assess these constructs. Examples include the Prototype-Willingness model (e.g., Dodge et al., 2013; Whitaker et al., 2014) and the Integrative Model of Doping Behavior (Barkoukis et al., 2013; Lazuras et al., 2015). Such literature will be considered in the current project, alongside Donovan et al.’s (2015) guide. The quality, validity, and reliability of multiple questionnaires measuring psychological doping related factors have recently been reviewed by Madigan et al. (2021). Madigan et al. (2021) recommend five different scales which have acceptable validity and reliability. These

The quality, validity, and reliability of multiple questionnaires measuring psychological doping related factors have recently been reviewed by Madigan et al. (2021). Madigan et al. (2021) recommend five different scales which have acceptable validity and reliability. These measure doping attitudes, doping willingness, doping susceptibility, and doping efficacy. However, Madigan et al (2021) concluded that there are several psychological doping factors (i.e., doping likelihood and doping intentions) that have no existing measures thatBrief Assessments of Psychosocial Constructs - 5 show sufficient validity and reliability. The present study aims to provide a broader set of doping-related constructs than those included in the Madigan et al. review and with fewer items per construct, as our primary aim is to develop brief scales for practical purposes. With regard to ASP, there has been no published equivalent project to that of Donovan et al. (2015) to collate instruments of psychosocial constructs that are relevant to anti-doping. As a starting point, a team of researchers at Leeds Beckett University (UK) have recently conducted a systematic review of all the coach-related anti-doping literature (Barnes et al., 2020). They identified three higher-order themes (individual, behavioral and contextual factors) consisting of five lower-order themes (self-reported behavior, hypothetical behavior, coach beliefs, knowledge, and psychosocial components). They concluded that the antidoping research addressing coaches has diversified for the past 20 years into a broader consideration of not just coaches’ knowledge and beliefs, but also behavioral and contextual factors. The Barnes et al. review informed the selection and development of constructs pertinent to ASP for our project. 2.2 Developing Short Versions of Questionnaires Developing short versions of existing longer questionnaires is a practice that is becoming increasingly common in psychology (e.g., Cortina et al., 2020; Heggestad et al., 2019), with recent examples in the doping literature (e.g., Boardley et al., 2018). In fact, of the different

increasingly common in psychology (e.g., Cortina et al., 2020; Heggestad et al., 2019), with recent examples in the doping literature (e.g., Boardley et al., 2018). In fact, of the different ways in which a questionnaire can be altered, Heggestad et al. (2019) identified the development of a short version as the most “serious” modification. One common problem in developing short versions is that authors tend to select items that are paraphrases of each other, in order to construct scales with acceptable internal reliability coefficients . According to Cortina et al., such practice results in grammatical redundancy-induced common method bias and a lack of representativeness. There are different ways in which a scale can be shortened. For instance, experts could be asked to rate the most appropriate items or evidence from previous studies could be inspected (e.g., items with highest factor loadings). However, the first option could be very resourceintensive, particularly if a large number of questionnaires need to be shortened. The second option is limited by what information is currently available, and by the fact that the available evidence comes from diverse samples completing some of the scales and not from one sample of participants completing all scales. Very recently, the procedures for item selection have taken advantage of developments in technology (e.g., machine learning; Noetel et al. 2019). In this project, we will use a procedure developed by Cortina et al. (2020). This procedure, implemented by a userfriendly app built in R Shiny called OASIS, assists in identifying an optimal shortened version of an existing measure. OASIS uses a psychometrically defensible optimization strategy that simultaneously considers multiple important criteria such as internal consistency reliability, part-total correlations, general factor loadings, convergent and discriminant validity, and content coverage of the items. Essentially, multiple criteria are used simultaneously to determine the optimal solution (i.e., short version) for a long version of a scale. The user

part-total correlations, general factor loadings, convergent and discriminant validity, and content coverage of the items. Essentially, multiple criteria are used simultaneously to determine the optimal solution (i.e., short version) for a long version of a scale. The user decides on the target number of items for the shortened scale (e.g., 3). The app then identifies all of the possible ways that the original set of items could be combined to make a scale of that length. For instance, the user could request from the app to show only those 3item short versions of an established scale for which the omega reliability coefficient is .70 or above, convergent validity correlations are .70 and above, and divergent validity correlationsBrief Assessments of Psychosocial Constructs - 6 are .30 and below. All derived short versions are then evaluated (e.g., for conceptual and grammatical redundancy) and one version is put forward for the final short version. 2.3 The Aims of the Current Project In brief, our project aims to address the lack of a concise measure that ADOs can use to evaluate the effects of their anti-doping education on related psychosocial variables. To this end, we aimed to develop short versions of assessment tools separately for adult athletes, ASP, and adolescent athletes. The key research questions were a) what are the core psychosocial constructs which should be captured by the assessment overarching tool for each target group?, b) can brief measures be developed to capture each of these core constructs?, and c) can evidence be provided for the internal reliability and validity of the scores of these brief measures?

3. Method 3.1 Procedure The project was divided into four phases. In phase 1, the research team reviewed the literature and selected the psychosocial constructs for phase 2. In phase 2, a survey with the selected constructs was sent to anti-doping experts and the results of this abbreviated Delphi poll informed the constructs chosen for phase 3. In phase 3, the questionnaires were distributed to adolescent athletes, adult athletes, and ASP. In phase 4, the final evaluation

selected constructs was sent to anti-doping experts and the results of this abbreviated Delphi poll informed the constructs chosen for phase 3. In phase 3, the questionnaires were distributed to adolescent athletes, adult athletes, and ASP. In phase 4, the final evaluation tool was created for each target group by selecting the best items for each construct using the statistical program R (OASIS package). Each phase is described in detail in the following pages.

3.1.1 Phase 1-Selection of Constructs The psychological constructs included in the three questionnaires were determined via a sequence of steps. First, two experienced anti-doping researchers (Profs. Boardley and Barkoukis) created a list of 59 relevant psychosocial constructs in collaboration with the research assistant (Rivold). The included constructs reflected psychosocial variables that were perceived by the researchers as malleable, if they were to be targeted by anti-doping education programs. We did not include constructs that measured clean sport behaviors, objective knowledge, or inadvertent doping, as research work on those constructs had been commissioned by WADA to other research groups. During a number of online meetings between the research team, these constructs were reduced to a total of 27 and were divided into 10 groups. The criteria used to reduce the number of constructs were related to conceptual overlap amongst them, whether existing questionnaires existed to measure each of them, and whether they were measured with Likert scales (e.g., constructs measured with scenarios were excluded, given their length). After the 27 constructs were selected, lay descriptions were written for each construct which were then used in phase 2. The list of the constructs, including the descriptions that were presented to the participants of phase 2, is presented in Table 1.Brief Assessments of Psychosocial Constructs - 7 Table 1 - Constructs, groups and definitions for the abbreviated Delphi Poll in Phase 2

Construct Group Definition for athletes (adults and adolescents) Definition for ASP Moral stance Morality The extent to which an athlete thinks that

Table 1 - Constructs, groups and definitions for the abbreviated Delphi Poll in Phase 2

Construct Group Definition for athletes (adults and adolescents) Definition for ASP Moral stance Morality The extent to which an athlete thinks that doping is right or wrong. The extent to which an ASP thinks that doping is right or wrong. Moral disengagement Morality Moral disengagement refers to the use psychosocial maneuvers that allow an athlete to transgress moral standards without experiencing negative emotions, such as guilt and shame. An example could be when an athlete thinks that it is OK to dope, if it helps provide for his or her family. Moral disengagement refers to the use of psychosocial maneuvers that allow ASP to transgress moral standards without experiencing negative emotions, such as guilt and shame. An example could be when an ASP thinks that it is OK to dope, if it helps an athlete to provide for his or her family. Moral affect Morality Moral affect refers to the emotional responses of guilt, shame, and embarrassment resulting from having violated one’s moral stance by engaging in doping. Not included for ASP Moral values Morality Moral values indicate an athlete's value system regarding the ethical side of using banned substances and methods. Moral values indicate ASP’s value system regarding the ethical side of using banned substances and methods. Moral norms Morality Moral norms are formal and informal standards and behaviors within an athlete’s social environment that Moral norms are formal and informal

system regarding the ethical side of using banned substances and methods. Moral norms Morality Moral norms are formal and informal standards and behaviors within an athlete’s social environment that Moral norms are formal and informal standards and behaviors within an ASP’s social environment thatBrief Assessments of Psychosocial Constructs - 8 suggest what is morally acceptable and morally unacceptable in relation to doping. suggest what is morally acceptable and morally unacceptable in relation to doping. Doping attitudes Motivation An athlete’s positive or negative evaluation and perspective on doping. An ASP’s positive or negative evaluation and perspective on doping. Descriptive norms Motivation Descriptive norms reflect an athlete’s perceived prevalence of other athletes’ use of doping. Descriptive norms reflect ASP’s perceived prevalence of use of doping among athletes. Subjective/injunctive norms Motivation Subjective norms reflect athletes’ beliefs about important others’ opinions towards doping. Subjective norms reflect ASP’s beliefs about important others’ opinions towards doping. Perceived likelihood of being tested in and out of competition Doping deterrents How likely an athlete perceives it to be that they will be tested in or out of competition. How likely an ASP perceives it to be that their athlete will be tested in an out of competition. Perceived severity of the sanctions for testing positive Doping deterrents How severe an athlete perceives the sanctions to be for testing positive. How severe an ASP perceives the sanctions to be if

be tested in an out of competition. Perceived severity of the sanctions for testing positive Doping deterrents How severe an athlete perceives the sanctions to be for testing positive. How severe an ASP perceives the sanctions to be if their athletes test positive. Perceived likelihood of evading detection if tested in and out of competition Doping deterrents How likely an athlete perceives it to be that he or she will evade detection if they dope in and out of competition. How likely an ASP perceives it that their athletes will evade detection if they dope in and out of competition. Perceived affordability of doping substances Doping deterrents An athlete’s appraisal of the economic costs of doping in relation to his or her economic capabilities. Not included for ASP APED safety beliefs Doping deterrents An athlete’s beliefs regarding the risk of experiencing mental An ASP’s beliefs regarding the risk of their athletesBrief Assessments of Psychosocial Constructs - 9 and/or physical health consequences from doping. experiencing mental and/or physical health consequences from doping. Perceived performanceenhancing effects of banned substances and methods Benefit appraisals An athlete’s appraisal of the extent to which the use of performanceenhancing substances and methods leads to improved performance in their sport. An ASP’s appraisal of the extent to which the use of performanceenhancing substances and methods by their athletes would lead to improved performance in their sport. Other nonperformance related positive effects of banned substances

An ASP’s appraisal of the extent to which the use of performanceenhancing substances and methods by their athletes would lead to improved performance in their sport. Other nonperformance related positive effects of banned substances and methods Benefit appraisals An athlete’s appraisal of the nonperformance related positive outcomes of doping, such as achieving celebrity status, sponsorship deals, and/or financial security. An ASP’s appraisal of the nonperformance related positive outcomes of doping for their athletes, such as achieving celebrity status, sponsorship deals, and financial security. Doping intentions Proxies of behavior An athlete’s intention to use banned substances and/or methods in the near future. An ASP’s intention to help one or more of their athletes to use banned substances and/or methods in the near future. Doping willingness Proxies of behavior An athlete's willingness to use banned substances and methods. An ASP's willingness to facilitate their athletes' use of banned substances and methods. Self-presentational concern Body image An athlete’s awareness of and concerns associated with how other people perceive his/her physical appearance. Not included for ASP Drive for muscularity Body image An athlete’s pursuit of cultural and gender specific body Not included for ASPBrief Assessments of Psychosocial Constructs - 10 shape ideals. In Western cultures males tend to desire a more muscular physique, whereas females desire a thinner physique. Drive for thinness Body image An athlete’s pursuit of cultural and gender specific body shape ideals. In Western cultures males tend to desire

Western cultures males tend to desire a more muscular physique, whereas females desire a thinner physique. Drive for thinness Body image An athlete’s pursuit of cultural and gender specific body shape ideals. In Western cultures males tend to desire a more muscular physique, whereas females desire a thinner physique. Not included for ASP Perceived behavioral control Confidence An athlete’s beliefs regarding their ability to control whether they engage in doping. An ASP’s beliefs regarding their ability to control whether their athletes engage in doping. Self-efficacy to refrain from doping Confidence An athletes’ belief in their ability to resist internal and external pressures to dope. An ASP’s belief in their ability to help their athletes to resist internal and external pressures to dope. Anticipated regret/guilt Emotions The degree to which an athlete anticipates experiencing guilt/regret as a result of using banned substances and/or methods. The degree to which an ASP anticipates experiencing guilt/regret as a result of helping their athletes use banned substances and/or methods. Perceived legitimacy of anti-doping Beliefs about antidoping system Athletes' perception of whether antidoping organizations are duly constituted and have valid authority to enforce anti-doping regulations. ASP's perception of whether anti-doping organizations are duly constituted and have valid authority to enforce antidoping regulations. Integration of antidoping into coaching practice Athlete support personnel behavior Not included for athletes The degree to which

whether anti-doping organizations are duly constituted and have valid authority to enforce antidoping regulations. Integration of antidoping into coaching practice Athlete support personnel behavior Not included for athletes The degree to which an ASP monitors, observes and/or provides advice toBrief Assessments of Psychosocial Constructs - 11 their athletes around issues such as doping control procedures, inadvertent doping, and risks to health. Diffusion/sharing of anti-doping behavior Athlete support personnel behavior Not included for athletes The degree to which an ASP displaces and/or diffuses their responsibilities towards anti-doping by asserting that anti-doping is not part of their job duties. Doping confrontation efficacy beliefs Athlete support personnel behavior Not included for athletes The extent to which a coach believes in their ability to effectively confront athletes regarding doping and offer appropriate solutions.

3.1.2 Phase 2Survey of Experts Next, the 27 constructs, including their descriptions, were sent via email as a survey to a total of 61 anti-doping experts who had been chosen by the research team and WADA. Thirty two of them were academic researchers with strong anti-doping experience and 29 of them were anti-doping experts from a number of different National ADOs.