ARTICLE IN PRESS Available online at www.sciencedirect.com Journal of Adolescence Journal of Adolescence xx (2008) 1e16 www.elsevier.com/locate/jado Peer victimisation and internalising difficulties: The moderating role of friendship quality Sarah Woods*, John Done, Hardeep Kalsi University of Hertfordshire, School of Psychology, College Lane, Hatfield, Hertfordshire, AL10 9AB, UK Abstract A cross-sectional study is reported in which loneliness and emotional problems are explored in adolescent victims of direct or relational bullying, together with the potentially moderating influence of friendship quality. Adolescents (N ¼ 401, mean age 13.5) completed the School Relationships Questionnaire, to identify bullying and victimisation roles, the Friendship Activity Questionnaire (FAQ), the Strengths and Difficulties Questionnaire (SDQ) to assess emotional problems, and the Loneliness and Social Dissatisfaction Questionnaire (LSDQ). Relational victims, but not direct victims reported significantly elevated feelings of loneliness and emotional problems compared to non-victims. Direct victims reported a significantly higher quality of friendship, compared to non-victims. Poor quality of friendship was also associated with high levels of loneliness, and vice versa for direct victims, but not for relational victims. This indicates that the higher quality of friendship found in direct victims is associated with the reduced levels of loneliness found in this group. Given the cross-sectional nature of this study, it is not possible to ascertain whether this association truly reflects the role of friendship quality as a moderator, and hence protective factor against adverse influences of victimisation. The different mechanisms underlying direct and relational victimisation are important for future research and intervention programmes. Ó 2008 The Association for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved. Keywords: Victimisation; Friendship quality; Loneliness; Internalisation problems; Intervention Abbreviations: SRQ, School Relationship Questionnaire; FAQ, Friendship Activity Questionnaire; SDQ, Strengths and Difficulties Questionnaire; LSDQ, Loneliness and Social Dissatisfaction Questionnaire. * Corresponding author. Tel.: þ44 1707 285057. E-mail address: [email protected] (S. Woods). 0140-1971/$30.00 Ó 2008 The Association for Professionals in Services for Adolescents. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.adolescence.2008.03.005 Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 2 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 Introduction Peer victimisation has attracted both research and media attention since reports have indicated that it is a common experience for some children and adolescents, with accounts of the incidence varying from between 8% and 46% (Espelage & Swearer, 2003; Schäfer, Korn, Brodbeck, Wolke, & Schulz, 2005; Smith et al., 1999; Wolke, Woods, Schulz, & Stanford, 2001). Victimisation behaviour is constitutionally different from one-off aggressive acts, and has been defined in the literature as deliberate, repeated negative actions towards a person with the victim perceiving an imbalance of power (Olweus, 1999). An association between peer victimisation and internalising problems (i.e. loneliness, anxiety, depression, suicidal behaviour or ideation), and low self-esteem has been well documented (Boulton & Underwood, 1992; Craig, 1998; Hawker & Boulton, 2000; Parker & Asher, 1987, 1993; Rigby, 1998; Salmon, James, & Smith, 1998). In response to the limitations of cross-sectional designs, several longitudinal studies have been carried out to determine the cause and effect relationships between victimisation and maladjustment. However, the results remain equivocal as some studies have suggested a bi-directional relationship between victimisation and adjustment (Egan & Perry, 1998), whereas others have reported that peer victimisation predicts later maladjustment (Bond, Carlin, Rubin, & Patton, 2001; Kochenderfer & Ladd, 1997). Children with internalising problems have also been found to be at greater risk of becoming victims in the first place (Griffin & Gross, 2004). Thus, the causeeeffect relationships between the experience of victimisation and internalising disorders are probably transactional (i.e. involving an interaction between the vulnerability of the individual and exposure to risk or protective factors) rather than simple in keeping with most developmental models of internalising disorders (Zahn-Waxler, Klimes-Dougan, & Slattery, 2000). One possible risk/protective factor is peer rejection/peer acceptance (Hay, Payne, & Chadwick, 2004). There is now abundant research evidence that peer relations provide an essential medium for children’s emotional, cognitive and social development, and those children who experience difficulties with peers may struggle in different areas throughout life (Gettinger, 2003). Victims of bullying are frequently reported as unpopular, having fewer friends, and find it difficult to make new friends (Griffin & Gross, 2004; Haynie et al., 2001; Young & Sweeting, 2004). Further, those reporting poor social relationships are at greater risk of developing depression (Bond et al., 2001). Controversy remains regarding how to most accurately measure peer relations, and it is important to emphasise the distinction between dyadic peer interactions (i.e. friendships) and groupbased peer interactions (e.g. popularity, peer networks). The differences are not trivial since methods used to understand dyadic relationships focus on interpersonal processes, rather than structural characteristics of the peer group (Gifford-Smith & Brownell, 2003). The majority of studies on peer relationships have used group-based measures which index popularity, peer group acceptance and affiliation, rather than quality of friendship. The limited evidence that does exist concerning the interaction between quality of peer relations, victimisation and risk of internalising disorder, suggests that when victims experience poor peer relations, then they are at increased risk of developing internalising problems (Petit, Laird, Dodge, Bates, & Criss, 2001). The converse picture, that good peer relations, or high quality friendships, act as a ‘buffer’ (Rigby, 2000) and provide ‘protection’ (Boulton, Trueman, Chau, Whitehand, & Amatya, 1999) from loneliness and internalising problems (Hodges, Boivin, Vitato, & Bukowski, 1999), has received little by way of empirical hypothesis testing (Hay et al., 2004). Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS S. Woods et al. / Journal of Adolescence xx (2008) 1e16 3 Simply measuring whether a victim has a best friend, or not, is an inadequate measure of the potential benefit, and hence protection offered by the friendship (Furman, 1996). Friendship quality is a multidimensional construct composed of both positive features (provision of security, companionship, help, intimacy or closeness) and a negative feature (conflict) (Bukowski, Hoza, & Boivon, 1994; Gifford-Smith & Brownell, 2003). An underlying advantage of measuring the quality of dyadic friendships concerns the focus on self-perceptions of friends, which is unique and special, and cannot be obtained by peers or adults (Furman, 1996). Further, it is methodologically impractical to consider reciprocated reports of the quality of friendships, as individual perceptions about particular friendships are likely to be very different. Adolescents typically have more than one good friend, and reciprocated ‘best’ friends are unlikely to always occur, as the quality of different friendships varies (e.g. one friend might be highly rated on ‘help’ and another friend could be highly rated for ‘companionship’). High quality friendship typically involves low levels of conflict and high levels of the positive features (Berndt, 2002). Friendships which provide little support can be those with high levels of conflict, or those with low levels of security, companionship, and help (Furman, 1996; Gifford-Smith & Brownell, 2003; Hodges, Malone, & Perry, 1997). Some studies have reported that friends of victims provide limited help and support (Champion, Vernberg, & Shipman, 2004; Hodges et al., 1997; Pellegrini, Bartini, & Brooks, 1999). Victims are also reported to experience more conflict problems with their best friend, and greater difficulty in managing confrontation in peer interactions (Champion et al., 2004). However, the evidence base is still somewhat limited in this important area of friendship quality, and the mitigating effect of friendships on the emotional consequences of victimisation. Whether the quality of friendships and peer relations act as risk or protective factors could depend on the nature of victimisation, which can be categorised into direct or relational bullying (Wolke et al., 2001). The former refers to physically aggressive acts such as being hit or threatened, whereas the latter refers to social exclusion, peer manipulation and rumour spreading (Crick & Grotpeter, 1995). In one large Dutch study (van der Wal, de Wit, & Hirasing, 2004), there was a marked difference between risk of depression in direct and relational victims. Victims of relational bullying had a substantial and significantly increased risk of depression, whereas victims of direct bullying were, at minimal or no risk of depression, following adjustment for confounding factors. Direct and relational victimisation are comprised of different social behaviours too, with the success of relational aggressive acts depending more heavily on the role of the peer group (Crick & Grotpeter, 1995). We are aware of only one study to have considered qualities of best friendships, together with direct and relational victimisation behaviour as possible risk/protective factors in depression and social anxiety among adolescents (Greca & Harrison, 2005). Results revealed that relational victimisation (but not direct victimisation) predicted high social anxiety and depressive symptoms, only if this was accompanied by negative interaction with a best friend (e.g. exclusions and peer pressure). In other words, the negative quality of friendships acted as a precipitating factor for relational victimisation. Given this background, the current study was designed to investigate a number of relevant hypotheses. Firstly, do victims report having a best friend, and is the quality of that dyadic friendship comparable to non-victims? Second, do victims report more loneliness than non-victims, and if so is this associated with the quality of friendship with a best friend. Thirdly, is the prevalence of clinically significant levels of emotional problems greater in victims, and if so is this related to friendship quality? Each of these three hypotheses was evaluated with respect to direct and Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 4 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 relational victimisation separately in a cross-sectional, caseecontrol (victims vs. non-victims) study in one secondary school in the UK. For each analysis gender was introduced as a predictor variable and potential moderator of any victimisation effect (Baron & Kenny, 1986), given that gender differences in quality of friendship (Johnson, 2004; Phillipsen, 1999) and internalising problems have been reported in previous studies. Method Participants Secondary school children from a south London school in the UK were approached to participate in the research programme. The school was located in a suburb of south London and had a catchment area of students from predominantly lower/middle socioeconomic status families. Adolescents (N ¼ 401) from 14 classes took part in the study from year groups 7e11 (mean age M ¼ 13.5, SD ¼ 1.5). Overall academic abilities for the sample indicated that 64% of students were in higher ability classes and 36% were from lower ability classes. Only two adolescents (0.5%) were not given parental consent to participate in the study, and thus 98% of the original sample completed the study. The school constituted a diverse ethnic mix of children. Table 1 provides the demographic make up of the sample. Measures School Relationships Questionnaire (SRQ) This questionnaire assessed the incidence of bullying behaviour over a 6 month time frame, and was developed from the ‘‘Bullying & Friendship Patterns’’ child interview (Wolke et al., Table 1 Sample characteristics Sample size (%) Gender Male Female Year groups 6 (11 years) 7 (12 years) 8 (13 years) 9 (14 years) 10 (15 years) 11 (16 years) Ethnicity Black Asian White Mixed Other 188 (47) 213 (53) 33 85 91 84 65 43 (8.2) (21.2) (23.0) (21.0) (16.2) (11.0) 169 99 93 33 7 (42.0) (25.0) (23.0) (8.0) (1.7) Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS S. Woods et al. / Journal of Adolescence xx (2008) 1e16 5 2001), which used similar questions to the Olweus (1991) Bullying Questionnaire (Woods & White, 2005). The interview was modified into a self-completion questionnaire and changes were made to reduce the number of questions and make them appropriate for adolescents (see Woods & White, 2005 for a full description). The questionnaire was subdivided into four sections; ‘Direct Aggression Received’ (e.g. ‘Have you been hit or beaten up?’), ‘Verbal & Relational Aggression Received’ (e.g. Have other pupils called you nasty names?, Have other pupils not wanted to hang around with you [to make you upset]?, Have other pupils said they wouldn’t be friends with you anymore, or said they would tell-tale [tell other people things about you]?, Have other pupils told lies, said nasty things, or told stories about you that were not true?, Have other pupils spoilt activities [e.g. sports games or class activities] on purpose [to make you upset]?), ‘Direct Aggression Given’ (e.g. ‘Have you threatened/blackmailed someone?’), and ‘Verbal & Relational Aggression Given’ (e.g. Have you not hung around with another pupil/other pupils, to make them upset?). Responses were scored 0e2 depending on how frequently the individual had been involved in a victimisation or bullying situation (‘‘not at all/seldom’’ ¼ 0, ‘‘frequently’’ ¼ 1 or ‘‘very frequently’’ ¼ 2). The SRQ allows the classification of four roles: (1) ‘pure bully’ (perpetrator of aggressive acts only), (2) ‘pure victim’ (receiver of aggressive acts), (3) ‘bully/victim’ (both perpetrator and receiver), and (4) ‘neutral’ roles (non-involvement in bullying or victimisation). For the purposes of this study, we included only those students who met the criteria for ‘pure victim’ and ‘neutral’ roles. Scores of 1 or 2 (frequently or very frequently) in the ‘Direct Aggression Received’ and scores of 1 or 2 in the ‘Verbal & Relational Aggression Received’ section resulted in categorisation as a direct and/or relational victim. Subjects were categorised as direct and/or relational neutrals if all responses on the SRQ were ‘‘not at all/ seldom’’ (score of 0). Strengths and Difficulties Questionnaire (SDQ) This scale consists of 25 items scored according to five subscales: pro-social behaviour, hyperactivity, emotional symptoms, conduct problems and peer problems (Goodman, 1997). Each item is scored 0e2 in response to ‘not true’, ‘somewhat true’ or ‘certainly true’ and a total score ranging from 0 to 10 is generated for each subscale by summing the scores for the five items that make up the scale. Cut-off scores are provided for those with likely clinically significant levels of behavioural/emotional problems using the following bandings; 90% of children in the sample are normal/borderline, and 10% (>90th percentile) are in the clinical range (Goodman, 1997). The clinical validity of both the cut-off scores, and the adolescent self-report version, have been documented in numerous papers (e.g. Becker, Hagenberg, Roessner, Woerner, & Rothernberger, 2004; Bourdon, Goodman, Rae, Simpson, & Koretx, 2005). Friendship Activity Questionnaire (FAQ) The FAQ was chosen for this study as it focuses on the special and unique dyadic peer relationship with a best friend (Bukowski et al., 1994). This is very different to the focus in the School Relationships Questionnaire that addresses general peer relationships and interactions. The FAQ is a self-report questionnaire comprising 37 items that are rated using a five-point Likert type scale (1 ¼ not true to 5 ¼ really true), for example, ‘My friend and I help each other’. Five dimensions of friendship (companionship, help, security, closeness, and conflict) are covered. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 6 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 Internal consistency for each dimension is high, and the scale has good criterion validity indicated by higher ratings for more stable friendships (Bukowski et al., 1994). Students in the study were asked to write the initial of their best friend on the front page of the FAQ, and then answer all questions with reference to the designated ‘best friend’. The Loneliness and Social Dissatisfaction Questionnaire (LSDQ) The LSDQ assesses adolescents’ self-perceptions about their own peer relationships (Asher & Wheeler, 1985). The scale is comprised of 24 items measured according to a five-point Likert scale (1 ¼ always true to 5 ¼ never true). The questionnaire consists of 16 primary items which focus on adolescents’ feelings of loneliness (e.g. I feel left out of things at school), feelings of social inadequacy (e.g. I get along with my classmates), and also eight filler items focused on children’s hobbies and interests (e.g. I like music). Scores on the LSDQ are often used as a continuous measure of loneliness and social dissatisfaction. Evidence suggests that the LSDQ is a reliable and valid measure (Cronbach’s alpha 0.90) (Asher & Paquette, 2003). Procedure Ethical approval for the study was granted by the University of Hertfordshire Ethics Committee. The school was selected on a convenience basis and involvement of the school was approved by the head teacher. All data were collected during regular class time and standardised instructions including confidentiality and withdrawal issues were read to students before they participated. The importance of answering each of the five questionnaires truthfully and privately was also communicated to the students. During data collection, teachers were asked to remain unobtrusive. The questionnaires took approximately 30e40 min to complete. Results Sample characteristics for direct and relational victimisation Table 2 reports the incidence of both direct and relational victimisation, and neutral status (Note: As the current study was concerned with victimisation status compared to neutral status, the incidence of bully and bully/victim roles are not reported). Twice as many adolescents Table 2 Incidence of direct and relational victimisation, and neutral status Direct victimisation frequency (%) Pure victim Male Female Neutral Male Female Relational victimisation frequency (%) 19 (10.1) 22 (10.3) 42 (22.3) 43 (20.2) 141 (75.0) 178 (83.6) 99 (52.7) 128 (60.1) The percentage refers to the whole sample (N ¼ 401) before the exclusion of bullies and bully/victim classifications. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 7 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 reported being relational victims compared to physical victims. No gender differences were found for the prevalence of self-reported direct and relational victims. A kappa coefficient was calculated to examine the agreement between self-reported direct and relational victimisation. This produced a low level of agreement (kappa ¼ 0.22), which reinforces the argument that direct and relational victimisation are distinct behaviours that should be investigated and analysed separately in relation to neutral (non-victims) when evaluating the current study hypotheses. Friendship quality in victims Two-factor MANOVAs were used to calculate whether friendship quality, as measured by the five dimensions on the FAQ, varied as a function of the two factors: victim status and gender. Separate MANOVAs were calculated for direct and relational victimisation. i Direct victimisation: The main effects of both victim status, F (5, 352) ¼ 2.9, P ¼ 0.01, and gender, F (5, 352) ¼ 7.6, P < 0.001, were statistically significant. The interaction between victim status and gender did not reach statistical significance though, F (5, 352) ¼ 1.05, P ¼ 0.39. Univariate ANOVAs were used to explore these main effects. Females had significantly elevated scores for all four positive dimensions of friendship quality (P < 0.001), and reduced scores for conflict (P < 0.01), compared to males. Direct victims scored significantly higher than neutrals on three of the four positive dimensions of the FAQ, namely Companionship, Security and Closeness. They also obtained higher scores on the Help scale, although this was not statistically significant (see Table 3). Contrary to expectations, this indicated that direct victims reported higher quality of friendship compared to neutrals across all four positive dimensions. Direct victims did not differ from neutrals on the negative dimension of Conflict (P > 0.10). ii Relational victimisation: The main effect of victim status was statistically significant, F (5, 304) ¼ 2.2, P ¼ 0.05, but appeared not to be a large effect. Again the main effect of gender was highly significant, F (5, 304) ¼ 17.7, P < 0.001. Females had significantly elevated scores for all four positive dimensions of friendship quality (P < 0.001) and lower Conflict scores (P < 0.01). The interaction between gender and victim status did not reach statistical significance, F (5, 304) ¼ 0.52, P ¼ 0.76. Results of the univariate ANOVAs revealed that relational victims scored significantly higher on Conflict (P ¼ 0.006), but did not differ from neutrals on any of the four positive dimensions of the FAQ (see Table 4). Table 3 Univariate comparisons of direct victims vs. neutrals for all five Friendship Quality (FAQ) variables FAQ variable Companionship Help Security Closeness Conflict Means (SD) Direct victims (n ¼ 41) Neutrals (n ¼ 319) 15.50 21.60 18.40 20.40 7.30 13.96 20.40 17.05 18.70 6.30 (3.3) (4.1) (2.9) (3.9) (4.4) (3.5) (4.0) (3.6) (4.5) (3.9) F value Significance Cohen’s d 7.3 3.3 5.6 5.2 1.8 0.007 0.070 0.018 0.023 0.185 0.44 0.30 0.38 0.38 0.25 Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 8 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 Table 4 Univariate comparisons of relational victims vs. neutrals for all five Friendship Quality (FAQ) variables FAQ variable Means (SD) Relational victims (n ¼ 85) Neutrals (n ¼ 227) Companionship Help Security Closeness Conflict 13.8 20.1 16.9 18.8 7.1 14.3 20.8 17.2 19.1 5.8 (3.8) (4.4) (3.9) (4.9) (4.2) (3.4) (3.7) (3.4) (4.4) (3.7) F value Significance Cohen’s d 1.60 2.35 0.70 0.37 7.70 0.20 0.13 0.40 0.54 0.006 0.14 0.18 0.08 0.07 0.34 Do victims report more loneliness, and is loneliness moderated by gender and friendship quality? To calculate whether victims reported more loneliness, multiple regression analysis (MRA) was used. LSDQ scores were initially regressed onto the victimisation factor. To explore whether gender and friendship quality were moderators of any association between victimisation and LSDQ scores the statistical method proposed by Baron and Kenny (1986) was used. According to Baron and Kenny, a factor is deemed to be a moderator if there is a significant two-way interaction between the factor and the independent ‘predictor’ variable. Using MRA, the two-way interactions between victimisation and gender, and victimisation and friendship quality were tested for statistical significance. Given the multicollinearity between the five dimensions in the FAQ, it was only necessary to use one dimension of the FAQ, namely Help, in the MRA. Separate MRAs were conducted for direct and relational victimisation, respectively. i Direct victimisation: The main effect of victim status did not meet criteria for statistical significance, F (1, 358) ¼ 2.7, P ¼ 0.10. Thus, LSDQ scores were not significantly elevated for direct victims taken as a whole. The two-way interactions between victimisation and gender, F (1, 356) ¼ 3.9, P ¼ 0.05, and victimisation and Help, F (1, 356) ¼ 3.7, P ¼ 0.05, were both statistically significant, indicating that when both gender and friendship quality are entered into the regression analysis then subtle associations between victimisation and the LSDQ come to light. In the case of gender, being a male victim increased LSDQ scores, whereas being a female victim did not (see Table 5). The interaction between Help and victimisation is presented graphically in Fig. 1. Although increasing quality of friendship was significantly associated with reduced levels of loneliness in direct victims and neutrals, this relationship was more marked in victims (R2 for the MRA were 0.26 for victims as opposed to 0.08 for neutrals). For lower reported values of Help, victims reported more loneliness than neutrals, whereas for higher values of Help, victims and neutrals both reported low levels of loneliness. Indeed it can be seen in Fig. 1 that direct victims with high levels of friendship quality reported lower levels of loneliness than neutrals with low levels of friendship quality. ii Relational victimisation: A significant main effect was found for victim status, F (1, 310) ¼ 17.45, P < 0.001. With an R2 ¼ 0.051, this indicates a substantial association between relational victimisation and loneliness. The two-way interactions between victimisation and gender, F (1, 308) ¼ 0.4 P ¼ 0.55, and victimisation and Help, F (1, 308) ¼ 0.4, P ¼ 0.53, were not Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 9 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 Table 5 Loneliness in both direct and relational victims, and neutrals Means (SD) Direct victims Neutrals Males Females Total 32.70 (10.1) (n ¼ 19) 26.50 (8.8) (n ¼ 22) 29.30 (9.8) (n ¼ 41) 27.70 (8.8) (n ¼ 141) 26.90 (7.7) (n ¼ 178) 27.20 (8.2) (n ¼ 319) Relational victims Neutrals Males Females Total 31.4 (8.8) (n ¼ 42) 29.6 (9.05) (n ¼ 43) 30.50 (8.9) (n ¼ 85) 26.3 (8.8) (n ¼ 99) 25.9 (7.7) (n ¼ 128) 26.00 (8.2) (n ¼ 227) F ratio P-value Cohen’s d 5.2 0.065 2.7 0.05* 1* 0.10 0.56 0.00 0.25 9.9 7.3 17.45 0.004* 0.016* 0.000 0.58 0.44 0.52 *Bonferroni correction, k ¼ 2. significant, indicating that neither gender nor friendship quality had any influence on the association between victimisation and loneliness. Do victims experience more clinically significant emotional problems, and are these moderated by friendship quality? Scores on the emotional subscale of the SDQ were classified into ‘borderline/abnormal’ versus ‘normal’ as described in the section Method. This revealed that 9.76% of direct victims had borderline/ abnormal emotional problems compared to 11.29% of neutrals. Relational victims were considerably more likely to have borderline/clinical emotional problems compared to non-victims (relational 80 70 LDSQ Score 60 50 40 Subject Status 30 Direct victim Rsq = 0.2640 20 Neutral Rsq = 0.0874 10 0 10 20 30 Friendship Quality (Help) Fig. 1. Scatter plot of LSDQ scores against Friendship Quality (Help) for direct victims and neutrals. Separate regression lines are provided to illustrate the significant interaction between the victimisation factor and friendship quality. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 10 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 Table 6 Emotional problems in direct and relational victims, and tests for the moderating effects of gender and friendship quality (FAQ) Risk factor Beta SE Wald df P-value Adjusted odds ratio (Exp B) 95% CI lower limit Unadjusted Direct victim Relational victim Gender 0.16 1.1 0.99 0.56 0.38 0.35 0.09 0.87 7.8 1 1 1 0.77 0.003 0.005 0.85 3.06 2.70 0.29 1.45 1.34 2.52 6.40 5.40 1.3 0.081 1.16 1.13 0.179 0.883 1.3 0.205 1.73 1 1 1 0.25 0.651 0.189 0.27 1.08 3.19 0.03 0.764 0.566 2.5 1.54 18.00 0.000 0.099 0.000 1 0.998 1.00 0.824 1.21 Interaction Direct victim gender Direct victim help Relational victim gender Relational victim help 95% CI higher limit Wald Statistic e this is used in logistic regression to determine whether a variable is a significant predictor of the outcome. Exp B e this is used in logistic regression as a indicator of the change in odds from a unit change in a predictor variable. victims: 18.82% vs. neutrals: 7.08%). Girls were nearly three times more likely to have emotional problems in the borderline/clinical range compared to boys (girls: 15.49% vs. boys: 6.38%). Logistic regression was used to calculate whether the prevalence of emotional problems was significantly elevated in victims compared to neutrals. To explore whether gender and friendship quality could be potential moderators, the two-way interactions between victimisation and gender, and victimisation and the Help score were entered into the logistic regression model (see Table 6). Again direct and relational victimisations were treated as separate factors. As expected, the odds for emotional problems were similar for direct victims and neutrals (OR ¼ 0.85), but significantly elevated in relational victims (OR ¼ 3.06). None of the two-way interactions between either of the measures of victimisation, and the two purported moderating factors were statistically significant. This indicates that neither gender nor friendship quality were moderating factors of the association between victimisation and emotional problems. Thus, the absence of emotional problems in direct victims, and the excess of emotional problems in relational victims were both unaffected by friendship quality. Discussion The present study investigated whether victims of direct and relational bullying would report poorer quality of friendships compared to non-victims (neutrals), and whether friendship quality is related to loneliness and clinically significant levels of emotional problems. The main findings of the study were Before any moderating factors were taken into consideration, relational victims, but not direct victims reported increased loneliness and greater risk of emotional problems. Direct victims reported significantly higher quality friendships, compared to neutrals, on all positive dimensions of friendship. Relational victims however, reported higher levels of conflict. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS S. Woods et al. / Journal of Adolescence xx (2008) 1e16 11 Friendship quality moderated the association between direct victimisation and loneliness, but it did not moderate the association between direct victimisation and emotional problems. For high levels of friendship quality, loneliness in direct victims was low, but for low levels of friendship quality, loneliness in direct victims was relatively high. Friendship quality did not moderate the association between relational victimisation, and either loneliness or emotional problems. The incidence of relational victimisation was approximately double than for direct victimisation among this adolescent sample, and the frequency of relational and direct victimisation was equivalent among males and females. This finding is in line with the developmental view of bullying proposed by Björkqvist and colleagues (e.g. Björkqvist, Lagerspetz, & Kaukiainen, 1992) that relational and indirect means of aggression are used more frequently once competent social and cognitive abilities have developed. Taken as a group, relational victims appear to be at risk of both loneliness and emotional problems, and this was true for both males and females alike. Direct victims showed no increases in loneliness, or any risk from emotional problems, compared to neutrals. This apparently greater risk for loneliness and emotional problems in relational but not direct victims is an important finding. A number of studies of emotional disorders and loneliness among school children, have reported increased levels of emotional problems (Bond et al., 2001; Denny, Clark, Fleming, & Wall, 2004) and loneliness (Nangle, Erdley, Newman, Mason, & Carpenter, 2003) among victims, but their criteria for identification of victims did not distinguish between direct and relational victimisation. However, our results are in agreement with the findings reported by van der Wal et al. (2004). In that study the impact of being bullied on depression was higher for indirect (i.e. relational) bullying compared to direct bullying. Similarly Paquette and Underwood (1999) found that adolescents reported that social aggression made them feel more sad and bad about themselves compared to physical aggression. It would appear from the current findings that whereas the experience of direct bullying might be coped with relatively well, dealing with relational bullying is more difficult. This raises the question, ‘‘Why might this be so?’’ The study reported here provides some evidence about the importance of dyadic friendships in relation to victimisation behaviour. Friendship quality acted as a moderating factor on direct victimisation, but not on relational victimisation. Direct victims who reported the lowest levels of friendship quality did indeed experience higher levels of loneliness, whereas those reporting high levels of friendship quality did not experience loneliness. In line with previous research it was hypothesised that victims would either suffer from not having friends or would report significantly lower quality friendships, and as such this combination would result in an increased risk of loneliness and emotional problems (Boulton et al., 1999; Hodges et al., 1999; Kochenderfer & Ladd, 1997; Kochenderfer-Ladd & Wardrop, 2001; Rigby, 2000). However, the opposite pattern emerged in the case of direct victims. As a group they reported significantly higher quality of friendship with their best friend, compared to neutral adolescents. This was true for all four positive subscales of the FAQ, and could have arisen for a number of reasons. Firstly, direct victims may turn to a close friend who is not a member of the bullying network as a way of coping with victimisation (i.e. companionship, security etc.), in keeping with the ‘Friendship Protection Hypothesis’ (Boulton et al., 1999). Thus high friendship quality may act as a protective buffer against the negative consequences of bullying. It seems that having a best Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 12 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 friend does not prevent bullying from happening but it might prevent feelings of loneliness, depression and alienation (Buhrmester, 1990; Ladd & Kochenderfer, 1996). Our results offer some initial support for this friendship protection hypothesis. A second possible explanation could be that direct victims have distorted views of close friendships and view a relationship with a peer in an overly positive manner, when in fact it provides no more companionship or help than that experienced by non-victims. One way to test this out would be to compare FAQ ratings from the best friend with those of the index case. However, the fact that victims come to perceive a relationship more positively is in itself a potential coping strategy. Thirdly, direct victims are more likely to be bullied by an individual(s) outside of their direct peer group which would make it easier for them to escape the torment of bullying, and perhaps derive greater pleasure from their friends than is possible for relational victims (Crick & Bigbee, 1998; Pellegrini, Bartini, & Brooks, 1999). In contrast, relational victims reported reduced levels of friendship quality compared to neutrals. Although they did not differ significantly from neutrals on any of the four positive dimensions, there was a substantially elevated level of conflict in their friendship. This again points to the importance of considering direct and relational bullying as two different types of aggression. Why might friendship quality differ between direct and relational victims? First, relational bullying behaviour is different from direct bullying in its social structure as it generally occurs within the peer group that can include the victims ‘best friend’ (Crick & Grotpeter, 1995). Unlike direct victims, relational victims may have fewer alternatives for friendship and may feel betrayed by the peer group as relationships that should be based on trust have been broken. This would not only increase the risk of feelings of isolation and loneliness (Crick & Bigbee, 1998; Crick, Bigbee, & Howes, 1996), but also lead to increased levels of conflict in their friendships. However, another potential reason for the increased risk of loneliness and emotional problems in relational victims could be that relational bullying is a more manipulative and covert means of behaviour compared to direct bullying, meaning that it could go unnoticed by peers, teachers and parents. There are reports that adolescents may feel unable to discuss relational bullying due to its covert nature (Juvonen & Graham, 2001; Naylor, Cowie, & del Rey, 2001). The results presented here on friendship quality among victims appear to be somewhat in contradiction to those obtained from sociometric measures of group-based peer interactions that consider various peer group classifications including popular, average, neglected, rejected, and controversial. Sociometric measures have generally reported that those children classified as rejected have a much poorer quality of peer interactions (Ladd, 1983; Newcomb, Bukowski, & Pattee, 1993). However, in the study reported by Champion et al. (2004), which examined dyadic friendships in victims, it was also reported that victims do not lack friends, but that their friendships were marked by more conflict with their best friend. The distinction between dyadic peer interactions (i.e. friendships) and group-based peer interactions (e.g. popularity, centrality within a social network) is not a trivial distinction. Rejected (i.e. unpopular) children often report at least one close friendship, whereas a third of popular children do not report a close friend (GiffordSmith & Brownell, 2003). Limitations of this study merit note. First, cause and effect relationships could not be addressed in this cross-sectional study, and therefore we cannot make any conclusions about whether dyadic friendships serve as a protective factor against different forms of victimisation, and feelings of loneliness. There is a need for longitudinal studies to tease out these Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS S. Woods et al. / Journal of Adolescence xx (2008) 1e16 13 relationships. This study was unable to determine whether an adolescent’s choice of best friend was a reciprocated friendship. However, we were specifically interested in the dyadic relationship of friendships from the adolescent’s own perspective and it has been argued that self-perceptions of friendship provide unique and special data that cannot be obtained by peers or adults (Furman, 1996). Self-report questionnaires were used throughout the study, which could have resulted in problems due to shared variance. However, the results provide evidence to substantiate that there was little overlap between the questionnaires in content and that the results do not fit with the shared method variance artefact. The Friendship Activity Questionnaire (FAQ) refers specifically to the dyadic relationship with a best friend, whereas the School Relationships Questionnaire (SRQ) relates to negative aspects of the peer group and the Loneliness and Social Dissatisfaction Questionnaire (LSDQ) to all social relationships (including family). Research using self-reports of victimisation has ascertained that they are a valid method (Ahmad & Smith, 1990) and teachers and/or parents are often unaware of bullying behaviour, particularly relational victimisation (e.g. Nicolaides, Toda, & Smith, 2002). The ethnic diversity of the current sample could have influenced the results. Future studies should consider a more homogenous sample from a wider range of schools. The results of this study have a number of potential implications. For future intervention strategies to be successful, consideration needs to be given to whether victimisation is direct or relational. Friendships, if offering a high level of quality across the positive dimensions, appear to confer protection against loneliness amongst those experiencing direct victimisation. However, where lower quality friendships are reported then there appears to be an increased risk of loneliness in direct victims. Friendship quality though did not appear to provide protection for relational victims. We speculate here that best friends are not involved in direct bullying but may be involved, or associated with those perpetrating relational bullying. Thus, best friends can readily be used to help to mitigate the negative consequences of direct bullying. However, interventions focusing on the support from best friends amongst relational victims may require a different approach, such as identifying sources of conflict in the friendship and educating the best friend about their potential contribution as a buffer against the relational bullying coming from within the peer group. Acknowledgements We would like to thank the teachers and students of St. Mary’s RC High School, Croydon, UK for agreeing to participate in this study. In particular, special thanks to Mr. Ujhabo and Mrs. Frankerviech. References Ahmad, Y. S., & Smith, P. K. (1990). Behavioural measures review: bullying in schools. Newsletter: Association for Child Psychology & Psychiatry, 12, 26e27. Asher, S. R., & Paquette, J. A. (2003). Loneliness and peer relations in childhood. Current Directions in Psychological Science, 12, 75e78. Asher, S. R., & Wheeler, V. A. (1985). Children’s loneliness: a comparison of rejected and neglected peer status. Journal of Consulting and Clinical Psychology, 53, 500e505. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 14 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 Baron, R. M., & Kenny, D. A. (1986). The moderator-mediator variable distinction in social psychological research: conceptual, strategic and statistical considerations. Journal of Personality and Social Psychology, 51, 1173e1182. Becker, A., Hagenberg, N., Roessner, V., Woerner, W., & Rothernberger, A. (2004). Evaluation of the self-reported SDQ in a clinical setting: do self-reports tell us more than ratings by adult informants? European Journal of Child and Adolescent Psychiatry, 13, 17e24. Berndt, T. J. (2002). Friendship quality and social development. Current Directions in Psychological Science, 11, 7e10. Björkqvist, K., Lagerspetz, K. M. J., & Kaukiainen, A. (1992). Do girls manipulate and boys fight? Developmental trends in regard to direct and indirect aggression. Aggressive Behaviour, 18, 117e127. Bond, L., Carlin, J. B., Rubin, K., & Patton, G. (2001). Does bullying cause emotional problems? A prospective study of young teenagers. British Medical Journal, 323, 480e484. Boulton, M. J., Trueman, M., Chau, C., Whitehand, C., & Amatya, K. (1999). Concurrent and longitudinal links between friendship and peer victimization: implications for befriending interventions. Journal of Adolescence, 22, 461e466. Boulton, M. J., & Underwood, K. (1992). Bully/victim problems among middle school children. British Journal of Educational Psychology, 62, 73e87. Bourdon, K. H., Goodman, R., Rae, D. S., Simpson, G., & Koretx, D. S. (2005). The Strengths and Difficulties Questionnaire: US normative data and psychometric properties. Journal of American Academy of Child & Adolescent Psychiatry, 44, 557e564. Buhrmester, D. (1990). Intimacy of friendship, interpersonal competence and adjustment during preadolescence and adolescence. Child Development, 61, 1101e1111. Bukowski, W. M., Hoza, B., & Boivon, M. (1994). Measuring friendship quality during pre- and early adolescence: the development and psychometric properties of the Friendship Quality Scale. Journal of Social and Personal Relationships, 11, 471e484. Champion, K., Vernberg, E. M., & Shipman, K. (2004). Non-bullying victims of bullies: aggression, social skills, and friendship characteristics. Applied Developmental Psychology, 24, 535e551. Craig, W. M. (1998). The relationship among bullying, victimization, depression, anxiety and aggression in elementary school children. Personality and Individual Differences, 24, 123e130. Crick, N. R., & Bigbee, M. A. (1998). Relational and overt forms of peer victimization: a multi-informant approach. Journal of Consulting and Clinical Psychology, 66, 337e347. Crick, N. R., Bigbee, M. A., & Howes, C. (1996). Gender differences in children’s normative beliefs about aggression: how do I hurt thee? Let me count the ways. Child Development, 67, 1003e1014. Crick, N. R., & Grotpeter, J. K. (1995). Relational aggression, gender, and socialepsychological adjustment. Child Development, 66, 710e722. Denny, S., Clark, T. C., Fleming, T., & Wall, M. (2004). Emotional resilience: risk and protective factors for depression among alternative education students in New Zealand. American Journal of Orthopsychiatry, 74, 137e149. Egan, S. K., & Perry, D. G. (1998). Does low self-regard invite victimization? Developmental Psychology, 34, 299e 309. Espelage, D. L., & Swearer, S. M. (2003). Research on school bullying and victimization: what have we learned and where do we go from here? School Psychology Review, 32, 365e383. Furman, W. (1996). The measurement of friendship perceptions: conceptual and methodological issues. In W. M. Bukowski (Ed.), The company they keep: Friendship in childhood and adolescence (pp. 41e65). Cambridge, MA: Cambridge University Press. Gettinger, M. (2003). Promoting social competence in an era of school reform: a commentary on Gifford-Smith and Brownell. Journal of School Psychology, 41, 299e304. Gifford-Smith, M. E., & Brownell, C. A. (2003). Childhood peer relationships: social acceptance, friendships and peer networks. Journal of School Psychology, 41, 235e284. Goodman, R. (1997). The strengths and difficulties questionnaire: a research note. Journal of Child Psychology and Psychiatry, 38, 581e586. Greca, L., & Harrison, H. M. (2005). Adolescent peer relations, friendships, and romantic relationships: do they predict social anxiety and depression? Journal of Clinical Child and Adolescent Psychology, 34, 49e61. Griffin, R. S., & Gross, A. M. (2004). Childhood bullying: current empirical findings and future directions for research. Aggression and Violent Behavior, 9, 379e400. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS S. Woods et al. / Journal of Adolescence xx (2008) 1e16 15 Hawker, D. J. S., & Boulton, M. J. (2000). Twenty years’ research on peer victimization and psychosocial adjustment: a meta-analytic review of cross-sectional studies. Journal of Child Psychology and Psychiatry, 41, 441e456. Hay, D., Payne, A., & Chadwick, A. (2004). Peer relations in childhood. Journal of Child Psychology and Psychiatry, 45, 84. Haynie, D. L., Nansel, T., Eitel, P., Crump, A. D., Saylor, K., Yu, K., et al. (2001). Bullies, victims, and bully/victims: distinct groups of at-risk youth. Journal of Early Adolescence, 21, 29e49. Hodges, E. V. E., Boivin, M., Vitato, F., & Bukowski, W. M. (1999). The power of friendship: protection against an escalating cycle of peer victimization. Developmental Psychology, 35, 94e101. Hodges, E. V. E., Malone, M. J., & Perry, D. G. (1997). Individual risk and social risk as interacting determinants of victimization in the peer group. Developmental Psychology, 33, 1032e1039. Johnson, H. D. (2004). Gender, grade and relationship differences in emotional closeness within adolescent friendships. Adolescence, 39, 243e255. Juvonen, J., & Graham, S. (2001). Peer harassment in school. New York: The Guildford Press. Kochenderfer, B. J., & Ladd, G. W. (1997). Victimized children’s responses to peers’ aggression: behaviours associated with reduced versus continued victimization. Development and Psychopathology, 9, 59e73. Kochenderfer-Ladd, B., & Wardrop, J. L. (2001). Chronicity and instability of children’s peer victimization experiences and predictors of loneliness and social satisfaction trajectories. Child Development, 71, 134e151. Ladd, G. W. (1983). Social networks of popular, average, and rejected children in school settings. Merrill-Palmer Quarterly, 29, 283e307. Ladd, G. W., & Kochenderfer, B. J. (1996). Friendship quality as a predictor of young children’s early school adjustment. Child Development, 67, 1103e1118. Nangle, D. W., Erdley, C. A., Newman, J. E., Mason, C. A., & Carpenter, E. M. (2003). Popularity, friendship quantity and friendship quality: interactive influences on children’s loneliness and depression. Journal of Clinical Child and Adolescent Psychology, 32, 546e555. Naylor, P., Cowie, H., & del Rey, R. (2001). Coping strategies of secondary school children in response to being bullied. Child Psychology & Psychiatry Review, 6, 114e120. Newcomb, A. F., Bukowski, W. M., & Pattee, L. (1993). Children’s peer relations: a meta-analytic review of popular, rejected, neglected, controversial, and average sociometric status. Psychological Bulletin, 113, 99e128. Nicolaides, S., Toda, Y., & Smith, P. K. (2002). Knowledge and attitudes about school bullying in trainee teachers. British Journal of Educational Psychology, 72, 105e118. Olweus, D. (1991). Bully/victim problems among school children: some basic facts and effects of a school-based intervention program. In D. J. Pepler, & K. Rubin (Eds.), The development and treatment of childhood aggression (pp. 411e438). Hillsdale, NJ: Erlbaum. Olweus, D. (1999). Norway. In P. K. Smith, Y. Morita, J. Junger-Tas, D. Olweus, R. Catalano, & P. Slee (Eds.), The nature of school bullying: A cross-national perspective (pp. 28e48). London: Routledge. Paquette, J. A., & Underwood, M. K. (1999). Gender differences in young adolescents’ experiences of peer victimization: social and physical aggression. Merrill-Palmer Quarterly, 45, 242e266. Parker, J. G., & Asher, S. R. (1987). Peer relations and later personal adjustment: are low-accepted children at risk? Psychological Bulletin, 102, 357e389. Parker, J. G., & Asher, S. R. (1993). Friendship and friendship quality in middle childhood: Links with peer group acceptance and feelings of loneliness and social dissatisfaction. Developmental Psychology, 29, 611e621. Pellegrini, A. D., Bartini, M., & Brooks, F. (1999). School bullies, victims, and aggressive victims: factors relating to group affiliation and victimization in early adolescence. Journal of Educational Psychology, 91, 216e224. Petit, G. S., Laird, R. D., Dodge, K. A., Bates, J. E., & Criss, M. A. (2001). Antecedents and behavior-problem outcomes of parental monitoring and psychological control in early adolescence. Child Development, 72, 583e 598. Phillipsen, L. C. (1999). Associations between age, gender, and group acceptance and three components of friendship quality. Journal of Early Adolescence, 19, 438e464. Rigby, K. (1998). Suicidal ideation and bullying among Australian secondary school students. The Australian Educational and Developmental Psychologist, 15, 45e61. Rigby, K. (2000). Effects of peer victimization in schools and perceived social support on adolescent well-being. Journal of Adolescence., 23, 57e68. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005 ARTICLE IN PRESS 16 S. Woods et al. / Journal of Adolescence xx (2008) 1e16 Salmon, G., James, A., & Smith, D. M. (1998). Bullying in schools: self reported anxiety, depression, and self esteem in secondary school children. British Medical Journal, 317, 924e925. Schäfer, M., Korn, S., Brodbeck, F. C., Wolke, D., & Schulz, H. (2005). Bullying roles in changing contexts: the stability of victim and bully roles from primary to secondary school. International Journal of Behavioral Development, 29, 323e335. Smith, P. K., Morita, Y., Junger-Tas, J., Olweus, D., Catalano, R., & Slee, P. (1999). The nature of school bullying: A cross-national perspective. London: Routledge. van der Wal, M. F., de Wit, C. A. M., & Hirasing, R. A. (2004). Psychosocial health among young victims and offenders of direct and indirect bullying. Pediatrics, 11, 1312e1317. Wolke, D., Woods, S., Schulz, H., & Stanford, K. (2001). Bullying and victimisation of primary school children in South England and South Germany: prevalence and school factors. British Journal of Psychology, 92, 673e696. Woods, S., & White, E. (2005). The association between bullying behaviour, arousal levels and behaviour problems. Journal of Adolescence, 28, 381e395. Young, R., & Sweeting, H. (2004). Adolescent bullying relationships: psychological well-being and gender atypical behaviour: a gender diagnosticity approach. Sex Roles, 50, 525e538. Zahn-Waxler, C., Klimes-Dougan, B., & Slattery, M. J. (2000). Internalising problems of childhood and adolescence: prospects, pitfalls and progress in understanding the development of anxiety and depression. Development and Psychopathology, 12, 443e466. Please cite this article in press as: Woods, S. et al., Peer victimisation and internalising difficulties: The moderating role of friendship quality, Journal of Adolescence (2008), doi:10.1016/j.adolescence.2008.03.005
© Copyright 2026 Paperzz