 |
Investing
in the power of teachers
to improve teaching and learning. |
     |
|
  |
Project
#9 November
1995 Abstract
This research was partially funded through a grant from the McDowell Foundation. However, the points of view and opinions expressed in project documents are those of the authors and do not necessarily reflect the views of the Foundation. The purpose of the Dr. Stirling McDowell Foundation for Research into Teaching is to fund research, inquiry, and dissemination of information focussing on instruction (both teaching and learning) in the context of the public elementary and secondary education system. Specifically, it will:
The Foundation is an independent charitable organization formed by the Saskatchewan Teachers' Federation in 1991. It is governed by a Board of Directors with the assistance of an advisory board of representatives from the educational and business communities. The selection and evaluation of projects funded by the Foundation is carried out by a teacher-led Project Review Committee. Inquiries concerning research supported by the McDowell Foundation may be directed to the following address:
We assessed the effects of a package consisting of self-evaluation via videotape feedback procedures on the number per hour of inappropriate and appropriate social behaviors of children with moderate cognitive challenges. The participants came from an elementary special class placement with integration in age appropriate classrooms in a small Canadian prairie city. The results showed substantial decreases in inappropriate social behaviors for the children. Social validation data indicated that observers blind to purposes of the study rated the social behavior of the participants as improved. In addition, the teacher reported the self-evaluation procedures to be more effective in improving student social behaviors than other social skills procedures used in the past. In a comprehensive review of inclusionary practices in Saskatchewan, Canada, McLeod (1995) found that regular and special education teachers are increasingly becoming concerned with how best to teach and manage children with cognitive and behavioral challenges. The emphasis on least restrictive placement of children with exceptional needs has fostered the current movement toward inclusive educational programming and placement. Children with moderate cognitive challenges frequently face social skills or social performance deficits in addition to their academic learning challenges (Saskatchewan Learning, Training & Employment, 1989a). Social development theorist, Hartup (1979, 1983) identified the demonstration of appropriate social behaviors as a key to a friendship building and maintaining. The absence of friendships often leads to escalating behavioral challenges demonstrated by the child in the classroom and on the playground (Bierman, 1987; Coie, Dodge & Kupersmidt, 1990; Gresham, 1982). Cowen, Pederson, Babigian, Izzold, and Trost (1973) report a relationship between how one functions socially and long-term adjustment. Several researchers have come to similar conclusion through extensive review and summaries of widely disparate literature (e.g., Strayhorn, Strain & Walker, 1993; Kupersmidt & Coie, 1990; Hymel, Rubin, Rowden & LeMare, 1990; Asher, Parkhurst, Hymel & Williams, 1990; Rubin, LeMare & Lollis, 1990; Kupersmidt, Coie & Dodge, 1990; Schloss, Schloss, Wood &Kiehl, 1986; and Roff, Sells & Golden, 1972). It is imperative that children with cognitive challenges be taught appropriate social behaviors to increase the likelihood of peer acceptance and friendship. Children with Moderate Cognitive Challenges Saskatchewan Education, Training and Employment's (SETE) Special Education Policy manual (1989b) states that "A pupil with mental retardation will demonstrate sub-average intellectual functioning and impaired adaptive behavior" (p. 65). This concurs with the American Association on Mental Deficiency (AAMD) which also defines mental retardation as subaverage general intellectual functioning concurrent with adaptive behavior deficits. (Grossman, 1983). When one considers adaptive behaviors, approximately one percent of the general population would be cognitively challenged with approximately one quarter of those in the moderate range (Lefrancois, 1991). Various IQ cutoff points are used to distinguish mild, moderate, severe and profound cognitive challenges. Taking into consideration AAMD, various standardized IQ measures, (Lefrancois, 1991) and SETE (1989b) classifications which all use an IQ marker in the 50 to 60 range to distinguish mild from moderate cognitive challenges, this project will define children with moderate cognitive challenges as having adaptive behavior deficits and an IQ score in the 40 to 60 range on a standardized intelligence measure. Increasingly children with moderate cognitive challenges are being educated for at least part of their day in integrated educational settings. With the growing movement toward closing adult institutional settings for people with moderate cognitive challenges, it is expected that children with cognitive challenges will be taught the skills necessary for survival in the communities in which they live. Those skills include interacting with others in socially acceptable ways including controlling behaviors and habits society views as socially acceptable ways including controlling behaviors and habits society views as socially unacceptable. Individuals who cannot function in socially acceptable ways will be at-risk for finding and keeping friends, housing and employment. "Preparation for participation in an integrated society is most effectively done in an integrated educational setting" (SETE, 1989a, p. 141). Children with moderate cognitive challenges need to identify and change their behavior which society views as inappropriate; consequently social skills programs should be an essential component of any educational plan. Social Skills Program Knitzer, Steinberg and Fleisch (1990 identified two major methods of teaching social skills: behavior management (operant) systems and cognitive-behavioral (multicomponent) programs. Operant systems involve providing a token or tangible reinforcement for appropriate behavior, or loss of a token or tangible reinforcement for inappropriate behavior. Multicomponent social skills programs would include those that teach a particular generic skill needed by students in a class, through modeling, drill and practice, role play, class discussion, feedback and guided practice (Bornstein, Bellack, Hersen, 1977). Both methods of social skill training are aimed at improving the behavior of children behaving in socially unacceptable ways. Knitzer and colleagues (1990) identify the reliance on external agents to administer contingencies in behavior management or multicomponent systems to be a major problem in social skills programs. Other weaknesses include: coercive and excess control by teachers; inability of teachers to observe all behaviors of a particular child; removal from regular curricular opportunities; reduced opportunities for students to spontaneously interact with peers ; lack of individualization and application to real life situations; poor generalization based on learned dependence on adult reinforcers; and interference in on-going flow of child-to-child interactions (see Dunlap, 1993; Walther & Beare, 1991; Knitzer, Steinberg & Fleisch, 1990; Friedman & Duchnowski, 1987; and Kazdin, 1975a, 1975b). The lack of individualization in traditional social skills research and programs may mean the idiosyncratic needs of each child are not being met. Lewis and Sugai (1993) and McConnell (1987) advocate for the inclusion of self-management procedures in all social skills interventions. Self-management procedures lessen the dependence on external agents, such as teachers, parents, or peers, to manage social behavioral programs (Nelson, Smith, Young & Dodd, 1991); and have been found to be effective in reducing off-task and disruptive behaviors in junior high school aged students (Smith, Young, West, Morgan & Rhode, 1988). Overall, the findings of studies employing self-management procedures have found the procedures to be "relatively powerful interventions, and may be a viable option to externally managed procedures" (Nelson et al., 1991, p. 177). A significant benefit of self management procedures is they are easily adapted in group contexts to the idiosyncratic behaviors and needs of children (Dunlap, Dunlap, Koegel & Koegel, 1991). Most self-management procedures require the participant to immediately self-record the occurrence or nonoccurrence of target behaviors. This is a disadvantage when the target behaviors involve social interactions with others (Kern-Dunlap, Dunlap, Clarke, Childs, White & Stewart, 1992). The use of videotape has been used as one method for incorporating self-management procedures without disrupting the natural flow of peer-to-peer interactions (e.g., DeRoo & Haralson, 1971). Videotape feedback procedures involve videotaping participants at a particular activity and then playing the video back for them (Osborne, Kiburz & Miller, 1986). A second advantage of video feedback is it offers a permanent record of what transpired thereby limiting defensiveness and arguments (Walther & Beare, 1991). A third advantage is it offers more accurate feedback to participants than traditional feedback methods (Booth & Fairbanks, 1984). Feedback that comes from an external source will naturally be biased, in that at minimum it will represent what the external source was looking for. Research has shown that children with externalizing behavior challenges consistently overestimate their levels of social and behavioral competence as reported bo other sources (Patterson, Kupersmidt & Griesler, 1990; Alvarez & Adelman, 1986). Videotaped feedback will facilitate children seeing for themselves the behavior others find inappropriate. A fourth advantage of videotape feedback is it can occur at a delayed time, out of the context of the situation that occurred during the taping. Given the advantage of time, an individual may be more prepared to objectively evaluate what occurred, and determine what can be learned from that experience (Walther & Beare, 1991). Several studies have demonstrated the benefit of employing videotape feedback in reducing off task and disruptive behaviors (Esveltd, Dawson & Forness, 1974; Kehle, Clarke, Jenson & Wampold, 1986; Osborne et al., 1986; Booth & Fairbank, 1984; Walther & Beare, 1991; Lonnecker, Brady, McPherson & Hawkins, 1994). In four studies involving similar procedures, one by Kern-Dunlap and colleagues (1992); two reported in Kern, Wacker, Mace, Falk, Dunlap, and Kromrey (1995); and one by Falk, Dunlap & Kern (in press) the focus was on improving peer interactions for students exhibiting social interaction challenges via videotape feedback. In the Kern-Dunlap and colleagues (1992) study, the participants were five males between the ages of eleven to thirteen. They were all described as having behavioral challenges and enrolled in one of two self-contained programs serving children with "severe emotional disturbance." A multiple baseline within schools and replicated across students with a withdrawal phase for one student was used. The intervention effects were demonstrated for each of the participants in substantially decreasing the rate of inappropriate peer interactions and increasing the rate of appropriate peer interactions. In the first study reported by Kern et a. (1995), the authors conducted a component analysis to determine if a simpler package would produce the same results as the videotape feedback package employed by Kern-Dunlap et al. (1992). Three boys between the ages of ten to twelve participated in the study. All three were described as having emotional and behavioral challenges and served in a self-contained classroom. A multiple baseline across students was used to assess the frequency of appropriate and inappropriate interactions while components of package were added one at a time. The components evaluated were rewards alone, discussion plus rewards, and self-evaluation plus rewards. The results from the component analysis clearly demonstrated that rewards alone, and discussion plus rewards were not able to achieve substantial decreases in inappropriate interactions and increases in appropriate interactions. Only the self-evaluation via video feedback plus reward package was able to achieve the desired results. The second study reported by Kern and colleagues (1995), assessed the efficacy and feasibility of implementing the self-evaluation via video feedback procedures in a group context. Specifically this was done to address concerns over the one-to-one time commitment involved if a teacher wished to implement the procedures. The study participants were from three classrooms in a partial hospitalization program. All the students in each classroom participated in the study, however, for practical purposes, data were collected on only two or three from each classroom. The eight students targeted for data collection ranged in age from eight to thirteen. In this study, the self-evaluation via video feedback with rewards condition occurred in this group. The class simultaneously viewed ten 30 second segments of the videotape from the previous activity session. The students were given the same instructions as in the other two studies; however, they were working toward a class reward rather than an individual reward. All the participants showed marked decreases in inappropriate interactions and increases in appropriate interactions following the implementation of the intervention. The study findings were supported by an assessment of treatment effects. It revealed that observers naive to the purposes of the study, significantly rated the participants' behavior as better during the self-evaluation via video feedback with rewards condition than during baseline. The teachers involved with the experimenter indicated they were satisfied with the procedures. The Falk et al. (in press) study extended the previous studies by distinguishing between children with externalizing and internalizing behavioral challenges in inclusive classrooms. Specifically this was done to determine the efficacy of the procedures in regular education classrooms with children with idiosyncratic needs as opposed to the more homogeneous needs of children in special classrooms. The study participants were from three classrooms in a K-8 school in a small Canadian prairie city. Eighteen students from three middle years classrooms were selected for the study. Six of the students exhibited externalizing behavioral challenges, four displayed internalizing behavioral challenges and eight students with no identified behavioral or academic challenge were included for comparison purposes. The eighteen students targeted for data collection ranged in age from eleven to fourteen. Each group simultaneously viewed ten 30 second segments of the videotape from the previous activity session. The students were given the same instructions as in the other two studies. The students were given the same instructions as in the other two studies. All the participants showed marked decreases in inappropriate interactions and increases in appropriate interactions following the implementation of the intervention. The study findings were supported by an assessment of treatment effects. It revealed that observers naive to the purposes of the study, significantly rated the behavior of participants' with either externalizing or internalizing behavioral challenges as better during the self-evaluation with rewards was no more effective than without rewards. Purpose of the Study The purpose of this study was to extend previous research demonstrating the efficacy of self evaluation via video feedback procedures to students with cognitive challenges. The strategy was designed to address the limitations previously discussed by integrating an individualized instructive approach with a natural type of interactive circumstance. This was accomplished by having participants evaluate the appropriateness of their own behavior (self-management via videotape) under circumstances the children might typically encounter. The objectives of the study included: assessing the efficacy of the procedures with children with moderate cognitive challenges (individually and in a group setting); assessing the durability of the intervention gains in the absence of daily exposure; assessing whether individuals naive to the purposes of the study judged the students' behaviors to be improved following the administration of the self-evaluation via videotape procedures; and assessing the likelihood teachers would use the procedures outside of an experimental setting. Participants The six students who participated in this study attended a public K-8 school in a small city in Saskatchewan, Canada. The students (ages 8 to 11) were in a special elementary life skills classroom with some integration in age appropriate regular education classrooms. Specific participant characteristics are presented in Table 1. Setting The video feedback and activity sessions occurred in the special education classroom. The video feedback took place at a table or a desk, so the participants had a solid surface on which to record their responses. Throughout the activity sessions the participants sat in a half-circle around a table. A camcorder was stationed approximately seven meters from the children. The regular classroom teacher or the data collectors were present during each of the activity sessions. The teachers and/or data collectors attempted to avoid interacting with the participants during the activity sessions and read or completed other work they had in the far corner of the classroom from the participants. They only interacted with the participants when clarification was requested regarding the rules of the game the participants were playing, when a participant could not be seen by the camcorder, or when behaviors became dangerous to self or others.
Procedures The independent variable (self-evaluation via videotape feedback procedures) had two conditions: absence of treatment and presence of treatment. All participants experienced both conditions during the course of the study. The study consisted of daily (Monday, Tuesday, Thursday and Friday) activity sessions where one of the treatment conditions was employed. During each activity session, the participants played one of the following table games: Go Fish, Memory, Pass the Bag, Number Bingo and Alphabet Bingo. The participants were videotaped for ten minutes while playing the game during each activity session. The activity session lasted approximately 15 minutes. During the self-evaluation conditions, the participants began each session by simultaneously viewing ten consecutive 30 second segments of the videotape from the previous activity session. At the end of each segment, the participants were asked to respond with a "happy face" or a "sad face" to the statement "I got along with my classmates." They checked their response on a self evaluation recording form adapted from Kern et al. (1995) (see Appendix A). When a participant responded with a "sad face", he or she was asked to describe their response (or lack of ), and how they could have responded more appropriately given the circumstances. Suggestions were provided by other group members and the adult if the student was unable to generate appropriate alternate responses. Occasionally responses were solicited from students who handled a difficult situation particularly well. They were asked to describe why they selected their response. To assess the accuracy of the participants' self-evaluations, the facilitator(s) matched with a participant during the video feedback sessions. The study began with a seven session desensitization period. The desensitization period was used for everyone to learn the games their group selected, allow the participants to get used to the presence of the camcorder, and to train data collectors. During baseline, data were collected from activity sessions described above. Data were collected on inappropriate and appropriate social behaviors as defined in the Dependent Variables section of this report. During baseline no feedback or contingencies were given to the participation. During self-evaluation, the participants began each session by self-evaluating via videotape feedback. As a means of encouraging students to set goals for their own behavior change, this condition started with each student selecting various behaviors of concern that they believed were particularly problematic in forming friendships. During the ten 30 second segments of videotape being watched, the facilitator evaluated the behavior of a participant. Those students who displayed difficulty in accurately self-evaluating their behavior were monitored and matched with frequently. Four of the participants (Mark, Mandy, Robbie and Andy) experienced this condition individually prior to experiencing this condition as a group. To assess the durability of the treatment procedures, the procedures were faded over a four week period with three of the participants. The activity sessions were conducted three to four times a week, however, self-evaluation occurred every second or third session rather than every session. Design This experiment employed a multiple baseline with reversal across students design (Kazdin, 1982). This design uses single-subject methodology to evaluate the effect of the independent variable (self-evaluation via video feedback procedures) on the dependent variables (inappropriate and appropriate social behaviors) with children with moderate cognitive challenges. The design for two of the students was ABABC with "A" being baseline, "B" being self-evaluation via videotape feedback, and "C" being fading of self-evaluation procedures. An ABAB design was used for two of the students. An ABC design was used for one of the students and finally a simple AB design was used for the sixth student. Visual analysis was used to analyze the data. Individual students functioned as the unit of analysis. Direct observation data were collected, scored and graphed. Data were analyzed for changes that occurred in response to the self-evaluation via video feedback procedures. The characteristics of a single-subject methodology which contribute to its particular relevance for the empirical investigation of this research proposal include: internal validity, external validity, applicability and data evaluation. In an empirical investigation, internal validity is said to be present when the changes in targeted behaviors can be attributed to a manipulated variable rather than extraneous variables, such as random or naturally occurring events (Borg & Gall, 1989). In single subject research, internal validity is enhanced through repeated measurement, and close monitoring of individual participant changes. Employing a reversal design, where the consistency of behavior changes is clearly observed and repeatedly measured when a variable is presented, withdrawn and presented again increases the likelihood that the variable accounts for the observed changes, and consequently increases internal validity. A variety of single-subject research designs have been designed to control for threats to internal validity (Gay, 1981; Hersen & Barlow, 1976; Kratochwill, 1978). This research study employed a reversal design with four of the six participants to demonstrate a functional relationship between the independent and dependent variables (Parsonson & Baer, 1978;1986). Specifically, the self-evaluation via videotape feedback procedures were withdrawn with four of the participants to demonstrate the functional relationship that existed between their presence and the occurrence of appropriate and inappropriate social behaviors by the participants. The extent to which results from an empirical investigating generalize to other participants, settings and target behaviors determines external validity of single-subject research designs (Borg & Gall, 1989). Researchers (e.g., Campbell & Stanley, 1963; Cook & Campbell, 1979; Kratochwill, 1978) have identified a variety of threats to external validity; however, careful construction of experimental procedures will minimize those threats regardless of research methodology. External validity is of particular interest with single-subject research because of the concern over the small number of subjects. Without replications with other participants, observed changes may result from idiosyncratic characteristics of an individual participant, limiting the generalizability of the findings. Several authors (e.g., Sidman, 1960; Hersen & Barlow, 1976; Kratochwill, 1978) have developed and expanded inter-participant replication procedures to provide adequate protection against threats to external validity. This research study, replicated the self-evaluation via video feedback procedures across six participants. Single subject research designs are particularly suited to research involving behavior modification, and working with low-incidence populations (Borg & Gall, 1989). This study addressed changing idiosyncratic social behaviors of children with moderate cognitive challenges. The design facilitated frequent and repeated observations of child behavior over time making it sensitive to changing behavior patterns due to the independent variable--self-evaluation via videotape feedback procedures (McReynolds & Kearns, 1983). Visual analysis of graphically-presented data is the widely accepted method for analyzing single subject research (Parsonson & Baer, 1978; 1986). Data are examined for differences that occur in response to the manipulation of the independent variable. In particular, changes in the level, trend, and/or variability in the dependent variable are analyzed. Well-established strategies and methods (Parsonson & Baer, 1978; 1986) for determining whether changes are functionally related to the manipulated variable provide empirical strength. The present research program employed visual analyses for data analysis. The participant functioned as the unit of analysis. Direct observation data were collected, scored and graphed. Data were analyzed for changes that occurred in response to self-evaluation via video feedback procedures. Dependent Variables The main dependent variables were: the rate of inappropriate social behaviors, and the rate of appropriate social behaviors. A social behavior consisted of any verbal or nonverbal behavior directed toward self or peer(s). A new social behavior was scored when one of the following conditions was met: (a) three or more seconds elapsed in the verbal or nonverbal behavior of a child; (b) a verbal or nonverbal interchange occurred between the target child and his/her peer; or (c) verbal of nonverbal behavior was redirected from one child to a different child. All social behaviors were categorized as either inappropriate or appropriate (Kern et al., 1995). Any verbal on nonverbal behavior which would not be acceptable in social settings, such as in a school or community, was considered an inappropriate social behavior. Derogatory, demanding, excessively loud, or inappropriate statements or gestures were considered as inappropriate social behaviors. Nose picking, sneezing or coughing without covering the mouth, touching private areas of the body, and picking scabs were also recorded as inappropriate social behaviors. Any verbal or nonverbal behavior which would be acceptable in social settings such as in a school or community was considered an appropriate social behavior. Validating, helping or neutral statements and gestures were considered appropriate interactions (Kern-Dunlap et al., 1992; Kern et al., 1995; Falk et al., in press). Instruments and Techniques for Measurement Direct observation of participant's behaviors provided information on the extent to which a participant's inappropriate and appropriate social behaviors were impacted by the independent variable. Direct observation data on student behavior were taken from the videotaped activity sessions using a frequency recording system where the frequency of inappropriate and appropriate social behaviors were assessed during 10 consecutive one minute intervals (e.g., Kern-Dunlap et al., 1992; Kern et al., 1995; Falk et al., in press). The frequency of inappropriate and appropriate peer interaction were then converted to a number per hour for data reporting. The rationale for examining the participant's rate and quality of social behaviors was that children who interact infrequently or inappropriately with the peer group are deprived of significant social learning opportunities. The goal of treatment was to increase the quantity and quality of appropriate behaviors so that social learning could take place (Asher, Markell & Hymel, 1981, Deutsch, 1974). Some children with a low rate-of-interacting, may be very competent when they do interact, while others may interact frequently but aversively, and consequently, are rejected by their peers. Methods which merely attempt to increase interactions and social behaviors in general, without regard for the quality of the behaviors, or methods which attempt to reduce inappropriate interactions and social behaviors without maintaining or increasing appropriate behaviors have limited utility (Kern et al., 1995; Asher et al., 1981). The data were collected by two data collectors who were trained in the observation and analysis of the operationally defined behaviors. Videotapes of activity sessions during the desensitization period were used to practice scoring videotapes. Interobserver agreement for both inappropriate and appropriate social behaviors was calculated on at least 20% of the activity sessions distributed across students and conditions. Interobserver agreement for data was calculated by using a Block-by-Block Agreement method (Bailey & Bostow, 1979) in which observer scores were compared interval by interval. Agreements were counted when both observers had recorded the same number of occurrences of the targeted behavior within the interval. For example, if both observers recorded three occurrences of inappropriate behavior in a particular interval, the total number of agreements for that interval would be three. Disagreements were counted when there was a difference in the number of occurrences recorded between the two observers. For example, if one observer recorded four occurrences of inappropriate behavior, while the other observer only recorded three, there would be one disagreement and three agreements in total. The percentage of interval agreement was calculated by dividing the number of agreements by the total of agreements plus disagreements for each interval, and multiplying by 100. The percentage of interobserver agreements for each interval were added together and divided by the number of intervals, which was 10. This formula yielded a percentage of interobserver agreement for each session (Kazdin, 1982). Interobserver agreement data are present in Table 2 by condition and child for both inappropriate and appropriate social behaviors. The mean interobserver agreement for inappropriate social behaviors was 88% (range=69-100%). The mean interobserver agreement for appropriate social behaviors was 87% (range=65-100%). During seven of the sessions interobserver argument fell below 75% for one of the participants. In each of those sessions, the targeted behavior only occurred a maximum of six times.
The social validity of the intervention effects were assessed using an adpatedsemantic differential developed by Kern et al. (1995) (see Appendix B). Seven respondents naive to the purposes of the study viewed four three minute segments (two from baseline and two during self-evaluation). The segments were purposefully chosen as representative samples of each child's assessment for the condition and were viewed in random order. Following the viewing of each segment, the respondents (two administrators, a behavioral program consultant, an educational psychologist, and three teachers) were asked to place a check mark on one of seven lines between a pair of counterbalanced antonym adjectives. For scoring purposes the authors assigned a numerical value to the seven lines between each pair of counterbalanced antonyms. The lines closest to the positive adjectives received a numerical value of 1, while the lines closest to the negative adjectives received a numerical value of 7 (similar to a Likert scale). Scores for each of the three children were totaled for each condition. Baseline scores were then subtracted from self-evaluation scores. A positive improvement in the appropriate social behaviors of the students would be reflected in a negative difference score. A final measure to assess teacher satisfaction with the procedures and the research process was completed. It was a two page questionnaire with the first page assessing teacher satisfaction with the self-evaluation via videotape feedback procedures and the second page assessing their satisfaction with being involved in an experimental research process. Social skills research has frequently been criticized for finding significant results with procedures that teachers would never use outside of an experimental setting (Schloss et al., 1986). In short, studies may report significant improvements in behavior from a statistical perspective, however teachers may feel the behavioral gains to still be at socially unacceptable levels or the procedures too cumbersome to warrant the effort required to experience those gains. The process of experimental social skills research often discourages teachers from participating in similar projects again; therefore the second page of the questionnaire was designed to assess the teacher's attitudes toward her involvement in the research process. The results are summarized in Figure 1 and Table 3. Figure 1 presents the graphed data showing condition changes and type of social behaviors (inappropriate and appropriate). Table 3 is provided to support the visual presentation of the data in Figure 1. It presents the condition means for inappropriate and appropriate social behaviors for each child. Table 4 summarized the social validity of treatment effects questionnaire used to assess whether individuals naive to the purposes of the study perceived the participants to have changed in some meaningful way over the course of the study. Direct Observation The data graphically presented in Figure 1 represents the six study participants. The introduction of the self-evaluation via videotape feedback procedures resulted in an immediate drop in the number per hour of inappropriate social behaviors for each participant. An examination of the condition means presented in Table 3 reveals that for Mandy and Robbie the mean number per hour of inappropriate social behaviors was cut to approximately one-third, while Heather and Carrie's were cut to approximately one-quarter. The ratios were even more substantial for Mark and Andy, whose number of inappropriate social behaviors per hour were cut to almost one-sixth and one seventh respectively. In addition, the implementation of the procedures also had a stabilizing effect on the number per hour of inappropriate social behaviors for each of the participants. Once the drop occurred, the number of inappropriate social behaviors per hour remained fairly stable for all six participants.
A multiple baseline withdrawal of the self-evaluation procedures for four of the participants (Mark, Mandy, Robbie and Andy) resulted in a return to baseline levels for each one. The reintroduction of the self-evaluation procedures produced the same effect as the first time for each of the four students. The reintroduction of the self-evaluation procedures for the four students served as an introduction of the procedures for Heather and Carrie. This time the procedures were implemented as a group with no discernible differences noted than when individually administered. The introduction of fading procedures to the treatment package with the three students (Mark, Andy and Heather) produced little change from when the procedures were employed at each session. Andy had two sessions during this condition where his number of inappropriate social behaviors climbed substantially; however, they just as suddenly decreased for the remaining five sessions.
The impact on the number per hour of appropriate social behaviors was less consequential, with the primary benefit being greater stability in occurrence. For all participants a decrease in the number per hour of inappropriate social behaviors were accompanied by a slight to moderate decrease in the number per hour of appropriate social behaviors. Social Validation of Intervention Effects The results from the social validity questionnaire of the intervention effects showed that individuals naive to the purposes of the study viewed the three participants' (Mark, Mandy and Andy) behavior as substantially improved following the introduction of the self-evaluation via videotape feedback procedures. On this measure, lower mean scores would identify students perceived to have fewer social behavioral challenges. Higher scores identify students perceived to have serious social behavioral challenges. Optimal mean scores would be close to 1 while highly problematic mean scores would be near 7.
An examination of the difference scores reveal that the participants experienced more positive ratings by the raters following the intervention. On the seven point Likert-type scale, behavioral improvements of 1.43, 2.87 and 3.01 were found for Mark, Mandy and Andy respectively. Table 4 provides a more detailed examination of the results. Consumer Satisfaction with the Treatment Procedures The teacher involved in this study rated the procedures easy to learn and implement. She thought they took a reasonable amount of time to implement and were highly effective in improving the social behaviors of her students not only during the experimental sessions but throughout the day. She clearly preferred the self-evaluation procedures over other social skills procedures she had previously tried. The teacher provided anecdotal notes on each student to accompany the consumer satisfaction measure at the end of the school year (two months after the conclusion of the study). The following is a summary of the comments she provided for each student. Mandy The number of repetitious and socially inappropriate behaviors demonstrated by Mandy substantially decreased and have remained low since the conclusion of the study. In particular, Mandy has not been seen to touch her crotch and then smell her fingers in months. The study saw a significant decrease in Mandy's striking out at others. However, two months later, while her striking out at others is not near as severe as prior to the introduction of self-evaluation procedures, it is on the increase. Mandy continues to be rude and argumentative with adults, but since the introduction of the self-evaluation procedures this behavior has substantially improved with her peers and has remained greatly improved. Mandy has not demonstrated any of the uncontrolled laughing or excessive head and body waving and shaking behaviors when she is excited or when someone else is in trouble since the self-evaluation procedures began. Carrie Carrie's repetitious use of the greeting "hi" still remains an area of concern, however, watching herself on videotape has reduced the number of incidents. The self-evaluation procedures were the most effective means of having Carrie see the inappropriateness of this and will be used again in the future. The self-evaluation procedures also enabled Carrie to admit that she sucked her thumb and has helped her consistently reduce the number of episodes even following the conclusion of the study. Carrie's bossiness and encouraging other students to misbehave all but disappeared during the self-evaluation procedures, but unfortunately have reoccurred as a major behavior management issue following the conclusion of the study. Carrie has become much more focused in her ability to follow and participate in games with her peers following self-evaluation. Heather Staying on task was a primary concern for Heather. Regardless of the activity or setting (free time, academic or playing with peers), Heather continually needed to be prompted to stay on task. The self-evaluation procedures greatly assisted Heather in staying on task and in fact, the procedures have been adapted in Math class where she had the greatest difficulty. Now, Heather will stay on task in Math as long as she suspects the video camera is on, if she knows it is off she will "shut right down." Andy Andy's primary behaviors of concern were being easily agitated by his peers, tattling, poor sportsmanship, crying when corrected, and being rude to peers. All these behaviors greatly decreased during the self-evaluation procedures and remained low for several weeks after the conclusion of the study; however, recently they have begun to be problematic again, particularly incessant tattling. Self-evaluation procedures have been successfully used during Math class to help Andy monitor how he handles corrective feedback when his assignments are marked. Typically Andy would cry if any of his work was marked as incorrect. Lately, even when a peer teases him about having to make corrections he is making an effort to keep on task and complete the correction without resorting to tears. Mark Mark's referral behaviors included: making inappropriate noises (including whining(, defying authority, cheating in games, tattling on others, leaving the room without permission, and being rude to peers. Mark's behavioral gains throughout the study have been fairly stable. He rarely makes inappropriate noises in class and when he forgets he catches himself and looks at the teacher and says, "oh, sorry." He has quit tattling on his peers and now asks permission to leave the room (e.g., to use the bathroom). He also has become much more accepting of his peers and has begun to offer to help his peers instead of ridicule them when they are having trouble with their work. Robbie Robbie's referral behaviors included anger management challenges, cheating in games with peers and refusing to acknowledge his own inappropriate actions. Interestingly, while Robbie's inappropriate social behaviors decreased during self-evaluation, he repeatedly refused to admit to the behaviors while viewing the videotapes until the last few sessions. For example, many times the video tapes would be repeatedly rewound so he could see himself hitting a peer, yet he would refuse to admit that he hit the peer. Near the end of the study he began to admit to his inappropriate behaviors which continued for the remainder of the school year. Robbie's mother recently commented that this was one area of tremendous growth they have consistently seen at home since Spring (when the procedures were implemented). Consumer Satisfaction with the Experimental Process The classroom teacher was positive about being involved with the experimental process. Based on the questionnaire, she found the purpose of the research to be clear, the design easy to learn and the length of the study to be reasonable. She did feel data collectors were essential for this study and she could not have been involved without their involvement. Her concerns primarily focused on the collection of baseline data and the need for experimental reversals (returning to baseline condition). She strongly stated a willingness to be apart of a similar research project in the future. The following are comments the classroom teacher provided at the end of the school year on being involved in this research project. "As a classroom teacher of special needs kids, I often wondered during this project if I was doing the kids any favors. The days we taped baseline were days from hell. The escalated unacceptable behaviors took forever to get back down to an acceptable level. (Note: the behaviors likely escalated because as part of the experimental condition, there was to minimal or no interference from the teacher). We were finally seeing great behavior improvements in March to May and then the study ended. I would like to follow this study by using the procedures without having to collect baseline data for experimental reasons, particularly for children with moderate cognitive challenges in integrated settings. I can see so many benefits if this could be done long term with this population of children." The results of this study show that self-evaluation via video tape feedback procedures were effective when administered in a group with participants with moderate cognitive challenges. Specifically, the results of this study showed that the self-evaluation via video tape feedback procedures were: (1) effective in consistently reducing the rate of inappropriate social behaviors for the children (regardless of whether administered individually or in a group setting); (2) durable in the absence of daily exposure; (3) viewed as socially significant by individuals naive to the purposes of the study who judged the students' behaviors to be improved following the administration of the procedures; and (4) viewed as well worth using by teachers who had students requiring social skills interventions. The results of the study also show that unlike similar research with children (albeit with social, emotional and behavioral challenges), the procedures were not effective in increasing their rate of appropriate social behaviors. Finally, the functional relationship between the independent variable (self-evaluation via video tape feedback procedures) and one of the dependent variables (occurrence and nonoccurrence of inappropriate social behaviors) was demonstrated further by the withdrawal analysis conducted with four of the participants. The results from the social validity questionnaire of intervention effects indicated the intervention yielded beneficial results. Individuals who were naive to the purposes of the study rated the participants as having more social behavioral challenges in the baseline videotape segments then following the implementation of the intervention. This rating was achieved despite the random presentation of the video tape segments, and no identification to lead them to suspect a condition change had occurred. The results of the measure do suggest that care givers are likely to view the outcome of the self-evaluation via video tape feedback procedures favorably. The results from this study are consistent with previous research findings on improving children's social behaviors through self-evaluation via videotape feedback procedures (e.g., Kern-Dunlap et al., 1992; Kern et al., 1995; Falk et al., in press). This study extended earlier analyses by demonstrating the effectiveness of the procedures when implemented with children with moderate cognitive challenges. A social validation measure strengthened the research findings as to their effectiveness. Improvements in the participants' social interactions may be the result of rule-governed behavior. Rule-governed behavior occurs when the contingencies are learned in the form of rules instead of through experiencing behavioral consequences (e.g., Reece, 1989; Poppen, 1989). In this study, the children's future behavior was likely guided by rules formulated during the videotape feedback sessions. For example, a participant may have decided that in order to record a "yes" on their self evaluation sheet completed during the video tape feedback sessions, s/he would have to not hit others in the group during the activity sessions. The rapid decrease in the frequencies of inappropriate social behaviors following the implementation of the intervention offers strong empirical support to the theory of rule-governed behavior. Generalization and maintenance of behavioral gains are essential questions with any intervention (Stokes & Baer, 1977). The self-evaluation via video tap feedback procedures were found to produce substantial behavioral gains in this study. The four weeks of data collected during the fading of the procedures suggests the procedures are durable (similar to the kern et al., 1995 study). Anecdotal information from the classroom teacher identifies her perception that transfer occurred to other settings during the school day. Recent parental reporting (Robbie's mother) suggests that the behaviors may be transferring to settings outside of school (the home) as well. The arrangements of the participants in a semicircle, the rules surrounding the selection of the games to be played, the presence of the teacher and data collectors, and the presence of the camcorder could all serve to remind the student that the setting is artificial. Each one likely functioned as a discriminative stimulus, reminding the participants when their behavior was being monitored. That certainly was suggested by Heather's behavior in Math class where when she believed the video camera was on she remained on task, yet when she believed it was not on, she frequently was off task. Goal setting was purposefully included in the study as a method of increasing the likelihood that the participants would become intrinsically motivated to change their behavior and maintain their behavioral gains (Nelson et al., 1991). Goal setting was included by having the participants select those behaviors which they believed most interfered with their ability to build and maintain friendships. The benefit of goal setting has been clearly demonstrated in the research (e.g., Paine, Hops, Walker, Greenwood, Fleischman & Guild, 1982). Ultimately it would be the goal of any social skills program to have the students involved intrinsically become motivated to achieve the objectives of the program. Personal goal setting would be a primary method for achieving the intrinsic motivation to change behavior and to maintain the behavioral gains. The lack of increases in appropriate social behaviors as the inappropriate social behaviors decreased is an area of concern. It is widely accepted that for behavioral gains to be lasting, inappropriate behaviors must be replaced with appropriate behaviors. One likely explanation could be the students replaced inappropriate nervous habits (e.g., thumb sucking, nose picking, etc.) with more acceptable nervous habits (e.g., tapping one's foot quietly or bouncing a knee). However, data distinguishing between inappropriate nervous habits and inappropriate verbal interactions were not collected. During the self-evaluation via video tape feedback procedures, the participants were all asked to describe an alternate appropriate behavior to the inappropriate ones they identified. The goal was to replace socially unacceptable behaviors with socially acceptable behaviors that served the same function (Lewis & Sugai, 1993). It was quickly evident with this population that while they had little difficulty identifying inappropriate social behaviors, they did have great difficulty generating alternative responses. Also, while the classroom teacher reported the return of some of the inappropriate social behaviors exhibited by her students she clearly believed the procedures to be highly effective. With this population she believed a greater length of time was required before new habits would be internalized. She also found it noteworthy that the four students with lasting improvements were the older four children in the study (Mandy, Mark, Andy, and Heather). All care givers involved with children with moderate cognitive challenges may find the results of this study important. The procedures support previous research findings by demonstrating their effectiveness in: (1) substantially reducing inappropriate social behaviors (e.g., Kern-Dunlap et al., 1992; Kern et al., 1995; and Falk et al., in press); (2) a group setting (e.g., Kern et al., 1995; and Falk et al., in press); (3) rapid treatment gains after only a limited exposure to the intervention (e.g., Kern-Dunlap et al., 1992; Kern et al., 1995; and Falk et al., in press); and (4) being viewed by teachers as relatively easy to learn and implement (Kern et al., 1995). It appears that while children with social behavioral challenges have very different presenting needs, self-evaluation via video tape feedback procedures allow idiosyncratic needs to be met. All the participants in this study were from a small city in the Canadian prairies, and therefore the results of the study may be limited to similar populations. While the multiple baseline design of the study reduced threats to external validity through inter-subject replication (Kazdin, 1982; Gay, 1981; Hersen & Barlow, 1976; and Kratochwill, 1978), additional research is needed to fully extend the procedures to other children described as having moderate cognitive challenges. The children involved in this study all were primarily served in a congregated special classroom with partial integration in age appropriate regular education classrooms. Future research should be targeted at children with moderate cognitive challenges served in either segregated or full inclusionary models of educational program delivery. As earlier studies using self-evaluation via video tape feedback procedures have found (Kern Dunlap et al., 1992; Kern et al., 1995; and Falk et al., in press), it is interesting to note the rapid improvements each child made in their quality and quantity of peer interactions. Students described as having moderate cognitive challenges were still able to quickly learn and employ behavioral expectations and contingencies. However, it is likely that maintenance of behavioral gains would require a substantially longer period of intervention than presented in this study. A few cautionary notes on this study include: (1) one must be careful not to simplify a complex challenge for children with cognitive challenges. Many of the children involved in the current study may have other more pressing issues in their lives which need to be addressed first or concurrently if meaningful behavioral gains are to be seen in all settings; (2) the classroom in this study was a teacher-volunteered classroom. It could be that the characteristics of teachers willing to implement new procedures with their students would be different from the general population of teachers. This may limit the generalizability of the study's results. Summary There continues to be a need to provide effective social skills programming for children with cognitive challenges. Several limitations with traditional social skills approaches have been discussed. In addition, previous research using self-evaluation via video tape feedback has not employed the procedures with moderate cognitive challenges. The current study attempted to address several of the limitations and extend the procedures to this population of children. The results of the study have demonstrated the effectiveness of the self-evaluation via video tape feedback procedures with children with moderate cognitive challenges. More importantly, the results demonstrate a practical method which may be presented in a group format yet addresses idiosyncratic behaviors. Further research is needed to address the portability/usability of the procedures in the absence of data collectors.
Support for this research was provided by the Dr. Stirling McDowell Foundation for Research into Teaching. Opinions expressed in this manuscript are those of the authors and no endorsement by supporting agencies should be inferred. The authors are grateful for the cooperation and assistance provided by Mr. Larry Booth, Mr. Bud Guidos, Dr. Sandy Kitts, Mrs. Lorraine Friesen, Mrs. Donna Dormer, Mrs. Evelyn Falk and Mr. Dennis Daniels. The authors also extend appreciation to Drs. Glen Dunlap and Lee Kern for their comments on the design and interpretation of the results. Appendix A: Self-Evaluation Student Recording Form
Appendix B: Social Validity of Intervention Effects STUDENT RATING This instrument contains five paris of antonyms describing student behavior. Place a check on one of the seven lines between the two adjectives indicating how you perceive the student's behavior during the videotape segment. Please mark as rapidly as possible. What is wanted is your first impression. This is not a test. The right answer is the one that is true for you. Be sure to make only one check mark for each pair of words and do not skip any pairs of words.
Appendix C: Social Validity - Consumer Satisfaction
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
