Which of the following actions illustrates the concept of respecting an infant or a toddler?

Infant Ment Health J. Author manuscript; available in PMC 2010 Jun 15.

Published in final edited form as:

PMCID: PMC2885814

NIHMSID: NIHMS173269

Abstract

This paper reports the construction and pilot reliability, validity, and psychometric properties of a new caregiver-child rating scale that emphasizes caregiver-child social-emotional interactions and relationships. While the scale was developed and studied in the context of orphanages for young children, it potentially could be used in non-residential early care and education settings as well as for parent-child interactions in the home. The intent was to assess a few dimensions that comprehensively cover the range of caregiver-child social-emotional interactions and relationships but could be administered in a relatively short period of time in a variety of situations and would not require extensive coder training, manuals, or materials. Results showed that the scale can be reliably administered even using observation periods as short as five minutes, reliability was replicated over seven different coders working in three different orphanages, and ratings of caregivers were similar across different types of caregiving activities (i.e., feeding, dressing/bathing, free play) and for caregivers attending to children birth to 4 and 4 to 8 yrs. of age. In the orphanage context, factor analyses showed the scale primarily reflects caregiver-child mutual engagement and relationship with subordinate components of caregiver punitiveness and caregiver- vs. child-directed behaviors and intrusiveness.

Early social/emotional relationship experiences, especially warm, caring, sensitive, and responsive interactions between adults and their infants and young children, are crucial contributors to promoting attachment (e.g., DeWolff & van IJzendoorn, 1997; van IJzendoorn & Sagi, 1999), which in turn is associated with longer-term positive child outcomes in social and mental development (e.g., Aviezer, Sagi, Resnick, & Gini, 2002; Landry, Smith, & Swank, 2006; Landry, Smith, Miller-Loncar, & Swank, 1997; Landry, Smith, Swank, & Miller-Loncar, 2000; Steelman, Assel, Swank, Smith, & Landry, 2002). Social/emotional relationship experiences have also been associated with the quality of early care and education environments, early childhood developmental gains, and longer-term educational success (e.g., Edwards & Raikes, 2002; Kontos, Howes, Shinn, & Galinsky, 1995).

Conversely, insecure attachment, especially disorganized attachment, is related to increased later problem behaviors, including externalizing behaviors in males and other social, behavior control, crime, and mental health problems, more so in high-risk children and those who continue to experience insensitive parenting and/or child care (e.g., Carlson, 1998; Fonagy et al., 1997; Greenberg, 1999; Lyons-Ruth, Alpern, & Repacholi, 1993; Rothbaum & Weisz, 1994; Shaw, Owens, Vondra, Keenan, & Winslow, 1997).

More specifically, institutional rearing environments for young children tend to provide quite minimal social/emotional relationship experiences (e.g., St. Petersburg-USA Orphanage Research Team, 2005, 2008; Rosas & McCall, 2008), and perhaps as a partial consequence of this deprivation such children tend to be substantially underdeveloped and have higher rates of behavioral problems even after being adopted into advantaged families (Blizzard, 1990; Gunnar, 2001; Johnson, 2000; MacLean, 2003; St. Petersburg-USA Orphanage Research Team, 2005, 2008). Further, interventions that emphasize improved early social/emotional relationship experiences in orphanages (St. Petersburg-USA Orphanage Research Team, 2008; Zeanah, Smyke, & Koga, 2003), provide high-quality foster care instead of institutionalization (Nelson, Zeanah, Fox, Marshall, Smyke, & Guthrie, 2007), or promote responsiveness in parents towards their own infants (Landry et al., 2006) have produced improvements in children's development in several domains.

The Need for an Assessment Instrument for Social/Emotional Relationship Caregiver Interactions

Despite the theoretical and empirical importance of early social/emotional relationships, there are relatively few measures of children's home or early care and education environments that focus specifically on those aspects of parent/caregiver-child interactions for children under 4-6 years of age.

Assessments of early care environments fall into roughly three categories (for a review, see Melhuish, 2001), those that primarily measure the total environmental setting, those that focus on individual children, and those that assess the nature of the caregiver-child interaction.

Measures of the total environmental setting

The most commonly used assessments of early care and education environments are comprehensive and reflect the total environment—the physical facility, toys and equipment, activities, staff support, and other aspects of the care environment in addition to parent/caregiver-child interactions. This comprehensiveness is a major asset, and these assessments typically do correlate with children's contemporary and future development (Melhuish, 2001). But the subscales of these instruments that pertain specifically to caregiver-child interactions are short and a small part of the total assessment.

For example, the HOME and ECERS are perhaps the most widely used general assessments. The HOME Inventory (Bradley & Caldwell, 1995; Caldwell & Bradley, 1984), originally designed to assess the home environment but adapted to group care (NICHD Study of Early Child Care Manual), has several subscales including Responsivity, Acceptance, and Involvement that reflect caregiver-child interactions. But these subscales involve caregiver stimulation of children and the responsivity of caregivers to children's initiatives to a greater extent than social-emotional relationships. Further, items are scored 0/1, so they are limited in their ability to reflect extent and thus are relatively insensitive to gradations of social/emotional interactions. Also, when the instrument is used in a group setting, the caregiver only needs to display the indicated behavior once to one child to receive credit for that item.

Similarly, the environmental rating scales designed especially for assessing early care and education environments (i.e., ITERS, ECERS, FDCERS; Harms, Clifford, & Cryer, 1998; Harms, Cryer, & Clifford, 1990) primarily assess the global environment (subscales of Space and Furnishings, Personal Care Routines, Activities, Program Structure, Staff Support) in addition to caregiver-child social behavior (subscales of Listening, Talking/Language-Reasoning, and Interaction). While scoring is on a 7-point (rather than 0/1) scale, only four items pertain to language-reasoning in children and only five items to social interaction; no items specifically measure social-emotional interaction or adult-child relationships. Moreover, both these measures require about an hour or more of observation and scoring.

Assessments of individual children

Assessments that focus on individual children (e.g., Melhuish, Mooney, Martin, & Lloyd, 1990; Pierrehumbert, Ramstein, Krucher, El-Najar, Lamb, & Halfon, 1996; Sylva, 1997) have the obvious advantage of providing detailed information about individual children, which the more general measures that aggregate over all children would obscure. However, they require more time and energy to conduct, and do not provide a total picture of adult-child interactions in group settings.

Assessments of caregiver-child interactions

Other scales focus more directly on caregiver-child interactions for individual caregivers (e.g., Arnett, 1989; Erickson, Sroufe, & Egeland, 1985; Goossens & Melhuish, 1996; Howes & Stewart, 1987; McGuire & Richman, 1987). Perhaps the most widely used is the Caregiver Interaction Scale (Arnett, 1989), which consists of 26 items directly aimed at assessing caregiver-child interactions. A principal component analysis of the items suggested that they reflected primarily authoritarian and stimulating interaction (van IJzendoorn & Tevecchio, 1998); the scale places less emphasis on the emotional component of interactions and relationships. Further, scoring is on a 4-point scale, and given the current scoring directions, the assessment can reach a ceiling that limits its ability to discriminate gradations in more positive behaviors (R. Fiene, personal communication, October 22, 2007).

Purpose of this Project

This paper reports preliminary information on a new scale, called the Caregiver-Child Social/Emotional and Relationship Rating Scale (CCSERRS) (pronounced “sirs”). The CCSERRS is an attempt to assess fundamental aspects of caregiver-child social/emotional interactions and relationships that is simple, contains only a few items (18), and requires a relatively short amount of time to administer. While the current scale was primarily developed in orphanage contexts, it can be used to rate individual parents at home as well as caregivers in non-residential group settings.

This paper describes the development of the CCSERRS, presents some pilot reliability and validity data, and gives reliability and psychometric data based on orphanage caregivers and children. Although these psychometric data involve relatively small Ns, the assessment is not fully standardized, and population “norms” are not provided (but neither are they for most other environmental scales), we present this preliminary report to try to fill the assessment gap described above and to stimulate other researchers to use the CCSERRS in different contexts, assess its reliability and validity, and contribute to its overall psychometric foundation and construct validity.

The Development of the CCSERRS

The Item Pool

Authors McCall and Groark made more than a dozen systematic observations of wards in three orphanages for children birth to 4 years of age in St. Petersburg, Russian Federation, over a six-year period beginning in 2001 as part of a large social/emotional relationship intervention project (Groark, Muhamedrahimov, Palmov, Nikiforova, & McCall, 2005; Muhamedrahimov, Palmov, Nikiforova, Groark, & McCall, 2004; St. Petersburg-USA Orphanage Research Team, 2005, 2008). The interventions produced substantial variation in caregiver-child social/emotional interaction and relationships. The notes of these observations on caregivers and children before and after the interventions were systematically reviewed, and all of the positive and negative caregiver and children's behaviors were listed. In addition, items pertaining to caregiver-child interactions from the HOME Inventory, the ITERS and ECERS Scales, the Caregiver Interaction Scale, and behaviors rated by Koren-Karie, Sagi-Schwartz, and Egoz-Mizrachi (2005) in their study of child care in Israel were added to the list. The items in this “pool” were then subjectively sorted into groups that were relatively similar within groups but dissimilar between groups, and then the general concept that characterized the underlying construct in each group was identified.

These basic constructs were further refined according to several subjective criteria.

First, it was desirable to have as few constructs as possible that nevertheless covered the maximum number of items in the pool to produce a comprehensive yet efficient set.

Second, the orphanage observations indicated that certain behaviors that appeared to be direct opposites actually were not simply two ends of a continuum and could vary somewhat independently of one another. Thus, separate constructs for each were included.

Third, caregivers sometimes deliberately behaved in ways they thought observers were looking for, or at least they displayed their “best” caregiving behaviors, but it was clear from the children's behavior that these “model” caregiver behaviors were unusual. Therefore, it seemed necessary to include some children's behaviors in a scale that represented caregiver-child interactions and relationships.

The CCSERRS

Appendix I presents the 18 items on the Caregiver-Child Social/Emotional and Relationship Rating Scale clustered into four caregiver categories of caregiver engagement (negative, positive), caregiver/child-directed behaviors, behavioral control (negative, positive), and caregiver affect plus three child categories of child engagement, child affect, and child relationship with the caregiver. Table 1 presents the general construct for each item followed by behaviors illustrating the concept that might be observed for caregivers attending to children a few months to approximately 6 years of age. The scale is also available in Spanish from the authors.

Table 1

Comparison of Item Ratings for Non-Intervention vs. Intervention

Ratings

Each item is scored on a four-point scale (0, 1, 2, 3) reflecting the frequency with which that behavior (positive or negative) occurred relative to the potential opportunities for it to occur. Items were rated 0=never, 1=rarely, 2=frequently, and 3=consistently in terms of their frequency during an observation relative to the opportunity for the behavior to occur. For example, a caregiver might not display behavioral control or punish a child unless a child happens to deviate from expected behavior. If no deviant behavior provoked punishment, the item would be scored 0, because the caregiver did not control or punish children. However, if only one child deviated from expected behavior and the caregiver punished that child, that would be scored 3 because it happened on each opportunity.

An exception to this rating strategy was made for items assessing caregivers' or children's responses to the other. If children played by themselves during the entire observation and never bid for a caregiver's attention, there was no opportunity for the caregiver to respond, but such an item was also scored 3 because the caregiver consistently “failed to respond” (i.e., there was no caregiver response to children). The same rating strategy was used for the item “child responsiveness and anticipation.”

Notice that items 1, 2, 5, 7-9, 11, 14, and 16 are all negative or undesirable behaviors while the remainder of the items are positive. Scoring, however, reflects the relative extent to which the behavior named in the item was observed, regardless of whether it was positive or negative. Consequently, negative behaviors must be reverse scored after the initial ratings have been completed but before scores are added to create a total or subscale scores. Of course, users of the CCSERRS may elect to score all items in a positive direction, but this leads to the awkward double-negative definition of negative items, for example, “the lack of caregiver detachment” or the “lack of failure to respond.”

Observation period

No “standardized” observation period and circumstances have been prescribed. Instead, the instrument is quite general, and could be used to rate caregiver behavior in a free play context for as little as five minutes for example, or caregiver behavior in feeding, bathing/dressing, and free play situations each for 5 minutes or longer. The lack of a standardized observation period has the liability of possibly limiting direct comparisons of scores across different contexts and observational procedures. However, such comparisons were made in this paper, and it is not clear how often researchers would actually make direct comparisons if the procedures were standardized. Further, the lack of standardized observation criteria means that the scale can be used in a variety of contexts and for different purposes specific to an intervention or site.

Scoring

After appropriate items have been reverse scored, the 18 scores can simply be added and then averaged, giving an overall index ranging between 0 and 3.

Pilot Reliability

Method

Authors McCall and Groark conducted pilot reliability assessments on 12 arbitrarily selected caregivers for children birth to 4 yrs., 4 in a no-intervention orphanage and 8 in an orphanage that had an intervention consisting of training plus structural changes aimed at improving caregiver-child social/emotional interactions (for a complete description, see St. Petersburg-USA Orphanage Research Team, 2008). Although small and unbalanced across intervention conditions, this pilot sample provided a wide range of caregiver-child interaction behaviors on most items on which to determine if the scale had the potential to be scored reliably.

Results

Percent agreement

On the 216 paired ratings of 18 items over 12 caregivers, the raters assigned identical ratings on 61% of the cases and were identical or within 1 point on 96% of the cases.

Total scores

For the Total Scores, the average difference between the two raters was 1.44, which was 4% of the average Total Score of 34.56 (the difference per item between raters averaged only .08 on a scale ranging from 0-3.00). The correlation between raters across the 12 caregivers was .94.

Item reliabilities

The correlations between raters and their mean scores across caregivers were calculated for each of the 18 items.2 Mean differences between raters and the correlation between raters provide two different kinds of indices of inter-rater consistency. Correlations were low (rs in the .50s, p < .10) for only three items, and no item showed both a mean difference and a low correlation between raters. Thus, even at the individual item level and on this small pilot sample, the potential to score these items consistently across raters seemed acceptable.

Pilot Validity

Method

Authors McCall and Groark rated 17 arbitrarily selected caregivers in the no-intervention and 36 caregivers in the training plus structural change intervention orphanages. The intervention emphasized warm, caring, sensitive, and responsive caregiver-child interactions, and was successful at improving HOME Inventory scores and improving children's physical, mental, and social-emotional development (St. Petersburg-USA Orphanage Research Team, 2008). Mean differences in ratings between these two orphanages would contribute to the proposition that the CCSERRS is sensitive to quasi-experimentally produced differences in caregiver social-emotional actions and relationship behavior (i.e., validity).

Results

Table 1 presents means and standard deviations for individual items for the non-intervention and intervention orphanages for each of the 18 items, the independent-samples t test of mean differences and its significance, and the partial eta2 reflecting the percent variance associated with the mean difference between the two orphanages.

Table 1 shows significant orphanage differences for all but four individual items, which had no or very limited variability. Most of the partial eta2 estimates of percent variance associated with the mean difference between orphanages were above .50 (except the items that had limited variability), meaning that half the total variability in individual item ratings was associated with orphanage differences. The Total Score was more than twice as great in the intervention group—a mean item score of 2.39 in vs. 1.06.

These results suggest that the individual items and the Total Score on the CCSERRS are sensitive to quasi-experimentally produced improvements in caregiver behavior.

Formal Reliability in a Nicaraguan Orphanage

Reliability was more formally assessed by several independent coders in an orphanage for children a few months to 8 yrs. old in Managua, Nicaragua. The orphanage environment was spartan with minimum furnishings, toys, and equipment. For example, the average Infant-Toddler Environmental Rating Scale and Early Childhood Environmental Rating Scale scores were 1.16 on scales ranging from 1-7.

Method

For purposes of reliability, there were five coders who formed ten pairs who coded 14 different caregivers who were responsible for providing all aspects of care for children.

Coder training

Coders were trained by first spending three days observing caregiving activities that had been videotaped in a different orphanage and in USA child care settings and discussing the nature of behaviors on the tapes with respect to the 18 items on the CCSERRS. This was followed by one morning of live practice observation in an elementary school center; each coder possessed definitions of the 18 items, pairs of coders observed a caregiver but assigned ratings privately, and then pairs of coders discussed any discrepancies between their ratings.

Observations

Ideally, each caregiver was observed engaged in three types of Activities—feeding, dressing/bathing, and free play. Within each Activity, the caregiver was observed for two 5-min. observations or Episodes while engaged in that Activity with two different children in each Activity. Thus, each caregiver was observed in six 5-min. Episodes of three different Activities for a total of 30 min. Mean scores were calculated for each Activity (the score from one Episode was used if it was impossible to obtain two Episodes on a caregiver; 27% of Episodes were missing as well as for Total Score.

Reliability Results

Percent agreement

Over all items and all Episodes, pairs of coders assigned an identical rating in 56.44% of the cases and were identical or within one point on 90.57% of the cases, figures only slightly lower than the 61% and 96% for the coders in the pilot study. Agreement was slightly higher for the Feeding Activity than for either the Bathing/Dressing or Free Play Activities. Percent agreements did not differ as a function of whether the caregiver was in charge of children birth to 4 yrs. or 4-8 yrs. of age.

For individual items, between 43% and 71% of the rating pairs were identical and 81%-98% were identical or within one point. Each coder had percent-agreement rates with each of the other coders similar to those reported above for the entire group, indicating that none of the five coders seemed to be at odds with any of the others.

Psychometric Information

The five coders individually rated all 55 of the major caregivers in the orphanage who attended to children 6 yrs. of age or younger. It was of interest to know how much variability in the scores could be attributed to differences between caregivers, coders, and the three Activities. An analysis of variance with Caregivers and Coders as random and Activity as a fixed factor revealed a significant effect for Caregivers (F=3.07, df=54/267, p<.001) and Activity (F=11.69, df=2/267, p<.001), but not for Coders (F<1). Differences between Caregivers accounted for 38% of the variance (partial eta2), Activity was associated with 8%, and Coders with 1%. This indicated quite clearly that the ratings reflected individual differences in Caregivers to a substantially greater extent than differences between Coders (38% versus 1%; Groark, McCall, & the Whole Child International Team, 2007).

Item—Partial Total Score correlations

The correlations between each item and the Total Score based on the 17 other items excluding the item in question were calculated. Except for “5. Caregiver-directed behaviors” (.315), “16. Child negative affect” (.426), “15. Child responsiveness and anticipation” (.457), and “11. Caregiver negative affect” (.476), all correlations were above .50 and eight items correlated above .60. Items with lower correlations are primarily negative behaviors, have relatively lower reliabilities, and occurrences of these behaviors were either so infrequent (e.g., children and caregivers rarely displayed negative affect) or very frequent (i.e., caregiver-directed behavior) that variability was substantially constrained. Generally, the Total Score seems to reflect a single general characteristic of caregiver behavior, but separate less prominent characteristics may also be involved (see below).

Factor analysis

The ratings of 55 caregivers by five coders were subjected a a principal component analysis with varimax rotation, and Table 2 presents all factor loadings .30 or greater. Negative behavioral items were reverse coded so high scores and positive loadings reflect the positive version of the behavior in question.

Table 2

Factor Analysis of the 18 Coding Items

The first of three factors retained in the model accounted for 34% of the variance. It seems to reflect Caregiver-Child Mutual Engagement and Relationship, including mutual responsiveness and positive affect and affection (recall negatively phrased items are reverse scored and a positive loading reflects a lack of those negative behaviors). Note that child items are also loaded on this factor, which suggests true caregiver and child reciprocal interaction, engagement, and relationship. Low scores on this factor may reflect caregiver-child detachment and very little meaningful warm, sensitive, responsive interaction, which is commonly observed in these orphanages.

The second factor, which accounted for 16.2% of the variance, reflects Caregiver Punitiveness, including both caregiver and child negative affect. The reverse scoring reflects the absence of these behaviors, so low scores would be associated with punitiveness while high scores would signal the absence of punitiveness (which often characterizes orphanage caregivers, at least while being observed by strangers).

The third factor, accounting for 10.2% of the variance, seems to represent Caregiver- vs. Child-Directed Behaviors and Intrusiveness (again the reverse scoring makes the factor positive so high scores represent many child-directed interactions and the absence of caregiver punitiveness). The structure of this factor is less simple than that of the first two, because three of the four items that defined it have moderate loadings (greater than .3) on the other two factors. This pattern seems explicable, however. Caregiver-directed behaviors, for example, are often displayed in orphanages at the expense of child-directed behaviors, they may be intrusive, and they may be part of disciplinary situations. Further, caregiver intrusiveness also occurs during disciplinary and punishment activities (Factor 2), and child-directed behaviors should occur more frequently in mutual engagement (Factor 1).

A factor analysis of the pooled sample from both St. Petersburg orphanages was also conducted after the orphanage item mean was subtracted from each item's rating (which removed any influence of orphanage mean differences from the correlation matrix). The results were quite similar to that reported above for the Managua orphanage, except the Managua third factor was now the second factor and the items “15. Child responsiveness and anticipation,” and “14. Child detachment and failure to respond” loaded on the Child- vs. Caregiver-Directed Behaviors and Intrusiveness factor to a greater extent than on the first factor. Given the inclusion of a very different orphanage with much improved caregiving in the St. Petersburg data, this degree of factorial consistency is encouraging.

Thus, the CCSERRS primarily reflects Caregiver-Child Mutual Engagement and Relationship, with subordinate components of Caregiver Punitiveness and Caregiver- vs. Child-Directed Behaviors and Intrusiveness. It should be noted that the caregiving in the Nicaragua orphanage that is the basis of this factor analysis was quite minimal, with a Total Score of 1.16 on a scale ranging from 0 to 3.0 (an average very close to the 1.06 obtained for the no-intervention orphanage in St. Petersburg), so the factor structure could change if caregiver behavior was more variable and ratings were spread more evenly throughout the 0-3 range (although the similarity in factor structure between the Nicaraguan and St. Petersburg data suggests the change might be minimal). The low levels of punitiveness and negative affect might also influence the factor structure, which was true in both samples.

Discussion

This paper reports preliminary reliability, validity, and psychometric information on a newly developed rating scale initially created to characterize caregiver-child social-emotional interactions and relationships in orphanages but that could also be used in non-residential early care and education settings and potentially parent-child interactions in a home environment. The intent was to identify a few dimensions of caregiver-infant/toddler/young child interactions that comprehensively covered the range of specific characteristics of caregiver-child social-emotional interactions and relationships, and to construct a scale that could be administered in a relatively short period of time and would not require extensive coder training, manuals, or materials.

The preliminary data presented on the Caregiver-Child Social/Emotional Relationship Rating Scale (CCSERRS) tentatively suggests that it can be reliably administered even using observation periods of as little as 5 or as long as 30 minutes spread over three kinds of caregiver activities (feeding, dressing/bathing, free play). Reliability was replicated over seven different coders working in three different orphanages. Brief training can prepare several coders (five in this study) to rate caregivers in a reliable and consistent fashion. Moreover, variability in scores was associated with individual differences in caregivers to a much greater extent than in coders (38% to 1%), and ratings of caregivers were similar across different types of caregiving activities (i.e., feeding, dressing/bathing, free play) and for caregivers attending children birth to 4 and 4 to 8 years of age.

The CCSERRS primarily reflects Caregiver-Child Mutual Engagement and Relationship with subordinate components of Caregiver Punitiveness and Caregiver- vs. Child-Directed Behaviors and Intrusiveness.

A pilot validity study demonstrated that most of the 18 items and the Total Score significantly discriminated between caregivers who received a special intervention of training and structural changes designed to improve their warm, caring, sensitive, and responsive interaction with children vs. orphanage caregivers who did not. This suggests that the CCSERRS likely has construct validity for reflecting positive social/emotional interactions and relationships between caregivers and children.

The CCSERRS has the advantage of being exclusively directed at caregiver-infant/toddler/young child social-emotional interactions and relationships, and it measures the extent of engagement, responsiveness, positive versus negative affect and affection, child- vs. caregiver-directed interactions, and support. These characteristics are only a small part of the more commonly used comprehensive home and group care environmental rating scales (i.e., HOME, ITERS/ECERS). Since these latter scales characterize the total physical and behavioral environment, the CCSERRS could be used in addition to the HOME or ECERS if a special focus on the social/emotional interaction is needed.

The authors encourage colleagues to try the CCSERRS and communicate the details of is use, reliability, validity, and psychometric properties to the authors. In this way a body of psychometric information can be amassed that can help overcome the limits of the present report (e.g., small Ns, only orphanage contexts, limited observation periods, specific caregiver activities, non-standardized observation procedures).

Clinical Implications

Early caregiver-child social-emotional interactions and relationships are theorized to have major implications for infant-toddler and longer-term mental health and behavioral competence, and the empirical literature generally confirms this principle (see above). A relatively simple scale that more directly rates such interactions has the potential of facilitating research and practice in this domain.

Appendix 1. The Caregiver-Child Social/Emotional and Relationship Rating Scale (CCSERRS)

Constructs to be Rated

Caregiver Engagement

1. Caregiver detachment

in the presence of a child or children, caregiver does not make eye contact; does not talk to children in a social way; does not engage children when that would be possible; ignores them when they are eating, dressing, playing; does other tasks during free time, such as talk to other caregivers, takes a break to smoke or groom herself; props bottle instead of feeding child.

2. Caregiver failure to respond

caregiver does not respond to child's vocalizations, smile, other social initiative, distress (crying, frustration); caregiver responds but in a non-social, business-like, mechanical manner. Caregiver does not psychologically engage the child. Caregiver fails to share enthusiasm, joy, or excitement of children; caregiver fails to display pride in children's accomplishment. If there is no stimulus from child, this is rated high (i.e., failure to respond is frequent).

3. Caregiver psychological availability and receptivity to children

caregiver looks child in the eye and holds child face-to-face while feeding or in play; caregiver listens attentively when children speak (e.g., looks at them, responds by nodding); caregiver gets at the level of the child (kneels, sits on floor with children) to have better eye contact, to hear a child, to talk or play with children; caregiver attends to and engages appropriately a child who is constructively engaged (she goes with the child's flow).

4. Caregiver responds to child

caregiver paces caregiving to match child's behavior (e.g., waits to give next spoonful of food until the child is ready), is patient with slow eaters or dressers; tolerates messiness; caregiver responds to child's behavior, signals, and communication attempts; caregiver makes empathetic verbalizations or facial expressions to child's actions and events; caregiver shares joy and excitement, pride in children's accomplishments; caregiver imitates child's behavior (vocalization, smile, actions); caregiver praises, encourages, rewards child for child's behavior. Mechanical response is scored low.

Caregiver/Child-Directed Behaviors

5. Caregiver directed behaviors

caregiver directs child physically or verbally in a non-disciplinary activity (e.g., how to eat, get dressed, wash, play) and expects the children to follow her directions or imitate her actions; caregiver talks at children; caregiver models use of an object for child to imitate in play or shows the child what to do (with object, how to dance); caregiver labels objects, parts of body, food, and expects imitation or no response; caregiver reads books to children with no child participation; caregiver teaches children about something, explains cause-effect, gives information without expectation of questions or discussion.

6. Child-directed behaviors

caregiver lets child lead and she follows (e.g., allowing the child to do something to the caregiver such as pulling hair or touching glasses) and caregiver responds in playful or positive way; caregiver engages in conversation back and forth (even if child responds non-verbally); caregiver plays reciprocal games with children (roll ball back and forth, play catch, peek-a-boo, goochie-goo); caregiver asks what the child wants to do and how the child wants to do it; caregiver asks open-ended questions; caregiver promotes peer interaction (turn taking, conversation, cooperation, sharing, pretend play). Caregiver appropriately allows children to “do their own thing.”

Behavioral Control

7. Caregiver intrusiveness

caregiver performs a caregiving action (feeding, bathing, changing) without engaging or preparing the child (“ready or not”); she catches the child by surprise; caregiver hurries caregiving without waiting for child to adjust (e.g., keeps spooning food into mouth, keeps putting bottle in mouth, keeps soaping child when child is upset). Caregiver stimulated child when child does not pay attention or respond to it. Caregiver intrudes on children who are appropriately engaged and prohibits what they want to do or redirects the child when it is unnecessary, she interrupts child activity to get the child to do something else.

8. Caregiver behavioral control/obedience/discipline

caregiver directs children individually or in group behavior (stand in line, sit against the wall, hold onto another child) and expects conformity to caregiver intentions; caregiver “corrects” behavior (e.g., keep in line, go over there, no food fights, etc.). Child is interested in one thing and caregiver reorients child to another.

9. Caregiver punishes a child

caregiver physically and negatively grabs a child to move him or her; corrects child in a clearly negative way; hits or spanks a child. Verbally abuses, yells or talks down to child in a disrespectful manner.

10. Caregiver support, empathy, guidance

caregiver kindly and positively encourages child to feed, dress, bath self; caregiver is sympathetic or empathetic with a child having difficulty or one who is hurt and crying; caregiver changes child behavior in a positive and supportive way by providing “guidance” or “suggestions;” caregiver explains a rule or consequences and calmly redirects behavior all in a positive and supportive way.

Caregiver Affect with Children

11. Caregiver negative affect

caregiver displays negative affect to a child (frowns, scowls); caregiver speaks negatively, harshly, with annoyance, hostility, and scolds or shouts at the child; caregiver is critical of children, puts them down, uses sarcasm, blames child, says “bad boy”.

12. Caregiver displays positive affect and affection

she smiles at child; talks to the child in a positive, warm, affectionate, supportive way; she hugs, kisses, and warmly holds a child.

13. Caregiver animation and expressiveness

caregiver talks or reads with animation and expressiveness; is animated or expressive when engaged with children; changes emotional expression to fit moment-to-moment activity both positive and negative (e.g., winces in animated pain when child falls, makes empathetic smiles or frowns to the child's actions); expresses positive emotion and enthusiasm to a child's accomplishments (praises and smiles to child's drawing or stack of blocks).

Child Engagement

14. Child detachment and failure to respond to caregiver

child seems unresponsive emotionally and physically to appropriate and positive caregiver behavior and caregiver attempts to engage the child (e.g., attempts to get the child to smile or laugh); child seems unresponsive emotionally to negative caregiver behavior or “discipline” or harsh words; child does not smile or “brighten,” get “excited,” wiggle with anticipation, or raise arms to be picked up when a caregiver comes over to attend to them or pick them up. If child responds in a forced, non-social, mechanical manner score high.

15. Child responsiveness and anticipation

child readily makes eye contact with caregiver, attempts to engage caregiver, asks questions or talks to caregiver, calls for caregiver; child anticipates caregiver engagement by smiling, “brightening,” shows excitement or wiggles, raises arms to be picked up when caregiver attends to him or her; child responds to caregiver attempts to engage the child by talking back, playing peek-a-boo; child anticipates comfort or help from caregiver when crying, frustrated, assaulted by other children or having toy taken away; child initiates talk to caregiver and clearly expects caregiver to talk back. If caregiver provides no stimulus to child, score 0.

Child Affect

16. Child negative affect

child cries or whimpers, shows anger or other negative affect.

17. Children's positive affect

child displays positive affect (smile, laugh) in interaction with caregiver during routine caregiving or play; child is “alive” and emotionally labile, matching emotions to caregiver's emotions; child smiles or laughs when tickled. Child is happy, smiling, laughing on own or with peers.

Child Relationship

18. Child relationship with caregivers

infant watches caregiver intently; infant follows with eyes caregivers deliberate movements. Child uses caregiver as a “secure base” by sharing a toy with her, asking a question, looking at the caregiver to share a positive or negative experience; child seeks caregiver when experiencing stress or is upset; child checks in with caregiver from a distance while playing (e.g., makes eye contact or vocal or verbal contact with caregiver); child shows separation anxiety or distress when caregiver leaves or turns head away, child attempts to re-engage caregiver; child shows wariness of strangers and may cling to caregiver or hold on to skirt.

Scoring

Each item is scored on a four-point scale (0=never, 1=rarely, 2=frequently, 3=consistently) reflecting the frequency with which the named behavior (positive or negative) occurred relative to the potential opportunities for it to occur. Note special scoring for items 2, 4, 14, 15 (see above). Items 1, 2, 5, 7-9, 11, 14, and 16 are negative or undesirable but scoring is high if they are consistently present; these items must be reverse scored before calculating total or subscale scores.

Footnotes

1This project funded in part by NICHD grant HD39017 for the St. Petersburg, Russian Federation activities and by Whole Child International (Karen Gordon, President) for the Nicaraguan activities. The authors are indebted to Auxiliadora Alvarado, Ian Downing, Juan Diego Rivas, Raquel Suazo, and Karla Urbina for conducting the coding in Managua, to their supervisors Mauricio Gaborit and Marisol Vanegas Guido of the Universities of Central America in San Salvador and Managua, respectively, and to Gabriella Serrano of Whole Child International for logistics management.

2A more complete report including tables of data is available at (www.educaton.pitt.edu/ocd).

References

• Arnett J. Caregivers in day care centres: Does training matter? Journal of Applied Developmental Psychology. 1989;10:541–552. [Google Scholar]
• Aviezer O, Sagi A, Resnick G, Gini M. School competence in young adolescence: Links to early attachment relationships beyond concurrent self-perceived competence and representations of relationships. International Journal of Behavioral Development. 2002;26:397–409. [Google Scholar]
• Blizzard RM. Psychosocial short stature. In: Lifshitz F, editor. Pediatric endocrinology. New York, NY: Marcel Dekker; 1990. pp. 77–91. [Google Scholar]
• Bradley RH, Caldwell BM. Caregiving and the regulation of child growth and development: Describing proximal aspects of caregiving systems. Developmental Review. 1995;15:38–85. [Google Scholar]
• Caldwell BM, Bradley RH. Home Observation for Measurement of the Environment. Little Rock: University of Arkansas at Little Rock; 1984. [Google Scholar]
• Carlson EA. A prospective longitudinal study of disorganized/disoriented attachment. Child Development. 1998;69:1107–1128. [PubMed] [Google Scholar]
• DeWolff MS, van IJzendoorn MH. Sensitivity and attachment: A meta-analysis on parental antecedents of infant attachment. Child Development. 1997;68:571–591. [PubMed] [Google Scholar]
• Edwards CP, Raikes H. Extending the dance: Relationship-based approaches to infant/toddler care and education. Young Children. 2002;57:10–17. [Google Scholar]
• Erickson MF, Sroufe LA, Egeland B. The relationship between quality of attachment and behavior problems in preschool in a high-risk sample. In: Bretherton I, Waters E, editors. Growing points of attachment Theory and Research. Monographs of the Society for Research in child Development. 1-2. Vol. 50. 1985. pp. 147–166. Serial #209. [PubMed] [Google Scholar]
• Fonagy P, Target M, Steele M, Steele H, Leigh T, Levinson A, Kennedy R. Crime and attachment: Morality, disruptive behavior, borderline personality disorder, crime, and their relationships to security of attachment. In: Atkinson L, Zucker K, editors. Attachment and psychopathology. New York: Guilford Press; 1997. pp. 223–274. [Google Scholar]
• Goossens C, Melhuish EC. On the ecological validity of measuring the sensitivity of professional caregivers: The laboratory versus the nursery. European Journal of Psychology of Education. 1996;11:169–176. [Google Scholar]
• Greenberg MT. Attachment and psychopathology in childhood. In: Cassidy J, Shaver PR, editors. Handbook of attachment: Theory, research, and clinical applications. New York: Guilford; 1999. pp. 469–496. [Google Scholar]
• Groark CJ, McCall RB, and the Whole Child Inernational Team . Managua orphanage baseline report. Pittsburgh, PA: University of Pittsburgh Office of Child Development; 2007. [Google Scholar]
• Groark CJ, Muhamedrahimov RJ, Palmov OI, Nikiforova NV, McCall RB. Improvements in early care in Russian orphanages and their relationship to observed behaviors. Infant Mental Health Journal. 2005;26:96–109. [PubMed] [Google Scholar]
• Gunnar M. Effects of early deprivation; Findings from orphanage-reared infants and children. In: Nelson CA, Luciana M, editors. Handbook of developmental cognitive neuroscience. Cambridge, MA: MIT Press; 2001. pp. 617–629. [Google Scholar]
• Harms T, Clifford RM, Cryer D. Early Childhood Environmental Rating Scale. Revised. NY: Teachers College Press; 1998. [Google Scholar]
• Harms T, Cryer D, Clifford RM. Infant/Toddler Childhood Environmental Rating Scale. NY: Teachers College Press; 1990. [Google Scholar]
• Howes C, Stewart P. Child's play with adults, toys, and peers: An examination of family and child care influences. Developmental Psychology. 1987;23:423–430. [Google Scholar]
• Johnson DE. Medical and developmental sequelae of early childhood institutionalization in Eastern European adoptees. In: Nelson CA, editor. The effects of early adversity on neurobehavioral development. Vol. 31. Mahwah, NJ: Lawrence Erlbaum; 2000. pp. 113–162. [Google Scholar]
• Kontos S, Howes C, Shinn M, Galinsky E. Quality in family child care and relative care. New York: Teachers College Press; 1995. [Google Scholar]
• Koren-Karie N, Sagi-Schwartz A, Egoz-Mizrachi N. The emotional quality of child care centers in Israel: The Haifa study of early child care. Infant Mental Health Journal. 2005;26:110–126. [PubMed] [Google Scholar]
• Landry SH, Smith KE, Miller-Loncar CL, Swank PR. Predicting cognitive-language and social growth curves from early maternal behaviors in children at varying degrees of biological risk. Developmental Psychology. 1997;33(6):1040–1053. [PubMed] [Google Scholar]
• Landry SH, Smith KE, Swank PR. Responsive parenting: Establishing early foundations for social, communication, and independent problem-solving skills. Developmental Psychology. 2006;42:627–642. [PubMed] [Google Scholar]
• Landry SH, Smith KE, Swank PR, Miller-Loncar CL. Early maternal and child influences on children's later independent cognitive and social functioning. Child Development. 2000;71:358–375. [PubMed] [Google Scholar]
• Lyons-Ruth K, Alpern L, Repacholi D. Disorganized infant attachment classification and maternal psychosocial problems as predictors of hostile-aggressive behavior in the preschool classroom. Child Development. 1993;64:572–585. [PubMed] [Google Scholar]
• MacLean K. The impact of institutionalization on child development. Development and Psychopathology. 2003;15:853–884. [PubMed] [Google Scholar]
• McGuire J, Richman N. Management of behavior problems in day nurseries [Special Issue] Early Child Development and Care. 1987;45 [Google Scholar]
• Melhuish EC. The quest for quality in early day care and preschool experience continues. International Journal of Behavioral Development. 2001;25:1–6. [Google Scholar]
• Melhuish EC, Mooney A, Martin S, Lloyd E. Type of child care at 18 months: I. Differences in interactional experiences. Journal of Child Psychology and Psychiatry. 1990;31:849–859. [PubMed] [Google Scholar]
• Muhamedrahimov RJ, Palmov OI, Nikiforova NV, Groark CJ, McCall RB. Institution-based early intervention program. Infant Mental Health Journal. 2004;25:488–501. [Google Scholar]
• Nelson CA, Zeanah CH, Fox NA, Marshall PJ, Smyke AT, Guthrie D. Cognitive Recovery in socially deprived young children: The Bucharest early intervention project. Science. 2007 Dec 21;318:1937–1940. [PubMed] [Google Scholar]
• Pierrehumbert B, Ramstein T, Krucher R, El-Najar S, Lamb ME, Halfon O. L'evaluation du lieu de vie du jeune enfant: Développement d'un instrument. Bulletin de Psychologie, XLIX. 1996;426:565–584. [Google Scholar]
• Rosas J, McCall RB. Characteristics of institutions, interventions, and children's development. Pittsburgh, PA: University of Pittsburgh Office of Child Development; 2008. [Google Scholar]
• Rothbaum F, Weisz JR. Parental care giving and child externalizing behavior in non-clinical samples: A meta-analysis. Psychological Bulletin. 1994;116:55–74. [PubMed] [Google Scholar]
• Shaw DS, Owens EB, Vondra JI, Keenan K, Winslow EB. Early risk factors and pathways in the development of early disruptive behavior problems. Development and Psychopathology. 1997;8:679–700. [Google Scholar]
• St. Petersburg – USA Orphanage Research Team. Characteristics of children, caregivers, and orphanages for young children in St. Petersburg, Russian Federation. Journal of Applied Developmental Psychology: Child Abandonment, Special Issue. 2005;26:477–506. [Google Scholar]
• St. Petersburg-USA Orphanage Research Team. The effects of early social-emotional and relationship experience on the development of young orphanage children. Society for Research in Child Development Monograph. 2008;73(3) Serial No. 291. [PMC free article] [PubMed] [Google Scholar]
• Steelman LM, Assel MA, Swank PR, Smith KE, Landry SH. Early maternal warm responsiveness as a predictor of child social skills: Direct and indirect paths of influence over time. Applied Developmental Psychology. 2002;23:135–156. [Google Scholar]
• Sylva K. Target child observation. In: Sylva K, Stevenson J, editors. Social behavior and competence in childhood. Windsor, UK: NFER-Nelson; 1997. [Google Scholar]
• van IJzendoorn MH, Sagi A. Cross-cultural patterns of attachment. In: Cassidy J, Shaver PR, editors. Handbook of attachment. New York, NY: Guilford Press; 1999. pp. 713–734. [Google Scholar]
• van IJzendoorn MH, Tevecchio LWC. Quality of center day care and attunement between parents and caregivers. Center day care in cross-national perspective. Journal of Genetic Psychology. 1998;159:437–456. [PubMed] [Google Scholar]
• Zeanah CH, Smyke AT, Koga SFM. Effects of foster care following institutionalization on cognitive development and behavior problems. In: Nelson CA (Chair), editor. The effects of early institutionalization on brain behavior development: The Bucharest Early Intervention Project; Symposium presented at the biennial meeting of the Society for Research in Child Development; Tampa, FL. 2003. Apr, [Google Scholar]