skip to main content skip to footer

Development of the Behaviorally Anchored Rating Scales for the Skills Demonstration and Progression Guide BARS GRS O*NET

Author(s):
Klieger, David M.; Kell, Harrison; Rikoon, Samuel H.; Burkander, Kri; Bochenek, Jennifer; Shore, Jane R.
Publication Year:
2018
Report Number:
RR-18-24
Source:
ETS Research Report
Document Type:
Report
Page Count:
38
Subject/Key Words:
Behaviorally Anchored Rating Scales (BARS), Behavioral Summary Scales, Graphical Rating Scales (GRS), Job Performance, Job Training, Job Descriptions, Workforce Readiness, 21st Century Skills, Noncognitive Skills, Socioemotional Skills, Psychosocial Skills, Soft Skills, Critical Incidents Method, Occupational Information Network (O*NET)

Abstract

In this research, we developed behaviorally anchored rating scales (BARS) to evaluate the job performance of Zone Three jobs requiring middle‐level skills and prior education ranging from vocational training to an associate's degree as well as work‐related skills or experience. We ultimately identified 7 relevant job performance dimensions mainly based on prior research literature. The dimensions are thought to reflect socioemotional constructs (soft skills, “21st‐century skills”) considered vital to success today in Zone Three jobs. Managers of Zone Three employees helped us develop the final behavioral statements to anchor the 6 BARS points by generating approximately 430 critical incidents on which we based the initial behavioral statements that we wrote. Another group of managers confirmed the relevancy of the statements to the Zone Three workers they supervised, retranslated statements back into dimensions to confirm the dimensions to which they belong, and provided ratings of the effectiveness level that each statement represented. We modified statements to fill any gaps in effectiveness level coverage. Then, we asked leading experts in assessment to confirm the adequacy of the final statements, and after they did so, we created the final anchored scales. Both past literature and our own analyses indicate that these BARS generalize across economic sectors and thus potentially can provide substantial value to organizations that wish to assess, in an efficient and cost‐effective manner, the performance of middle‐skills employees with many different types of possible job descriptions.

Read More