The following is an excerpt from Rikoon, S. H., Liebtag, T., Olivera-Aguilar, M., Steinberg, J., & Robbins, S. B. (2015). Anticipating college enrollment: Adapting SuccessNavigator for high school samples (ETS Research Report No. RR-15-37). Princeton, NJ: Educational Testing Service. Click here for the full report: https://onlinelibrary.wiley.com/doi/epdf/10.1002/ets2.12084
Prior research has demonstrated positive relationships between the expression of noncognitive or psychosocial skills and both academic success and persistence in higher education (e.g., Porchea, Allen, Robbins, & Phelps, 2010; Poropat, 2009; Robbins, Allen, Casillas, Peterson, & Le, 2006; Robbins et al., 2004; Trapmann, Hell, Hirn, & Schuler, 2007). Specifically, the SuccessNavigator® assessment, created by Educational Testing Service (ETS), has also been shown to predict collegiate academic success (Markle, Olivera-Aguilar, Jackson, Noeth, & Robbins, 2013; Rikoon, Liebtag, Olivera-Aguilar, Robbins, & Jackson, 2014). While this body of research focuses on postsecondary outcomes, it does not diminish the importance of developing strong behavioral skills earlier in one’s academic career. In particular, the demonstration of strong noncognitive skills and appropriate academic behaviors prior to enrolling in a postsecondary institution is vital for long-term success beyond the high school level (Farrington et al., 2012).
Venezia and Jaeger (2013) highlighted the importance of holistic approaches to high school intervention, suggesting that the most effective programs designed to encourage college readiness are able to identify and target specific areas of need. Furthermore, Lleras (2008) found that the exhibition of specific personality and noncognitive factors (i.e., conscientiousness, motivation, and cooperativeness) among high school students related to higher levels of educational attainment. High School SuccessNavigator (HSSN) was developed as an extension of the original SuccessNavigator assessment (Markle et al., 2013), which was focused on students who have already completed the transition to college. The broad goal of HSSN is to facilitate the standardized identification of noncognitive strengths and potential vulnerabilities among late–high school (11th and 12th grade) students.
In this report, we detail the conceptual and methodological development of HSSN along several lines. We first discuss contextual issues motivating the expansion of the SuccessNavigator assessment to high school environments, proceeding to detail its multiple intended uses and utility to the field. We conclude by noting the limitations of the current study and highlighting future directions for research.
Counseling Burden in U.S. Public Schools
The National Association for College Admission Counseling (NACAC) estimates that the average national student to counselor ratio in U.S. public K–12 schools is 478:1 (McDonough, 2006)1compared to the 250:1 ratio recommended by the American School Counselor Association (ASCA) and the 100:1 ratio recommended by NACAC (American School Counselor Association, 2011, 2012; Clinedinst & Hawkins, 2011). Further complicating the issue of high school student to counselor ratios is that, although ASCA provides recommendations defining the role and responsibilities of professionally certified counselors,2 those of a typical practicing counselor are difficult to determine and must (by virtue of differing access to resources) vary widely between schools and districts. Examples of duties assigned to counselors beyond the traditional tasks of course planning and managing behavioral or disciplinary issues include (a) fulfillment of requests for student data, (b) standardized test coordination and administration, (c) leading partnerships with community organizations, (d) development and implementation of intervention strategies, (e) working intensively with students deemed at risk of academic failure, (f) managing budgetary restrictions, and (g) precollege advisement (Burnham & Jackson, 2000; Perna et al., 2008; Woods & Domina, 2014). This is not to suggest that any of the preceding activities are inappropriate for counselors to engage in, only that the scope of their responsibilities has the potential to become burdensome even before considering any time spent interacting with students.
With specific attention to college advisement, McDonough (2006) estimated that “students in public school can expect less than an [one] hour of postsecondary education counseling during the entire school year” (p. 3). Research by Engberg and Gilbert (2014) showed that the amount of time high school counselors spend on college-related activities and advisement is related positively to college attendance rates. Counselor availability, their level of student interaction, and the extent to which schools offer institutional support and college planning resources are all key contributors to the likelihood of a high school student pursuing higher education (Belasco, 2013; Engberg & Gilbert, 2014; Perna et al., 2008; Woods & Domina, 2014).
The impact of high school counselors on the pursuit of higher education appears to be particularly important among students from low-income backgrounds. Belasco (2013) found that students with a lower socioeconomic status (SES) “may have yielded more benefit from their relationship with a school counselor” (p. 795; emphasis added) compared to students with a higher SES. Perhaps not surprisingly, given the positive relationship between educational attainment and income levels (e.g., Bailey & Dynarski, 2011), lower individual and institutional SES have also been associated with a lower likelihood of pursuit, enrollment in, and completion of a postsecondary program (Belasco, 2013; Oseguera, 2013). This finding is perhaps a direct result of higher student to counselor ratios and increased burdens of responsibility in schools primarily serving students in low-income situations, relative to schools located in affluent areas (Engberg & Gilbert, 2014).
In addition to the potential institutional obstacles faced by students in low-income situations, they also experience a significant degree of academic “undermatch.” An undermatch occurs when student’s academic credentials would have gained that student access to a college or university more selective than the postsecondary institution at which the student actually enrolled (Smith, Pender, & Howell, 2013). Research over the past decade has found that students in low-income situations are especially prone to undermatching (Avery et al., 2006; Dillon & Smith, 2013; Griffith & Rothstein, 2009; Hoxby & Avery, 2013). Most often, undermatching is not the direct result of a higher education admissions process but is derived from students not submitting an application to a more selective institution in the first place (Dillon & Smith, 2013). Prior studies have revealed several factors contributing to student undermatch, including a lack of information about application and admission processes, concerns about the affordability of selective colleges, and greater distance between a student’s home and selective versus less selective institutions (Avery et al., 2006; Griffith & Rothstein, 2009; Hoxby & Avery, 2013; Roderick, Nagaoka, Coca, & Moeller, 2008).
Utility of High School SuccessNavigator
The combined importance of assisting high school students reach their full educational potential and alleviating the current pressure on overburdened counselors to serve all of their students effectively (e.g., by working to prevent undermatch) has motivated nationwide initiatives to support students in low-income situations or those otherwise in need.3 As these types of programs, organizations, and high school counselors in general strive to achieve their respective missions, they naturally imply (if not explicitly claim to deliver) efforts to cultivate important noncognitive skills among the students they serve. For example, a student without prior plans to apply to competitive colleges or universities might be required to develop increased levels of perseverance, organization, and stress management to complete the process successfully.
It is in these and similar contexts where an assessment such as HSSN can potentially meet an essential need by enabling student support staff to gauge each student’s areas of greatest need and focus their efforts accordingly. Furthermore, the counseling process may become more efficient through the implementation of an assessment that students complete on their own, thereby freeing up their already limited time in direct contact with a counselor for the actual delivery of guidance. Recognizing that counselors will naturally form a casual impression of student noncognitive skill levels and behavioral characteristics (e.g., through direct interactions or others’ reports), the development of HSSN provides a more rigorous metric on which these qualities in a supportive, low-stakes context can be evaluated.
HSSN is an assessment carrying low or no stakes. While assessments that help in making high-stake decisions are associated with the offer or denial of a specific outcome or opportunity for test takers dependent on their performance (e.g., graduation from high school, admission to a particular college, advanced course placement, employment offer), a low-stake classification implies no such direct impact of assessment results on the individual level. The primary intent underlying the development of HSSN was to provide an advising tool demonstrating psychometric rigor to facilitate individualized student counseling. In practice, HSSN should provide a common language, series of metrics, and supportive materials to aid counselors and students alike in solidifying the skills and behaviors necessary for success in higher education. More specifically, an implementation of HSSN could serve four primary purposes:
- Assess student strengths and areas of need on a broad spectrum of noncognitive skills and behaviors identified as significant contributors to academic success on the postsecondary level.
- Provide usable feedback to high school guidance counselors and college advisors in the form of HSSN score reports and associated resource guide materials. Counselors could make use of such feedback to coordinate the individualized delivery of institutional resources and interventions to support student needs.
- Provide specific details regarding students’ stated intentions toward higher education. Combining these data with noncognitive skill levels would help gauge students’ motivation to pursue postsecondary opportunities. In addition to post–high school plans, HSSN reports may also include related background data on each student’s grade point average (GPA) and standardized test scores (if available).
- Aggregate and report student data at the school level. Administrators could use aggregate reports to determine the relative strengths and weaknesses of their specific student body as well as where their institution lies in comparison to the large sample of students who participated in the HSSN field trial (described in detail in the next section). Similar to their use on the college level, high school leaders could employ institutional reports to inform program development catered to the specific needs of their student population.
Development of High School SuccessNavigator
HSSN encompasses four general construct domains (see Table 1 for a summary of all targeted constructs). These are academic skills, achievement motivation, self-management, and social support. With some modification (see the next section for detail), these domains are similar in scope to those assessed by the original college-level SuccessNavigator assessment (Markle et al., 2013), which was theoretically anchored to the five-factor model of personality (also known as the Big 5; McCrae & Costa,1987). HSSN, being an adaptation of this assessment, incorporates focal constructs that remain primarily aligned in theory with three of the Big 5 personality constructs: conscientiousness, emotional stability, and extraversion.
| Domains and Subskills | Sample Assessment Item |
|---|---|
| Academic Skills: Tools and strategies for academic success | |
| Meeting Class Expectations: Doing what’s expected to meet the requirements of your course including assignments and in-class behavior |
|
| Organization: Strategies for organizing work and time
Perseverance: Determination and effort to pursue tasks/goals despite difficulty or delay |
|
| Achievement Motivation: Active pursuit of personal and academic goals | |
| Commitment to College Degree Goals: Perceived value and determination to succeed and complete college |
|
| Academic Self-Efficacy: Belief in one’s ability to perform and achieve in an academic setting |
|
| Self-Management: Internal reactions and the belief in a personal ability to succeed | |
| Managing Test Anxiety: General reactions to test-taking experiences, including negative thoughts and feelings (e.g., worry, dread) |
|
| Managing Stress: Tendency to feel frustrated, discouraged, or upset when under pressure or burdened by demands |
|
| Social Support: Connecting with people and student resources for success | |
| Connectedness: A general sense of belonging and engagement
School-related Help Seeking: Attitudes about and tendency to seek help from established resources |
|
| Lacking Barriers: Financial pressures, family responsibilities, conflicting work schedules, and limited institutional knowledge |
|
Previous research has shown higher levels of conscientiousness to be positively associated with academic success at all levels of education through to adulthood (Ackerman & Heggestad, 1997; de Fruyt & Mervielde, 1996; Noftle & Robins, 2007; O’Connor & Paunonen, 2007; Shiner, Masten, & Roberts, 2003). Typical descriptions of the construct include characteristics such as goal striving, drive, effort, and an ability to organize and follow through with plans. These and similar elements are conceptually related to the academic skills and achievement motivation domains included in HSSN. Academic skills include individuals’ organizational capabilities (e.g., managing schoolwork and time), their ability to meet classroom expectations (e.g., behaviors directed toward meeting course requirements), and the extent to which they persist in their work and tasks despite any difficulties encountered. Achievement motivation centers on the extent to which students maintain a focus on pursuing higher education (e.g., perceiving college as valuable to their future) and believe in their own ability to succeed or achieve their academic goals (i.e., academic self-efficacy; Bong, 2001; Bong & Skaalvik, 2003; Chemers, Hu, & Garcia, 2001).
Emotional stability (interpreted as exhibiting low levels of the Big 5 dimension neuroticism; i.e., being even-tempered, maintaining calm when faced with stress) has been found in previous meta-analytical research to be related to academic achievement and persistence (Poropat, 2009; Richardson, Abraham, & Bond, 2012; Robbins et al., 2004). Indicators of emotional stability (such as the extent to which students are able to control their responses to external stress and anxiety) are assessed in HSSN’s self-management domain. Self-management is defined by students’ responses to and their ability to manage general personal stressors as well as the severity of their reactions to the experience of test taking (Chapell et al., 2005). Test anxiety represents a specific and important challenge to negotiate (if not overcome) for high school students preparing to apply to colleges or universities (Hill, 1984; Zeidner, 1998). This issue is relevant for students preparing to apply to postsecondary institutions where their performance on a standardized high-stakes assessment (e.g., SAT®, ACT) is likely to serve as an initial threshold for admission as well as those who may be required to take academic placement exams (carrying similarly high stakes for course placement) upon enrolling in less selective institutions (Charlesworth, Fleege, & Weitman, 1994; Cizek & Burg, 2006; Segool, Nathaniel, Mata, & Gallant, 2014).
Extraversion is a personality trait characterized by an individual’s aptitude and willingness to engage and connect with other people. Meta-analyses have demonstrated mixed results in terms of relationships between extraverted attitudes or behaviors (e.g., social involvement) and success in higher education, generally showing them to be either near-zero or moderately positive (Poropat, 2009; Richardson et al., 2012; Robbins et al., 2004; Trapmann et al., 2007). HSSN addresses this area indirectly through its assessment of the social support mechanisms students may have access to as they work to complete high school. Attitudes and behaviors used to define social support in HSSN include the extent to which students feel connected with their peers and community at large as well as students’ knowledge of and comfort with utilizing the institutional resources offered by their school (e.g., academic support or other counseling).
Apart from the four primary domains assessed by HSSN, the instrument also includes several items designed to assess the extent to which students experience external barriers to their academic success (e.g., family issues, other extracurricular responsibilities, illness). These items address specific challenges to students’ time and attention, by definition exerting a negative influence on their ability to complete the required schoolwork and focus on furthering their education. All else held constant, experiencing fewer such barriers would be expected to relate positively to academic achievement.
Future Directions
Perhaps the most important direction for future research is to attempt to gather college enrollment and/or performance data for the current sample (e.g., admission, enrollment status, institution type, collegiate GPA). Students participating in the HSSN field trial study were still enrolled in high school during this report’s production. Compiling college enrollment data will be vital to generating predictive evidence for the validity of HSSN scores to the extent that they portend divergent postsecondary outcomes. It may also be of interest to use enrollment data to develop a predictive index capable of gauging a current high school student’s likelihood of successfully transitioning to higher education.
One clear limitation of the current study is its focus on students enrolled in traditional public and charter high school environments as opposed to other populations of precollege students. Future studies to expand the potential applicability of HSSN might investigate its adaptation to adult learners in pursuit of a high school equivalency credential (e.g., the HiSET® test, GED) or those following other alternative pathways to postsecondary education.
As a newly available assessment targeting noncognitive attitudes and behaviors, HSSN provides an opportunity to pursue multiple lines of future research (beyond those discussed previously) with significant potential value to the field. Chief among these may be studies deploying the assessment longitudinally in an effort to understand the stability of its measurement characteristics over time as well as its sensitivity to developmental change. A similar line of research would see HSSN employed as an interim or outcome assessment within the context of an intervention study (e.g., to gauge the impact of a program designed to improve noncognitive skills levels).
There are also several other applied studies that would be interesting to conduct using HSSN data along with student outcomes. As an example, latent profile analysis is expected to prove a useful strategy for developing typologies of late high school noncognitive skill expression. Such analyses should in turn enable the targeted delivery of student support services customized to the specific needs of empirically identified student subgroups (Olivera-Aguilar, Markle, & Robbins, 2014; Pastor, Barron, Miller, & Davis, 2007). Further studies making use of multilevel modeling would also provide a more nuanced picture than is currently available of the extent to which both school- and student-level noncognitive characteristics (in combination with other contextual factors) contribute to the prediction of student academic success in high school and beyond.
Recent Comments