Authentic Goal Setting with ABE Learners: Accountability for Programs or Process for Learning?

Sally Gabb
SABES Regional Coordinator for Southeastern Massachusetts

Adults who enroll in adult basic education classes know why they have come back to school. Each states her/his purpose clearly: . . . "to learn English, to get a GED, to make a better life for me and my family." These purposes have propelled adult learners, despite the stresses of family, community and work, to dedicate time to gaining needed basic skills. But for most, this clarity of purpose does not easily translate into crafted 'goals,' outlined as a metered time line with identified benchmarks, and 'documentation' of achievement towards a stated goal or goals.

During its ten-year history, Adventures in Assessment has provided a rich record of creative and dynamic efforts by Massachusetts ABE practitioners to devise classroom and program support systems for learner goal-setting support that will enable learners to redefine their purposes, and to understand why and how to set goals. In 1994, Dulaney Alexander wrote:

"Language learning is a lengthy process for most of us. None of our students will master English before leaving our program. Some students will leave to work, some because they must move too far from school, some because their families can't afford their time for school, and some because they decided the "perfect" English is not a realistic short term goal. The skills that enable the student to treat learning English as a personal project, whether in school or out, begin with assessment and goal setting. These are probably as valuable as anything else we teach."

Alexander's eloquent description of the importance of classroom self-assessment and goal setting could be applied to any ABE, GED or ESOL classroom, to complement and build on the purposes which learners can easily identify. His description embraces what many consider truly learner-centered goal setting, with timelines and outcomes that are as mutable as life experience. A test or a time limit will not easily capture the value of such a process for the learner.

Top

Goals and Outcomes: What Are we Counting?

In recent years, the Massachusetts Department of Education (DOE) has integrated learner goal setting and outcome assessment as part of the data requirements for the performance accountability system. The system requires funded programs to elicit primary and secondary goals for each learner, with performance to be measured during the one-year grant period for 'primary' goals. In part, this element in the Massachusetts performance reporting system was in response to the National Reporting System, which designates and defines 'core outcomes'. While 'educational gain' is the 'Core Outcome Measure' #1, others include: 2) entering employment, 3) retaining employment, 4) acquiring a secondary diploma and 5) entering post secondary education or training. The NRS requires learner goal setting as a measure of program performance as stated in the official NRS Implementation Guidelines. "The NRS will assess program performance by comparing students' outcomes to their stated goals."

According to seasoned practitioners in the Massachusetts system, goal setting is fully supported by the field in concept. Practitioners across the state have encouraged recognition of learner-generated goals, and outcome measures other than pre-post test statistics for purely academic gain. But the purposes for goal setting as envisioned by practitioners, and the process through which learners should be supported in setting goals, differs radically from the procedures outlined in the NRS Measures and Methods. The NRS has as its stated goal 'to establish an accountability system for the federally-funded adult basic education programs.'

SABES has been given the responsibility for supporting program efforts to meet dual goals: 1) to encourage classroom-based learner-centered formative goal setting and goal attainment evaluation with learners, and 2) to usher in collection of goal setting and goal attainment data through the SMARTT system. A group of practitioners in the Southeast region of Massachusetts recently responded to questions about the challenges in encouraging learners to set goals during the intake process for their participation in the ABE, ESOL, or GED class:

"No matter how limited their reading skills are, or how limited their English is, the first response is always "to learn English", or "to get my GED". Learning how to set goals is part of instruction."

"You have to get to know a learner, gain trust, establish dialogue before that learner will be able to articulate what he or she hopes to accomplish, what the 'goal' is. In the beginning, most learners don't even understand the question if you ask, 'what are your goals in the class for this year."

According to Elsa Auerbach and others, assessment of Adult Basic Education student outcomes is 'authentic' only when based on learner identified purposes for attending ABE classes. 'Authentic assessment' describes goal setting and self evaluation as a process, to be carried out over time in dialogue with the practitioner, measuring skills and content knowledge in the context of real life tasks.

In the still-evolving SMARTT data collection system, the Adult and Community Learning Services division of DOE has created a lengthy list for programs to use as a pre-post measure, with a range of possible 'goals', but the requirement that goal attainment be measured and verified. The list was drawn from both the NRS designations and a survey of the field, from work with programs and their experience with learner-centered goal setting (see Adventures in Assessment, Volume 4, April 1993). But programs and practitioners suggest that the very nature of such a pre-post system seems to contradict reflective ongoing self-assessment. The procedures defined by Massachusetts DOE are directly counter to a process that enables the learner to develop goals and learning objectives over time, and to change those goals and objectives as he/she grows as a learner.

Classroom Self Evaluation and SMARTT System Goal Setting: A Not So Complementary Relationship

While the user's manual for the SMARTT system also encourages programs to be truly 'learner centered' and 'open ended' in the goal setting process, most report difficulty in carrying out the activity as suggested in the manual. For some classes, particularly lower level monolingual (English only) ESOL classes, such activities are limited severely by lack of vocabulary and ability to express ideas in English. Even in ABE classes, most students are hard pressed in the initial interview to say more than 'to read better', or 'to get my GED in the long run'. The first is measurable by reading tests, but such tests may not reflect the kind of advances lower level learners make in the beginning.

The outcomes of these behavior changes may only be evident several years after the initial changes. In addition, the learner's primary goal for attending classes may be difficult for ABE programs and practitioners to measure, such as 'improving my communication on the job', 'being able to help my children more with homework', or even 'feeling better about myself as a human being because I'm coming back to school.'

Top

An Ambitious Example: Evolution and Devolution at One Program

In 1993, Amesbury Even Start described a unique and complex goal setting and evaluation process for its Even Start family literacy program (see Adventures in Assessment, Volume 4, April 1993). This system included in depth intervewing of each learner, and regular self-assessment on the part of the parent learners. The content was often directed to a combination of self-esteem issues, practical applications of literacy, and application to parent/child interaction.

The Goal Sheet uncovers a variety of assessment information, but at the same time acts as an extremely inviting, unintimidating way to assess initial capabilities, attributes and interests. The Goal Sheet is a visual representation of an eight-week cycle. It is broken up into categories representing program requirements at Even Start, including adult education, parenting skills, child development and parent child together time. These categories are then subdivided into short term and long term goals.

The goal setting process at Amesbury Even Start, according to Susan Martin, Sandra Hall and Jeanette Bahre, provided initial information to case managers, teachers and learners "without the use of extensive formalized testing or frequently inaccurate initial self evaluations."

The ongoing interaction around strengths, challenges and goals enabled the case manager, instructor and learner to work towards understanding how eight-week program options could lead to long term goals. The Amesbury authors stressed that effective goal setting could only occur when a process was in place that allowed learner and staff to revisit such affective areas as dreams and wishes, and do an analysis of "felt" and "real" strengths and challenges. A key aspect of the process was the way that standardized testing (the CASAS) was integrated with the extensive interview process as a way to enable the learner to choose program components, e.g., her short-term goals.

As noted by the authors . . . "the system is used in self assessment as an adult learner builds [short and long term goals) based on the information he/she reflects upon, and as the work is continuously defined and redefined for the future."

In a phone interview, this writer learned that the Amesbury program no longer has a family literacy grant, and the well-developed model is no longer in use at the center. Nevertheless, the model stands as an impressive attempt not only to develop a systematic process for enabling learners to develop appropriate and attainable goals over time, but also to learn some strategies for setting appropriate goals.

Top

In-Class Assessment as a Mediated Teacher/Learner Dialogue: A Practitioner Targets Barriers to Persistence

In 1994, Loretta Pardi and Estelle Williams of the Harborside Community Center worked together to develop effective in-class self-assessment/teacher assessment tools. Loretta designed the 'Record of Participation' that both she and each learner completed on a weekly basis. This instrument (see Adventures in Assessment, Volume 7, December 1994) allowed the learner to rate him/herself on a wide variety of areas, including attendance, willingness to try new work, independent and group work, class participation, and accomplishment of job skills. In addition, it provided space for the learner to identify specific academic skills mastered in math, writing and social studies.

"In the first weeks of using the record with my class, I found that most of the students tended to give themselves lower ratings than I gave them. This not entirely unexpected result gave rise to a useful class discussion about low self-esteem. As weeks progressed, however, and the class became accustomed to the weekly routine of reflection and assessment, I found that we usually concurred."

Loretta reported that she set up conferences with students when noticeable discrepancies occurred between a student's self assessment and her own. This surfaced issues besides actual performance, including fear and attachment to the class.

In a recent follow-up phone interview with the Harborside Community Center, Loretta confirmed that she still incorporates an ongoing goal setting process into her practice. Loretta explained that she currently focuses on enabling learners to establish their own 'comfort zones' as the most important component of goal setting.

"I work with many learners who are getting ready to transition from ESOL to the pre-GED program. I spend a lot of time talking about what they might expect from the new program. I ask them to consider issues such as work schedule and childcare, which could interfere with their ability to learn. I ask them to consider their learning styles and strategies, and to consider what is realistic for them.

For example, when students tell me they have a hard time remembering what they learned from one session to the next, I ask them to look at how to set short term achievable goals. I stress that setting new goals is a successful accomplishment when the original goal isn't accomplished. I help students learn how to break goals into manageable pieces."

Loretta noted that the whole idea of enabling the learner to find his/her 'comfort zone', and working to achieve goals outside the class that will set the stage for participating in class and reaching academic goals are not, and probably can not, be a part of the SMARTT data. She reported that in her program, as in many others, the initial goal setting for reporting in the SMARTT data is carried out by the counselor, and that she does her own goal setting work in class. This separation between SMARTT system goal setting and classroom goal setting seems to be a common occurrence. But, as Loretta described, without the developmental process for goal setting, the recorded goals will not reflect a valid student purpose, or a true picture of learner goals attained.

Goal Setting: In Conflict with Language and Culture?

Katy Hartnett, ESOL specialist for the Adult Literacy Resource Institute in Boston, says goal setting as a pre-post summative system is a limiting picture of the ESOL student audience and the many goals they have for seeking education. She explained:

"This is especially true for ESOL learners, who can't express in English either short or long-term goals. Most ESOL programs settle for 'improve English' because it's the only goal most new learners can express, and it fits the SMARTT guidelines. And while it IS the primary goal for most ESOL learners, it doesn't capture the other attending goals. The cultural journey for ESOL learners is complex and multi-dimensional."

In his 1998 action research project, practitioner Glenn Cotton described several 'authentic' ways to identify learner 'purposes' for improving English. The teacher may show the learner with very limited English a group of photographs or drawings depicting people in a variety of social situations. Learners are asked to choose those that are most important to them for learning English. Or learners may be asked to draw their own pictures of situations in which they feel it is most important to be able to use English.

Glenn noted: . . ."For a goal setting activity to be truly useful, especially with beginning level ESOL learners, it seems to me that it needs to be integrated into a learning activity which assists learners to understand the questions being asked of them, and which guides them to reflect on and discover what their authentic needs/goals are."

Both Katy and Glenn observed that always recording the 'improve English' goal doesn't provide valuable feedback for either the learner or the program.

Top

Rethinking Goal Setting and Outcome Assessment: What Progress since 1996?

In the introduction to Volume 9 of Adventures in Assessment, Alison Simmons wrote:

"Teachers know how to measure progress, but get stuck trying to find a tool or reporting mechanism to put their data in that would satisfy all the different audiences. The question goes beyond, 'How will I measure the progress of my students so it is meaningful to them, informs instruction and the curriculum, and satisfied the funder'.

"Rather the questions should be, 'How do I report the progress I have seen to satisfy different audiences? How do I report the progress my students have seen [in themselves]? What information is useful to me in my program, and what information do I need to satisfy other audiences?"

Alison referred to the many forms of alternative assessment she had seen developed over the years by Massachusetts practitioners, including checklists, portfolios, and benchmarks, as well as the standardized tools. She also forecast the likely requirement for more systematic and standardized assessment to satisfy both public and private funding sources. For this reason, she concludes:

". . . we need to validate and encourage teachers to continue to develop systems that inform instruction and show students where they are making gains, and how far they have come in achieving their goals."

Through my limited research, I have learned that Massachusetts practitioners do continue to develop and utilize creative tools and methods for formative assessment, and classroom processes for identifying, evaluating and revising/expanding learning goals. As noted, many feel that these in-class and/or program-based formative processes are in conflict with the state mandate for summative goal assessment.

Top

The Position at the Top: Process Can Complement Outcome Data Collection

In a recent exchange on the National Literacy Advocacy listserv, Massachusetts Director of Adult Basic Education Robert Bickerton wrote the following:

"For several years in Massachusetts, we have made the recording of student articulated goals for their participation in ABE the centerpiece of what we mean by 'performance accountability' - predating the Workforce Investment Act [and the NRS]. The policy is to record whatever the student says is the reason or reasons why she/he has decided to enroll in the program and to update these goals as the students' skills, abilities, dreams, aspirations and perspectives evolve and, possibly, solidify.

"Not only have programs been under NO pressure from the state to curtail how ambitious and multiyear many of these goals are, but our office has often been in the position of encouraging and counseling programs to not curtail what students articulate at their end. High expectations are embraced by everyone - in theory. But are hard for some to accept in practice. With WIA came the expectation that the student's 'primary goal' would be achievable within a single program year.

"I and others in our state strongly objected to this requirement arguing that it would pervert having student-articulated goals . . . and could foster 'creaming.' For a while it appeared as if this would become a sticking point in our negotiations with the USDOE and we were prepared for a battle. Instead we found a way to tweak the existing policy. WE still honor the goals articulated by each student regardless of how long it may take to achieve them. We now also ask programs to discuss with each student what goals (or benchmarks) he or she would like to achieve in the program year."

ACLS Director Bickerton wrote, "Massachusetts DOE, including its professional development arm, the SABES system, needs to work with the field to develop benchmarks and standards for learning gains and goal achievement that are responsive to the needs and circumstances of the very diverse students who come to us for services."

There are rich resources and knowledge of ABE learner purposes and goals within the community of practitioners and learners throughout the ABE system in Massachusetts. Creative practitioners resist the pressure to make 'goal setting' into an exercise isolated from the teaching/learning process. Dynamic practitioners such as those quoted in this article want to demonstrate accountability for their work with learners. The challenge to ABE lies in promoting and sharing the many authentic models and practices for identifying learner purposes and goals in their own terms, and at the same time providing outcome data for learner achievement that will confirm the 'performance accountability' that is alive and well in ABE classrooms across the state.

Originally published in Adventures in Assessment, Volume 13 (Spring 2001),
SABES/World Education, Boston, MA, Copyright 2001.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.