The Alternative Assessment Aspect of the G.O.A.L.S. Project

The Process of Component #3

Lindy Whiton
Western Mass. SABES; former Coordinator, Component #3

This issue of Adventures in Assessment is dedicated to discussing the process a group of teachers involved in alternative assessment went through over the past year. The group that put together the articles contained here were originally drawn together to complete part of the Greater Opportunities in Adult Learners Success (G.O.A.L.S) Project.

The original intention of the G.O.A.L.S. Project, which was developed by Sandy Brawders for the Bureau of Adult Education, was to investigate and develop an accountability system which would be a true reflection of the field, one that intrinsically asks teachers, directors and learners their opinion on “What are ya doin’? What would be helpful? When are you having problems? How do you make changes? Did you discover anything new this year that you would like to share?”

The G.O.A.L.S. Team interviewed the staff of more than 70 programs. We talked to both teachers and directors. We interviewed 250 learners, both ABE and ESL (see Adventures in Assessment, Vol. 1). We asked the field to be a part of a discovery process. Although this project has come from the Bureau of Adult Education and not the field, it has adopted several different techniques to get at the language the field uses, to get at the needs of the field, to find a true reflection and voice of the field.

Although the original intention of gathering this group together was not for staff development nor as a teacher research project, it has been both. What resulted was both good staff development and a form of teacher research.

The Alternative Assessment Aspect of G.O.A.L.S.
The objectives of the G.O.A.L.S. project were as follows:
• Have an accountability system that reflects the entire field;
• Provide an opportunity for all voices to be heard;
• Give individuals an opportunity to impact on state policy making;
• Develop a learner-centered accountability system1;
• Look for an organic language of evaluation that is accessible to the learner, not ‘secret knowledge’ only decipherable to bureaucrats;
• Be able to pay for what it asks;
• Create an appropriate framework, rather than rely on already-existing frameworks that may be inadequate;
This project assumes a staff development network and system in place where there is support for change (training) in the field (SABES).

Component #3 is the section of the G.O.A.L.S. project that looks at alternative means of assessment. The Depart- ment of Education chose to run this component as a teacher-based research project to develop a state run accountability system, one that would foster a dialogue with adult learners, teachers, and other staff, legislature, the federal government and the public at large.

The questions that were important to us throughout the entire project were:
• What must the Bureau know in order to fund successful programs?
• How does the Bureau open lines between programs and the Bureau so that learners benefit from everyone’s expertise?
• What do the programs need from the Bureau to best do their jobs?
• Can we develop a system of accountability that truly represents the variety and richness of programs, thus reflecting the variety and richness of learners?
• In Component #3, can we develop a document that similarly represents our variations and that allows for one piece of the puzzle to be placed, so that when we step back and look at all the components as a whole, alternative assessment is clear and helpful?

The majority of the participating programs already had in place curricula that reflected the principles of alternative assessment, and were attempting to develop tools to inform teachers and learners as to the effect of these curricula. (See Adventures in Assessment, Volume #4 - Whiton and McGrail.) This group believed learners’ goals were not only important but a driving force in curriculum, thus, these assessment practice are based on the individual goals of the learner as well as the goals of the individual programs. Whether we are teaching ABE, GED, or ESL literacy practices have to directly relate not only to the goals of learners, but to their immediate lives as well. Our beliefs are backed by the works of Lytle (1989, 1990), Auerbach (1990), Spruck Wrigley (1992) and Fingeret (1989). To discuss learners’ growth by talking about standardized increases in reading levels “ignores other legitimate criteria for evaluating a literacy program, like the quality of the curriculum, teaching, or its connection to significant social issues relevant to students’ lives and interests; and, it fails to recognize that increases in reading scores have little to do with the way adults live and use literacy in the real world” (McGrail 1991). And with ESL literacy students, standardized assessments “fail to distinguish between language, literacy, and culture; that is, they don’t tell us whether the learner has trouble with an item because he or she (a) is unfamiliar with the cultural notion underlying the task (b) lacks the requisite knowledge of English vocabulary or sentence structure or (c) does not have enough experience with reading and writing to complete the task” (Spruck Wrigley, 1992).

Reading assessment must reflect recent advances in the understanding of reading process. The International Reading Association is concerned that instructional decisions are too often made from assessments which define reading as a sequence of discrete skills that students must master to become readers. Such assessments foster inappropriate instruction.2

Thus, based on the educational theories that drive the program’s curriculum, each of the programs involved in this project developed assessment/ evaluation tools that consider who their learners and teachers are, the goals and interests of their learners, and how they used literacy in their daily lives.
The development of a community of learners is an extremely important factor in the success of learners’ reaching their goals and our curricula and our assessments/evaluation tools need to reflect this, too. These practices inspire and cultivate community involvement and ownership of educational decisions in the classroom.

The articles that follow reflect the hard work of these programs. They combined what they knew about alternative assessment with the needs of the Bureau, and the needs of other programs. These articles, or notebooks, are the product of the process that Component #3 went through while wrestling with the issues of accountability, assessment, and evaluation. They don’t say, “Here, just duplicate and standardize these tools.” Rather, they relate the process they went through in developing the tools to help other practitioners develop their own tools based on their own curricula, learners, and classrooms.

The Process of Assessing Assessment
The first meeting was in April, 1992. We introduced ourselves, then immediately sat down to the task of defining assessment and evaluation. First, we attempted to get the entire group to settle on one definition of assessment and one of evaluation. We could not. We agreed to disagree. (Since then, we have reached agreement.)

At the end of the first meeting, we decided the goal of the next meeting would be to share the tools the group’s members had already developed and to describe our programs to each other. This was done in early June, 1992. It was fascinating to see the variety of tools. Even more impressive were the variations on the same tools.4 Teachers had adapted models to make tools that were appropriate for their sites and needs. By the end of our sharing, we knew we needed some format to bring us all together. I was then asked to write a philosophical statement and some ideas for direction, distribute this paper to everyone, and then call us back together. At this point I developed the chart (Figure 1) adapted from Rena Sofer’s work (Soifer, Rena, Complete Theory-to-Practice Handbook). It lists all the different types of tools that had been adopted by the programs and fits them into a framework that addresses accountability issues. The paper also divides tools into the categories “Starting Up,” “On-Going,” and “Looking Back” (Auerbach, McGrail, Adventures in Assessment, Volume 1).

At our next meeting, we decided to split up into groups that represented these three categories. These groups would develop a tool kit including examples of tools in each category and describe when, why, how, and who might use these tools. This discussion raised the question of what the tool kit had to do with accountability. Could the tool kit gather the type of information that DOE was requesting? It was felt that for the majority of these programs, reading levels, acquiring a job, or entering college were not the only outcomes nor were they the only measures of success. Actually, in terms of reading levels, we thought just the opposite, that to talk about reading levels was irrelevant to literacy programs. The types of measures we felt were important were not addressed by either state or federal reports.

The teachers wanted to talk about changes in attitudes or smiles, involvement with school systems, or finding a home. Where was the mechanism to talk about community involvement, or the changes in empowerment, or the willingness to believe you can make a change as a measurement of success for programs? These measures do not fit neatly onto a linear chart that denotes increase in something. For most of these programs, linear charts contained the information they believed the Commonwealth wanted and that contradicted what they believed was their mission. At this meeting the “Looking Back Group” decided to tackle the task of discussing the larger community changes within their category and create a fourth group. (This discussion is reflected in the articles.)

For the next several months, people met in their smaller groups and attempted to visit each other’s programs. In early October, we regrouped, gave updates and began the topic of Quality Indicators, sharing a copy of the Federal indicators and a copy of the indicators developed at ACBE by Sondra Stein and Susan Rosenblum. We decided to read all the articles and return in November to put together recommendations from the Massachusetts quality indicators.

At the November meeting, we struggled with what we saw as the real issue, namely that the concept of accountability needed to be changed from a hierarchical linear concept to a more cyclical concept. One suggestion was that programs needed to be better ap-prised of the needs of the State DOE and how they were accountable to the Federal DOE so they understood the Bureau’s constraints. Although the group felt it was difficult to improve the ACBE job, the teachers did establish a list of recommendations and also generated proposals for an accountability system that would reflect those quality indicators.

After our November meeting, there was a long break until February. We decided to finish the “Notebooks” and to create documentation about the notebooks for others in the field, and to document some changes that individuals have gone through. It was at this point of the process that I went on to a new job, leaving the group to write up their experiences and their feelings.

The Fruits of Our Labor
This work was used to develop the indicators for the Commonwealth of Massachusetts. These teachers’ articles and the Alternative Assessment Notebooks reveal all the earmarks of quality staff development. The process:
• Networked teachers together across disciplines, programs, and the state;
• Provided a chance to share materials, methodology and ideas for further programming;
• Gave an opportunity to discuss issues at a time in our profession when we rarely have time to stop and reflect upon any form of assessment;
• Enabled them to read other people’s work (including but not limited to Lytle, Fingeret, and Stein), and
• Allowed them to begin to document their ideas.

Much attention has been given to the use of teacher inquiry groups as good staff development. Lytle and Cochran-Smith argue it is a “way to generate both ‘local knowledge’ and ‘public knowledge’ about teaching; that is, knowledge developed and used by teachers for themselves and their immediate communities, as well as knowledge useful to a larger community” (1992). So, even though the original intention may not have been to create a successful staff development experience, it was a natural by-product of the experience.

Because of the different perspectives of the teachers, it was similar to a multi-level classroom. However, all of the teachers had years of experience from which to draw. Sometimes it was difficult to balance the task of looking at the issue of accountability and what the Bureau had hired me to do and to engage in this staff development activity.

As I have said previously, I found this experience to be very similar to the process of inquiry and to the process of staff development. The clearest proof is that the practitioners who came into this component in April of 1992 claiming they had no tools in place, but were really interested in learning, are the ones with the most energy to see this group continue. They are the ones committing themselves for another year. Those practitioners who had already given the topic a lot of thought and creation seem to be interested in moving to another topic. It is the sign of good staff development when there is excitement remaining in those who still feel they have something to gain from the experience.

As for an accountability system, I believe the point of all this work is to show that accountability cannot rest on the learner’s achievements. Comparing learners does not make a program accountable. The idea is to work with the program, inclusive of the learner, to create quality programs. Those indicators of quality programs that are agreed upon should be measured by program development and not by looking at standardized reading scores. Component #3 has helped to better define the roles of assessment evaluation and accountability, and it is important for us to stop using those words synonymous and interchangeable. We must be accountable to our own knowledge and not give in for the sake of convenience.

Footnotes
1 Public policy is usually created by administrative and governmental offices and then individuals are asked to be responsible and implement that policy. This project is attempting to get around making that mistake and systematically and intentionally asked for individual and group input.

2 International Reading Association. (1988). Resolution on reading assessment. Newark, DE: Author.

3 A method of training which causes participants to first define for themselves a concept and then, by joining with more and more people, have to agree upon a definition with a group.

4 Examples of the types of tools can be found in Adventures in Assessment, Volume 3, article by Eileen Barry.

Figure 1

Top of page

This article was published in Adventures in Assessment, Volume 5 (October 1993), SABES/World Education, Boston, MA, Copyright 1993.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.