Assessment Issues: Research and Practice

Loren McGrail
SABES, Boston, MA

In recent years the subject of testing and assessment in adult literacy and education has focus of growing concern, both nationally and locally. Decisions about the nature and purpose of assessment are critically important to literacy policy and practice at all levels. Adult learners, teachers, administrators, and funders all agree that what programs choose to assess and the methods they employ affect the quality of teaching and learning. Assessment in adult literacy is a central issue with high stakes: "The authority vested in these tests can determine the way programs are de- veloped, what is taught, and the climate of teaching and learning. It shapes legislation and funding policies of public and private agencies. It is tied to welfare eligibility for young parents. It drives government job training programs. It can deny entry into military, or crucial access to a diploma or ajob." (Business Councillor Effective Literacy Newsletter No. 22, January 1990)

The Mandate

In April of 1988 Congress passed legislation which called for the use of standardized tests to evaluate all adult basic education (ABE) and English as and Language (FSL) programs funded under the Adult Education Act. These Adult Education Amendments (public law 100-97) and the implementing regulations of the u.s. Department of Education (August 1989) require that the results of standardized tests be used as one indicator of program effectiveness and that at least a third of local programs which apply for federal ABE funds (administered in Massachusetts by the DOE) should be evaluated through the use of standardized tests. (see BCEL Newsletter, No.22, January 1990)

While the federal mandate for standardized assessment in ABE and ESL is objectionable to many, it is also a sign that adult education has come of age, for the need for accountability often accompanies the investment of significant resources.

The mandate, however, also highlights the growing concern of literacy practitioners, theorists, and test designers about what standardized tests actually test and for what purposes.

As standardized tests have come into sweeping use, so have complaints about them and their validity. For many the most important question is, do the test results tell us anything of real value either about learner progress or program effectiveness?

The teachers evaluated me to find out how much I knew and compared me with the rest of the class. It made me feel small.
--Carmen Ferino

The Debate

The use of standardized testing most likely de- rives in part from their relative ease of adlninistration and what is viewed by some as their reliable and objective, quantitative data on the achievement, abilities, and skills of students-data that is free from individual judgement. Because the tests and the conditions are (theoretically) constant, they are thought to be useful indicators of individual progress over time (pre- and post- testing). Thus, by extension, they are thought to be useful indicators of program effectiveness.

According to Susan Lytle of the University of Pennsylvania, the chief spokesperson and advocate for alternative forms of assessments, these charges of" quantifiable and reliable data " for program evaluation are questionable at best. For Lytle, ("From the Inside Out: Reinventing Assessment" in Focus on Basics, Volume 2, Number 1, Fall 1988), the equation of learner progress assessment--defined as increased reading levels--with program effectiveness is problematic for two reasons: 1) It ig- nores other legitimate criteria for evaluating a literacy program like the quality of the curriculum, teaching, or its connection to significant social issues relevant to students lives and interests; and, 2) it fails to recognize that increases in reading scores have little to do with the way adults live and use literacy in the real world.

So, despite the relative ease of administrating tests, few adult educators are satisfied with the quality of information revealed and many more are deeply dissatisfied with the effects such testing has had on teaching and learning. In a specia11989 issue on alternative assessment from the Literacy Assistance Center in New York, this grievance was articulated clearly: "There seems to be considerable agreement among adult literacy practitioners that the TABE (whether it be the 1976 or 1987 edition )is not only an inappropriate instrument for individual assessment, but it does not inform the teaching and learning process, and it in fact may act to discourage students as they reinvolve themselves in the educational process. "

Other practitioners and theorists agree with Lytle and Johnston (1986) that by assessing only decontextualized subskills like word recognition, paragraph comprehension, and vocabulary, we fail to measure the ways in which adult learners already can and do use literacy in their daily lives. Still others argue that because the tests focus on product rather than process, they tell us very little about the affective or metacognitive factors in literacy acquisition.

Even Dr. Thomas Sticht in his report to the U.S. Department of Education, "Testing and Assessment in Adult Basic Education and English as a Second Language Programs," advocates that, "It may be desirable to separate testing for program accountability from testing for instructional decision making. " His advice should provide some guidance to literacy and ABE programs caught in the eternal conflict of how to be accountable to the funders and the public in a quantifiable way and how to create ways for teachers to use assessment tools and processes that are an integral part of teaching and learning and that can inform curricula.

The call for alternative forms of assessment is coming from all directions, including many mainstream and professional organizations and educators. Elsa Auerbach of the University of Massachusetts in her book, Making Meaning, Making Change, quotes the Board of Directors of the International Reading Association's its 1988 statement: "Reading assessment must reflect recent advances in the understanding of the reading process. The IRA is concerned that instructional decisions are too often made from assessments which define reading as a sequence of discrete skills that students must master to become readers. Such assessments foster inappropriate instruction. "

Alternative Assessment Approaches/Tools

How is student assessment conducted?

-oral interview
-tests (what kind? standardized/program-developed?)

-performance standards (measure of competencies?)

-collections of student work (writing samples?)

-observation of classroom interactions?

-self -report
-on-going documentation (teacher reports?)

What counts as progress?

-reading level gains?
-test scores?
-affective gains in self-confidence, etc..
-ability to use language/literacy outside the class?
-ability to make personal, family, school, classroom and/
or community changes? (Auerbach, 1990)

A few key principles guide what constitutes al- ternative assessment. The first is that it must be "program-based" and "learner-centered". The second is that it should help the learner achieve his or her goals. In other words, what is assessed must reflect what the learner wishes or needs to accomplish . Third, the process must build upon the learner's strengths, not deficits; it must be a process that is done with the learner and not to the learner. Fourth, assessment should be part of the learning experience--"an on-going collaboration between the teacher, the learner, and the text, to review, and refocus what should take place in light of progress being made. " Fifth, it should not depend upon a single procedure but a variety of procedures. And most important it should provide feedback that will make programs more effective (BCEL Newsletter , January, 1990).

By definition, learner-centered assessment differs from program to program, teacher to teacher, and l~er to l~er .However, a tool kit of assessment procedures that communicates respect for adults, for what they bring with them to the learning experience and for why they have come might include the following:

In-take or start-up activities to get a sense of student strenghts, interests, goals and needs. These initial interviews can be helpful in planning lessons as well as providing data about what students can already do with language and literacy and how they think about it. They can be done atthe beginning of a cycle (pre-instruction placement tool) or they can be integrated into instruction. They include:

-informal interviews

-reading samples

-writing samples

-goal-setting activities.

Along the way in-class activities that docu- ment learning as it takes place. These activities are integrated into instruction on a regular basis. They include:

-journals kept by students

-journals kept by teachers

-portfolios or writing folders

-anecdotes.

Activities at the end of the cycle that reflect learning, teaching, curriculum and program design. They often involve both teachers and learners to- gether and include:

-student self -evaluation charts or check lists

-peer interviews

-student-teacher conferences

-class or program evaluations. (Hemmendinger, 1988)

The Future: Directions and Implications

Along with learner-centered or participatory approaches to adult education, the alternative assess- ment movement is based on recent cross-cultural and ethnographic research. This research provides support for the notion of "many literacies" or "multiple literacies " --diverse literacy practices where by learners connect literacy to their everyday life and find ways to determine for themselves the conditions under which they will use reading and writing. Brian Street, Shirley Brice Heath and Scribner and Cole, by focusing on the social, political, and economic nature of these practices, tell us how standarized tests don't reflect universal literacy but rather, " attribute value to particular literacy conventions. "

The measurement (standardized tests) of these conventions are what funders and legislators use to determine program accountability and effective- ness. Lytle believes this is due in part to the fact that funders lack good information about the qualitative effects of programs on learners' lives. She advocates two basic strategies to remedy this situation. The first is to invite wider participation in the conver- sation about alternative assessment. The second is to conduct program-based practictioner research simultaneously across the country to strengthen these new conceptual frameworks and to exchange and critique innovative practices. Auerbach concurs with Lytle that the cornerstone of qualitative research is documenting what happens, when it happens, or the teacher's ability to know the students, and to notice and record their development in a variety of areas.

References

Auerbach, E., Making Meaning, Making Change (Boston: University of Massachusetts, 1990)

Belifiore, M.E. and B. Bumaby, " Assessment and Evaluation" , Teaching English in the Workplace (Alberta,Ontario: Oise Press 1984).

Martin, Rachel, Editor, Focus on Basics , Vol. 2, No.1 (Boston: World Education, 1988).

Gillespie, M., Many Literacies: Modulesfor Training Adult Beginning Readers and Tutors (Amherst: University of Massachusetts, CIE Publications, 1990).

Hemmendinger, A. , Tool Kit: Self Evaluation Exercises for Students and Literacy Workers (Ontario, Canada: East End Literacy, 1988).

"Initial Assessment of Reading Skills" and "Initial Interview and Assessment in ESOL " (ALBSU Newsletter, Summer 1989).

Johnston. P. ." Assessment in Reading," Reading Research Handbook (Longman, 1984).

Kucer, S. , Using Informal Evaluation to Promote Change in the Literacy Curriculum (Los Angeles: University of Southern California, 1990) .

Lytle, S. , "From the Inside Out: Reinventing Assessment" , Focus on Basics. Vol. 2. No.1 (Boston: World Education, 1988).

Lytle, S. , Tharmor, T, and Penner, F., Literacy Theory in Practice: Assessing Readings and Writings of Low-Literate Adults (Philadelphia: University of Pennsylvania, 1986).

Padak, N.D., Davidson, I. L., and Padak, G. M.; Exploring Reading with Adult Beginning Readers, " Journal of Reading, 34(1), Sept. 1990.

"Standardized Tests: Their Use and Misuse," BCEL Newsletter (22); January 1990.

Sticht, T. , Testing and Assessment in Adult Basic Education and English as a Second Language Programs.

Tirane, P. L. , "Teaching Under the Tyranny of Testing: Can We Keep Our Balance?" (unpublished).

Originally published in Adventures in Assessment, Volume 1 (November 1991),
SABES/World Education, Boston, MA, Copyright 2003.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.