Read/Write/Now Adult Learning Center Assessment Adventures

Janet Kelly
Read/Write/Now Springfield, MA

The Program Read/Write/Now is a small, library-sponsored program for adult beginning readers and writers in the north end of Springfield, Massachusetts. It was started in 1987 as a pilot project with a grant from the Massachusetts Board of Library Commissioners to provide literacy services and to test a packaged computer literacy curriculum. Since then, it has become a whole language-based adult learning center which uses its six computers for producing and publishing learners' writing.

Curriculum and program development are continual processes that involve the contributions and collaboration of many talented teachers and commit- ted learners. The materials that are included here are a result of this collaboration between teachers and learners. There is a much bigger file of things we no longer use than of things we use. If it does not work, we ask ourselves if we need it and if there is a good answer, we revise it. If it still does not work, we put it in the archives. I consider these assessment mate- rials works in progress in the sense that, although they have been tried and revised, there is always room for improvement.

We have tried to develop assessment tools that work for learners and teachers as well as funding sources. What follows are some samples of assessment tools and descriptions of how we have used them for initial and early assessment. Materials for ongoing assessment and "then & now" assessment are referred to but not included with this article.1

Assessment in a Whole Language Classroom

Assessment in a whole language-based classroom is ongoing. It is an integml part of the program design and helps drive the process of developing curriculum. One of the goals of a whole language program is to empower learners by helping them to become more self-directed, to identify and work towards their own goals for learning, literacy, and life. A logical outgrowth of that goal is toincorporate learners' self -evaluation into the overall assessment activities of a program. Both learners and teachers need to know why they do what they do in a classroom so that they can have a sense of progress, as well as make decisions about future directions. One great motivator, especially in something as challenging as adult basic education, is for learners to clearly see and feel their own growth. Standardized tests do not measure growth in self -esteem, life skills, empowerment, community involvement, and self -confidence, nor do they truly measure the literacy skills that adult learners make in many programs.

Evaluation in a whole language-based program is a learning activity as well as a process of adding to information about learners' progress, teachers' effectiveness, and ideas for curriculum development. Whole language assessment includes observation, interviews, learners' self-evaluations, and formal and informal analysis. Examples of each of these aspects of evaluation are included in this article.

Alternative assessment tools which are learner-centered and holistic rather than standardized and quantitative are being developed and used in many programs. Some of these programs describe themselves as whole language, some have a Freirean approach, and others have an eclectic approach to literacy education. All are concerned with evaluation for the purposes of increasing learners' and teachers' sense of empowerment as participants in the process ofleaming and explicitly understanding their own growth and progress towards meeting their goals. This kind of assessment is not concerned with measuring changes in behavior, but may use changes in behavior as evidence of increasing understanding, knowledge, and competence.

The question we try to keep asking ourselves about assessment activities is WHY? There should always be an answer to that question that relates to helping learners and teachers become empowered and active participants in the process of their own education and work. If there is no good answer to the WHY question, or the answer does not relate to the development of the program, the benefit of the learner and the teacher as members of a learning community, it is time to scrap the particular assessment activity.

Portfolio Assessment Read/Write/Now uses a portfolio approach to maintain a comprehensive record of ongoing assessment. A learner's assessment portfolio contains: initial screening and placement interview, including a first writing sample, middle and later pieces from a learner's writing folder, dialogue journals, learning contracts, goals checklist, writing conference records, writing progress checklists, reading miscue analysis, book lists, reading conference records, student learning logs and other self-assessment materials, and publications of the learner's writing. Anecdotes and observations related to learners' progress are often kept in a running teacher's log and are included as part of the portfolio.

The learner's portfolio is added to and shared with the learner during regular conferences throughout the cycle and at other times when a learner needs to see concrete evidence of his/her progress. The portfolio can also be used to document progress to funders. Although this approach tends to be more comprehensive than many funders have time or interest in seeing, it helps to organize evaluation materials in a way that makes writing narrative summaries for funding reports easier.

(See hardcopy for more sample materials.)

INTITIAL ASSESSMENT

Screening Process

Initial Contact. There is generally a 10-20 minute telephone contact, before potential new students come in for a screening interview, during which learners have a chance to tell their story and describe their needs while a staff person listens and offers encouragement. If it seems appropriate, the staff person describes the program and the intake process. If the learner's needs cannot be met by our program, care is taken to make referrals to other resources and the learner is encouraged to call back for more help if needed. If appropriate, a screening interview is scheduled at a time that is convenient for learner and staff and bus route information and direc- tions are given.

Screening/Placement Interview. This usually takes 30-45 minutes, sometimes longer. The overall purpose of this interview is to make learners feel positive about their decision to pursue their goals for literacy, to introduce them to the way the program works, and to get a picture of the learners, goals, experience with education, and some of the skills they will bring to learning.

The interview is conducted in this framework:

a. Make the learner feel as comfortable as possible; this may involve offering coffee, acknowledging the fact that s/he has already done the hardest thing, which was to take the first step by coming to the interview.

b. Tell the learner what will happen and how long it will take. Encourage the learner to ask questions at any point. Assure her/him that everything s/he discusses with the interviewer is confidential and that if s/he is uncomfortable at any time in answering a question or doing something that is part of the interview, to say so.

c. Ask the learner if s/he would mind writing her/his name, address, and phone number on a sheet of paper. Offer help as needed. This serves the dual purpose of getting information and letting a learner who can write these things without help feel good about doing it. If a learner cannot write her/his name, it lets the interviewer know how to adapt some of the other tasks involved in the rest of the interview.

d. A three page interview form is used to ask the learner about her/his goals, uses for reading and writing in her/his life, and experiences with learning in the past. The interview also includes some questions designed to get at the learner's commitment to learning and any barriers s/he feels s/he will need to overcome to reach her/ his goals. The last item on the interview is a program policy statement regarding use of drugs or alcohol during class time by staff or learners which requires the signatures of learner and interviewer. The interview form is used as a guide, with staff asking the questions, adapting them as needed, and writing the learner's responses on the form.

e. Describe the approach that the program takes, what classes are like, and encourage the learner to ask any questions s/he has at this point.

f. Teacher-made reading assessment exercises. For learners who were able to write their name and reported some reading ability and experience in the interview, this is the next step. A teacher-made reading exercise is used to get a very general picture of a learner's comprehension, fluency and word recognition skills. This can be done in a number of ways. One way is to use a reading sample that is accessible to most adult learners (1.5 -2.5 grade level), which is written in a natural, predictable style, and is a connected, comprehensible text. We have used a selection from the Tana Reiff book, The Door is Open, (Fearon). Because it is about a woman returning to school after many years, learners can often identify with her fear in meeting the challenge. Discussing the reading becomes an opportunity to discuss the learners' own feelings about coming to school as an adult. It is often the first book learners ask to take home with them. It is written at a 1.6- 2.9 grade level, according to the publisher, and we have found it is accessible to most, though not all of the learners we meet.

Be sure to let the reader know what this testing is for and how it will work. Explain that the reading sample will be hard for some people, and easy for others, and that it is only a small part of assessing their reading skills. Introduce the sample to the learner by showing her/him the book, telling her/him the title, looking at the cover together, and previewing the piece s/he will be reading.

g. Slosson Oral Reading Test. For learners who were able to read the sample for the comprehension exercise with some fluency, the next step is the Slosson Oral Reading Test. The interviewer explains the test; that it isn't always easy to read isolated words on a list, the test isn't perfect, but it's another part of the picture of what skills someone has for reading. The word lists are given to the learner one at a time and s/he is asked to read the words s/he knows out loud. The interviewer marks each word on a the sheet, using a plus ( + ) for correct words and minus ( -) for incorrect words. When a learner makes seven miscues on anyone list, the testing stops. After the stopping point, the interviewer often comments on the, skills the reader showed, such as knowing the beginning consonant sounds.

The results, for the purpose of screening and placing learners in our program, are useful but limited. The Slosson yields a grade level equivalent, which has been important for some funding sources. It also gives a limited picture of learners' decoding, pronunciation, and sight word recognition skills. The greatest value of using this test for us is that the "grade level" number is often an accurate gauge of the level of material a learner will be able to read independently. We fmd that learners who start below the "2.0" level as reflected on the Slosson pre-test and go on to make a great deal of progress in connected reading, add to their sight word vocabulary and word recognition skills, and begin to see themselves as readers and writers, do not necessarily have that progress reflected in a Slosson post-test. The Slosson is best used as a placement test and not to measure significant growth and progress. However, when we have used it as a post-test for funding sources, we have included other documentation of reading development for all learners, but especially for the lower level readers whose growth hardly shows on the test.

h. Letter Identification: For a learner who is not able to read any of the words on the first list of the Slosson, or who has indicated s/he cannot read or write at all, the interviewer may ask if s/he knows some of the letters of the alphabet. A letter chart is used to have the learner point out the ones s/he knows. Some learners will report seeing things backwards or having been told they are dyslexic. A letter chart might be used to find out which letters, if any, learners reverse. If this is a severe problem, consultation is sought with a learning disabilities specialist for further assessment.

i. Writing Sample: The last part of the Screening/Placement process is to ask the potential student to try to write something. This is often not the learner's favorite part. The interview discusses the fact that writing is hard and that spelling is a part of writing that everyone finds difficult. The learner is encouraged to try it, and not to worry about their handwriting or spelling. A suggestion is made that they might want to write about their goals for themselves in connection with reading and writing, because it's probably on their minds; but anything they choose to write about is fine. They are seated at a table, given a lined sheet and a pencil with an eraser, and encouraged to write as much or as little as they want, and to do the best they can. The interviewer usually walks away to do something else while the learner writes for as long as s/he wants. Most learners who can, do write. Those who have not been able to write their names are not asked to do this by themselves. They may be asked if they would like to dictate something about their goals and then have it read back to them as a sample of the way they will be learning to read in class. Learners who write something independently are encouraged to read it to the interviewer and are given positive feedback on their courage and skills. Often the interviewer focuses on the content of the writing in making positive comments. The writing sample helps to assess what skills a learner will bring to writing as well as give us an idea of a learner's understanding of the writing process. It's often an oppor- tunity to surprise and please the writer by being able to read what they write and giving them support for writing it It is also helpful in documenting a learner's progress. In part, we like to have a writing exercise in the intitial screening as an introduction to what will be happening in class each day: writing will be part of it.

j. Closure: If a decision can be made on the spot, it is. If not, a clear message is given to the learner as to when a decision will be made. Learners who are appropriate for the program are invited to join a class and if they accept, a card with the time and date of their first class and the telephone number of the program is given to them. Transportation and other concerns are dealt with at this time. Learners are told they need to call if they are unable to come to class. If they wish to borrow a book before class starts, they can sign one out. Learners who are not appropriate for the program are told why and given a list of resources that might better meet their needs. Often these learners are identified before the end of the interview and not made to complete it.

EARLY ASSESSMENT

Read/Write/Now Goals List

Using a goals checklist enables us to help leamers identify their interests and name their particular goals, a crucial step towards self-directed learning. Learner's responses to the checklist are used to articulate goals and the steps to reach them. The goals, in turn, are translated into action plans through short-term learning contracts. It is also a tool for curriculum development, since our curriculum is based in part on the identified needs and interests of participants in the program. The common interests that are identified by the goals lists, such as driver's education, voting information, or using a checking account, can often be developed into an extended small group learning activity.

We have used the goals list in a variety of ways. It is always part of the activities early in the class cycle. We usually have a group lesson/discussion on goal setting. We have also used readings from learner-written publications or oral histories which touch on educational experiences and life experiences that many learners can identify with as a starting place for analysis and discussion about where we have been, where we want to go, and how we can get there.

We have introduced the goals list to the whole group, and also to small groups facilitated by teachers and tutors. The group goes through each section of the checklist as learners check off their own responses. The teacher is available to help as needed. In some cases the facilitator also reads and writes the learner's responses on the form. We hesitated about doing the checklists in groups because it seemed like a potentially revealing situation that might make some learners uncomfortable. However, it turned out to be another opportunity for learners to offer each other support and comfort as they shared some of their problems and hopes around reading and writing. Sharing their responses was a spontaneous occurrence, not orchestrated by teachers, and those few learners who felt uncomfortable in doing so were able to just write their answers, ask for assistance from the teacher if needed, and hand over the finished checklist. Obviously, goals checklists can also be done on an individual basis, with a teacher or tutor assisting as needed while one learner goes through the items and marks her/his responses.

Burke Reading Interview -Modified

What? An interview that explores the learner's understanding of what reading is and what a good reader does.

Why? To help teacher and learner understand what ideas about reading a learner is starting with; to aid in planning reading activities and discussions designed to explore the reading process and metacognative aspects of reading development.

How? Usually done individually, with teacheract- ing as interviewer and scribe; can be adapted to a group activity.

When? Early in class cycle; can be repeated at end of cycle-comparison of responses can be interesting for learner to reflect on.

Looking at Your Own Reading Behavior

What? Self-assessment questionnaire for learners

Why? To help learners become more aware of their own reading behavior as part of the process of setting own goals for reading and becoming more self - directed learners

How? Individually answered after group discusssion & clarification of questions

When? After 2 -3 weeks of classes in a cycle; can be repeated at end of cycle for purposes of compar- ing responses to assess growth.

Reading Progress Checklist

What? Teacher's short checklist for assessment of individual learner's reading behavior and development over period of class cycle

Why? To organize and categorize reading assessment information in an accessible format to help teachers plan strategy lessons and activities for individuals and reading groups

How? Teacher considers learner's reading behavior in class, reading conferences, miscue analysis, learner's self -assessment and responses to modified Burke interview and completes checklist

When? After the learner has been in classes for about three week and after above assessments are done; repeat at end of the cycle to assess changes.

Writing Progress Checklist

What? Checklist form to assess various aspects of writing development, both in terms of writing skills and self-concept as a writer

Why? To provide a coherent, comprehensive picture of a writer's progress for the learner and for teachers to assess individual and group needs for mini-lessons

How? Can be used by teacher after looking at a writer's dialogue journal and writing folder and shared with learner afterwards, or filled out with the learner, looking over the learner's writing together

When? After a few weeks in classes, or when the learner has been writing for three to four weeks; repeat and compare results at end the of the cycle.

Miscue Analysis

At the core of whole language reading assessment is the Reading Miscue Inventory,developed by Goodman, Burke, and Watson and adapted by practitioners in many settings. Miscue analysis looks at the effectiveness of a reader's use of strategies and cues in making sense of text. Inefficient use of language cues and strategies interferes with reading comprehension, which is the heart of the reading process. The purpose of miscue analysis is to discover the reader's strengths in using Various reading strategies to make sense of texts and to assess which strategies the reader needs to develop and strengthen to improve his/her fluency and reading comprehension. The results of miscue analysis can be used by teachers to plan strategy lessons and reading activities which address the particular needs of readers. Results can also be reviewed with students to help them understand, their own miscues strategies. Generally, miscue analysis is done at times when it is useful to assess the strengths and strategies a learner has and address the areas that are not yet developed. We try to do this early in a cycle of classes so that the information we get from it can be put to use in planning reading strategy lessons for learners. Subsequent miscue analyses can be used to evaluate growth in developing certain reading strategies by comparing them to earlier miscues and making a learning plan based on that information. Miscue analysis is only effective with readers who can independently read connected texts that are approximately 500 words long . The text can be written at a basic level, but there needs to be enough text for the reader to read for about 15 minutes and potentially make 25 or more miscues. (See next article, "Down and Dirty" Miscue Analysis for procedures. )

Learning Contracts: Encouraging Ownership of the Learning Process

At Read/Write/Now, we have been experimenting with using learning contracts for several years. One of the great benefits we have found in using them with adult learners is that they help learners to feel more directed and focused on their goals, and therefore to have more success in meeting them. The contract can organize and articulate a learning plan that may be happening anyway, but it is clarified and supported by writing it down. Learning contracts help teachers, too, in the sense that they help us to be aware of individual interests and needs of leamers in the process of negotiating and monitoring the contracts. Another positive effect of learning contracts is an increased sense of ownership that many learners develop about the process of their own education. It is impossible to see education as the passive receiving of information from others when, as a learner, you have written down a plan of action for meeting your own goals in reading and writing and the plan involves you doing things, not just listening while a teacher tells you about doing things.

Learners are encouraged to name goals for themselves to work on in class and outside of class. We usually do a whole group brainstorm to come up with alistofwhatkindsofreading, writing, or math activities people can do when they're not in class. The most creative ideas have often come from learners who are already doing a lot of reading or writing on their own, at home, at work, anywhere and everywhere. Quite a few of our students have taken copies of their learning contracts home with them to put them up on the refrigerator to remind themselves of what they decided to do. The learning contract is an agreement between a learner and a teacher, and also between a learner and her/himself. The learner and teacher together decide on some personal goals for reading, writing, and other goals. Two questions are answered about each goal: What do you want to do? How do you want to do it?

The learning contract period is usually about eight weeks. A short Learning Contract Review conference is set up at the end of the eight week period to discuss how the learner and teacher think the contract went -were the goals met? Why or why not? Are there some better ways to try to meet the goals that weren't meant? Does the learner want to continue with the same goals or change some or all of them ? At that point a new contract is written up or notations are made on the standing one to indicate that it is still in effect. Our experience has been that, for most learners, each successive contract period seems to make it easier for learners and teachers to fine-tune the process of naming reachable goals and reasonable steps to meeting them. The first contract conference is always the hardest in terms of making it a process that is really owned by the learner. Learners' ownership of their own learning seems to develop over time with the experience of setting and meeting goals: The more ownership learners feel over their own education, the more success they seem to have in meeting learning gools, and the more success they have in meeting goals, the more ownership they feel for their own learning.

Goal Setting and Learning Contracts

The steps outlined below typically take place after a learner has been in the program for three to four weeks and has had time to get comfortable in the group, get used to the way the program works, and identify some goals, likes and dislikes. Subsequent Learning Contract Review Conferences do not require quite as much preparation time on the part of the teacher .

A. Before Meeting with the Learner:

1. Review learner's Initial Screening & placement Interview, including fIrst writing sample and answers to interview questions relating to goals.

2. Review learner's Goals Checklist for interests, specific goals named by student.

3. Writing Assessment: Review dialogue journal and writing folder, using Writing Progress Checklist to help categorize progress; add categories as needed.

4. Reading Assessment: Teacher's observations of learner's reading behavior and development, together with these three tools:

a. Miscue Analysis: do one or review the latest one done to see where learner's strengths are and which strategies need to be developed further.

b. Looking at Your Own Reading Behav- ior: look over learner's responses and write a few notes to yourself as to any reading gools students may have stated or implied in their answers.

c. Burke Reading Interview (Modified): for teacher's use in understanding how a learner views the reading process, which may help in plan- ning activities for that person or the reading group.

d. Reading Progress Checklist: for teacher's use in assessing individual reader's progress as observed in reading groups and individual conferences.

B. Orientation to learning contracts is done as a group activity' before conferences begin.

C. Individual Conferences:

1. Materials needed: Learner's assessment portfolio, learning contract form.
2. Procedure:

a. Review purpose of conference & contract, briefly.

b. Explain timeframe (about eight weeks) for contract and need to choose short-term goals that can be accomplished in that time.

c. Learners may have several reading and writing goals, some for in class and some for outside of class.

D. Choosing Goals:

Learning about helping learners to choose goals and trying to empower them as decision-makers in their own learning has been an evolutionary process since we first began using goals checklists and learning contracts several years ago. We have gone from a stance of very limited interference and influ- ence in the process of learners' decision-making about their educational goals to the role of full participants in a learning community. Full participation means listening to each other, sharing our opinions, knowledge, and advice in the process of negotiating the decisions that we often make together. Setting goals as individual learners and deciding what our goals as a group are going to be are negotiated decisions. Teachers and learners both have a voice in the way it all turns out. If we did not negotiate over these decisions, there would be a greater rate of frustration among both learners and teachers. If, for instance, a learner who could only read a few words decided he was going to get his driver's license within the contract period of 8 weeks and the teacher meeting with him simply said that sounded like a fine goal, without considering what kinds of materials were available and how much time it would take the learner to have enough understanding of the material covered in the test to take the oral exam, chances are the learner would not meet his goal and the teacher would not feel like she had done her best to help him meet it. What we have learned about goal setting is that both the learner and the teacher need to have input into the process. Goals are made to be reached, so the more realistic we can be, the better all concerned feel about the process. But it is a delicate balance longer term goals and aspirations need to be nourished. All of us need far off goals to aim for, and all great accomplishments start one step at a time. As teachers, we try to help learners figure out what steps they can start now to get to where they want to go, whether they arrive next week or next year.

1. Start with what the learner names as goals for her/himself. If they have no ideas, suggest goals based on their responses to one of the self -assessment tools.

2. Reading Goals: Make goals as specific as possible. If the learner has a general goal such as "reading better", but has no ideas as to how to do this, suggest a goal based on what you know about their reading based on the Miscue analysis, participation in groups, and individual conferences. "Read bet- ter" might translate into working on a specific reading strategy like using pnxliction and contextual cues to make sense of what they read. "Reading more" might translate into choosing a book to read at home and making a commitment to turn off the TV for 30 minutes a night to have the quiet to read it.

3. Writing Goals: Make these goals as specific as possible, too. Ask the writer to think about what's good about their writing, what they like about it. Tell them about some positive aspects of their writing that you have noticed -content and process, as well as mechanics. Ask the writer to tell you some things they would like to do better in writing. If "spell better" is a goal, help the learner come up with specific ways they can work on their spelling and make that one of the goals. Try to make some of the writing goals process or content oriented and some of them skills oriented.

4. Other Goals: These may be interests indicated on the Goals Checklist or they may be related to meeting the learner's immediate needs, or steps to meeting longer term goals, such as getting a driver's license. In class and outside work could be part of the plan. Goals may also come out of work in options chosen by learners over eight week period, such as math, computers, or helping kids with homework.

Notes

1 All of the materials included in this article, as well as other curriuculum materials, can also be found in the ABE 0-4 (grade level) Curriculum Kit sponsored by SABES.

2 There are many possibilities for teacher-made assessment exercises. This is only one example of an exercise that has been part of our quick, to the point, but necessarily superficial assessment of potential learners' ability to read and understand a piece of connected text. It helps us to place a learner in a very large ballpark, certainly not to pinpoint exactly where s/he is sitting or exactly which reading strategies s/he has strengths and weaknesses in.

Read/Write/Now is grateful to the Adult Literacy Evaluation Project, at the Center for Literacy in Philadelphia, the source of the original goals check- list that we used, adapted and are continually revising to fit the needs of the learners.

4 See Many Literacies: Modules for Training Adult Be- ginning Readers and Tutors, "Life Goals: Maria's Story , " for an example of a group activity for setting goals.

5 Original interview developed by C. Burke (1987)

6 Adapted and expanded from Sylvia Greene's "Writing and Spelling Progress Sheet" from the Basic Literacy Kit (1989).

Originally published in Adventures in Assessment, Volume 2 (May 1992),
SABES/World Education, Boston, MA, Copyright 2003.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.