Three by Three by Four: Ongoing Assessment at the Community Learning Center

Karen Ebbit, Priscilla Lee, Pam Nelson, and Joann Wheeler
The Community Learning Center

T he Community Learning Center/SCALE Assessment Project began when a group of ESL teachers in Cambridge and Somerville
realized they shared an interest in exploring alternative methods of assessment. This article documents the process followed by the assessment team at the Community Learning Center: Karen Ebbitt, Priscilla Lee, Pam Nelson, and Joann Wheeler. Priscilla, Pam, and Joann conducted the field tests with their ESL I/literacy classes. Karen, who was not teaching a level I/literacy class during the project, did not participate in field testing but participated in the planning, development, and revision of the tools and process.

Assessment at CLC At CLC, assessment had largely been a matter of individual teacher's choice. Some tested weekly; some did teacher-made tests at the end of a cycle; and others did not test formally, but assessed progress on the basis of classroom performance. The ESL department had discussed assessment as something the students valued but had not devel- oped a consistent overall process. In working with SCALE, CLC staff hoped to create a process which would be comfortable and effective for students and teachers, and which would also satisfy reporting requirements for various funding sources.

The teachers began a series of four two-hour meetings at which project staff from both centers set general goals and examined options for alternative assessment. The group first looked at the relationship between assessment, goal-setting, and curriculum and then reviewed various tools for alternative assessment options including tools developed by other Massachusetts programs.

The teachers decided in the first goal-setting meeting to limit the scope of the project to developing an assessment tool which could be used at the ESL I/ESL literacy level. Their rationale was that if a tool was usable at a minimal1anguage level, it could easily be adapted for use with students who possessed more advanced skills. The overall goal was to address not only the assessment of language skills acquired in a given period of time, but also that of learning strategies and study skills.

A Three-Part Process After the four research meetings, project staff met separately at each center to design assessment tools that would suit the needs of their staff and students. The Community Learning Center created a three-part process consisting of a goal-setting exercise, a weekly self-assessment, and a progress record. The assessment tools were field tested in three ESL classes during two five-week summer sessions in July and August, 1991. Accounts of these field tests by the three teachers who used the process in their classes appear below. The forms, both current and revised, appear also.

Part 1 .The Goal-Setting Exercise

In the initial goal-setting exercise, students prioritize, in pairs or as a group, the language survival areas they would like to explore in their ESL class. Tools for this exercise consist of five 8 1/2 x 11 drawings of survival areas and one sheet on which all the drawings are reduced so teachers can copy and cut them to make decks of cards. The pictures provide a context in which students can discuss where they need to use English and which survival areas they think are most important for them to learn. A box is drawn on each card for students or pairs to write in a priority number--one (for "very important") through five (for "not so important").

Teacher A: Goal-Setting Exercise

Objective: To help students think about situations where they need to use English and choose which area they want to work on.

I began the goal-setting exercise by posting the large topic pictures and giving the students a chance to read the captions and talk about the pictures. Then we discussed where English was necessary. Many of the students agreed that they did not need to speak much English to go shopping; some said that English was necessary at their workplace; others were not working. Of the three remaining cards, we talked about whether using English on the telephone, in the doctor's office, or in conversation with classmates was easy or difficult.

I passed out the sets of small topic cards and asked the students to prioritize the topics, using one for "very necessary" through five for "easy for me. " The pairs got to work. I thought the activity was going well when I realized that one man had instructed everyone in Spanish to fill out their cards in a particular way--and he had completely misunderstood the exercise! He thought that I wanted to know whether he knew the Spanish translation for the captions on the cards, so he labelled the cognates, like "telephone" and "doctor," as easy, and marked the others as difficult. Everyone had followed his lead, so I decided to abandon the pair work and return to the large pictures on the board.

We repeated the discussion of situations where English is difficult and then voted as a group on each picture. The doctor's office and the telephone got an equal number of votes for difficulty, so that was what we determined to work on for the five weeks of the class.

This goal-setting exercise took 20 minutes.

Teacher B: Goal-Setting Exercise

Objective: To prioritize from a list the topics students want to study.

I began the goal-setting exercise by writing on the board: "What do I want to study?" After the students read the question to themselves and aloud, I began to go through the large topic cards with the class. We discussed them in turn, identifying the items in each picture. I allowed time for students to say whatever they wanted about a particular topic before moving on.

When we had gone through the topics, I turned to the board and read the question again. I spread all the topic cards along the blackboard ledge and had one student ask me the question. I modeled how to prioritize what I wanted to study by writing the five topics on the board and putting them in order of preference. I explained that" one" designated the topic they most wanted to study and "five" what they were least interested in. Students paired and took their own set of topic cards. I asked the pairs to choose what they wanted to study and write it on a separate piece of paper .

I walked around the room during this part of the procedure and talked with students, hoping to get an idea whether they understood the task. The 13 students came up with the following "votes. "

Shopping 415541554554113

Jobs 121111111312233

Doctor 332233222225442

Telephone 243322333313123

How are you feeling? 544544454445554

I assumed my explanation of how to prioritize was understood, but thought differently when I reviewed the students' lists. I saw that they had copied the order I had written on the board even though I had erased my numbers before the students chose their own. I had left the list of five topics on the board and the students seemed to adhere to the order of that list.

Nevertheless, I decided that whatever we worked on would be useful, so we decided on the topic of jobs. I talked with the class before figuring out how to teach this topic because I wanted to get a feel for what the students wanted. The class was composed mainly ofwomen who either worked in the home or worked elsewhere as cleaning women in nursing homes, offices, or universities. I decided to focus our attention on vocabulary having to do with tasks/ jobs/chores. My decision to limit the scope of our work was also based on the fact that our class met only twice a week for five weeks.

Teacher C: Goal-Setting Exercise

Objective: To implement a quick prioritizing exer- cise so learners can direct the topical content of their course.

I began by explaining the purpose of the goal- setting activity: "This will help me know what you want to study. " I asked the learners to spread out the topic cards and order them in answer to the question: "What is important for you to learn in English?" I modeled this exercise for the students.

The students were paired and I asked them to decide, with their partners, on the sequence of the cards, substituting their preference for those just modelled by me.

The students' cooperative work took about five minutes. While the pairs copferred, I walked around the room and observed that the students didn't have problems interacting in English with each other or agreeing on how to prioritize the cards.

The topic cards were numbered by the pairs of learners, with one signifying the topic as "most important for us to learn " and two through five designating the remaining topics in descending order of importance. This is how the five pairs of learners sequenced the topics:

Doctor 2 2 2 100

Shopping 21200

Jobs/Work 11120

Telephone 00131

How are
you reeling?010004

The prioritizing tool was easy to use and helped the students collectively choose the topical content of their ESL class.

Part 2. The Weekly Self-Assessment

The weekly self-assessment is a one page record of new words, grammar , etc. learned in class and outside of class, intended to increase the students' perception of what they learned, how they learned, and where they learned.
The original form was too difficult for the students to use so it was shortened and simplified. We wanted a simpler format, one which left out questions we had judged uninformative, and one which appeared less intimidating to learners accustomed to short sentences. I tried out a shorter form-pared down from 14 to six questions. This revised form retained questions which had previously been answered with relative ease and which had been helpful for the instructor and, apparently for the students. Using the form in class, the teachers found it useful to go through the revised form as a group for the first few weeks, having the class brainstorm new words or possible responses. After it became routine, students were able to complete the form by themselves.

Teacher A: Weekly Self-Assessment

Objective: To give students an opportunity to reflect on the learning process in and outside of class, and to provide the teacher with feedback on what the students think they are learning.

I decided to use the revised and simplified form which had been developed during July. When we had completed one full week of classes, I introduced the self-assessment. I explained that their answers would help me know what they had learned. I drew a large version of the form on the board.

Everyone filled in their name and the date without problems. I had anticipated trouble with filling in " # of classes last week, " so I asked the students to write how many classes they had attended so far. In two cases, students claimed to have been in class when they weren 't--maybe because of the school's strict attendance policy; but, I decided not to make an issue of it. We read the first question together from the board, and I clarified by saying, "Before, no," "Now, yes." At first, no one was willing to admit to understanding anything--either "before" or "now. " So I prompted them with some of the things we had worked on, such as body part words and remedies. They agreed that they understood those words, and some students wrote a few vocabulary words in the space provided.

I had left half an hour at the end of the class to do this assessment, and we ran out of time after completing the first two statements. I asked the students to take the forms home with them and bring them back to class next time. The hurricane intervened, however, and I decided not to return to this exercise.

Observations: I think the revised form could be used without difficulty at higher levels, but my students and I found it a frustrating exercise.

Teacher B: Weekly Self-Assessment

After we had met for four classes, I handed out the revised self-assessment form. I was mainly interested in how the students answered the first question. I explained that the form would help me know what they had learned during our first four meetings. Then I handed out the form and drew a version of it on the board.

The students did not have any trouble filling in name or date. However, there was some confusion about the "# of classes last week" because of classes missed during the hurricane and because the class met only twice each week. To simplify the process, I decided to go through my attendance sheet and tell the students how many days to write down.

I wrote the first open-ended statement on the board and asked students to read it. I explained the statement and asked students to list those words which they now understood because of our lessons. We wrote a list of those new words on the board and students chose from this list for their own form. We did not go further with this form than the first question.

Observation: The weekly self-assessment form proved problematic for my class. The concept of "now I understand" seemed strange to them and I wondered, after the class, whether they would have preferred "now I know." Perhaps the idea of writing a list of words was confusing. Whatever the cause, this activity left my class scratching their heads. I continued to scratch my own head, wondering why the questionnaire had failed. Initially I thought that I would bring back the form in a new version for the next class. However, I decided against it due to the very limited number of class hours we had; filling out the form took too much time. I believed that the progress record would provide the information we were looking for.

Teacher C: Weekly Self-Assessment

I distributed the original weekly self-assessment form to my students after six hours of classroom activity centered around" doctor, " their most preferred topic. My brief introduction was, "This will help me to know what you learned last week and it will help me to teach you better." Students filled out the form as a group. We used the blackboard to record some of the responses that came out of a collective interpretation of the open- ended statements under "in class" and "outside class."

Students wrote in their names with no trouble. For the week ending, I asked the students to write in "July 7,1991," indicating the Friday of the past week. This caused some initial confusion because some of the students anticipated writing the current date in the space, and two others had already done so. Once I explained that the date was to signify "the end of last week," students were further confused, pointing out that weeks end on Saturdays! "How many hours in class this week?" was filled in with some hesitation. If a student left class 10 minutes early on Tuesday, did that need to be accounted for in the total? I felt that some students were reluctant to record their absence(s) due to CLC ' s strict attendance policy.

The first two "in class" statements: "Now I know-new words" and "Now I understand " took some time as the students had to decide whether "new words" meant words they had never heard before the past week or words/ concepts which, recently reinforced, were now better understood. At this point, students referred to (and some copied) their classmates' responses to be sure that they were on the right track. They seemed uncertain about what I wanted from them. How much writing would satisfy the teacher? The task was becoming increasingly cumbersome.

When we came to the third" in class" statement, "1 don't understand-," they asked if they could leave this space blank. Only three students wrote in this space. Answers to "This week I liked_________" and "This week I didn't like__________" seemed to me to be biased and therefore uninformative. When asked, "What did you like?" most students answered, "Everything!" Half of the students had been in my classes for over six months, and we had shared many good feelings. They wanted me to know that they had been pleased with the classes. No one admitted that they didn't like something.

For the questions, "I listened to_________speak English" and" I spoke English to________, " I listed possible answers on the board. Most students wrote some variation of "myself, the teacher, everybody, and my classmates. " The students easily completed the last "in class" statements: "I read-in English" "I wrote______in English " and "I needed help with__________. "

The students also had no trouble filling out all four of the "outside class" statements: "This week I spoke English to____________. " "This week I listened to-speak English" "This week I read_________in English" and "This week I wrote _____ in English."

Observations: This stage of the field testing was much more time consuming than the goal- setting exercise. I had allotted the last 30 minutes of class for a task which eventually required our class to go 12 minutes over time. I suspect my vague introduction, as well as the structure of the form itself, made the task more laborious than was necessary .But, filling out the self-assessment form was, if tedious, a good experience for the students. The "in class" section required the learners to recall what was studied during class and their answers reflected what information I assume was most strongly retained. The "outside class" section caused them to consider situations in which they had (or had not) used English at work, home, and in their community.

Some of their responses were:

Yelena: At Stop & Shop. Please where is the milk? Where is the butter?

Ana: Before when I went to the doctor, I needed an interpreter, now I do it myself!

Jose: I made a phone call in English for the first time yesterday. I called Connecticut to order something for work.

Vitanie: I read English signs at work.

Students mentioned they had spoken English to family, friends, and neighbors at home, work, and in the supermarket. They had read English in the dictionary , at home, and at the library .One student wrote, "I read the bus name! " They had listened to English on the radio, on television, and at the supermarket. Not all students had written English outside of class, but those who had had done so at home or work.

Once completed, the forms were a great source of information for me. The students' answers, particularly the answers to "I need help with...," offered me useful knowledge. As a supplement to the assessment of each learner which I gain through "teacher instinct," the form showed my students' views on what had been learned. The form also helped me to focus on my teaching techniques and to consider ways in which my students' learning could be more enjoyable and efficient.

Teacher C: Weekly Self-Assessment

During the third week of class, I gave the revised assessment form to my students. By now, having used the original version of the self-assess- ment, they were familiar with what we were doing and why we were doing it.
Observations: The students filled out the new form individually and with far greater ease then they had its earlier incarnation. Compared with 45 minutes spent on the form the week before, the students tookless than 15 minutes to fill out the new form. While the students' familiarity with the format no doubt hastened the process, the improved form clearly helped to make the weekly self- assessment an easier and more enjoyable task.

Part 3. The Progress Record

The progress checklist is to be filled out by the class as a group at the end of a unit and kept as a record of what the group has worked on and how well each individual feels s/he has mastered the topics. The teacher and class determine what they have worked on and fill in the left side of the list. The students then check whether they feel they have mastered each topic or whether they want to work more on this area.

Teacher A: Progress Record

Objective: To enable the student to look back over a session and record what s/he learned, with an evaluation ofwhether or not review is necessary.

The progress report activity worked well with my class. Perhaps my students were getting used to the idea of thinking about what they had learned; or , perhaps the checklist format was easier for them than completing open-ended statements. To begin the activity , we established, as a group, what we had worked on. Each student contributed something and I recorded the words on the board--I didn't eliminate any, but I did classify them as we went along. I wrote all the body parts in one list, all the symptoms in a separate list, etc. Then we talked about the categories of words such as body parts, symptoms, and medicines. After that, students filled in all the blanks on the progress record. Some wrote general categories; others wrote the vocabulary words they had originally contributed. When everybody had filled in the "We worked on. .." column, we discussed briefly the two remaining columns, "I can do this" and "I need more." Students usually checked "I can do this" for individual vocabulary words and "need more" for categories. A few students checked "I need more" for everything, on principle, I suspect. One student did not complete the columns at all.

Teacher B: Progress Record

I reserved 25 minutes at the end of our final class for the progress record. I wrote, "What we worked on" on the board and asked the class to name the things we had studied. It took them a few minutes to understand what I was looking for and then they began to call out the new words they had learned. I recorded on the board whatever was said, trying to get everyone to offer something to the list. I gave each student a progress record and then explained how to fill it out. The "I can do this" column seemed confusing because of what we had studied. The phrase seemed more appropriate for talking about a task rather than a list of vocabulary words. I changed the columns to read "I understand" and "I want to study more. " The students copied from the list on the board for their own forms and then checked the appropriate column.

Observation: Perhaps what is important about this form is that the class (collectively and the individually) could list the terms that we studied.

Status: Ongoing
The process of defining and developing assessment tools at the Community Learning Center continues. The current status of alternative assessment at CLC is "ongoing." We hope this account of our experiences proves useful to other ESL teachers. We would be very happy to hear from anyone who has used, changed, or expanded on the tools we have described here.

Karen Ebbitt, Priscilla Lee, Pam Nelson, and Joann Wheeler teach at the Community Learning Center in Cambridge.

Originally published in Adventures in Assessment, Volume 2 (May 1992),
SABES/World Education, Boston, MA, Copyright 2003.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.