Analyzing Self-Evaluation Checklists:
A Starting Point for Dialogue

Andrea Mueller
Workplace Education Project
Labor Education Center,UMass Dartmouth

The Workplace Education Project of the Labor Education Center offers ESL, Pre-GED and GED classes in New Bedford and Fall River. Our students are primarily Portuguese, although some are Cape Verdean or Polish. About half our students are employed as stitchers or machine operators. Some work for small businesses and the number of unemployed students is rising.

Our program worked on alternative assessment in the GOALS Accountability Project, focusing on different options to document progress through portfolio assessment. In this process we revised our old assessment tools, and adapted and created new ones. We field-tested these tools and then discussed their usefulness for our particular group of learners and our project.

One of the tools in our portfolio is a progress checklist which we adapted from Janet Kelly’s forms. To make the checklist suitable for second language learners, we based the items on our students’ goals, as expressed over the last few years. The form asks learners to reflect on the frequency of their use of speaking, reading, and writing skills in various settings. Students can choose from “not yet”, “a little”, “sometimes”, and “usually”.

The progress checklist is two pages long and initially takes a long time to fill out. As students become more familiar with it, however, the process becomes quicker. Our program has multi-level classes and therefore the advanced students can usually help the beginners. Students complete this checklist every two to three months. We thought that this interval would make it more likely that learners would have made progress and that they would have noticed it themselves.

We expected this self-evaluation tool to show the students their improvements in English. This kind of self-assessment usually helps students think about their own progress and is also a step in taking responsibility for their learning. We expected to see general movement toward increasing use of English in most of the skill areas in most settings. This was not always the case, as the following example illustrates.

One student, Maria, completed the form on November 9, 1992. On all but the last skill Maria checked “a little”. She felt that she didn’t understand and couldn’t use the conditional tense yet (Figure 1).

Maria completed this form again on January 25, 1993. Usually students don’t see their initial form before they complete a new one. This way they have to think about how they feel about their progress at that particular time (and not just replicate what they checked off last time). If we compare the responses from November to January, however, we see Maria felt that she made progress in all but one category. Looking at grammar, she now feels she uses the tenses “sometimes” rather than “a little”. For the conditional tense, she feels she is now using it “a little” as opposed to not at all. She doesn’t feel that she understands English any better.

After completing the form in January, Maria looked at her form from November to realize she felt she was making progress. Maria’s progress checklists confirmed our expectations. Half the class displayed response patterns similar to Maria’s.

The responses of the other half of the class, however, didn’t change with time in the anticipated way and puzzled us. Carlota is one example from this group (Figure 2). Her form from November 9, 1992 shows that she speaks with her friends at work “a little” but not with her supervisor or her union representative. She also speaks “a little” with her family in English but only “sometimes” at the store and never at the doctor’s. She didn’t check off at all how frequently she talks to her neighbors.

Figure 1

Figure 2

Carlota filled out this form again at the end of January. Most of her ratings stayed the same. For example, she still doesn’t speak to her supervisor or her union representative. She moved from “a little” to “not yet” concerning speaking with friends at work. However, this time she says she speaks with both her neighbors and the doctor “sometimes”. She also feels she is speaking English more with her family.

Forcing Us to Reconsider
Examining the inconsistent results, my colleagues and I reconsidered the value of this tool. Our dissatisfaction stemmed from the differing response patterns we saw in some students’ forms and that we had difficulty in interpreting. Our main question was: Is it important that students move in a linear progression in their language acquisition or English usage? If we are going to use a subjective tool, shouldn’t we accept all subjective responses? What could we learn from students’ self-perceptions of their progress? Our conversation reinforced and strengthened our conviction that we valued knowing how the learner felt about his or her progress. We decided, however, that we also needed to probe deeper into the reasons for the responses that indicated a lack of progress.

Going back to the classroom, I talked to Carlota about her form and this conversation clarified some of her responses. Her increase in talking to the doctor from “not yet” to “sometimes” was a reflection of the changed family situation. One of her family members had some serious health problems and she had to accompany him to the doctor’s office often. If I had known this, I could have introduced more health-related material into the class or could have had her work on it individually. Carlota doesn’t speak to her supervisor or union representative in English because they are Portuguese.

Her job in a predominantly Portuguese-speaking workplace makes it harder for her to use English in that setting. In retrospect, she and I could then have reflected on how she feels about this situation. If she had preferred to speak more English at work, we could have thought about some strategies to do so. We could also have concentrated on how she could practice her English in other settings.

Knowing how learners feel about their progress enables us to find a meaningful starting point for dialogue. If big gaps exist between their perceptions of their abilities and ours, then it is important to “dig deeper” into the “why.” For example, the most advanced student in my ESL class consistently rates herself very low. I hope it is helpful for her to compare her subjective ratings with other more objective measures to see that she is progressing. Knowing how she feels about her language development, I can make sure that I point out to her specifically and often the gains she is making. More importantly, I need to involve her more consciously in identifying progress in her reading and writing since these are her strongest skill areas. Ultimately, only she can convince herself of her progress but I hope that these dialogues about our different perceptions could contribute to a better self-image of herself as a learner.

In contrast to the above student, I had another learner in my class who rated himself consistently high with this tool, even from the beginning of the class. He checked off “usually” for most of the settings and skill areas. Although his spoken English was rather fluent, my assessment of his reading and writing skills did not show the same skill level he felt he had. So far, I have not figured out how to discuss our different perceptions. I know this will be necessary for him to adequately understand where and how he needs to work on reading and writing.

Is Progress Always Linear?
Initially, we assumed that progress would be a linear movement from lower to higher English usage in all skill areas and in all settings. Now we see that progress is not linear but rather moves in jumps and starts and is based on the contextual conditions. For example, you can’t expect an increase in speaking English with your supervisor if your supervisor speaks the same language you do. You can expect a change in speaking English at the doctor’s if there is an increased need to do so.

We need more experience using this checklist, especially involving students, to develop its potential. But it is clear that to interpret the results of this tool, additional dialogue with students is required. In fact, the dialogue which emerges from this process is probably the most important outcome of this tool.

Top of page

This article was published in Adventures in Assessment, Volume 5 (October 1993), SABES/World Education, Boston, MA, Copyright 1993.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.