16 Spring 2004
In their groups they discussed the differences among the prompts and answered the following questions: "Whom am I writing this for?"; "Whom/what am I writing about?"; and "How many questions do I have to answer?"'
Afterwards, they returned to the whole group for a discussion of the differences among the prompts. I asked the learners how they could use the pre-writing activities to their advantage in responding to each prompt. All of sudden it made sense to them! The comments were overwhelming. "My transition was so easy, but I could use what Mayra said about her life to help me answer the third prompt." By referring back to the rubric, we could point to how these pre-writing activities might lead to higher levels of writing quality.
Step 6: Administering the prompt
Finally, the learners took the practice REEP, under much the same circumstances as an official program assessment. All in all, these first six steps took about three hours, or one class session. But the work had just begun!
Step 7: Scoring
I scored the essays myself and, using a simple form, highlighted
the strengths of each and offered concrete suggestions for improvement
based on skills that I knew the learner had been working on and
had demonstrated some ability with. (See Figure
2 & Figure 3.)
For example, if the learner had previously demonstrated the ability
to write complex sentences but didn't do so throughout the essay,
I suggested places where simple sentences could have easily and
effectively been combined. During learner-teacher writing conferences,
we discussed the scores and comments. I also gave the learners the
opportunity to rewrite their essays,
I felt that I had achieved my primary objectives for the lesson: The learners gain increased awareness of the standards by which their writing is judged, and they had some initial ideas about how they can improve their writing. But how could I use this experience to inform my writing instruction? I'm still working on this, but I did spend some time looking at the scores of the class overall, and found a few surprises. Some of my work had been effective. For example, the majority of learners demonstrated good control over the verb tenses we had been studying! However, many learners were having difficulty with paragraph formation and needed a review of the punctuation rules. (Try again, Joanne!) I felt like I had a new place to begin, both with each learner individually and with the class in general.
I have also noted a few unexpected results, including reduced test anxiety and improved REEP scores. But best of all, my learners feel like they have more control over meeting their mandatory "improve writing skills"' goal. Teaching from the REEP has been very empowering.
Using the REEP for ABE Instruction: Carey Reid
I have been co-teaching a pre-GED writing class at the Jamaica
When I learned that Joanne Pinsonneault was using the REEP in her mid-level ESOL class, it occurred to me that the learners in David's class might be able to handle it. David was game to try it, and so were the learners. We decided to use the REEP materials as instruments for periodic formative assessment. Here's how we did it.
Stage One: Unpacking the rubric
We gave each learner a copy of the scoring rubric and spent about 90 minutes going through it in a whole group discussion. David and I explained that the rubric taken as a whole represented most of the elements that make up writing. We explained further that the rubric as an assessment tool provided good indicators for levels of quality and sophistication.
At that point, we started to just cruise through it, pointing out indicators at random and giving oral examples of how these indicators might align with a particular piece of writing (e.g., "If the writer provides a lot of supportive detail and can use complex sentences, then she's writing at such-and-such a level") After this exploratory phase, we asked the learners if they were willing to look at some sample essays and try their hand at scoring them. I am not exaggerating when I report that the learners were very intrigued by the rubric and equally eager to try applying it to actual writing.
Stage Two: Scoring the anchor essays in small groups
David and I really kept our fingers crossed with the next stage. Could the learners apply the rubric to actual writing with some accuracy, or would they become frustrated by such a task? We reminded ourselves that one of the reasons we decided to use challenging materials and exercises in our class was our determination to put into actual practice ideas and theories we professed to believe in-e.g., that adults learn best by doing, that they know what they want, and that they have acquired tons of knowledge through their life experiences. We did not want to appear to be the harbingers of special knowledge ("We know all the grammar rules, but you don't"), or superior in position ("The REEP is for teachers, but not for learners"), or unconsciously condescending by coddling or "protecting' our learners ("Concepts and materials from standardized testing are much too complicated for you to understand"). In fact, in our class-planning sessions, David and I would call each other on perceived instances of these kinds of presumptions.
For this stage, we decided to use the essays currently used in the training of Massachusetts teachers for REEP scoring. As in that training, we decided to use the six "anchor essays," each of which represents the same score across all levels-for example, a "3" all the way across. Specifically, the six anchor essays represent scores from 1 through 6. We told the learners that they would be looking for that consistent score level for each essay, explaining that this would simplify the exercise a bit, adding that this was the same process used to train Massachusetts teachers to understand the REEP.
David and I then organized the class into three workgroups of five learners each. We gave each learner a packet of anchor essays and a blank scoring sheet and asked that they read a particular essay together, score it together using the rubric, and try to reach agreement on a score. For some time, we'd been using collaborative learning approaches, so the learners stepped right into this task. One by one, the workgroups read and scored a particular essay, and then when all three groups were finished David and I asked the facilitators to report the scores. Then we discussed the scores and the reasons behind them.
Frankly, David and I were surprised by the results of this exercise.
In nearly every case, the workgroups gave the same score to a given
essay as the REEP "experts" assigned to it. In those cases
where their and the experts' scores were different, the variance
was only a single point. In addition, the three workgroups nearly
always agreed on the same score, and when they varied it was only
by a single point. In full class discussions, the reasons for giving
scores were discussed, and where scores differed the workgroups
defended their choices. Occasionally, an individual learner would
disagree with the other members of her group; in the full class
discussions, she would get a chance to defend her choice. (A great
anecdote to share: One time a very shy learner revealed that she
did not agree with the score the other members of her group had
given. At that point, the other members asked her to speak up and
defend her score. As it happened, her score matched that of the
experts. Needless to say, the shy member's standing in her group
went up a notch
Stage Three: Using the rubric to score learner essays
The class explorations of the REEP really intrigued the learners. When David and I said we hoped to apply the rubric to their own writing, they were completely supportive of the idea. At this point, David and I began to realize that because the REEP is not used as an official learning gains assessment in Massachusetts for non-ESOL learners, such as those in our class, we were free to adjust the materials and approaches as we felt we needed to. For example, we could alter the rubric if we wished, to make it more appropriate for our class. We were also free to design prompts that we felt suited our class better than those that are more appropriate for ESOL learners, which are based, for example, on personal letters or narratives. It struck us that it might be better for our learners to use prompts that resembled those used for the GED Writing Test.
The first prompt we developed basically centered on the question, What is the most important profession in modern society and why? To get the ball rolling, we started with an open discussion and some brainstorming, as Joanne Pinsonneault had done. Then we asked learners to write original essays for 30-45 minutes in response to a prompt based on the brainstorm question. (See Figure 5.)
David and I then read over these essays and, again as Joanne had done, scored them and noted those scores on a single-learner score sheet. We decided to list for each learner only a few Notable Strengths and one Area for Improvement. Our idea was to provide the learners with a Next Step for improving the draft, based on ideas about process writing we'd been applying all along. At the next class, David or I sat down with each learner and went over the scoring and the notes we'd made together. (We've provided an example of an essay and score sheet as Figures 4 & 5.)
Stage Four: Using the essays as "authentic materials"
At this point, everything we were doing with the REEP and with our overall authentic materials approach came together in a nice neat package: David and I decided to use the learners' essays as source materials, as the stuff that future lessons could be based on. For example, we gleaned fifteen sentences with errors from the various essays, listed them on a single sheet, and then asked the learners to discuss and edit them together in workgroups. The learners were much more motivated to edit samples of their own work than materials from workbooks.
Next steps: Using the rubric more widely
David and I are just beginning to experiment with using the rubric as a self-assessment and peer-assessment tool. For example, the class is now knee-deep in a several-week project in which we're applying the NCSALL research on learner persistence to the learners' own lives. We began by reading and discussing together the research findings (Focus on Basics, Volume 4, Issue A). The learners are now writing essays composed of an initial summary of the research, a commentary on how the findings apply or do not apply to their own lives, and a final section on ways in which our own writing class might be altered to better support their persistence(!). Now that most of the learners have a first draft, we will be asking them to use the rubric to self-assess the piece and, perhaps in conference with one of us, plan out the second draft.
David and I are also just beginning to try to integrate the rubric with a peer-editing process that the class is developing together. We've come up with rules and guidelines around constructive criticism; now we're wondering if the rubric would be a good tool for promoting objective and constructive critiquing. We shall see!
A final note...
David, Joanne, and I have presented these ideas in conferences and workshops over the past year. A common concern among participants is that the REEP scoring rubric might be set too high for new readers and beginning ESOL learners. Gradually, however, we've all come to realize that teachers are free to adapt these materials for their own learners. They can simplify the indicators, make them more positive sounding, reduce the number of levels, and so forth.
We would like to offer one caveat, however. Using the REEP rubric
has, in our opinion, substantiated the claim among many theorists
and experienced practitioners that adult learners know a lot more
about writing than might be readily discernible. The learners often
attach, constructivist-fashion, their working knowledge of writing
Originally published in Adventures in Assessment,
Volume 16 (Spring 2004),
Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.