What We Had to Think About Before We Could Do Portfolio Assessment

Kathy Sikes
Durham Literacy Council
Durham, NC

The Durham Literacy Council is a community agency with a partici-
patory focus. Staff members and volunteer tutors work in a variety of settings from community centers, university worksite programs, residential substance abuse centers, jails and employment training programs. Volunteer training focuses on developing lessons from learners goals and authentic materials, and learners work either in small group settings, one to one tutoring sessions or both.

The Lila Wallace/Literacy South Portfolio Project inspired staff members to examine the role of assessment in a learner centered practice and to create the program structure necessary for implementation. The importance of participatory planning and ongoing reflection became a major focus as authentic assessment practices were explored. I am the program director and have worked for the Literacy Council for seven years.

Introduction
The portfolio project has helped us at the Literacy council think about a way to integrate assessment and instruction. If you’re going to start into a portfolio, then you have to have some plan in the beginning about what you’re after in your learning, beyond, “I want to learn to read and write better” or “I want to get a GED or get a drivers license.” Tutors and students are having to do a lot more planning together initially. Even though we’ve always said we were participatory, and I think we’ve done the best job we could do with that, sometimes tutors kind of fumbled with
focusing on students’ goals. I think it was difficult for them to always see the prog-ress they were making towards those goals, especially if someone didn’t read and write very well and they wanted to
get a GED or something. Doing the kind of planning that makes sense and there’s something there to attach it to, it’s almost like some kind of mutual contract, mutual decision making.

We’re not where we would like to be in terms of full program participation. It was difficult for us to get started and I’m not sure why. I think we really struggled with how we train volunteers about portfolio, and we’re still struggling with that somewhat. The piece that seemed to be missing when we did the portfolio training was the planning piece. Even our small group leaders were saying “We’ve done so many kinds of things, and there is nothing that really ties it together.” I realize that they weren’t planning very well and couldn’t do portfolios if they weren’t doing better planning. So we’re just now getting to the point that we’re going to start seeing people have real portfolios to talk about.

Introducing Portfolios
Our teachers, other than the other two people on our staff, are all volunteers and so part of my charge was to go back and train and disseminate information from the portfolio assessment project. I think we did a good job with that. We held different workshops – on Saturday morning, one in the evening. Early on the workshops were an introduction to the whole portfolio process because we were really new at it too. We just wanted to get started and we handed out some things like cover sheets and portfolios, and said, “Well, go and try it and we’ll meet in a couple of months and see what your questions are.” That couple of months passed and not very many people did anything. Then we evaluated how we would do the training differently.

Next, we invited volunteers and learners who were trying out portfolio assessment to come and talk with other tutors about it. It wasn’t perfect in the beginning. People just looked at the portfolios and talked about how they chose things or why a certain piece was in there. We decided we’d provide ample opportunity for people who had been in the program for
a long time to come and get information about portfolio assessment. We wrote about it in newsletters, we did everything we could to incorporate existing tutors. Then we decided to put it in the training. Now, it’s part of the tutor training that we do rather than an in-service workshop. Like pre-service training rather than in-service and staff development.

When we planned the first portfolio training workshop, we tried to do way too much. In some ways, I think we’ve made this an overcomplicated thing. When we first started having conversations in the portfolio assessment project about portfolios, there was a lot that I didn’t understand. I did not understand the concept of “criteria.” I’m not sure why not. Sometimes what I want to learn comes to me in the middle of things. It seemed like “criteria” was an esoteric thing. I didn’t know the process well enough to feel like I could change it, or we could add other things. I know at one point I felt very limited about what I had chosen for my criteria. And then I got over it. I realized, this is mine! I can change this. It’s fine, right? That part took a long time.

We struggled with how to talk to tutors about portfolio assessment, and basically what we said was, “this is a good thing. Look at all that can be done by using portfolios.” We gave tutors some sample lessons reflecting on change. If I had to do it over again, I would just skip all these discussions about change and start talking about “how do you know you’ve learned something?” rather than it being some kind of introduction into “Now we’re going to prepare you to participate in this process.” I don’t know, I think I was a little too careful, honestly, and not with learners with tutors. Then finally, I realized that part of it was that there was no organizational structure that supported portfolio assessment other than just a philosophical commitment.

Our tutors weren’t having ongoing discussions about progress with their students, and they didn’t have anything to base it on either. All they knew was some kind of large goal that this person had or didn’t have. I hesitated to provide models because there I didn’t want teachers and tutors to name the learning for the learner. I was concerned that we could interfere with the learners’ right to choose and describe their learning any way that they wanted.

Top of the page

Doing Portfolios
As far as the impact on students, that’s hard to know yet. I think that the students will increase their desire to be more participatory because tutors and students are supposed to work together to plan their work for the quarter. People are sending us more complete plans. And tutors and students are filling them out together to plan their work for the quarter. I see two different people’s handwriting on things. So I think that part will benefit the students.

Students will also benefit from looking at their work. In our first meeting, I say that it became really clear to people what they hadn’t done. I mean, They were really proud of the work that they had done, but they kept seeing these gaps of things that were goals they had not met. We had two people who said “I don’t have any writing to put in this,” and “what happened to that writing class we used to come to?” And I said, “you stopped coming, and we closed the class. You know, we can’t just run it forever with one person in it, or that needs to be a one-to-one match. Oh, well maybe we should do that again.”

There were students that came to that meeting that only worked in workbooks. They had much less to put in their portfolios; for example, they saw people putting in their portfolios evidence that they had registered to vote or had been to conferences and had collected lots of material. We had talked about meeting with students individually to look at their portfolios. That evening, four students out of twelve asked me, “when are we going to do that?” So they felt invested in the project.

I’m looking forward to when we have or next meeting. We’ll have people who are a little further along in looking at their portfolios than others. But my guess is that rather than everybody taking the same amount of time to get to that point, people will get there faster by seeing the examples of other people doing it.

I’ve loved doing my own portfolio. I’ve Loved it. Particularly the last time that we all got together in the portfolio project and I got to spend some time really looking at my folder. Until then, I just collected things really. The other times that we had to sort things ,I put a few things in my portfolio, but this time I really knew what I wanted in it. I also didn’t edit when I was bringing things. I brought a box full of different kinds of reading, writings and work.

Writing the cover sheet has been really informative for me. The first thing that I think was important to mention is that it’s not so much what I write on the cover sheet. It is the process of doing it. I mean, I’ve written handouts for years. I’ve written training workshops. I’ve never gone back and said, “why do I like this? Why am I keeping this now?” It’s made me feel really good about the kinds of things that I’ve accomplished in my thinking and my job.

I tried to watch my progress and my learning and thinking about original criteria specifically. But, not everything I’ve included in my portfolio is about that. For example, I included part of what I wrote for a small-group inquiry based workshop, because I made an important connection. I realized how the planning part was missing for the tutors. They were saying “We love the idea of portfolios, because we weren’t sure whether people were making progress, and it’s a hard time getting learner input about what to do, and I thought, we’re not very good at that, but we are getting somewhere now.”

Top of the page

Staff Development
Portfolio assessment has made us look at our training. Volunteers are, in essence, our staff and as such we expect some accountability for what’s happening with their work. In some ways I which we had a 20 hour a week person who just paid attention to helping people plan, being a resource to them in their planning and then visiting them about their portfolios. We don’t provide a lot of leadership on how to get input, how to ask good questions, and how to plan together. We just say “Do it. Good luck. And bring back a beautiful portfolio.” So it was the connection between those two things, planning and ongoing assessment, that made me realize where the training needed to be done.

Right now in training, we do a three-hour session on assessment, mostly talking about what happens when people come in the door, what kind of initial assessment. Then we talk about what we do with that information, what we tell the tutor. We talk about the planning sheets and how the tutor can take the information given them and then have this more in depth conversation with the learner. Part of it’s for your information lecture type stuff, but then usually at some point we say, “OK, now here’s some information we would have given you on the phone about learners,” and have them create lessons based on it, so they see the connection between the assessment piece and the lesson piece. People do quite a good job. I mean, I’ve been impressed. We don’t do very much on what happens between the planning and evaluating how well it went, and that’s the sort of thing that we have to work on in our training. Then we say . “OK, this was our goal, what might you put in a portfolio? What would be the range of things?” All we can do at this point is brainstorm that. We don’t have anything to real to show people. We created sample portfolios that are very different from each other and show them to people in training. That seems to help. It wouldn’t be anything like if learners were coming and talking about their portfolios, and I think we can pull that off by the next training. I do.

One of the other activities volunteers do in training is to help us create a training portfolio. They answer two questions and then talk about what they would give as evidence. We ask questions like, what did you learn about teaching and learning that you didn’t know before? What did you like most about the training? We’re getting much better feedback about what people are getting out of training.

Conclusion
My colleague Lee and I do all the training, so much of it is still in our heads. If we can create some kind of organizational memory of this project by keeping learners doing it— and by January we should have people who have pretty good starts on good portfolios—we will have started that process. If we can keep that going and have students help us lead portfolio workshops for other learners and volunteers then in a couple of years this will be our primary assessment tool.

It has been a real process for the organization, and if it took us two years to feel confident about it, I think we need to give ourselves that much time. Initially, we thought, “this will be a great thing to do because, one, this is the way we think anyway and learners will have that much more ownership over another piece of what they’re involved with at the literacy council, and it’ll be great to show the funders. We’ll have all these really tangible things for people to see.” Actually, the biggest impact has been an organizing effect — making us take a real look at training and saying — “what are we saying about assessment? How can we get people to buy into the process? How can we get them to feel like it’s not some kind of esoteric thing, or additional work, but make it as real as we’ve made writing and reading strategies?” That’s were we are. It has allowed us to create an instructional organization that we didn’t have before and to learn how to put things into a portfolio. That’s an important step for us.

Top of the page

Originally published in Adventures in Assessment, Volume 10 (December 1997),
SABES/World Education, Boston, MA, Copyright 1997.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.