The Tale of the Tools

A Matter of Stance and Dance

Loren McGrail
SABES / World Education

Learners Logs will never be a finished product.
Gear

Examining inconsistent results, my colleagues and I reconsidered the tools.
Mueller

I am still uncertain about the best ways to document the development of self-esteem and community, but I do feel that my teacher log will play an important part.
Barry

Obviously nothing that is based on human beings’ changing needs and goals is ever finished or perfect.
Kelly

I have a lot more questions than answers. In fact I am going to be revealing a lot of (gulp) mistakes , which Lord knows teachers do not really make.
Trunnell

Never finished or perfect, uncertainty, inconsistent results, more questions than answers. These are just a few of the voices of reflection from the authors who participated in Component #3 of the Greater Opportunities in Adult Learners Success (G.O.A.L.S.) Project developed by Sandy Brawders for the Bureau of Adult Education and coordinated first by Lindy Whiton, then Charlotte Baer and Caroline Gear. Each of the programs involved in this project developed assessment and evaluation tools that were based on educational theories or approaches, on who their learners and teachers were and, finally, on what the goals and interests of the learners were and how they used literacy in their daily lives.

These reflective writings which are the focus of this volume of Adventures in Assessment are documentations of inquiry, teachers researching their practice. As Whiton says about the writings in Adventures in Assessment in general, “This is the place where other teachers can listen not only to the ‘“tale of the Tools” but to the narrative, the story of who, why, and how come” (Whiton, Documenting the Voices, paper delivered at AAACE, 1993).

Just as authentic assessment asks learners to develop the habit of pausing to reflect before moving on, so too must teachers adopt the practice of “taking time to stop and think, to observe and make sense of the activities and progress of their students” (Lessoules and Gardner, 1992). This sentiment is echoed by Stephen Brookfield when he says, “Becoming skillfull teachers is a matter of stance and dance. It is a process that involves us taking a critically reflective stance on practice, studying our actions and reasoning about teaching as open-ended, problematic phenomena. It also engages us in a dance of experimentation and risk-taking in which we explore how learners experience learning while living out our pedagogical convictions and trying to realize democratic values in our practice (Brookfield, 1993).

I want to understand the students progress.
I want to see and I want the students to see
the complicated reality of learning.
Trunnel

Documenting Learning: What the Learners Get
How learners experience learning is a theme that reoccurs throughout all these teachers’ stories. For Mueller in Analyzing Self Evaluation Checklists: A Starting Point for Dialogue, the concern is how can students take responsibility for their own learning? Her careful analysis of the data her checklists revealed about Maria and Carlotta lead her to ask “Is it important that students move in a linear progression in their language acquisition or English usage?” This question is of particular importance to teachers working with ESOL students who are interested in documenting acquisition, not just language learning. This question also reflects Mueller’s understanding that a lack in linear progress may also indicate a significant difference in how learners perceive their own progress. “If big gaps exist between their perceptions of their abilities and ours, then it is important to dig deeper into the ‘why’.” The self-evaluation checklist then becomes the starting point for dialogue, not the end point.

For Barry, writing about and with her student Pat in Reflections on On-going Assessment: Documenting Self-Esteem and More, the issue of how learning is experienced was focused on how Pat felt she was making progress and how she knew when she was. Barry’s main concern was that while Pat’s development of self-esteem and sense of community were apparent from her accomplishments and conversations with Barry, the tools and procedures used in the program did not reflect this back to the learner. Barry’s reflection is a critical looking back at what her tools both documented and revealed to her as a teacher and what they revealed or did not reveal to her student Pat.

Barry’s comment “We need to have more frequent conversation about learning moments and more time together to analyze the data gained from these checklists” is echoed by many authors but probably most strongly by veteran tool maker and mentor, Janet Kelly. In A Reflection on the Ideal vs the Real Kelly admits, “I think we have relied more on what we as teachers thought was important to learners, what we were interested in knowing about their progress, what we thought we could document, and what we thought funders would want to know.”

She goes on to say that these beliefs still form the basis of choices in assessment practices but that she hopes the program will find ways “to invite learners to share more effectively in decisions not only about program and curriculum evaluation but also in the selection of and critique of materials for their own assessment portfolios.”

Like Barry, Kelly’s reflection piece also examines and critiques previously published tools at the Read/Write/Now Program and discusses the development of new tools. Of particular note is the discarding of learning logs (a tool much favored in Barry’s program) in favor of a shared teacher’s log where “Significant anecdotes and observations could be recorded on a joint teacher/learner from each learning contract period or month.” Kelly’s experience, like Barry’s, reminds us that what seems to work best is time and trust. “The longer learners are part of the learning center community, the better they get at telling us what works for them, what does not, what they like and do not like and why.”

Trunnel, in Danger: Road Construction Ahead, like Barry and Kelly, is also concerned with how the learners experience the tools he has created. “If you asked them about these tools, they would likely have much different responses and potentially much different assumptions about what should be assessed and what indicators they need.” Similarly he finds the teacher log to be an invaluable tool that records more completely the emotional and highly complicated task that learning is beyond skill acquisition. His use of the teacher log demonstrates one of the key principles of learner-centered assessment — assessment must feed back to instruction. Trunnel suggests that the notes he keeps are also for his own improvement, what strategies he has used, what has worked, what has not. He even suggests, “Perhaps there is a way it can be used too in teacher-sharing at my center or in a teacher self-evaluation.” Critical self reflection is a hallmark of all these reflective writings, however Trunnel stands alone in his ability to see and then identify the different kinds of checklists, “The checklists that I use are skills based, not strategy based, and thus do not record all the information I want.”

Like Trunnel, Gear in writing about The Learning Log: Evolution of an Assessment Tool was interested in how to create a tool that recorded information that was useful to the teacher and the student alike. Her needs and questions are different but just as important: “How can we get students to look at their own learning? How can we see their learning as a process and evaluation of their progress as part of the process? How can we get the students to measure their own progress rather than rely solely on the instructor? How can we get our students to answer the weekly evaluation questions with more than one sentence? How can we get our students to critique the classes and know it isn’t going to be taken as an insult to the instructor?

Revising tools was a similar concern for Oesch. In Adapting Tools To A New Program, Oesch describes how tools designed for one program were revised or adapted to another at the same agency. Some of you familiar with Martha Germanoski’s article, in the first issue of Adventures in Assessment, will recall her goals checklist and the daily log. These tools have been adapted and revised to assist learners in taking responsibility for their learning, not fix the learner but help the learner build on his/her strengths. Oesch also states that the revision in the EGAP (Educational Goals Assessment Package) led to changes in the curriculum. This is a good example of how a change in the assessment “tail” can wag the curriculum “dog.”

Documenting Community Development Inside and
Outside the Classroom
For Judy Hofer and Pat Larson in Reflecting on the Links Between Literacy Practices and Community Development, the issue of building on learner strengths is a collective endeavor. “We want to set forth right from the start that people coming to an adult education program must take an active role in their own learning, and that working within a group will enhance that process… Getting a GED and improving reading and writing skills barely scratch the surface of what is needed for people to get out of poverty and exercise more control over their lives.” Hofer and Larson are concerned with bringing the larger community into the classroom and the classroom to the larger community and how to create tools that measure not only individual progress but how to assess what people do as a group/community. They have benefited and borrowed from Sondra Stein’s Framework for Assessing Program Quality designed for community-based programs for the Association for Community-based Education. This framework and a view of literacy as practice helped them to generate a list of guiding questions that guided their inquiry. Some of the questions are as follows: How do we assess improvement and change as a continuous process not only for the individual but for the community of individuals? How do we assess if there is movement or change on a continuum in terms of community development by people participating in programs and by the program? How do we develop a language of assessment which takes into account the development of community and collaborative efforts at adult education centers which also speaks to the needs of funders?

The Stance and the Dance
Becoming skillful teachers in learner-centered approaches to assessment is a matter of stance and dance. It is a process that requires us to take a reflective stance on our practice (both past and present), to be clear about what we stand for or are trying to achieve, yet to pose questions rather than seek simple solutions, to be open to engage in a dance of experimentation and revision, and finally to risk finding out not just how our learners experience learning but how we help or fail them in that process. The G.O.A.L.S. Project allowed a handful of practitioners in the state of Massachusetts the resources, time, and money to carefully map their route and compare their travels. The toolkit, according to Trunnel, is the “Atlas, that we hope will help others explore other roads and paths.” To make the trip a little smoother, we have included these reflective writings as a kind of legend to understanding why some roads are paved while others are still dirt roads.

As editor of Adventures in Assessment, I invite you all to contribute your reflections or reactions so that we may learn from each other, discover new roads together.

Top of page

This article was published in Adventures in Assessment, Volume 5 (October 1993), SABES/World Education, Boston, MA, Copyright 1993.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.