Assessment in ESOL

The Haitian Multi-Service Center Experience

Maria Kephallenou
Haitian Multi-Service Center
Dorchester, MA

The Adult Education Program at the Haitian Multi-Service Center (HMSC) has developed a program-wide assessment process that responds to the needs and demands of our program. As part of the on-going development of this process we have found that assessment is an important programmatic issue that continually needs to be addressed. As we develop and refine our process, we ask ourselves numerous questions and address many issues about our processes and assessment in general. We always face new challenges and new perspectives on how to use assessment more efficiently.

Some issues arise time and time again. In a class of 15 students with varying educational backgrounds, individual needs, ages, priorities and language skills, for example, assessment of progress is difficult. Students do not progress at the same rate and the same time. Also, individual skills do not progress at the same rate. How do we account for all these different skills and factors and assess progress? Most likely, teachers begin by focusing on the skills: they compare a starting point — the beginning of the class — with the point of time of the final assessment. The different factors (backgrounds, schooling, age, etc.) are then taken into account to better understand and explain why progress was/was not made.

Another issue deals with the movement of students from one level to the next. Generally, a student is ready to move up when an overall better understanding and use of the language is demonstrated. It includes an initial assessment for the student, a point of time later with another assessment and a comparison between these two. However, “an overall better understanding and use of the language” is very vague. For example, how does one assess it? Is it the same for all students or are there variations? If so, how does one account for them? And how does it tie to the progress of individual language skills? Because teachers think differently, it is amazing that most students move with few problems.

When we talk about student progress at our meetings, the discussion is dynamic with contributions that generate plans to address issues. The discussion becomes difficult, however, when we explore ways to capture all of these dynamics in a form that is simple and easy for students to understand and for other teachers to translate.

Resolving this issue depends largely on one’s position in a program, the purpose and need for assessment, and how it fits into the whole program. Everybody agrees on the importance of assessment for the students, the teachers, and the program, but we all have different perspectives on its purpose.

A Student’s Perspective. Students assess their progress continually as an integral part of learning and as a guide. Assessing progress for them may also imply active participation and responsibility for their education. It is like checking where they are according to their plan and where they need to go to next. An articulated plan always helps as a point of reference. In our program we make efforts to help students articulate a plan through counseling.

A Teacher’s Perspective. Through assessment, teachers can address different issues such as: methods and effectiveness of their teaching; effectiveness of teaching styles with students of different cultural and educational backgrounds; the specific needs and goals of a student; their own beliefs about what constitutes progress; a specific student’s abilities and strengths; and the student’s capabilities and progress versus the capabilities and progress of the rest of the students in class.

An Intake Worker’s Perspective. An intake worker assessing and placing students must have a pretty good understanding of the range of classes, as well as their curricula. Without this understanding, accurate assessment and successful placement of students is extremely difficult. The intake worker seeks to find out whether a particular student shows signs of skills and capabilities usually shown by students who have been placed in a particular class/level. These skills/capabilities are considered as a “starting point,” but only to determine placement and to ensure a smooth immersion of the student in the system. The rest is left in the hands of the counselor and the teachers.

A Counselor’s Perspective. A counselor’s assessment involves working closely with students to help them set realistic goals, set a time frame in which these goals can be achieved, quantify progress in and outside of class (job search, resume, referrals, further education, etc.) and offer appropriate programmatic response to expressed needs: referrals for drop-in babysitting, referral to another department for information and/or concrete assistance, etc. Assessment at this level does not really deal with specific language skills, but rather how mastering language skills is relevant to the overall life, educational, and vocational plans of the students.

An Administrator’s Perspective. An administrator’s view about assessment addresses programmatic concerns, such as class size, outcomes, attendance, terminations (especially if terminations are not job and education related), etc. An administrator may use assessment to (a) measure and assess the effectiveness of the program, (b) capture and document outcomes, (c) project numbers and outcomes for the future, and (d) make the program (curriculum, assessment, teaching methods and materials, etc.) better respond to students’ needs for progress. As an administrator,
I look for confirmation that the program offers its students what we believe it does: the necessary tools to achieve their educational and vocational goals and to move on to social and economic self-sufficiency.

Having been a teacher not too long ago, I still recall the perspective of a teacher. As a teacher, I thought of assessment in a very limited way: it had to do with my students and my class — not necessarily with the whole program — and it was as much about my students as it was about my teaching and the materials I was using.Often, I considered assessment a burden, a task I had to do and once completed nobody would take another look. As a result, I completed the relevant forms but I was neither detailed nor very explanatory. Going back now to some of those assessments I think they were poorly written and would not be of much help to others.

My perspective on assessment as an administrator is a lot broader than that as a teacher; I have a clearer and deeper understanding of the program as a whole and how its various components support and complement each other. I also have a better picture of what the program is expected to do according to our proposals and funders’ expectations. When I review a teacher’s assessment of a student, I expect to see that the teacher knows and understands a student’s progress. The student’s self-assessment and the teacher’s assessment should be close. I expect the student to express some satisfaction with the class, progress made, and his/her teacher. If the teacher identifies problem areas, I expect to see a plan that addresses them.
Through the assessment, too, I have an indication if the teacher’s work is effective. Effective teacher’s work means to me that the teacher has put some thought into developing a curriculum and preparing for his/her class, and knows the strengths and weaknesses of his/her students. Student attendance, drop out and termination rates, and enthusiastic (or not) student comments are also indicative of the class in which active learning takes place.

Another important purpose that assessments fulfill is enabling students to participate in the educational process by assessing their progress. I value their assessments and opinions and take them into account when I assess the effectiveness and work of the program. There is a strong connection between assessment and program functions, such as outcomes, curriculum development, and teaching methods. Just like the students, the program must reflect upon its operations and assess its effectiveness and results. These results should feed back into the program and lead to decisions that will improve the program and benefit its students.

I believe that assessment is a process that does not remain static but evolves together with the program because it is the thread that connects and keeps all the components of a program in place and in check: teaching, curriculum, outcomes, goals and counseling. Unfortunately, many times, due to lack of resources and restrictions of time, we do not utilize assessment as a tool for check, change and improvement to its fullest.

The HMSC Experience

Capturing the dynamics of all assessments and accounting for all different perspectives, purposes and issues may take many creative forms, especially because most funders require that the program incorporate initial, on-going and final assessment of students’ accomplishments, but do not require any particular assessment tool. At the HMSC we have developed in-house assessment tools to capture the information we need and answer the questions we raise. They include intake/placement; counseling: orientation and goal-setting; student/teacher conferences: initial, on-
going and final; and exit/entrance criteria checklist.

Intake/Placement Assessment
The Intake worker determines the level/class in which students should be placed using the placement test. The placement test was developed by the program with the assistance of Loren McGrail through a SABES mini-grant and includes: (a) applications, (b) an oral interview to determine the students’ speaking/listening capabilities (at this point initial assessment for non-literate students also takes place), (c) reading materials to determine the students’ reading capabilities (reading materials include materials for different levels and a variety of topics; the test includes comprehension questions, writing the story in students’ words and/or responding to the ideas expressed), (d) writing, which includes either responding to the reading or writing about students’ experiences, and (e) a grammar test may be given to students depending on their level (see “The Toolkit for Authentic Assessment” from SABES for a detailed description and relevant forms).

This assessment, although not the only one developed, may be unique and exceptionally successful in placing students: students choose to work with the reading(s) they like or think are difficult or easy enough for them. The reading and writing materials are culturally sensitive (all are pieces of writing written by students at the HMSC) and so offer the students a familiar frame of reference. The students also have the freedom to choose the topic they want to write about. During the time they take the test, they are supported, guided, and helped by a bilingual intake worker. This intake/placement is time consuming, taking an average of 50 minutes to two hours. The intake worker needs approximately 15 to 30 minutes to read through and assess the level.

Counseling
The second assessment comes after the level of students is determined and before they enter class. The Program Counselor meets with students individually for approximately one hour to address two issues: program orientation and goal-setting.

Program orientation. This includes necessary information about other services that the HMSC offers, support services (e.g., drop-in day care, referral to other agency departments for services), information about elective classes that the Adult Education/ESOL Program offers, program policies, and specific information about the class the student will be placed in (times, where the class meets, and who the teacher is).

With the orientation, students feel they are part of a bigger agency with additional available services. This initial meeting introduces the role of the counselor to provide support to the teachers outside of the classroom and provide needed information and referrals to students to help them
accomplish their goals.

Goal setting. The counselor completes the Educational/Vocational Plan with the students. This document includes background educational and employment information about the students and includes their long- and short-term goals, as well as personal goals (this document has been updated to include information needed for the DOE MIS system).

More often than not, students find it very hard to set goals. It is especially hard for low-level students to articulate specific goals besides “to learn English.” The counselor is instrumental in engaging them in a discussion on why they need to be in an ESOL class (or in any other class for that matter) and he helps them to articulate goals. The counselor has to lead this discussion carefully (a balance of articulating real needs and imposing them) because the students should set their own goals, not the counselor. The skills to set short- and long-term goals take a long time to develop as they include certain language that the counselor — and the student — must master.

Student/Teacher Conferences
In these conferences, progress and evaluation are discussed by both parties. The conferences may take many forms depending on the level of the students (lower level classes may have more group-like meetings) and time. Ideally, they are held three times each session. The student/teacher conferences consist of three parts: initial, on-going, and final.

Initial. The first conference takes place within the first two weeks of the students’ placement in a class (either right after intake/placement or after movement from another class). The relevant document is completed by the students and identifies their specific goals for the session. The difference between this goal-setting and the goal-setting with the counselor is that the students think of goals that are achievable in a short period of time — the teaching session. These goals tend to be geared more around what they perceive as weaknesses and needs.

On-going. The second conference occurs around the middle of the session, or/and any time that is deemed necessary or useful for the teacher or the students. The document has two parts, one that is filled out by the students (their evaluation of their progress and goal achievement) and one by the teacher (the student’s evaluation of progress). Often, the students talk about possible problems with their learning, with teaching and materials, etc. These problems are addressed by the teacher and possible solutions are explored.

Final. The last conference takes place during the final week of each session. It, too, has two parts: one that is filled out by the students and one by the teacher. The students are asked to evaluate themselves in terms of progress and goals accomplished or not; give examples of their progress; reflect on whether they have different or additional goals after attending the class; and evaluate the program’s help and support in accomplishing these goals. The teacher is asked to provide a formal evaluation of the students’ progress in terms of each of the applicable basic skills (listening, speaking, reading, writing, computer literacy, math) and to provide an evaluation of students’ attendance throughout the session.

The purpose of this assessment is to provoke discussion, thinking, and evaluation and become a learning experience for both parties. The teacher’s input and opinion is as valued as that of the students’. Through the discussion, the students become as responsible for their learning as the teacher is for his/her teaching. This assessment, too, is time consuming. It is discouraging, however, that the forms often do not capture all the rich discussion that they may invoke either because the teachers do not write enough/well or the students view their part as not very important or both. It is very frustrating to read assessments of progress that are poorly written and that do not give a clear idea of where the students stand in terms of learning and progress. They do, however, add to the
incredible amount of paperwork accumulated through the years.

Exit/Entrance Criteria Checklist
Student/teacher conferences may involve the use and completion of the exit/entrance criteria checklist when a student is ready to move from one level to the next one. As its name suggests, a checklist summarizes the capabilities/skills mastered in a class by students, and, at the same time, summarizes the capabilities/skills needed for those students to advance to the next class. (There are as many checklists as there are classes.) The checklist is completed by the teacher.

A checklist can, in a short time, quickly display the most important aspects of teaching at one level and the capabilities of the students and, at the same time, can reveal the capabilities of the students as a starting point for the more advanced level. These checklists are curriculum-driven as they include skills that are taught/required in each level. Checklists by themselves are weak evaluative tools. In reality, a checklist is only one element taken into account when evaluating students before they move to a more advanced level (samples of their work, especially writing, and the initial in-class assessment by the new teacher ensure the right decision).

Despite the problems mentioned (time consuming, some resistance on every-body’s part to keep up with all this paperwork that results in very poorly written assessments, etc.), assessment is a very important aspect of a program, a crucial tool for evaluation and self-evaluation, and an important part of teaching because it touches all aspects of a program and can be used to confirm and validate findings about the program, its services, and its
students.

[Author’s note: “I wanted to acknowledge Alison Simmons for her assistance in the thinking and writing of this article.”]

Originally published in Adventures in Assessment, Volume 10 (December 1997),
SABES/World Education, Boston, MA, Copyright 1997.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.