Assessment in the ESOL Experience

Elizabeth Santiago
SABES Central Resource Center / World Education

“We have what I call a chat with very basic interview questions to evaluate the student’s oral and verbal capabilities. Now we have a mandatory test we have to give from the state. With the new test, I sit down and go through each question. Sometimes, if I speak their language, then I help. But fifty percent of the time they come with an interpreter. Sometimes I will ask to call a family member. Sometimes I am dry. I have nothing.” Cecil Demorney, Community Learning Center, Cambridge.

Traditionally, ESOL (English to Speakers of Other Languages) teachers have had to create original assessment tools to place low-level English language learners into their classrooms. Often times, standardized tests can be inappropriate for ESOL learners. Certain funding sources, however, require scores derived from tests like the BEST (Basic English Skills Test) for funding consideration and some funders ask for other documentation showing how learners have progressed. Teachers have had to “dance” with what is required (or what they perceive is required) by the funders and what they feel will help them place and move students more accurately.

How does an ESOL professional utilize what is available to them when placing beginning level students into their classrooms? How about when they are transitioning ESOL students into a higher level class, working with a multi-level class, and/or documenting each learner’s progress for multiple funding sources?

In August and September of 1996, Alison Simmons and I visited four programs in Massachusetts: Hamden County House of Correction in Hamden, Lutheran Social Service in Springfield, International Institute in Boston, and the Community Learning Center in Cambridge. In those diverse settings, we talked to six ESOL professionals about how they assess low-level ESOL students and what concerns they have within that context.

BACKGROUND
ESOL services for adults have been gaining national attention in recent years mostly because more and more adults have enrolled in ESOL classes and waiting lists for services are growing. Recent con-troversies surrounding immigrants and English-only legislation have put a spotlight on ESOL learners. Even with the current media coverage of English language learners, there is little about ESOL assessment being documented, published, and filtered into the classroom. National organizations like TESOL (Teachers of English to Speakers of Other Languages) and individual states and programs around the country have been researching and piloting assessment tools, but it is not as well known or widespread as it should be. The Meeting Summary on Current Research and Theory on Effective ESL Instruction put out by the U.S. Department of Education says “The issue is not so much that outcomes do not exist, but that the ESL field has varying objectives for instruction and does not know how to measure outcomes across groups” (page 10).

In a discussion I had with the editor of Adventures in Assessment, I expressed the confusion and frustration I was feeling in trying to assess a young man that I had recently begun tutoring. At our first session I wanted to get a sense where I should begin instruction and what he expected from me. I knew that he had just arrived in this country and spoke little if any English. I went to the SABES library at the Central Resource Center and found a book on low-level ESOL assessment. The book used pictures and leading questions to assess. I decided to try this approach with my learner.
I thought the first few pictures might be too simple for him but I decided to follow the format of the book and asked him the word for each object in English. After three objects, he turned to a friend and interpreter and asked him to tell me in English that he found the questions too easy. What was I to do with the realization he was unable to express to me in English his concerns, but insisted what I was asking him was too easy? I learned that my issue was one of many with which classroom teachers have to deal on a daily basis.

LIMITATIONS OF AVAILABLE TESTS
In ESL and the American Dream, authors Chisman, Wrigley, and Ewen state, “There are no standardized tests on the market that can adequately measure the proficiency of the great diversity of adults who need ESL service. This is no surprise given the complexity of language skills and the range of abilities that adult learners demonstrate. Yet funders continue to insist that standardized assessments be used to judge learner progress, and may use such measures of program effectiveness as well. This is true in spite of the fact that the most commonly used standardized tests have severe limitations” (page 58).

What are some tests currently in use by ESOL teachers and what are their limitations? There are both ABE (Adult Basic Education) and ESOL tests. ABE tests include: Test of Adult Basic Education (TABE), the Adult Basic Learning Examination (ABLE), and the Comprehensive Adult Student Assessment System (CASAS). These tests may be used with ESOL learners, but have obvious limitations because they were not designed for that population. There are tests specifically designed for the ESOL learner which may or may not be more appropriate. The most popular of these tests include the BEST test, the Comprehensive English Language Skills Assessment (CELSA), and the Test of English as a Foreign Language (TOEFL).

The BEST test assesses a learner’s basic survival skills such as listening comprehension, conversational ability, and literacy for placement purposes. In many settings, the BEST test is required by funders as a means for assessing learners on an ongoing basis as well as for initial and final assessment. In my discussion with the six teachers, each agreed the BEST was one of the better initial placement tools for their students.

On the other hand, teachers complained that students were able to memorize the items when the test is re-administered, therefore making it difficult to tell what knowledge a learner has actually acquired. Another criticism of using the test as an ongoing assessment tool is the idea that you have to teach to the test. “Sometimes if you don’t teach to the test, the students aren’t going to pass and move on to the next level,” said Lola Reid at Lutheran Social Settlement Services.

The CELSA assesses how well an English learner can deal with written English tests representing varying levels of difficulty. “The CELSA is useful inasmuch as it provides some indication on the lan-guage proficiency level of ESL students… however, it does not tell programs and funders if students can use English to achieve their own purposes in a variety of situations. Therefore, it has only limited use as an ESL proficiency assessment,” wrote Chisman, Wrigley, Ewen in ESL and the American Dream.

The TOEFL is the most comprehensive of the available standardized ESOL tests. TOEFL scores are used for admission and placement into thousands of colleges and universities across the nation. But the test’s purpose is narrow, as suggested above, and not broad enough to use with all of our learners.

To organize my conversations with the six ESOL teachers, I categorized the discussions following the three stages of assessment. The categories are initial, ongoing, and final assessment.

INITIAL ASSESSMENT
“I use the Oral BEST test. If they pass, I move to the written BEST. I also ask lots of questions, but nothing personal. I will not ask what they have done to get here. I don’t need to know that information. I also use Scrabble letters to make words in their native language. If they use an alphabet which is different than the English language, I will use pictures to assess. If their skills are very limited, I will try to interpret myself or ask another student to interpret.” Eileen Witkop, Hamden County House of Correction

“I have an informal oral interview so I can gauge their level. After that, there is an application tool with ten sequenced per-sonal questions. I see how they respond and it helps me select which reading sample to give them. I have them read the sample silently and then out loud. There are three or four comprehension questions. Then there is a writing sample. I ask them to write about their family or something else familiar. People usually get nervous about spelling so I tell them up front that’s not what I’m looking for now, but how they can write their ideas. We also have an assessment of basic math skills, but applicants often opt out of this saying they want reading and writing.” Leona Breslow, Inter-national Institute

“If I ask their birth date, for example, I will sing, ‘happy birthday to you, happy birthday to you...’ Most people know the tune so they give me the answer.” Cecil Demourney

Many teachers have various initial assessment tools they use with their students, but there is not one tool which all the teachers follow. Usually, if a teacher uses a particular test to satisfy their funders, they also come up with some program-based alternative form of assessment to use for themselves. “Our students come in at all levels from 0-above. The BEST test is used to determine the initial level of a student. The numbers generated from the BEST test are the numbers acceptable by the funding sources. We also have an informal assessment of our own once we place the students into our classrooms. We’ll list the activities that we did at the end of a class. Students vote on what they like best. This helps them think about their own learning and where it is more helpful to them: reading or writing. Low-level is more tricky. They say that was good. They tend not to be too critical. If you ask them what they want to do, they look at you like, ‘this is ridiculous, aren’t you the teacher?’” Lola Reid, Lutheran Social Services Settlement

The tests also raise issues of inappropriateness where the tests are concerned. “Our students are mostly given the TABE test, but the timing is extremely bad. They are given the test, it’s corrected and put on file. We can’t really go by that test because it is given at the worst time. It is usually given right after someone enters the facility and at that point they are usually
in a state of shock.” Eileen Witkop

Eileen’s sentiment may, at first glance, seem specific to incarcerated individuals. However, ESOL teachers at other settings have similar concerns when dealing with issues of trauma and shock. Some ESOL learners are coming from war-torn countries and may have lost family members or have endured other tragedies. Those issues may arise in all ESOL settings.
Another issue in assessing students initially and getting an accurate measure
of their levels is the reality of waiting lists. Some programs test students when they sign up for classes, then place them on a waiting list until a spot opens up. This approach also has problems. “Waiting lists make it difficult to determine their initial level. Because they have been waiting for so long, their English gets better. Sometimes we have to make changes. What we decided to do was network with other community organizations in Cambridge and help them set up English classes. Waiting lists are shorter which is great.” Cecil Demourney

Initial assessment is by far one of the most difficult areas for most ESOL teachers, but it has been one of the areas that has garnered more documentation (see previous volumes of Adventures in Assessment). There are also many variables associated with ongoing assessment.

Top of page

ONGOING ASSESSMENT
Each teacher with whom I spoke had more ideas for ongoing assessment, but there were equally as many concerns.

“For my low-level students, I try and teach them survival skills. I take them to places like the supermarket and point out fruits and vegetables, show them how to shop, what the numbers and signs are. I show them how to look for an apartment, learn how to read the abbreviations. I talk about how to take the subway and how to deal with getting lost. We also talk about body parts and medical care. It’s easier to gauge this type of knowledge through everyday observation, homework, participation in class and what the teacher perceives.” Cecil Demourney

The teacher’s perception arose over and over. How do you know someone is making progress?

“That has always been a question. How much learning time should I spend on assessment? Informal assessment is always in your head. One thing that has been positive for me is after a few weeks when they have a little more vocabulary, I show them a picture. They learned the who, what, when, etc. prior to this. I’ll ask them, ‘who is this person, what are they doing,’ etc. I ask them to talk about it, but not to write it. At the break, I write the story they discussed and give it back to them with blanks. I would ask them more questions. I mention to them to look at all the English they learned. It’s very validating.” Lola Reid

“A lot of it is work that they do, teacher observations, class work, informal interviews, and end-of-the-cycle interviews. I will reassess students who are having trouble. We do see some students with learning issues so progress can often be slow.” Leona Breslow

“For lower levels, I list the activities we have just finished on the board and use a happy face, a neutral face, and a frowning face as the determinant of each activity. They would vote on which they preferred. After a while, I said to myself why am I doing this? After that, the students went to the board and rotated the assessment. Then I tried it individually at their desks, but this was for higher levels. The lower levels always did it on the board. Then I tried an assessment sheet because I felt the exercise was taking up too much time in a predictable way. I would give them this sheet and they would bring it back at the end of the week. People didn’t like it, in fact they hated it. So I redid it to show their changes. I still think they don’t like it, but they like this one better.

It’s helpful for me also to figure out what to do in class. Because of the new assessment sheet, we had a conversation about what constitutes reading, mainly because there is a question that asks what have you read this week. They would come back and say, ‘I didn’t read,’ but they would have read an ad, the Yellow Pages, cooking instructions, etc. It’s very easy to tell, as a teacher, whether somebody gets it or doesn’t get it. It’s hard to say what I do exactly, but I just know.” Rebecca Schiffrin, Lutheran Settlement Services

Teachers couldn’t always pinpoint exactly how they know a learner has made progress. Each teacher was also at different levels with documentation. Some used portfolios to document learner’s progress, some kept informal notebooks, and many talked about goal setting sheets they had developed. “I use portfolio assessment and goal setting sheets to tell if my students have progressed. I revisit the goals list with the students. The students get so excited when they can check something off the list. Whether it is the achievement of writing their name in cursive or watching
a TV show in English, it doesn’t matter because they always feel a sense of power when they check off something. I also try to establish if there is a learning disability present, but that is often difficult to determine. I am building a kit that is appropriate to ESL learners which might help me see if there is a learning disability.” Eileen Witkop

“I started doing some portfolio assessment. I would put job applications and other accomplishments in it. It doesn’t look the same for each student because of absences. Sometimes I sit with students and go over what they have done over the last few months.” Barbara Lippel Paul, Lutheran Social Service, Springfield

Some teachers use technology in their on-going assessment tools. “Upon completion of a basic skills/computer class, we did on-line classroom assessment. I prepared a disk for each student. On each disk was a written evaluation the student would respond to. I would have leading questions at the bottom (see attached tool). The students loved it and you can see learning styles come out behind the computer. It was also a learning experience for me and I was able to modify as I went along. I let them go at their own pace. Computer class was directly affected by what went on in class. I keep a folder with a written evaluation. I also put in writing assignments and other evaluation materials.” Leona Breslow

The pattern in these interviews seemed to be that there isn’t one way to do assessment and there is a level of intuition on the part of the teacher. Barbara Lippel Paul stated, “You’re doing ongoing assessment all along in your class. Everything is to get them out the door to the next level, or job, etc. We are constantly assessing.” The next section will discuss that very thing: How do you know when a student is ready for the next level or job or whatever?

FINAL ASSESSMENT
“I always hear when a student wants to go to a lower level, but rarely will a student tell me they want to go to a higher level. When I suspect someone can go into a higher level, I tell them and they are rarely surprised. They sort of already know they are ready to move on.” Leona Breslow

Knowing when a student is ready to move to another level, whatever that level may be, takes on many forms. Some teachers have informal interviews with each student at the end of a class cycle while some mentioned checklists.

“Some students like to stay in the same level because they know everyone. Some others want to move when they are not ready. Now, we are trying to work harder on more assessment tools. It’s challenging. We have a curriculum for each level, but we don’t follow it blindly. We do what’s best for the group. We make sure the students master some grammar because we base judgement of level changes on grammar. The problem that comes up is different levels. It’s always a multi-level class. Some may be better in writing than in reading. You try to take that into consideration when you are planning your lesson. You try to work on the primary skills. Sometimes you have a student with real literacy problems so you have to be more creative. At the end of a cycle, we are asked to send a list to the counselors of people who are ready to move. It’s tough though. For ABE there are standardized tests. For ESL there really isn’t anything.” Cecil Demourney

Cecil’s sentiment sums up a frustration these ESOL professionals continue to feel. What can we make of all this? Well, Barbara, Lola, Rebecca, Eileen, Cecil, and Leona shared with us their thoughts and ideas for assessing their low and intermediate ESOL students, but there should be more documented, researched, and published on this very real issue.

WHAT’S GOING ON?
Adventures in Assessment certainly has a commitment to exploring all forms of alternative/authentic assessment. What other resources do we have available and what is being done to research this topic? As you can see from the teacher’s work, a lot of thought and hours have gone into assessing ESOL learners. Unfortunately, there aren’t many vehicles for teachers to share what they’ve done with each other.

Or worse yet, teachers don’t feel as if they are doing anything special. Almost all the teachers I spoke with said something like “well, I’m sure everyone is doing this...” Sometimes what may seem obvious or simple to one may spark a whole new approach to assessment in someone else.
For example, this experience has helped me with my learner. For one, I decided to have my learner guide me. If he found something too easy, I pushed him to somehow communicate to me why he felt that way and what I could do to make my instruction more satisfying to his needs. It turns out that he was an advanced writer and reader of English, but never really
had the opportunity to speak English in his native country. Once this was established, I was able to work on his oral abilities. I also realized that the interpreter was my friend, so to speak. I learned many valuable insights from the interpreter although he was not at every session. He came every now and again at the beginning of our sessions. We need to share more of what we do with each other.

Adventures in Assessment is a good place for that, as is the Massachusetts newsletter Bright Ideas or other state newsletters, but we should definitely have other outlets. Lenore Balliro of the Adult Literacy Resource Institute in Boston is starting an ESOL listserv on e-mail for Massachusetts ESOL teachers. That should be a great forum for expressing ideas about assessment. Adventures in Assessment will be printing another journal midsummer of next year; maybe there could be more articles published on ESOL assessment in the upcoming issue.

There also needs to be more research what learners need to be taught and what are the most effective ways to do that. Along with that, we need more documentation on how variables like shock, trauma, and native language literacy (among others) influence assessment practices.A Center for Applied Linguistics study found that children who had literacy in any language fared better in acquiring another language than those who have no literacy (Meeting Summary on Current Research and Theory on Effective ESL Instruction, issued by the U.S. Department of Education, page 8). Studies like this should be made available to ESOL classrooms.
On a national level, the TESOL conference is a good place to share ideas and find workshops on ESOL assessment as well as individual state adult education conferences. In Massachusetts, conferences like MATSOL and Network '97 may have ESOL assessment components embedded.

Top of page

Looking at Mandates

Funders I talked with do not mandate a particular assessment test. Some teachers have the perception, however, that funders are mandating the BEST or TABE for example.

Why? I discovered that the wording of documentation requirements can sometimes make the intent confusing. A funder, for example, may require a student SPL level when they enter a program and when they exit or move up, but not specify a test. One of the most reliable and popular ways to arrive at SPL levels is the BEST. Therefore, a teacher may translate a requirement of an SPL level as being a mandate for using the BEST.

Sometimes, too, programs write proposals including an assessment tool and as part of the program design. What is often related to the teachers administering the tests is not the funder’s requirement, but what the program said it would do.

There are funders who require specific tests. “The Massachusetts Department of Education requires that every program funded to provide ABE services (including basic literacy through adult secondary education and English for speakers of other languages) incorporate an initial an ongoing assessment of each student’s skills, capabilities, and goals. The Department expects such assessments to be appropriate, reliable, and valid measures. The Department does not, however, require programs to use any particular assessment instruments. Programs may choose from a wide variety of standardized, norm or criterion-referenced1 commercial tests and/or use center-developed assessments, including those referred to as “alternative assessments”, structures, portfolios, etc. Our goal is to ensure that students are appropriately placed and that their progress is appropriately documented. Assessments that enable our students to become partners in managing their own learning are particularly encouraged.” (Robert Bickerton, ABE State Director, MA)

The Industrial Service Program in Massachusetts requires Worker Assistance Centers to administer the TABE survey or the Job Corps Reading Screen with the WRAT 3. The Department of Labor (original source of funding) requires that there be a reading level score and a math score, not an equivalent, for each of the people enrolled in their programs. They offer a list of options to arrive at these scores. In Massachusetts the ISP program narrowed down the list to conduct adequate training on these tools. Some programs may use other tests like the ABLE. ORI (Office for Refugees and immigrants) requires SPL levels for entry and exit but does not require its programs to use the BEST. If a specific test is not required, and it is an SPL level or a grade level equivalent, then we need to explore other valid and reliable ways to achieve the same outcome.

1 Norm-referenced tests measure how well learners perform in relation to a norming group. Learners are tested to determine how their skills/scores compare with those of others who have taken the test, not how well they have mastered certain knowledge or skills. Criterion-referenced tests measure how well learners perform in relation to some absolute standard. Learners are tested to determine how well they mastered certain knowledge or skills, not how they performed in relation to a norming group or to others in the class. ( Testing and Assessment in Adult Basic Education and English as a Second Language Programs)

Originally published in Adventures in Assessment, Volume 9 (December 1996),
SABES/World Education, Boston, MA, Copyright 1996.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.