Voices from the Field:

The Basic English Skills Test

In this section, we look at various voices from the field around the much-used BEST. Although Adventures is a journal about learner-centered approaches to assessment and evaluation, it is time to look more deeply at other assessment tools" that we all use. This section is not intended to encourage or discourage the use of the BEST, but to ask practitioners in the field how they view the test from the historical, teacher, and volunteer perspectives. We should all reflect on the tools we use to see if they fit with our programs and our learners.

| The History of the BEST | Why I Think the Best Isn't Good Enough |
| The BEST is Workable, But It's Not the Only Choice |
| An AmeriCorps Volunteer's First Impressions of the BEST |

The History of the BEST

Moria Lucey
International Institute, Boston, MA

The history of the BEST is an interesting one. It began in the early 1980s when the Federal Office of Refugee Resettlement (ORR) saw a need to more effectively move refugees from welfare to economic self sufficiency through the "provision of a coordinated and structured English Language Training Program." In order to make this happen, they invested in a national initiative called the Mainstream English Language Training (MELT) Project which brought together ESL professionals from throughout the U.S. The goal of this initiative was to develop and design not only a test, but curriculum guidelines and Student Performance Level (SPL) definitions.

The International Institute of Boston was one of the contributors to the MELT Project. As part of the Project, the BEST (Basic English Skills Test) was developed and field tested in a number of locations, including Boston. This test was designed to assess "elementary" listening, speaking, reading and writing skills reflecting a competency based approach to language teaching and assessment. The test could be reliably used if an individual had low literacy skills. The oral test could be used by itself if an individual did not have the literacy skills needed for the written component.

The BEST is life skills- and task-based, evaluating a student's ability to use English in real life situations. It was thought to be a tool that could provide useful information in determining an initial class placement or determining progress in some specific functional areas such as telling time or counting money. It was never intended to test general language proficiency or to be especially useful in programs that did not use a survival life skills-based curriculum.

Much of the ESL being offered in the early 1980s was not life skills-based and did little to introduce learners to the world of work. Often the emphasis was grammar rather than functional language skills. Even more important to teachers, there were no resources for working with adults who were non-literate or had low native language literacy skills.

Likewise ESL assessment tools tended to be more academic with an emphasis on testing grammar. Very few tests could effectively assess learners with low literacy and/or little or no knowledge of English. As a result, many of the materials being used in the field were either inappropriate or were being generated by teachers with few guidelines or standards.

ORR, together with ESL professionals from many parts of the United States, decided there was a need to develop some common tools and common language for the Refugee ESL programs which were overwhelmed by the large numbers of adults arriving each month needing both English and the skills to get jobs.

Another interesting fact about the MELT Project is its relationship to the oversees refugee camp programs. In the late 1970s and early 1980s following the Cambodian genocide and the aftermath of the Vietnam war, large number of refugees were filling camps in countries throughout Southeast Asia. Many of them were applying to emigrate to the United States. The Department of State funded processing centers in a number of sites in both Southeast Asia and Africa where ESL and orientation prior to arrival to the U.S. was required for all adults accepted into the U.S. refugee program.

Close to 35,000 people graduated from these programs each year. In places like Galang, Indonesia, Bataan, Philippines and Phanat Nikon, Thailand, programs were set up and intensive efforts were made to develop common curriculum, common assessment processes and uniform class level definitions. As a result, the curriculum development, staff training and resources produced in these overseas programs was impressive.

Even more impressive was that the work done in these refugee camps was coordinated with the MELT material developed stateside. For example, when a refugee arrived in the U.S., his or her class level was stamped on the I-94 card giving stateside ESL programs an indication of a learner's SPL upon completion of the overseas program. Since the BEST was correlated with the SPL, it was possible to verify an SPL using the BEST and more easily place a student into a class. Overseas ESL and Cultural Orientation curriculum was shared with teachers in the U.S so teachers often knew what material students had covered in the camps; the MELT curriculum was designed to build on what was covered in the camps.

Now, almost 12 years later, the BEST and other MELT material have gone well beyond ESL programs targeting incoming refugees. As accountability and outcomes measurement become increasingly important, many funders have adopted the SPL and the BEST as required tools for ESL programs. Obviously the framework and "common language" that these tools provide give funders a better ability to quantify progress.

At a program level, however, assessment has to carefully consider both learner and program goals. So while the BEST is often thought to be one of the better tests, especially for learners at the beginning levels, it is not adequate for all programs and learners. The test was developed as an alternative to traditional paper and pencil tests, with an emphasis on assessing very basic life skills. If a program is not working with a basic survival life skills focus, this test may not be particularly useful when determining initial class placement or when measuring progress. If the emphasis is prevocational, for example, more specific prevocational skills may need to be measured rather than survival skills. In addition, for learners who have lived in the U.S for a long time, this test may not be appropriate. Tasks like telling time, counting money, writing checks or circling dates on a calendar may simply not be challenging enough.

Programs often use the BEST in conjunction with other tools. This enables teachers to individualize testing by drawing on a variety of tools, not just one test. Testing needs to match both the learner and where he/she is coming from culturally, linguistically, and academically and what the program needs to know to adequately address the learning needs of the student. The BEST by itself may or may not satisfy all of this.

Top of Page

Why I Think the BEST Isn't Good Enough

Dulaney Alexander
Operation Bootstrap, Lynn, MA

According to its documentation, the BEST was designed for determining classroom placement, for assessing individual or class progress, and diagnosing "language-use tasks" (like telling time) that need teaching and remediation.

As a placement tool, the BEST attempts to cover a lot of ground, including not only grammar and vocabulary but also cultural familiarity and the American idiom. Were all prospective students nearly alike in background, the BEST might be more useful. But consider two possibilities: a Russian immigrant with post-secondary education (including formal English studies) who arrived in the U.S. two days ago; and an immigrant from the Dominican Republic who completed only primary school and has lived in the US for twelve years, all the while working on an assembly line with little opportunity for contact with native speakers.

The Russian is likely to miss all the money questions, never having seen American currency before the test; most of the other questions will be incomprehensible because the examiner's American accent is so unlike the British accent the Russian encountered in previous studies. Yet, this person may have considerable vocabulary and exposure to written English, not to mention the benefit of having already studied English.

Meanwhile, the factory worker has been shopping, watching television, and seeing her children through school. She has learned to speak enough pidgin English to get through day-to-day language encounters, as well as several questions on this test.

Consider the possible core evaluation scores for these two students. The newlyarrived Russian is so mystified by the American accent that she can understand few of the questions, although she eagerly names objects in each picture of the test. She scores eight points, although with a couple of trips to the grocery store she might have gotten another four or five points for the shopping portion of the test. Her core evaluation results: SPL 0 (Beginning Literacy ESOL).

The second student's performance reflects cultural familiarity and a lot of (fossilized and incorrect) language. She gets five points for the shopping portion of the test, she does well on the fluency questions (where volume of production is measured without penalty for grammatical inaccuracy), earning 13 more points. These 18 points place her at SPL 2 (Beginning ESOL). Both students are in some sense beginners, but their language needs are very different. The difference is far more complex than 10 points worth.

As a student gains more language, the scoring of the BEST test becomes more punishing. The listening vocabulary needed to earn the nine "listening comprehension" points is not difficult, although an inability to follow a map may cost even an advanced student two of these points. Eight questions are scored for fluency. The three points for a "fluency question" are accessible to a student who has only a minimal degree of grammatical control but who is expressive and uninhibited by a concern for perfection; a reticent personality can easily diminish the score of a more capable student. But the 24 questions on Form B that are scored for grammatical and idiomatic accuracy are truly problematic for the ESOL student.

Consider the question "What is she doing?" accompanied by a picture of a woman watching TV. "She watch TV," "Her watching TV," and "She watching TV" all get scored as a one-point answer (correct information in a response that would not be used by a native speaker), while "Watching TV" is a two-point answer. For these "communication questions" there is a one-point risk in attempting anything more sophisticated than the phrasal response "watching TV." A two-point answer requires subject-verb agreement, verb tense, articles, prepositions and word choice being grammatically and idiomatically correct, too. For a student who is in a class that encourages communication over grammatical precision, the BEST changes the rules by awarding more points when the student gives an answer terse enough that the inevitable slip of article or preposition never occurs. The BEST fails to capture how well the student has mastered any of the pitfalls that lie between "watch TV" and "she's watching TV."

The BEST's third goal -- as a diagnostic tool -- is only somewhat better met. The BEST can point out which of the several areas of "survival" vocabulary (and culture) are needed by a student. However, given its all-or-nothing approach to grammatical accuracy, it is of no use in determining the structural issues of English that need to be addressed.

The MELT curricula framework, of which the BEST was the final piece, defined the task of the ESOL class as being tied to "language use tasks" in the American culture, necessary for functioning in the US. It is a useful tool for the ESOL teacher. The long form of the BEST is a cumbersome attempt to quantify language skills that fails to account for the diversity of our students' backgrounds and for the immensity of the language learning task. An ESOL program would do well to consider other tools for placement, progress, and diagnosis of the language learners' needs.

Top of Page

The BEST is Workable, But It's Not the Only Choice

Barbara Lippel-Paul
Lutheran Social Services, Worcester, MA

I began using the BEST in the early 1980s when I was an ESL coordinator for the Refugee Employment and Education Program (REEP) in Massachusetts. I used it to help assess the oral English language skills of the newly-arriving adult refugees.

At that time, there was only one version each of the core section and the literacy skills section. The program was a competency-based program which taught survival skills and pre-vocational skills, and the BEST tested these areas. It provided a baseline score which could be compared to all learners for placement purposes. When used as an ongoing assessment tool, it tested mastery of the materials, showing the need for more practice, movement to a higher level or, ultimately, ability to handle employment.

When used in this manner, the BEST was quite effective, but its biggest drawback was that it took too long to administer. The core section had to be administered to each student individually, which often took half an hour. Only several years later did a short version appear which could be given in ten to fifteen minutes. The literacy skills section was still rarely used because it took at least an hour and the program emphasized aural/oral skills.

In the late 1980s, refugee programs were funded through local Service Delivery Areas (SDAs) which had a central assessment unit that administered the BEST to all program participants. Individuals who never used the BEST or had no experience testing limited English-proficient clients were now responsible for program referral and placement. I was asked by the local SDA to train their assessment staff in the use of the BEST. However, since this subjective test was open to the opinion of the tester, the scores often made no sense. Students who could repeat their personal information, tell time, and identify money would score higher than they really were in overall English Language Skills. Conversely, students soon learned that high scores could prevent them from attending ESL classes, so they often would not answer questions in order to score low. As a result, I had to develop an in-house assessment tool which would be more reliable for placement purposes.

Most recently I was still associated with a refugee education program, and assessment has gone back to the programs. The BEST, in its long form, is still the chosen tool of assessment. It is used for placement purposes, but it is also used as the only ongoing assessment tool to identify student achievement. The same version of the test was administered every ten weeks (until recently the test was given every five weeks).

Since the success of the program is linked to the increase in the number of Student Performance Levels (SPLs) , the curriculum is centered around the basic skills tested in the test. When a student reaches an SPL level 6, he is no longer eligible for English classes. This has led to many questions about the validity of the test in my mind. While participating in the program, a student could take the same test six or more times. Also, many people overlook the fact that this is an oral interview for assessing basic life skills. If you have been in this country for a year or more, you most likely will be able to answer questions about survival topics but do you really have enough language skills to successfully obtain employment or advance in your position?

I do understand and support an "early to work" philosophy for limited English speakers, but I feel that students should be allowed to continue their education to achieve advancement. The BEST is a good assessment tool for placement, but it should not be the only tool used to show off individual achievement or program effectiveness. In the late 1980s when the Federal government passed legislation calling for standardized tests to evaluate ABE and ESL programs, there was much debate about the purpose of assessment and interest in the field for alternative assessment tools. The conclusions made were that assessment should focus on the needs of the students and measure how we are helping students meet their goals. It is an ongoing partnership between the program and the student.

Assessment can take many forms, including for example, student portfolios, teacher observation, student feedback, and student participation in discussions, simulations, and demonstrations.

Top of Page

An AmeriCorps Volunteer's First Impressions of the BEST

Rachael Donnelly
International Language Institute, Northampton, MA

I have been an AmeriCorps*VISTA volunteer at the International Language Institute of Massachusetts in Northampton, MA for three months. One of the most exciting parts of my assignment is learning more about the field of ESL education.

I was recently trained by another VISTA volunteer at my site to administer BESTs to incoming and current students. The training procedure seemed quite lax to me: my training involved reading the BEST booklet to become familiar with the questions, observing several testings, scoring the observed tests, and then comparing my scores with those of the tester. I noticed that my trainer skipped or reworded some questions that were in the test booklet. Upon asking about this, my trainer explained to me that those questions were "worded too hard" and that the woman who trained her "didn't like" certain questions and so skipped them or worded them differently.

After several observations, I was deemed ready to begin testing students myself.

Two groups of people in our school take the BEST. Incoming ESL students take the BEST to determine in which class level they belong. Students already in our program take the test to determine their progress and, if progress has been made, to reassign classes.

It seems more natural to administer the BEST to incoming students. Students who have never met me or other members of the staff don't think it strange if I ask them their name, where they are from, or what their native language is. But it sometimes strikes current students as strange to be asked questions we already know the answer to and have on record. At best, the student realizes this is part of the test, and plays along. At worst, the student does not take the tester or the test seriously.

Students at the ILI long enough to be retested are usually comfortable enough to want to talk expansively to their tester. While we certainly encourage our students to share their thoughts and feelings with us, while taking the BEST students may sometimes, through conversation, give an answer before the question is asked. This prevents the tester from determining whether the student does or would understand the question.

The BEST does offer a truer assessment of communication skills than a multiple choice test. Because questions are open ended, students may answer questions in a way that better demonstrates their skills. However, students may speak with fluency and skill about topics not covered by test questions and there is no way for students to be "credited" for these skills in the BEST.

The BEST seems to work better as an assessment tool for those at rudimentary levels of English. Because it doesn't address more sophisticated English speaking skills, the BEST offers no place to go for a student who has mastered basic communication skills. The BEST score also, while it has a few literacy questions, does not comprehensively test reading and writing levels, so those who have adequate speaking skills but lower literacy levels may fall through the cracks.

The scoring of the BEST depends greatly on the tester. While being trained, there were several instances in which my trainer and I gave different scores on the same question. Because the score depends on the tester's perception of "good" or "bad," the test score cannot be compared consistently with BESTs administered by other testers. For true consistency, the BEST needs to be administered in the same way each time by the same person. If the test is being used internally at one agency, skipping one question will not affect class levels if all of the tests have the same question omitted. However, because we work with other agencies and often refer students back and forth, test scores may not be consistent with those given at other agencies.

I feel that the BEST is appropriate for what it was designed for: as an intake tool for those with a low level of English. For levels of English higher than basic survival skills, however, a new assessment tool needs to be found.

Top of Page

Originally published in Adventures in Assessment, Volume 10 (December 1997),
SABES/World Education, Boston, MA, Copyright 1997.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.