How Much and What Kind?

One Family Literacy Program's Assessment Story

Sylvia Greene, Nancy Hoe, and Lally Stowell
Community Learning Center, Cambridge, MA

The topic of assessment is a fascinating, challenging, and daunting one for any adult education program, no matter what its scope and setting. In the field of family literacy, however, assessment must broaden to include children as well as parents, and curriculum components such as parenting, parent-and-child time and home visiting along with the traditional conversation, reading, grammar, writing and math.

At the Cambridge Even Start program, we have tried to deal with these challenges by doing everything on the assessment map: standardized and non-standardized; group and individual; formal and informal; teacher and self. You name it, we do it: anecdotal running records, work sampling, conferences and I.E.P.s journals, locked confidential files, open accessible portfolio files. The result is, as our local evaluator put it recently, we are "drowning in data."

If assessment is always a work in progress, the above-mentioned evaluator, Elizabeth Brach, is the latest chapter in our assessment story. (Elizabeth works for the Office of Resource Development and Assessment at the Cambridge School Department, directed by Barbara Black.) Every program should be so lucky as to experience an outside evaluator as positive, appreciative, fair, rigorous, professional, and willing to look at existing assessment protocols and ask hard questions. Our latest revisions and attempts to streamline owe a great deal to her.

Cambridge Even Start is a collaboration between the Cambridge Community Learning Center and the Cambridge School Department (primarily its HomeBased Early Childhood, Primary Education and Title I programs). Families attend five mornings a week from 9 to 12 at the Gately Shelter in North Cambridge, a multi-use city building housing youth programs in the afternoon and evening. Even Start is a "center-based" program in that most of the components happen in the same place at the same time; however, each family also receives a weekly hour-long home visit at a time convenient to them and their home visitor, and we also take some field trips off-site. We are designed to serve 30 Cambridge families a year. Since our beginning in 1993, we have served 84 families from 20 different countries speaking 12 different languages, reflecting the diversity of Cambridge.

For the parents, we offer an ESL class (Level 2/3 in the Community Learning Center's sequence) taught by Lally Stowell, and an ABE class (Intermediate/ Pre-GED level) taught by Sylvia Greene, on Tuesdays 9-12, Thursdays 9- 11, and Fridays 9-11, for a total of seven hours a week. Monday and Wednesday are for computer classes (taught by Javier Aponte), with parents coming one day or the other for three hours. On those days, other offerings are an hour-long phonics and spelling class, individual tutoring, group homework time, and time for parents to volunteer in the preschool class. Fridays from 11-12 we have a parent discussion / support group for ESL and ABE parents combined. Thursdays from 11-12 we have Parent and Child Time (PACT) involving all parents, all children, and all staff. Parent and child activities centered around a book, toy, or game are also the central focus of the home visits.

Our preschool class meets every morn- from 9-12, and is open to children 2 years 9 months to kindergarten age. It is taught by Nancy Hoe, Estalina. Rodriguez, and June Ramdewar. (We provide babysitting reimbursements for children under 2 years 9 months.) About half the parents have their children in our preschool class, and the others have their children in Head Start or other preschool settings, family daycare, or elementary school. We run from September to the end of July, and are open-entry, open-exit.

Even Start is staffed by two Co-Coordinators (Nancy is Early Childhood CoCoordinator and Sylvia is ABE Co- Coordinator) who each administer half-time and teach half-time; three half-time teachers (Lally who works with Sylvia, and Estalina and June who work with Nancy); a 10-hour-a-week computer teacher (Javier); and a Harvard Graduate School of Education intern and four volunteer tutors who help out in different parts of the program. Counseling is provided by the teaching staff and agencies to whom we refer parents as needed.

Top of Page

Initial Assessment

Parent
New families come to us in several different ways: through agency referral, word-of-mouth, cable TV announcements, targeted AFDC mailings, and so on. Over the telephone, it is usually possible through evaluating conversational ability and asking a few questions about previous education to determine whether the parent is more appropriate for Lally's ESL class or Sylvia's ABE class, or not appropriate for Even Start at all, in which case we try to refer the person elsewhere. This initial phone call is the first step in the assessment process.

Next comes an appointment with Lally or Sylvia, in the parent's home or at the Gately Shelter. A Community Learning Center registration form is filled out, yielding the usual demographic data, plus educational history of the parent, and goals the parent chooses for themselves from the range provided on the form (which is useful, but not complete). A separate Child Intake Form is filled out, to be discussed later in this article. At this point, the ESL and ABE initial assessments diverge.

ESL: The next step in the ESL assessment is a teacher-made series of graded readings designed by teachers at the Community Learning Center that the parent reads aloud, with Lally asking set questions after each reading. She notes numbers of hesitations and mispronunciations, and level of comprehension, which are recorded on the form and filed eventually in the parent's folder. Based on this reading inventory and informal conversation with the parent, she determines if he or she is appropriate for her class. If the parent scores too low, Lally refers him or her to another class at the Community Learning Center -- ESL 1 or ESL Literacy 1 (for people who are also not literate in their own language) -- another adult education program, or, in some cases, loans a series of her own tapes to help the parent reach the level of her class. ("Too low" would mean ESL 1 level: someone who speaks hardly any English and would not be able to sit in a group of adults from many different countries and have a simple conversation about a topic like discipline or bedtime routines.)

There is no second form of this informal reading inventory to re-administer later in the year to check progress, so Lally, along with the ESL department at the Community Learning Center, will be working on designing new ESL assessments that have two forms, one for pre-testing and one for post-testing.

ABE: For ABE parents, Sylvia next uses two interest inventories: one is general, asking questions about the parent's workrelated reading, writing and math needs, topics s/he likes to read about, what s/he does well, who s/he most admires, and where s/he would most like to travel. The second is a more specific menu of possible GED-type topics, divided into the areas of Science, Social Studies and Literature, which the parent can rank in order of interest. The topics are ones like the civil rights movement, poetry, stress reduction, geography of the Caribbean, and so on, that have been of interest in the past to many adult learners, not just GED students. (At Even Start, a majority of the ABE students are interested in getting a GED. The others want to improve their reading, writing and math, usually with job training or job advancement in mind.)

Next, the parent does a writing sample, and then with Sylvia analyzes it for strengths and needs using a checklist of writing components, including handwriting, mechanics, spelling, vocabulary, use of standard grammar, ability to stick to the topic, ability to combine thoughts and feelings, and so on. This is kept in the parent's portfolio folder, for diagnostic purposes and for comparison with later writing samples chosen by the parent to measure progress. The Community Learning Center staff is presently in the process of developing a "writing rubric" to facilitate wholistic scoring of writing samples, and Even Start looks forward to benefiting from this effort.

Usually, this is enough for the first meeting, as 1-1 /2 to 2 hours have often passed. In the second session, the parent is given the Diagnostic Assessments of Reading, or DAR (Florence G. Roswell and Jeanne S. Chall, The Riverside Publishing Company, 8420 Bryn Mawr Ave., Chicago, IL 60631). This standardized but informal, individually-administered test yields grade levels and diagnostic information in five reading components: isolated word recognition, word analysis, word recognition in context, oral vocabulary, and silent reading comprehension, as well as spelling. A relatively short, flexible, easy-to-administer instrument, the DAR results in a profile extremely useful in designing instruction specific to the parent's needs. The instrument was chosen by the ABE Department at the Community Learning Center as most useful for looking at several components of reading.

We also give the TABE (Tests of Adult Basic Education, CTB/McGraw Hill, 20 Ryan Ranch Rd., Monterey, CA 93940) in order to satisfy our funders, be consistent with the Community Learning Center, and provide transition to job training centers that use TABE scores. We give the TABE Locator, then the indicated level of the test. (We give only the Reading subtest.)

Next comes the Community Learning Center's informal math diagnostic. As with the writing sample, Sylvia analyzes the results with the parent, noting strengths and needs using a checklist of skills ranging from basic computation skills up to geometry and algebra.

The DAR, TABE and math diagnostic tests are all filed in the parent's locked, confidential file. This is plenty for the second session. At this point the parent can begin class.

The last step in initial ABE assessment happens after s/he has been in class a week or so. The Literacy and Numeracy Practices Questionnaire is a fancy name for a list of real-life reading, writing, and math survival skills, grouped into those used at home, at work, and out in the community. They include such things as reading the newspaper, figuring tips, writing a resume and filling out an accident report. For each skill there are columns for "Can do" already (and if so, "How often?" and "Easy or Hard?"), and "Would like to learn."

Based on the preceding information from all the assessments done so far, it is now possible to sit down with the parent and have an initial conference to set goals. Goals can be chosen from the "Needs work" column of the checklists for the writing sample and math diagnostic, from the "Would like to learn" column of the questionnaire, or the topics ranked high on the interest inventories. The form used for the conference in the ABE class (a different one is used in the ESL class) is on the next page.

All this assessment, then, leads to the parent's conference record, which informs both group and individual instruction and helps track progress. When a parent completes one of their short-term goals as stated on this form, he or she receives a certificate, a copy of which may go in their portfolio file.

Top of Page

Child
During the first meeting with the parent, a Child Intake Form is filled out for each child in the family aged seven or younger. This yields birthdate; parents' names; names, ages and school placement of all siblings; favorite activities; allergies; other services and agencies involved with the child; and the parents' goals for the child.

During the first home visit to the family, Nancy Hoe or the Home-Based Director (Ellen Grant Valade) takes a developmental history for each child aged seven or younger. They also give the Peabody Picture Vocabulary Test (PPVT), a wellrespected standardized instrument which tests the child's receptive vocabulary and concept development.

Family
Soon after parents begin class, they are asked their goals for the whole family. These are often difficult to articulate, and require discussion and modeling in the Friday parent discussion group. Parents often take time to articulate their goals, so a list of examples from other parents is often helpful. (An example of a family goal is to spend more time together on the weekends.) Their goals are filed in their family portfolio folder. For each family, there are two sets of fairly comparable files, one kept inside a locked cabinet, and one in a portable milk carton on top of the cabinet accessible to parents and teachers. In each place, there is a hanging file for each family containing four differently-colored folders, one for the parent's adult class, one for each child seven or younger, one for home visiting and one for the family.

Ongoing Assessment

Family
The most obvious ongoing assessment information is attendance records. We were keeping separate attendance forms for parents, children in our preschool class, and home visits; now we are consolidating these into one family attendance form to be filed at the end of each month in the locked family file. Also, family goals are updated every five months during individual conferences or in a parent discussion group.

Parent
Another obvious source of assessment data is a record of homework assigned and either completed or not completed. Discussion of both attendance and homework records is included in the conference. Conferences happen in January and June and include notation of goals met and setting of new goals.

The TABE is re-administered to ABE parents right before these conferences and discussion of (hopefully!) progress as shown by TABE scores is included. Lally uses a group of five parenting issues (such as discipline and giving children responsibility) for ongoing assessments every two months during the year. She pairs two ESL students to discuss the topic, then has them separate and write on the topic; discussion of these writing samples is part of her individual conferences.

ABE parents keep journals which they write in each Friday, answering two questions: "What did you learn in your adult class today?" and "Give an example of something you did this past week to help your child learn," the second question having been suggested by our first local evaluator as a positive, non-judgemental and open-ended way for parents to think about their parenting.

Copies of all writing samples are kept in the parent's accessible portfolio file, and at conferences can be compared using the same checklist used for the initial writing sample to show changes.

Since math instruction in the ABE class is by necessity individualized, each student has a running Individualized Math Record Sheet, with columns for book, page number, concept worked on, date given, and date received. These are kept in a folder in the classroom.

For each parent, Sylvia also keeps a running Teacher/Tutor Comment Sheet, where she or one of the volunteer tutors who help in the classroom can note anything important. These sheets are confidential and are kept in a folder, eventually filed in the parent's folder in the locked cabinet.

Recently, thanks to feedback from our local evaluator Elizabeth Brach, it became clear that there was no opportunity for the parent to evaluate all five components of Even Start at once, thus reflecting the comprehensiveness of family literacy. Therefore, we came up with the following form, to be filled out by the parents every other Friday during Parent Discussion time. The form has been modified several times, and the final product owes much to the valuable feedback of Home-Based Director Ellen Grant Valade. We have tried it once, and hope it will help us keep an eye on the program as a whole. After the parent has filled it out, the form will be filed in the locked family file.

Top of Page

"This Week I..." Form

The history of the form on the previous page is an example of the ongoing adaptation of our assessment practices. We used to have a separate home visit report form, and a separate school contact report form and a separate form to report books parents read to children during the week. Now the school contact and home visit forms are absorbed into the above one and thanks to a suggestion from Ellen Grant Valade, the home visitor will now help the parent keep a running record of books read, since it will be easier to do in the home where the books are right there in plain sight.

Each July, during home visits, Sylvia fills out a form with parents asking them to comment on their own progress, their children's progress, and their plans for September.

Child
Work samples are collected throughout the year for each child in the Even Start preschool class, and extensive running anecdotal records are kept. A developmental checklist is filled out for the child at midyear, then again at the end of the year. (Nancy is presently working on a specific literacy development checklist covering birth to 7 years linked to an instrument the Cambridge School Department is beginning to use which is based on Marie Clay's Concepts About Print, or CAP.) Preschool staff write short monthly goals.

In July, an End-of-the-Year Report is written for each child using the Cambridge School Department's Kindergarten Transition Form. A copy of this report is given to the parent. This coming year, we will begin keeping a parent-child spiral notebook for each child in the preschool class, in which staff can write down information about the child, and where parents can read and respond. These will be kept in an accessible box in the preschool room. Finally, the Peabody Picture Vocabulary Test is readministered by Nancy during the July home visit.

Exit Assessment

Since parents rarely leave Even Start with plenty of notice, this aspect of our assessment protocol is weakest. Ideally, we hope to give ABE parents a final TABE all children a final PPVT, and all parents our Exit Interview, which includes questions about their future plans and their ratings of the different components of the program.

After all is said and done, what is the purpose of all this data collection? Within our program, it clearly helps parents document and celebrate their own and their children's progress. Even Start also needs to make sure, however, that their assessment practices are meaningful to potential funders, since our federal grant runs out the year 2001. We need to demonstrate that we are doing some good in terms that make sense to taxpayers and school departments and state legislators and whoever else might be interested in eventually adopting us.

For this reason, the eleven Even Start programs around Massachusetts -- already accustomed to working together, thanks to a caring and energetic statewide coordinator at DOE (Arlene Dale) who assembles us regularly -- have begun to discuss the idea of looking at family literacy assessment from a statewide perspective.

In addition to the local and state layers of assessment, there is a national layer. Like the other 600 or so Even Start programs around the country, we have to report each July to the Even Start Information System (ESIS) standard demographic and attendance data, among other things. For two years, we were also part of something called the National Sample study, and for a I mited number of our families, reported TABE or CASAS scores for parents and Preschool Language Survey scores for their preschool children.

The resulting National Even Start Evaluation was disappointing because it did not capture at all the daily progress practitioners were seeing. Partly in response to this situation, a newly-formed national organization of practitioners called the National Even Start Association is moving toward doing its own evaluation, which will hopefully document this progress. Nancy is on their Advisory Board.

Our assessment practices seem to require constant tinkering, and we find ourselves tying this and rejecting that, seeing what works and doesn't work, and listening, reading, sharing, and certainly borrowing. Please feel free to adapt any of the instruments mentioned in this article, or contact us with any questions or ideas. (Sylvia Greene and Lally Stowell can be reached at the Community Learning Center, 19 Brookline St., Cambridge, MA 02139, tel. 617-349-6363; Nancy Hoe is at the Cambridge School Department, 159 Thorndike St., Cambridge, MA 02141, tel. 617-349-6493.) Cambridge Even Start looks forward to feedback to this article, and to the continuing dialogue.

Top of Page

Originally published in Adventures in Assessment, Volume 10 (December 1997),
SABES/World Education, Boston, MA, Copyright 1997.

Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection.