The National Reporting System (NRS) has had the healthy effect of surfacing many questions about assessment and accountability in Adult Basic Educa-tion (ABE). The required use of standardized assessments for both placement and measures of progress has caused concern at both the state and program level. Typical standardized tests simply do not capture all the positive outcomes gained from adult literacy instruction.

Reality shows us that even if these tests could measure every-thing we want to document, many adult students are not available for post-testing. These issues prompted adult educators in Ohio to explore types of standardized assessment beyond the standardized tests. This led to the development of Ohio's Uniform Portfolio System.

Ohio's Uniform Portfolio System (UPS) strives to integrate the positive aspects of portfolio assessment into a standardized assessment system through the use of uniform competency checklists. Working together, consultants from the Ohio ABE office, researchers, and practitioners created reading, writing, math, and ESOL checklists for each of the six NRS levels based on the NRS educational functioning level descriptors. Students are placed into an NRS level for report-ing purposes based on scores from a standardized test, but progress is measured by both a standardized post-test and by the percentage of items mastered on the competency checklists.

Although the actual portfolio may look different in various pro-grams around the state or even in different classrooms within a program, it is uniform in that it must contain student goals, an indi-vidual learning plan, and one or more of the competency checklists (depending on the student's goals). As students master competencies from the lists, they check them off and include documentation showing mastery (such as a test or work sample) in the portfolio. Teachers and students have freedom to decide how they will demonstrate mastery, but the items on the checklists are the same across the state, which is what makes the system standardized.

Concerns were raised early on that the checklists not become a mandated curriculum. Because success is measured by how many competencies on the list are checked off, care was taken to make sure the competencies are general enough to be taught in the context of a variety of individual student needs and interests. For example, one of the items on the level 4 reading log says "Draw conclusions based on details in the text." It is easy to see how you can vary the text to meet the student's needs and interests and yet still meet the checkoff. One student might be reading a parent information sheet from her child's preschool and drawing conclusions about what she needs to do at home to help her child while another student might be reading an article on job hunting and drawing conclu-sions on what steps he needs to do to get the job he wants.

All teachers in the state are required to review the uniform portfolio with each student a mini- mum of every three months. At this review each student's NRS level is also reassessed in one of two ways. Students who have completed enough hours to make standardized testing appropriate and meaningful take a post-test. Levels are deter-mined for students who haven't completed enough hours for mean-ingful post-testing based on the percentage of items mastered on the checklists. The state has designated that when 75% of the items on the checklist are mastered the student has progressed to the next level.

This system reduces the number of students who can not be counted because they do not have a standardized post assessment (a require-ment of the NRS), without the problem of giving standardized tests too often in order to be sure students don't leave before post-testing. Because the competencies on the checklists and the percent mastered for completion of a level are standardized across the state, the level determined at the last portfolio review can be used as a standardized post assessment if the student leaves without a post-test.

Top of Page

The UPS supports research on best practices in assessment because it allows and encourages the use of multiple measures using a variety of assessment tools. Use of more than one assessment to measure performance verifies the reliability of the assessment. Anxiety from former negative school experi-ences can effect the performance of many adult literacy students in traditional testing situations, but these same students may be able to demonstrate competence when measured with other forms of assessment. A standardized test usually shows a snapshot of what the student knows at one particular moment in time. A truer picture emerges by assessing skills over a period of time with several different measures. For example, although a standardized test alone is allowable to measure a student's math level, a better demonstration of what the student knows and can do with math could be documented by a portfolio including an end of the chapter test on fractions, percents, and decimals; a personal budget illustrated with a pie graph; a week's menu with nutritional analysis; and a comparison of the cost of using credit cards, renting to owning, or paying cash for appliances.

Portfolios can document attainment of a broader range of skills and use of skills in context than can be measured on a multiple-choice test of decontextualized skills. They link assessment closely with instruc-tion and engage adult learners as active partners in the assessment process. This allows teachers to plan instruction and assessment around issues of interest and importance to students. For example, a student interested in a political election could demonstrate a writing competency by writing an essay comparing two candidates, while a young mother could compare and contrast two preschool programs she is considering for her child.

Implementing the UPS in time to meet the NRS deadline has produced several challenges. The portfolio checklists, which are the core of Ohio's system, need to be validated. Current use in the field will show whether the checklists measure what they are supposed to measure, that is, at what NRS level the student is functioning. Teachers are comparing the performance of students at NRS levels as identified by CASAS, TABE, and AMES with the items on the Ohio UPS checklists and with the diagnostic profiles of the standardized tests. Plans are already underway to adjust the checklists.

Reliability, or consistency of measure, is another issue that must be addressed. The UPS will only really become standardized when teachers around the state understand and agree on what each item on the checklist means and what its standard of mastery will be. For example, an item from the level 2 reading checklist states that the student will "interpret abbreviations commonly used in documents." As a field, Ohio teachers need to decide what abbreviations those are so that the criteria for mastery is uniform throughout the state. This does not mean that we intend to create Ohio standardized tests to measure each of the competencies, but that we need to agree on what it is exactly we are measuring and what mastery will look like, although students may demonstrate it in different ways.

A third challenge is staff training to ensure that all teachers are using the UPS in a standardized fashion for external accountability purposes and to help teachers develop effec-tive techniques in using portfolio assessment to meet the needs of students and staff. Staff develop- ment began last spring with videotapes produced by the state ABE office, which were required viewing for all ABE teachers in Ohio. The state is following up with a series of portfolio workshops for staff development. Local programs are encouraged to study the portfolio process and the UPS checklists and submit ideas for improvement.

Although Ohio's Uniform Portfolio System is still in the developmental stage the idea shows great promise. Continued work to ensure validity and reliability, coupled with ongoing staff training on use of the system and alternative assessment techniques, will ensure Ohio develops a system built on sound assessment strategies that meets the need for external accountability for the NRS, and, at the same time, provides useful information for students and teachers to drive the continuous cycle of assessing, planning, and teaching/learning.

Jane J. Meyer is Coordinator of the Canton City Schools Adult Basic Literacy Education (ABLE) Program in Canton, Ohio. She can be reached by email at meyer_j@cssd.striet.org