![[SABES logo]](../../../images/sabes1.gif)
| About SABES Contact Us Site Map |
|
|
What's New | SABES Resources | SABES Calendar | |
| Teacher's License | Other Resources | Other Sites |
A Performance Framework for Teaching and Learning with the Equipped for the Future (EFF) Content StandardsPeggy McGuire
|
|
|
Volume
12:
CONTENTS Introduction:
Successful Supervision: Three Perspectives Connecting the ESOL Framework to Actual Practice Learning
from Experience Learning and Change: A Phase Two North Carolina ESOL Framework Inquiry Project |
Equipped for the Future (EFF) is a grassroots and collaborative initiative of the National Institute for Literacy, aimed at the reform of this country's Adult Education and Lifelong Learning System so that the latter becomes thoroughly and consistently standards-based and customer-driven. Building consensus on the results that matter to all the customers of the system is the first and most important component of this reform since it provides the foundation for real change in teaching and learning. Through research and consensus-building over the past several years, we have been able to describe the knowledge and skills all adults need to be effective in carrying out activities central to their roles as parents and family members, citizens and community members, and workers. The EFF Content Standards express this consensus on what adults need to know and be able to do. And teachers all over the country, in ABE and ESOL classes, in Even Start and Welfare-to-Work programs, have been using the tools of the EFF Content Framework - the Role Maps, the Common Activities, the Skills Wheel, the Standards - to translate learner goals into instruction. They are able to do so because these components of the Content Framework reflect a dual focus on building skills and applying those skills to achieve real-life "results that matter". Having established what adults need to know and be able to do, our current task is to use a similar research and consensus-building process to develop an Assessment Framework that supports the Standards. This Assessment Framework will allow us to measure how well students are able to use what they know and report it in a meaningful way. It will move us from "EFF Content Standards" toward "EFF Performance Standards". One of the key tasks of this new phase of development is to define levels for each of the individual EFF standards - specifically, to build a research-based performance continuum for each EFF standard that will support the identification of level descriptors for all 16 standards.
We began by looking at other frameworks that have attempted to define a similarly broad continuum of adult performance, including the National Adult Literacy Survey (NALS) and the qualifications frameworks developed by Australia, England, Scotland, South Africa, and New Zealand. We also looked at cognitive science research on expertise and transfer, and data from EFF field development sites that included teacher descriptions of student performance. Our goal was to identify a theory-based set of dimensions for describing performance at both ends of the continuum: adults with many years of formal education and advanced degrees at the high end, and adults with few years of formal education and low English literacy skills at the beginning. We were sensitive to the failure of existing adult frameworks to adequately discriminate among performances at the low end of the scale. We paid very close attention to data from our field sites that defined the kind of evidence of progress teachers looked for and how they described student performances. At the same time, we made the assumption that the goal of the adult literacy system for adults at the low end of the scale (as for all other adults) is to facilitate increasingly more effective performance in the world. We wanted to be sure to build one continuum, and not to strand low-literate adults on a special, developmental continuum cut off from movement along the main pathway toward mastery and expert performance. Since research over the past 20 years has been building a greater understanding
of the cognitive and metacognitive strategies used by expert performers
and how they differ from those used by novice performers, we began to
examine whether we could build our continuum on this theoretical foundation. What is a knowledge base and how do we build it? Traditionally we think of a knowledge base as "what you know". The cognitive science research on expertise and transfer asks us to think not only about how much one knows (the number of facts, procedures, concepts, etc.) but also how the knowledge is organized. The goal is to assure that as an individual's store of knowledge relative to a particular domain or skill grows, the structure of the knowledge base also develops, becoming increasingly coherent, principled, useful, and goal oriented. This means that what a person knows – at whatever level of knowledge – is organized for efficient retrieval and application in everyday life. She has access to that knowledge, and can draw upon it for effective action in the world. As an individual moves along a developmental continuum from "novice"
toward "expert", then she develops more and better strategies for organizing
the contents of her knowledge base around principles and concepts. To
"organize" is to see, and eventually develop new "patterns" of information.
By "patterns" we mean connections or relationships between 1) facts, 2)
facts and concepts, and/or As what an individual knows becomes more organized, growing expertise is also marked by increased ability to identify what information is relevant to a particular task or problem. Further, the individual becomes more and more able to identify the conditions under which particular kinds of knowledge are useful. The "expert", then, has many strategies to retrieve and use information that is appropriate to whatever work she is trying to accomplish, in whatever context she finds herself. We see evidence of such developments in the knowledge base in improved performance along the other three dimensions we have identified. In other words, a more coherent, principled knowledge base supports performance with greater fluency, greater independence, under a greater range of conditions. What do we mean by building greater fluency and flexibility of performance? We are all familiar with the axiom "practice makes perfect." EFF defines this dimension as the level of effort required for an individual to retrieve and apply relevant knowledge. Points along the continuum range from "a great deal of effort" through "some effort," and "fluent" to "automatic." Why performing with increasing independence? An important indicator of an adult's increasing skill is the extent to which he or she needs direction or guidance in using that skill. EFF borrows DeFabio's definition of independence for this dimension: "an individual's ability to select, plan, execute, and monitor his or her own performance without reliance on the direction of others." Points along a skill development continuum for this dimension of performance would look at a decreasing need for assistance in carrying out these executive (or metacognitive) functions of performance, whether a person is acting alone or in collaboration with others. What about increasing the range of performance? This dimension gets to the heart of defining how well an individual can use a skill. Included in our concept of "range" are variables related to both task and context. These variables include the type as well as the number of tasks and contexts in which one can use the skill. Variables to consider include the degree of familiarity/unfamiliarity of a task or context; the structuredness/unstructuredness of the task; and the complexity of the task. Increase in range, like increase in independence, is directly related to the growth and more principled organization of one's knowledge base. We have focused on these four dimensions of performance because they address not only what people know but also how well they can use what they know. Together, they comprise a simple, coherent, research-based picture of performance that makes sense within programs as well as to all the many publics that care about what people can do (and where their limits are) as a result of their learning.
The data collection template being used for this field research was developed through the efforts of a group of teachers, EFF staff and Technical Assistance Team members who began meeting in Summer, 1999. We began with a template that articulated the four key dimensions of performance as a guide for placing these descriptions on a developmental performance continuum. Teachers found the template too confusing and suggested that it would be easier to use if the dimensions were embedded in categories that reflected how teachers think about planning and instruction. Guided by this insight, we reorganized the template to focus on four categories: task, context, knowledge, and performance. Task and context break out two aspects of range of conditions to reflect how teachers begin planning for appropriate instruction. The focus here is what learners can do with a particular skill or combination of skills. Thus, EFF teachers will likely use key components of the EFF Content Framework - Role Maps and Common Activities - to develop meaningful learning opportunities in which knowledge and skills are linked to real-life tasks and contexts. Knowledge remains a single dimension. Degree of fluency and degree of independence are combined into one performance category since teachers found that they looked at these two aspects together. All three relate to what learners know. Furthermore, as noted earlier, growing fluency and independence relate directly to the extent to which knowledge is organized around important concepts and principles. Thus, a "well-organized" knowledge base is the bridge between "knowing" and "doing." To assist teachers in using the four dimensions in planning and instruction, we asked questions about each category, as if we were developing an "observation rubric." Field development participants are using this revised template to develop an ongoing record of what learners can do with specific skills, a collection of evidence "moments" over time that together create the "big picture" of real-world outcomes. We hope the template will help teachers draw pictures of learner performance which capture the complexity of what learners are capable of performing, and communicate it in a way that is easy to understand. A PERFORMANCE FRAMEWORK To that end, the categories and their related performance questions from
the data collection template have also been organized into a one-page
document, the "EFF Performance Framework" (see Figure 1) which is available
now for any EFF teacher to use in planning, implementing and assessing
instruction around the EFF Content Standards. The "EFF Performance Framework"
integrates the four dimensions into a set of observation protocols, or
questions. The questions focus on the most important aspects of learner
performance for developing expertise. These are the questions that we
hope that every teacher who is teaching to the EFF Content Standards will
ask herself This is a research-based framework through which to look at learner performance and answer the questions "What do students know?" and "What can students do with this knowledge?" We hope teachers will carry this framework around, refer to it often, and finally integrate it into their thinking about teaching, learning and assessing with the EFF Standards so well that it becomes second nature - that they become "experts" at focusing their thinking on the four dimensions of performance. Furthermore, the framework is usable for any/all of the standards, and for collecting detailed evidence in a way that it can be compared. Using the Performance Framework in practice requires teachers and learners to stay focused on the components of performance for each standard. A lot of the power and promise of the framework comes from the fact that the Content Standards are consensus statements of what is important to teach and learn for each skill. The components of performance break down what needs to be taught and assessed to ensure that learners develop and can use the EFF skill. Staying focused on the components of performance allows EFF teachers and learners to be sure that everyone is focused on the same important things when planning, implementing and assessing learning activities. In other words, focus on the EFF components of performance allows for greater standardization of instruction without sacrificing flexibility, creativity, or accommodation of diverse learning styles and needs. It enables us to reliably say: "Adults with this skill can do these things - to act flexibly, with a range of options and choices, to meet the goals in their lives - and this is how we know." The Performance Framework, then, is a starting point for focusing teaching
and learning on EFF Standards and on the four key dimensions of skill
development and application. By establishing the four dimensions as the
basis for our Performance Framework as well as for our developmental continuum,
EFF aims to help teachers and learners keep simultaneous focus USING THE PERFORMANCE FRAMEWORK
TO PAY ATTENTION TO CRITICAL ASPECTS OF TEACHING AND LEARNING 1) Tasks What kinds of tasks can learners perform? What are the activities that learners engage in that require use of the skill? These tasks might be carried out in the instructional setting and might be contained in a discrete lesson or a series of lessons. Or, learners may be performing the tasks in other settings, such as at home or in their communities. In either case, the task will be identified because it requires use of the targeted EFF skill as well as relates to a learner's goal or purpose. How complex is the task? Is it simple, one-step, brief, short-term, well-defined and highly structured, requiring very little judgment or prediction of outcomes? Or is it more complex, longer and more sustained, involving multiple steps, less defined or structured, or even self-defined and self-initiated, requiring careful judgment and accurate prediction of outcomes? How familiar is the task to the learner? Some guiding questions would
be "Has the learner done this task before? Has the learner seen this task
done before?" Complexity: A simple task might be safely turning on and off the computer, or somewhat more complex, safely entering and exiting a word processing program. As the learners'skills develop, they will engage in ever more complex tasks, from writing and saving short written documents, to editing and rearranging text; highly complex tasks, requiring strong skills, might include using graphics, multiple fonts, even a desktop publishing program to write and produce a professional newsletter. Familiarity: Has the learner ever seen a computer before? Ever used a computer? Ever used a word processing program? Ever worked with publishing software? At the high end of the continuum, the learner has performed the task at hand many times before, or has seen it done often before; thus it is a familiar task. Further down the continuum, tasks are less and less familiar to the learner - all the way to those that the learner has never before performed or seen done. 2) Contexts In what contexts can learners perform? How familiar are the learners with the context? Carrying out a task in a very familiar situation where there is a lot of assistance and support is easier for learners than carrying out the same task in a less familiar, less supported context. At the high end of the continuum, the context is very familiar - for instance, exclusively in the classroom or instructional situation. Further down the continuum, the context becomes less familiar - on the job, in a community meeting, in the home where the learner has never tried to accomplish this particular task before. In how many different situations can the learners perform? At the low end of the continuum, the learner can only accomplish the task in a single situation (such as the instructional setting). Movement along the continuum is marked by ability to perform the task in a growing number of different contexts - indicating also that the learner is transferring the use of EFF skills from one activity or role to another. At the highest point, the situations in which the learner can perform are multiple and varied; learners can perform the task to meet a variety of needs and purposes, and their transfer of skill use from one context to another is systematic. How much risk is involved in the situation? How high are the stakes? The external environment in which learners are trying to accomplish goals can often present significant challenges to achievement. Some of these challenges have to do with "why" the learner is performing a task. What is at stake? Is this about successfully following directions in class? passing a credentialing exam, like the GED? getting/keeping a good job? Another aspect of "risk" has to do with more personal and/or societal challenges that learners often face in their efforts. Is there a threat of abuse in the home? Are there issues of racism/sexism/homophobia/etc. in the community? Are decent jobs unavailable, or in settings that pose risks to workers? At one end of the continuum, such risks are minimal, and the stakes are low. At the other, risks/stakes increase. EXAMPLE: Familiarity: At the low end of the continuum, the context for speaking in front of a group will be most familiar to the learners - in the learners'own instructional setting, in front of a small group of fellow learners, perhaps. Further along, the context becomes less familiar - in a different classroom, in front of a different group of learners, or the agency staff, or its Board. The context of the state hearings would be high on the continuum, assuming learners have never spoken in front of state legislators and so are unfamiliar with that context. Number of different situations: At the low end, learners can speak before a group only in one situation - the instructional setting in this example. But as development along the continuum proceeds, learners can perform outside the instructional setting and in increasing numbers of situations - in other classes, at Staff or Board meetings, before the Home-School Association, at state hearings. Level of risk: Relatively low-risk/low-stakes contexts, on the
low end of the continuum, might include a classroom full of supportive
co-learners or before other audiences who will not make major decisions
about the learner based on the performance. Even at the state hearings,
where learner performance may have a profound impact on listeners, the
risk to the individual learners is not at the highest end of the continuum.
But learners may soon need to use their speaking skills in much higher-stakes
contexts - before potential employers or in an attempt to gain a
seat on the local school board, for instance. 3) Knowledge Base When we consider the Knowledge Base necessary for any EFF skill, remember that we are looking at skills in the contexts of purpose (what adults need to know to meet their expressed goal) and performance (what adults need to be able to do with what they know). So we need to look at content knowledge, but we also need to look beyond content knowledge to how we organize and apply the content knowledge in meaningful contexts. This focus on use of skills to meet specific purposes may explain why so much "EFF Teaching" so often looks like "authentic task" or "project" based learning (content + application) instead of "drill and practice" (content only). The following two questions refer to what is contained in the knowledge base for the targeted EFF skill; the third question, and its subset of questions, refer to specific strategies for organizing and applying the contents of Knowledge Base for use in a meaningful context. What do learners know? What vocabulary do the learners have related to the skill? related to the subject area? Depending on the task to be accomplished, learners will need to understand different amounts and kinds of language used in the subject matter, as well as language about the skills being developed in the task. At the low end of the continuum, learners will only have minimal and simple vocabulary; as they move along in development, their store of vocabulary will grow and begin to include less familiar and more technical terminology. What content knowledge do the learners have related to the skill? related to the subject area? "Content knowledge" for any skill or subject area includes familiarity with facts, operations, concepts, rules, protocols, practices and/or conditions of use essential to the skill/subject area - including the purpose and audience for skill performance. At the low end of the continuum, learners have minimal or no familiarity with these essential content aspects of the skill/subject. As learners develop the skill, they become more and more familiar with an increasing amount of content knowledge that is useful in a greater variety of tasks and for a greater variety of purposes. What strategies do the learners have for organizing and applying content knowledge? Before we can use what we know about a skill, we have to have ways to organize all those discrete bits of skill-related information that come into our brains in different ways and at different times. That way the information "bits" can eventually be retrieved in the right combinations or "chunks", at the right time, for the right purpose. Can learners recognize relationships or connections? Can learners create new relationships or connections? These questions refer to strategies that overlap in the developmental process so we deal with them together. The strategies have to do with the growing ability to first see, then develop, new "patterns" of information. By "patterns" we mean connections or relationships between facts, facts and concepts, and/or prior knowledge and newly acquired facts, connections or relationships that are based on some bigger themes or concepts that tie the bits of information together. At the low end of the continuum learners have very few such strategies and are limited to simple recall of previously-learned bits of information. Initial "pattern recognition" is evident later when learners can achieve some low-level understanding of meaning by explaining, interpreting, translating, summarizing, paraphrasing, restating, and/or using examples. As learners develop more strategies, their understanding increases and becomes more complex; they can regularly recognize patterns, some simple and some higher order. They can recognize cause and effect relationships, for instance, and can join prior knowledge with new information to solve some problems. They begin to generalize, draw conclusions and predict outcomes in some cases. As they move toward the upper reaches of the continuum, learners begin to move beyond "pattern recognition" to "pattern creation." They move from accurate analysis (seeing the relationships between concepts and related details, between content and organizational structure - using strategies such as comparison/contrast, analogies, generalization, inference and prediction) to synthesis (organizing information in new ways and proposing alternate systems of knowledge - using high-order strategies such as abstraction, criticism and justification). Can learners identify what information is important to the task/problem? At the low end of the continuum, learners who do not have data/information organized around principles and concepts have a very difficult time deciding what information is relevant to solving a problem or completing a task. Further along, the strategies learners have for organizing information enable them to consciously retrieve important information for a clearly-defined purpose, then for multiple purposes, then for wide-ranging purposes and contexts. Can learners understand when information or concepts apply? This question refers to a learner's ability to decide which procedures, concepts or principles are applicable to which situation/task/problem - in other words, the conditions under which procedures or concepts are useful. At the low end of the continuum there is very little understanding that procedures or concepts are not universally applicable. Further along, learners develop a growing repertoire of strategies linked to specific situations. Eventually, learners are able to flexibly choose from among a range of appropriate strategies those that are most effective under the specific combination of circumstances represented by task and context. EXAMPLE: Vocabulary and content knowledge: Perhaps the learners need to begin with relatively simple tasks, like making shopping lists before going to the grocery store so that they will use their limited funds more carefully. This task would appear at the lower end of the continuum as it requires a limited set of vocabulary words, basic content knowledge, and limited familiarity with writing rules and practices. Further along the continuum, tasks will require a larger store of vocabulary, more diverse content knowledge and broader familiarity with writing rules - a note to a child's teacher requesting a meeting, for instance. While this may not be highly complex, it nevertheless involves sentences, punctuation, coherence, etc. as well as attention to the needs of an external audience to whom you are conveying information. Still further on, a written description of one local child care program may require a good store of words, writing conventions, and knowledge of child care concerns. At the high end, a task such as writing a guide to local child care options would probably require use of previously unfamiliar and more technical vocabulary, considerable mastery of writing conventions, and significant content knowledge in the areas of child development, state licensing regulations, etc. Relationships and connections: At the low end of the continuum, learners may be able to do no more than copy in writing the words that they need for a grocery list from a master list (or later, from a newspaper circular). A little further on, they may be able to write a simple note to a teacher after practicing doing so in class, but this activity still represents little "understanding" beyond recall. However, a task such as rewriting the contents of a brochure advertising a local child care program so that it is easier to read requires at least "low-level" understanding. That understanding is reflected in the ability to interpret, paraphrase and restate information in writing, and so belongs further along the continuum. As learners progress in skill development, they are able to see patterns and use strategies to express relationships. For instance, after rewriting several brochures, they are ready to write an essay in which they compare the programs that have produced the brochures - what do the programs have in common? how are they different? And high on the continuum, learners will be able not only to recognize patterns but to create new ones. To write a useful guide to child care options, their writing skills will allow them to summarize their own research and prior knowledge, critically analyze choices according to key points of interest to parents, and express their conclusions and recommendations as to the best options. What information is important, and when? At the low end of the developmental continuum, a learner might copy exactly a master grocery list or a "generic" note to a teacher, without realizing that the list can be tailored to individual needs at specific times, or that a note will be more effective if it shows awareness of a particular teacher's context and concerns. Even rewriting a brochure loses some impact if the rewrite doesn't edit out information that is not useful for the audience's purpose (e.g., trying to make a decision about which child care option to choose). As skill develops, the learner becomes more and more able to choose and communicate information based on key points of interest (the "right" foods for my new, healthy diet; two specific matters I want to discuss about my child's reading; adherence to state child care regulations) and the particular audience being addressed (myself; the Reading Resource Room teacher; single moms in search of quality and affordable child care).
How well can learners perform? "Performance" refers to the dimensions of skill development that move the focus of teaching and learning beyond "what we know" and "how we organize what we know" to "what we can do with what we know". It is about what the use of an EFF skill looks like in practice. How fluently can learners perform? How much effort is required? How consistently do the learners start and finish, getting to the desired outcome? The slowness and difficulty in performance at the low end of the continuum is matched by inconsistency of performance. At this point, learners will make a lot of "errors", will produce little work, and will have a hard time finishing the task. Further along the continuum, learners begin to show greater consistency in use of the skill; they complete tasks more often and with fewer errors. At the high end, effective skill use is systematic, work is completed and errors are rare. How well are barriers controlled or overcome? "Barriers" here refer to immediate adverse conditions that get in the way of effectively using a skill to perform a task. Such barriers may differ in nature and degree depending on the task and context (is the room too noisy? do I need glasses? do I never find time to work at home? have I misplaced the instructions?), but the key question is if/how the learners act to address them. At the low end of the continuum, learners will be easily diverted from the task by such problems, will be defeated and give up. Further along in skill development, learners will start to more often strategize about how to overcome identified obstacles; at the high end, regularly addressing and overcoming barriers becomes part of the learning process EXAMPLE: At the early stages on the continuum, learners will need to be taught, and constantly reminded of, the math operations that they need to complete necessary planning activities (how much space do we have? what shape will the garden take? how many different plots can we put in? how many plants can be placed in each plot? etc.). They will work slowly and struggle to "get it right". They may have a difficult time with obstacles that get in their way (do we have the right measuring tools? how do we get them? One plant requires more space than another, so how do we plant them both in the same plot?). It may be easy to abandon their plans at this point if too many problems present themselves. But as skills and experience with the math increase, learners will need fewer reminders and less help with addressing barriers. At the high end of the continuum, their work is systematic and they get the garden planted according to a clear and successful plan - using math to achieve their goal. How independently can the learners perform? How much help is needed from others? When we use "Independence" as a dimension of skill development, we are not suggesting that working alone is better or "smarter" than working in collaboration with others. However, one indicator of developing mastery has to do with how much assistance learners need in order to use a skill to perform a task. At the low end of the continuum, learners need substantial help from others in order to use the skill even in the most familiar and simple tasks. Then, as the skill develops, learners will still need help, but more often with tasks that are difficult or unfamiliar. At the high end, no assistance is needed; rather, learners at this level are ready to assist others. How much initiative is shown in getting started? At the lower end of the continuum, learners need a "push" to begin a task; as they develop greater skill, they will need less prompting and will often get started on their own. At the high end learners need no "push" to get started; in fact, they will often initiate new tasks on their own, identifying and pursuing new opportunities to learn. How often do learners generate their own strategies to complete the task? Once learners get started, how much and what kinds of help do they need to complete a task? At the low end of the continuum, learners need to be offered a great deal of structure, clarification and guidance. They need to be able to "copy" strategies and approaches that others have used. Further along the continuum, learners can sometimes come up with strategies on their own, without strong guidance; at other times they still need approaches to be imposed and guidance to be offered. At the high end, learners can invent their own strategies, adapt approaches from outside sources of information, and justify their choices of the most appropriate ways to complete the task. They don't need guidance but can offer guidance to others. EXAMPLE: At the low end of the continuum, learners have little or no experience with effective advocacy, so they will need a great deal of instruction, prompting and assistance from you in how to begin. They will need you to give them a highly structured strategy, for instance, for recruiting other learners to join them and for identifying the issues that they want to address as a group. They will need your active involvement in order to insure success at reaching even a relatively simple goal such as asking Board members of the agency to donate used books for a small lending library. As their skill development moves along the continuum (and as they gain experience - and success!), they will need less assistance and fewer instructions from you, though they may still want your help with more complex advocacy opportunities (getting permission to reconfigure current space, or raising funds to add on a new room!). At the high end of the continuum, learners will be so skilled in advocacy, and will need so little help, that they can assist other, new program participants in joining the group and learning the necessary skills. CONCLUSION We have tried here to describe some of the key work that is under way in this next phase of the EFF development process:
We are deeply grateful to the many development partners whose efforts have brought us this far and are moving us forward. Hard work lies ahead, but we look forward to the opportunities and potential for real change that accompany it. For more information on Equipped for the Future and its publications, contact 1-877-433-7627 or go to its Web site: www.nifl.gov/lincs/collections/eff/eff.html Originally published in Adventures in Assessment,
Funding support for the publication of this document on the Web provided in part by the Ohio State Literacy Resource Center as part of the LINCS Assessment Special Collection. |
| | SABES Home Page | SABES Resources | SABES Calendar | Other Web Sites | |
