Home
Youenn Leborgne's Library tagged → View Popular
** Writing and using learning outcomes - Moon 2002 **
VERY USEFUL READ
learning outcomes, criteria, objectives, aims, benchmark, threshold
Aims (which provide a useful addition to learning outcomes) refer to teaching intentions while learning outcomes refer to expected learning
Examples of aims are given
Learning outcomes should be written at threshold (minimum acceptable standard: pass-fail line)
When outcomes are precise, assessment criteria are less critical
List of some uses of learning outcomes
A definition and examples are given
Some form of module may seem problematic for description of learning outcomes in advance
Use of "expected to be able"
3 parts:
- a verb that indicates what the learner is expected to be able to do
- a word or words that indicate on what or with what the learner is acting (if outcome is about skills then the words may describe the way the skill is performed)
- a word or words that indicate the nature (in context or in terms of standard) of the performance required as evidence that the learning has been achieved
Since, learning outcomes are written at threshold:
- if one component of a multiple outcome is not met by a student, he *theoretically* fails the whole outcome
- if a student fails to attain one or more outcomes, he *theoretically* fails the whole module
Another common fault is when outcomes refer to the learning instead of the representation of learning
It's important for learning outcomes to be written at threshold so that students have a "safe" place for exploring new ideas, creative approaches, etc.
** A list of vocabulary is given for various levels of learning **
While all learning outcomes need to be assessable, they may not all be assessed in practice (eg. when a student has a choice as to which question to answer in an exam)
** Aligning teaching for constructing learning - Biggs 2003 **
Notions
constructive alignment, intended learning outcome, activities, assessment, declarative, functioning
In a good system, such as constructive alignment, all aspects of teaching and assessment are tuned to support high level learning
An example of a poor system is given
"Constructive": constructivism
"Alignment": key is that all components in the system, especially the teaching methods used and the assessment tasks, are aligned with the learning activities assumed in the learning outcomes
1. Defining the intended learning outcomes (ILOs)
Declarative knowledge is content.
Functioning knowledge is skills in applying that content appropriately (more essential)
The first step in designing the curriculum objectives is to make clear what levels of understanding we want from our students in what topics, and what performances of understanding would give us that knowledge.
2. Choosing activities likely to lead to the ILOs
Beyond lectures and tutorials, activities that take place outside of class such as interactive group work, peer teaching, independent learning and work-based learning are rich sources of relevant activities
3. Assessing students' actual learning outcomes to see how well they match what was intended
As far as the students are concerned, the assessment is the curriculum.
Making sure the assessment tasks mirror the ILOs is critical
4. Arriving at a final grade
Matching perfomance against the criteria is not a matter of counting marks but of making holistic judgments
Assessment: a critical perspective - Reynolds and Trehan 2000
Notions:
e-learning, assessment, design, control, power, authority, peer, social, participation, hierarchy
Assessment is a primary location for power relations
Development of less hierarchical approaches to learning and teaching.
Corresponding changes in the practice of assessment are harder to find
** Sharing the procedures of assessment doesn't necessarily result in more democratic processes: **
- may cause anxiety and frustration
- ambiguity resulting from the redefinition of the tutor's role
- make reasonable expectations as to what the students will be able to do
- if students know that the tutors will intervene if the marking is unsatisfactory, then the marking can't be said to participative or empowering
- fear of receiving a bad mark if a bad one is given
- differences present in learning groups: granting of freedoms by the teacher can result in that control being taken by another agency
Individuals' status and influence informed by who they are in the wider society, in relation to age, gender, sex and race.
Perceived ability may also play a role
Operating assessment methods which encourage learners to be supportive to fellow learners while developing their skills in critically evaluating the work of others, is a challenging, complex process
Putting into practice participative assessment requires tutors to be prepared and able to work with the complex social processes which are generated. If not, traditional practice may be preferable.
Empowering pedagogy doesn't dissolve the authority or power of the instructor. It does move (if previously mentionned barriers successfully addressed) from power as domination to power as creative energy.
Classrooms needn't always reflect an equality of power, but they must reflect movement in that direction.
** Writing and using assessment criteria - Moon 2002 **
** READING THE ARTICLE ITSELF IS ESSENTIAL **
Criteria may be developed more closely with either:
- the learning outcome (leaving freedom as to the choice of method to be devised)
- the assessment methods
Criteria implies criterion-referenced assessment
Criteria help students understand what's expected of them
Precision in writing the criteria is sometimes more appropriate than others: mustn't detract from the challenge of the task
Use the form "The student will..."
Assessment criteria should relate to learning outcome in a clear and transparent way (vocabulary used may mean different things to different people)
Weighting system may be superimposed in many assessment situations
Two main forms:
1. threshold assessment criteria: standard to reach to demonstrate achievement (pass-fail line)
More detailed representation of the learning outcome.
All should be reached
2. grade assessment criteria: standard to reach to get a particular grade (more motivating and rewarding for the learner)
Relationship between outcome and criteria should be overt at the pass-fail line but thereafter can be separate
Helpful to write and communicate "desirable learning outcomes": grade assessment criteria are then guided by both learning outcome (provides the pass-fail point info) and desirable learning outcome
If weighting is part of the development of the development of grade assessment criteria, the conditions set by the learning outcomes all have to be achieved successfully
** Quite useful figure illustrating the basic model of module/development taking into account grading criteria **
** Lots of example of different forms of assessment criteria and learning outcomes **
Online Assessment Wiki
Invite key: edinburgh
Name: Youenn Leborgne
Email: youenn.leborgne@ed.ac.uk
Laying a Foundation for Lifelong Learning: Case Studies of E-Assessment in Large 1st-Year Classes - Nicol 2007
Notions and references:
formative assessment, feedback, self-regulation , self-assessment, autonomy, reflection, peer instruction, vote, confidence testing, time on task
References about the 7 principles of good feedback practice in relation to learner self-regulation.
References about 11 conditions under which assessment supports student learning.
The concepts of self-regulated learning and academic success are central to this paper.
Starting assumption is that students are already engaged in self-regulation but that some students are better at self-regulation than others.
Two case studies showing how ICT can support the development of learner self-regulation. Also provided are some illustrative examples of how learner self-regulation might be supported using multiple-choice tests.
For each case study, outline of mapping between the new settings and the 7 principles.
Example 1: psychology
Task questions are progressively more difficult, responses move from individual to group response and a model answer for comparison at each stage.
Opportunities for constructive formative assessment (scaffolding) linked to supportive peer discussion.
Students positive about experience (collaboration, self-confidence, understanding)
Findings have given them the confidence to propose a radical redesign of the 1st-year class
Example 2: mechanical engineering
Active-learning sessions
Peer-instruction: a form of Socratic Dialogue or ‘teaching by questioning'
Typically:
- teacher briefly explains concept
- MCQ by EVS
- "convince your neighbour that you have the right answer"
- retest or class-wide discussion
- teacher clarifies correct answer
Alternatives:
1: "just-in-time-teaching"
- MCQ: show areas of weakness
- focus of the EVS session is based on these areas of weakness
2
- confidence testing (CBM): students engage in metacognitive thinking
Huge success
More power when assessment principles underpin implementation (as in EVS) and when the implementation blends online/offline interactions (as with just-in-time-teach
Assessment and Learning: Contradictory or Complementary? Assessment for learning in higher education - Boud 1995
There is probably more bad practice and ignorance of significant issues in the area of assessment than in any other aspect of higher education.
We see that we must see both summative and formative aspects of assessment together, at all times.
Every act of assessment gives a message to students about what they should be learning and how they should go about it.
Assessment can encourage passive, reproductive forms of learning while simultaneously hiding the inadequate understanding to which such forms of learning inevitably lead.
Very little attention has been given to the compounding effects of ssessment even when we know that it is the total array of demands in a given period which influences how each one is tackled.
We should develop assessment procedures of high consequential validity
Well-designed assessment practices should be oriented around the key concepts and ideas that students should be able to deal with
The stages of development which illustrates the evolution of our ideas are given details of.
Good assessment now is that which both closely reflects desired learning outcomes and in which the process of assessment has a directly beneficial influence on the learning process
We need to look at the impact of the total package of learning and assessment and not simply at fragments of assessment.
In the holistic conception, the traditional power relationships between student and assessor must be challenged.
This power relationship combined with the judgement vocabulary used in assessing can profoundly absuse learners as it is often hard to make the disctinction between judgement about the work and judgement about the person.
- give descriptive and elaborate feedback which stays away from categorical/"final" statements and language which can abuse students
- shift away from assessment which occurs only at the end of the period of study or which doesn't allow for a response
Notions:
Assessment interpretation, outcomes, consequences, language
Consequential validity
** Including young people with disabilities: Assessment challenges in higher education - Hanafin et al 2007 **
Notions:
accessibility, assessment, design, attitudes, tool, participation, élitism
A MUST
Under-representation of people with disabilities in HE is the consequence of attitudinal and environmental barriers, within and external to higher education
Little understanding about how disabled students experience assessment nor of its effects on them
Assessment practices are designed, picked from a huge amount of alternatives. Yet, they don't take account of the variety of ways people learn
Participants in the study had by far the most difficulties in the domain of attitudinal change
Final exams, build-in environment and getting lecture notes are particularly problematic
Competitive individualism intrinsic to an assessment structure: a way of excluding disabled students and an anti-learning mechanism
The removal of a practice (note provision) from its proper structural position to the position of a private ‘grace and favour’ arrangement reinforces perceptions of disabled students as objects of charity and nuisances
Assistive technology
Dyslexia sometimes perceived as a strategy to confer unfair advantages
** List of strategies participants used to cope **
Limited range of assessment practices
Assessment always leads to learning: What kind of learning?
Alternative assessment
Taken-for-granted nature of the assessment mode makes it easy to explain under/achievement in terms of individual deficit rather than unjust and partial practice
Emphasing understanding rather than rote learning would be more appropriate with HE's goals
The excellence movement in education is fundamentally concerned with "how to exclude rather than with how to include"
** 5 ways to improve formative assessment are listed **
More inclusive assessment practices would be of benefit to all
The creative and inclusive nature of assessment developments in the special education sector has much to contribute to mainstream education practices
Reappraisal of scholarship to include not just the traditional research model
** Inclusive assessment - McClenaghan 2006 **
Notions:
accessibility, assessment, inclusiveness, activity
Individuals should be able to engage to their full potential
Difference between accessibility and inclusiveness is explained
There's no one-size fits all solutions to these issues
The greater the variety in assessment methods, the greater the chance to meet a wide range of students' needs
In the UK, almost 5% of all enrolled students have some form of disability
A list of broad groups of disabilities is given with potential consequences on assessment.
The Obsessive Compulsive Disorders section is of personal importance
All students benefit from clear written instructions and questions that are written simply and clearly.
Carefully judge the equivalence and validity of your modified or alternative assessment against learning outcomes and assessment criteria
5 ways of making assessment inclusive and accessible are listed and ** very interesting suggestions ** are given for each:
1. Inclusive design:
Involves the design stage, consists of increasing clarity, etc. (everyone benefits from it)
2. Universal design (multiple formats):
Being flexible with the assessment modes. Technology makes it easier and everyone benefits from it (accomodates various preferred learning styles)
3. Adjustments and adaptations
Same assessment - different methods
4. Additional arrangements
Making changes to the physical environment, providing additional tools, etc. (involves planning and being aware of administrative problems)
5. Alternative assessments
Assessment doesn't have to be identical - but equal
Sometimes, only alternatives can make an assessment accessible. Why not make it available to all students?
*** Guide tables about (generic as well as very specific) possible accomodations to both various assessment modes and disabilities ***
The Needs Assessment is done by specialist staff
Disabled students in higher education: Experiences and outcomes - Fuller et al 2008
Notions:
accessibility, disability, assessment
A marked increase in the proportion of disabled students participating in higher education
Support for disabled students has expanded and is now managed through centralised support units. In line with disability equality legislation, lecturers are expected to make reasonable and anticipatory adjustments to curriculum, pedagogy and assessment practices and students are legally entitled to these adjustments
=>
Although lecturers are generally supportive of disabled students, they sometimes feel overwhelmed by requests for individualised adjustments and unsure about the balancebetween maintaining academic standards and accommodating the needs of disabled students
The adjustments which were made tended to be limited and formulaic, particularly in the area of assessment
=>
The development of inclusive and flexible curricula would require fewer adjustments to be made and would ensure fairness for all whilst ensuring the maintenance of academic standards
Disclosure and acceptance of the label of disability was problematic for some students, especially those with unseen impairments
=>
Better communication between and within central services and academic departments would ensure more effective support for all disabled students, not just those who contact the disability support office
Disabled students are a heterogeneous group and experiences and outcomes are variably linked to the nature of the student’s impairment
=>
More effective monitoring of the experiences and outcomes of disabled students by impairment should lead to more effective support for those most at risk of academic failure
** Learning to Learn: more than a skill set - Rawson 2000 **
Notions:
emotions, mental models, self-reflection, context, identity, peer assessment, negotiation, collaboration
"learning to learn" involves a far deeper and more personal self-reflexive process
Questionning of givens and perceptions and the mutual influence of the one on the other are essential
1. Development, engagement and exploration of the whole person
2. Assessment offers the potential leverage on learning to learn but is also inextricably linked with it
HE carries the belief that participants can develop the rational power necessary for mastery of a given discipline, but the assessment process seems not to admit this rationality
An authoritarian and summative approach to assessment reinforces the power differences between staff and students, measures a limited range of abilities and encourages surface approches to learning
Assessment is central to the development of autonomy
Shift from traditional mode of control of educator to a more facilitative role
How can the process of knowledge acquisition and assessment achieve the transfer of knowledge and not stifle creativity? Reconciling "public" and personal knowledge
The acquisition of personal knowledge is for the individual at one level an attempt at meaning making, and at the level of learning, to learn an attempt at understanding that process of meaning making
The "accuracy" of collaborative, peer, and self-assessment (underrepresented in HE) provides the possibility of greater formative value, for students and educators
There are reservations to both ends of Heron’s continuum (unilateral <-> self assessment strategies): more collaboration but institutions still have a role to play
The pitfalls of non-involvement of the learner are much more serious than those of involvement
The curriculum needs to be negociated between learner and educator
The learning process is more fundamental than its results
** List of a number of principles of assessment practices **
We may need to turn the process of learning to learn upon ourselves
Review of "Towards inclusive learning in higher education" - Lindsay 2007
Notions:
accessibility, assessment design
“Stop adopting practices which predominantly focus on adjustments and start thinking about inclusive curriculum and assessment design which offer all students choices that align with their abilities.”
The essays cover a good range, some primarily inspirational, but most dealing with specific application issues—from student learning, through assessment to staff development, with useful asides on virtual learning and multimedia, placements and fieldwork, postgraduate study and professional training.
The d/D does not, it seems, indicate a secondary interest in stuttering, but is a device for inclusively referring to “the Deaf ” who regard themselves as a distinctive cultural/linguistic minority, and “the deaf” who—through the use of signing, lip-reading and so on—regard themselves as integrated with the hearing community
No more than one-sixth (3% out of 19%) of the disabled population currently study for degrees.
Variations in the form of a test are virtually certain to increase variability in outcomes, so reducing the reliability and discriminative capability of the assessment as a whole
** Enthusiasm for removing barriers can make us forget that many stakeholders (employers, for example) primarily value assessments as discriminators: precisely the function that is undermined by excessive enthusiasm for “flexibility” **
The Continual Assessment of Confidence or Knowledge with Hidden MCQ? - Phil Davies 2005
Notion:
MCQ, summative, formative, assessment, cumulative, confidence
Phil Davies implementation of cumulative confidence tests to make MCQs:
- more reliable
- more formative
Certainty-Based Marking (CBM) for reflective learning and proper knowledge assessment - Gardner-Medwin 2007
Notions:
confidence, certainty, MCQ, reliability, reflection, formative, self
Their author used them mostly for formative tests and pre-exam revisions. To encourage self-assessment early in the year alongside coursework, they use follow-up tests that are closely tied to specific practicals or classes
Students see it as helpful and fair
Lucky guesses aren't knowledge and it's incorrect and inefficient to mark an assessment as if it were. Worse, it encourages sloppy habits of thought in students
CBM rewards those who can distinguish their more reliable and less reliable answers. It places a premium on being able to think through a thorough justification for an answer, and it rewards reflection that leads to the conclusion that an answer is less certain than initially thought
CBM aren't ideal and should probably not be used as a sole form of assessment. But in large classes, where there is critical core material (medicine for eg.), they're very helpful for substantial component of assessment (particularly of self-assessment to support learning). Other forms of assessment can stimulate deeper-learning (by being more probing) but can be more difficult to set up on a large scale. Best to use CBM + other forms of assessment
1. Objective testing need NOT simply test factual knowledge and encourage rote learning.
2. Objective testing is for some (not all) purposes BETTER assessment than essays or problems.
3. The notion that you should use 'modern' question formats like single-best-answeror extended matching questions rather than 'outdated' True/False questions is often generalised far beyond any valid supporting evidence we know of. T/F questions are often BEST PRACTICE.
4. It is (common) BAD PRACTICE to include a 'Don't Know' option with T/F or Best-Option Qs.
List of common wrong opinions about objective tests is given
** List of specific staff reactions to CBM and answers from the authors **
Objective tests - Bull and McKenna 2003
Objective: no judgement has to be made on the correctness or otherwise of an answer at the time of marking. Only as objective as the test’s designer makes it.
Saves marking time and make more regular assessment possible
Possible to assess a wide range of knowledge (social sciences, humanities, etc. too)
Well suited to certain tasks:
- test on wide range of materials, provide feedback
- questions that are constructed imaginatively can challenge students and test higher learning levels.
Limits:
- can't assess ability to communicate, construct arguments, ...
- care must be taken to avoid decontextualisation
- wise to use it with other assessment methods
Can be used in a number of ways:
- diagnostic
- self-assessment
- formative assessment
- summative assessment
** Advantages and disadvantages of objective testing are listed **
All 6 levels of Bloom's taxonomy (knowledge, comprehension, application, analysis, synthesis, evaluation) can be tested through objective tests
Objective tests can test across a wide range of knowledge and abilities; however, designing questions to test higher order skills can be time-consuming and requires skill and creativity.
It is important to determine where and how objective tests can be used most productively and effectively within a given course/module.
** Assessment 2.0 Assessment in the age of Web 2.0 - Elliott 2007 **
Notions:
collaboration, digital immigrants, natives, traditional assessment, activities, PLEs
List of problems with traditional assessment methods and with regards to current change in our personal lives
Traditional assessment is expensive, inflexible, prompts memorisation, time-consuming for staffs, individualistic and perceived by students as something external to them.
E-systems mostly imitate traditional systems but online, holding back progress in assessment.
Some considerations of web 2.0 for assessment:
- user-generated content
- power of the crowd (knowledgeable users can sometimes make better decisions than individual experts)
- architecture of participation: easy to use tools, improving as more people use them
- openness (free sharing of information and resources)
Biggest problem is the opposition between digital immigrants and digital natives
** Characteristics of digital natives with respect to their learning styles **
** Characteristics of assessment 2.0 **
** List of web 2.0 tools suited to the characteristics of assessment 2.0 **
Assessment could provide real challenges using real tools
Web 2.0 is collaborative, inexpensive, easy to maintain, very scaleable
Older students and teachers aren't using web 2.0 services routinely.
Considerations about the future
If education is to change, that change has to be led by the assessment system
** E-assessment by design: using multiple-choice tests to good effect - Nicol 2007 **
Notions:
EVS, MCQs, self-regulation, peer
A key message is that the power of MCQs (to enhance learning) is not increased merely by better test construction. Power is also achieved by manipulating the context within which these tests are used.
List of limitations to MCQs: better for lower-order skills (?) limited feedback, recognition rather than construction of answer, no role for students setting the goals and standards
The 7 principles of good feedback practice framework provide a clear lens through which to design and evaluate practice
Applying the 7 principles in relation to MCQs:
1. Clarifying goals, criteria and standards
Having students construct the tests themselves
2. Self-assessment and reflection
Administering MCQ in an open-book situation (quality of the questions particularly important). CBM can be used to increase reflection
3. Delivers high-quality feedback
Enhacing MCQ feedback by relating it to other classroom activities
4. Encourages dialogue around learning
Having students work in small groups to construct tests or to comments on tests. Having students to discuss their answer as they're taking the test or to initiate a class discussion of answers to tests
5. Feedback and motivation
Repeating opportunities to take MCQ tests (highly motivating). Motivation further enhanced when this formative procedure is linked to later summative tests of a similar format
6. Closing the gap
Repeating MCQ tests until a satisfactory performance is reached
6. Feedback shaping teaching
MCQs might be presented before students come to a lecture and even linked to homework assignments. The teacher then use the results of tests to identify areas of learning difficulty and to decide where to focus teaching effort in class or in further online tasks (form of "just-in-time teaching")
** Highly interesting list of case studies discussed against those 7 principles **
Increased power can be leveraged from MCQs when they are linked to a clear pedagogical goal and implemented in relation to a coherent set of principles
Supporting student learning: the use of computer-based formative assessment modules - Peat and Franklin 2002
Notions:
formative, summative, MCQs, self-assessment, connections
** Gives very interesting practical information **
More students, higher diversity, less resources, more support needed for students having higher expectations about formative feedback, especially because of other commitments (job, ...)
Changes implemented in a first year Biology course:
- summative and formative weekly quizzes
Commercial product has saved time and money
Benefits: instant feedback, control over order of questions, preference towards questions including diagrams, photo or graphs, multiple choice format, quick to complete
- mock exam to clarify what good performance is
Paper-based, marked by students in their own time from paper-based or web-based info, enter their answers in the web-based version which gives them feedback (helps them identify their understanding, which in turn might indicate the need for some remedial action). Depending on mark, they might be encouraged to use web-based revision materials
- self-assessment modules (SAMs) designed to draw together related parts of a course to help students make connections between topics and to promote a deeper learning strategy through feedback and reinforcement
Each SAM tests on 4 level of understanding of increasing difficulty, with students comparing their work with sample answers.
At the beginning of each SAM, students are directed to a statement of educational rationale (value of self-assessment mainly) and are informed of what each level is testing. They're encouraged to reflect on their experience.
Benefits: being able to choose the level of difficulty, receiving formative feedback even if they quit the module before the end, helps in revising, in understanding the material (especially for level 1 and 2 more concerned with content itself), in indicating areas of improvement, offering diagrams of ideas and being a different approach than the textbook, *relating concepts*
Mix of formative and summative assessment is recommended
(Most materials mentionned are accessible
Types of e-assessment items - Crisp 2007
Notions:
objective testing, norm-referenced, criterion-referenced, diagnostic, summative, formative, validity, reliability
Criterion-referenced assessment: teacher predetermine the acceptable standard that must be reached
Norm-referenced assessment: compares the performance of students agains others in the cohort (relative ranking is important)
Diagnostic assessment: undertaken before learning has occured to identify knowledge and skill level of students so as to adapt learning acivities.
Formative assessment is inseparable from teaching and provide useful feedback to individual students at a critical point in the learning process (collaboration is beneficial)
Test reliability: a test is reliable if it is consistent within itself and across time
Test validity: the degree to which the test actually measures what it claims to measure
Types of objective tests: text input, numeric input, true/false, MCQ, hot spot, ...
One issue that frequently arises when online assessments tasks are marked is that of relative or comparative equivalence in different student responses (eg: when asked to fill in the blanks)
Objective tests provide a number of practical advantages t oboth student and teacher
** List of a several examples of the use of e-assessment applications that test higer order capabilities **
** List of many of the common forms of e-assessment item types **
Selected Tags
Related Tags
Sponsored Links
Top Contributors
Groups interested in assessment
-
General ICT
Sites for ICT implementatio...
Items: 91 | Visits: 57
Created by: Robin Brigden
-
Technology Assessments
List of assessment tools of...
Items: 23 | Visits: 103
Created by: Lucy Gray
-
21st Century Skills
Items: 170 | Visits: 82
Created by: Michael Walker
Diigo is about better ways to research, share and collaborate on information. Learn more »
Join Diigo