Ennio Fioramonti's List: psychometrics

•  

     

  Cover Story
• uropsychologist A.R. Luria, as is Kaufman's
• Naglieri's own test, the CAS, is based on the theories of Soviet neuropsychologist A.R. Luria, as is Kaufman's K-ABC
• the challenge is to convince people to give up the traditional scales, such as the WISC, with which they are most comfortable.
• The challenge is convincing people that tests such as the CAS--which do not correlate highly with traditional tests--still measure something worth knowing
• their main concern is with the way tests are used, not with the basic philosophy of testing. But other reformers have launched more fundamental criticisms, ranging from "Emotional Intelligence" (Bantam Books, 1995), by Daniel Goleman, PhD, which suggested that "EI" can matter more than IQ (see article on page 52), to the multiple intelligences theory of Harvard University psychologist Howard Gardner, PhD, and the triarchic theory of successful intelligence of APA President Robert J. Sternberg, PhD, of Yale University. These very different theories have one thing in common: the assumption that traditional theories and tests fail to capture essential aspects of intelligence.
• linguistic, logical-mathematical, musical, bodily-kinesthetic, spatial, interpersonal and intrapersonal. (He has since added existential and naturalist intelligences.) But that formulation has had little impact on testing, in part because the kinds of quantitative factor-analytic studies that might validate the theory in the eyes of the testing community have never been conducted.
• Should we even be using intelligence tests in the first place?
• the child's actual behavior in the classroom and at home is often a better indicator of a child's ability than an abstract intelligence test, so children might get educational services that are more appropriate to their needs if IQ tests were discouraged
• "The movement that's trying to get rid of IQ tests is failing to understand that these tests are valid in the hands of a competent practitioner who can go beyond the numbers--or at least use the numbers to understand what makes the person tick, to integrate those test scores with the kind of child you're looking at, and to blend those behaviors with the scores to make useful recommendations," he says.
• That, he says, removes the focus from a single IQ score and allows for an assessment of the child as a whole, which can then be used to develop individualized teaching strategie

9 more annotations...

• The CHC model was expanded by McGrew (1997) and later revised with the help of Flanagan (1998).
• Recent advances in current theory and research on the structure of human cognitive abilities have resulted in a new empirically derived model commonly referred to as the Cattell-Horn-Carroll theory of cognitive abilities (CHC theory). CHC theory of cognitive abilities is an amalgamation of two similar theories about the content and structure of human cognitive abilities. The first of these two theories is Gf-Gc theory (Cattell, 1941; Horn 1965), and the second is Carroll's (1993) Three-Stratum theory.
• Recent advances in current theory and research on the structure of human cognitive abilities have resulted in a new empirically derived model commonly referred to as the Cattell-Horn-Carroll theory of cognitive abilities (CHC theory). CHC theory of cognitive abilities is an amalgamation of two similar theories about the content and structure of human cognitive abilities. The first of these two theories is Gf-Gc theory (Cattell, 1941; Horn 1965), and the second is Carroll's (1993) Three-Stratum theory.
• Recent advances in current theory and research on the structure of human cognitive abilities have resulted in a new empirically derived model commonly referred to as the Cattell-Horn-Carroll theory of cognitive abilities (CHC theory). CHC theory of cognitive abilities is an amalgamation of two similar theories about the content and structure of human cognitive abilities. The first of these two theories is Gf-Gc theory (Cattell, 1941; Horn 1965), and the second is Carroll's (1993) Three-Stratum theory.
• Recent advances in current theory and research on the structure of human cognitive abilities have resulted in a new empirically derived model commonly referred to as the Cattell-Horn-Carroll theory of cognitive abilities (CHC theory). CHC theory of cognitive abilities is an amalgamation of two similar theories about the content and structure of human cognitive abilities. The first of these two theories is Gf-Gc theory (Cattell, 1941; Horn 1965), and the second is Carroll's (1993) Three-Stratum theory.

3 more annotations...

• Generalizability theory, or G Theory, is a statistical framework for conceptualizing, investigating, and designing reliable observations. It is used to determine the reliability (i.e., reproducibility) of measurements under specific conditions. It is particularly useful for assessing the reliability of performance assessments. It was originally introduced in Cronbach, L.J., Nageswari, R., & Gleser, G.C. (1963).
• The focus of classical test theory (CTT) is on determining error of the measurement. Perhaps the most famous model of CTT is the equation X = T + e, where X is the observed score, T is the true score, and e is the error involved in measurement. Although e could represent many different types of error, such as rater or instrument error, CTT only allows us to estimate one type of error at a time. Essentially it throws all sources of error into one error term. This may be suitable in the context of highly controlled laboratory conditions, but variance is a part of everyday life. In field research, for example, it is unrealistic to expect that the conditions of measurement will remain constant. Generalizability theory acknowledges and allows for variability in assessment conditions that may affect measurements. The advantage of G theory lies in the fact that researchers can estimate what proportion of the total variance in the results is due to the individual factors that often vary in assessment, such as setting, time, items, and raters.

• Cronbach's alpha and internal consistency
• Cronbach's alpha will generally increase as the intercorrelations among test items increase, and is thus known as an internal consistency estimate of reliability of test scores. Because intercorrelations among test items are maximized when all items measure the same construct, Cronbach's alpha is widely believed to indirectly indicate the degree to which a set of items measures a single unidimensional latent construct. However, the average intercorrelation among test items is affected by skew just like any other average. Thus, whereas the modal intercorrelation among test items will equal zero when the set of items measures several unrelated latent constructs, the average intercorrelation among test items will be greater than zero in this case. Indeed, several investigators have shown that alpha can take on quite high values even when the set of items measures several unrelated latent constructs (e.g., Cortina, 1993; Cronbach, 1951; Green, Lissitz & Mulaik, 1977; Revelle, 1979; Schmitt, 1996; Zinbarg, Yovel, Revelle & McDonald, 2006). As a result, alpha is most appropriately used when the items measure different substantive areas within a single construct. When the set of items measures more than one construct, coefficient omega_hierarchical is more appropriate (McDonald, 1999; Zinbarg, Revelle, Yovel & Li, 2005).

• Summary

   

  Graphic of person and item distributions for a test produced in RUMM2020 software, RUMMlab, Perth: www.rummlab.com.au
• 

• Rasch model

• Rasch models are used for analysing data from assessments to measure things such as abilities, attitudes, and personality traits. For example, they may be used to estimate a student's reading ability from answers to questions on a reading assessment, or the extremity of a person's attitude to capital punishment from responses on a questionnaire.

   

  Rasch models are particularly used in psychometrics, the field concerned with the theory and technique of psychological and educational measurement. In addition, they are increasingly being used in other areas, including the health profession and market research because of their general applicability.
• Scaling
• Interpreting scale locations
• Features of the Rasch model
• Other considerations
• A criticism of the Rasch model is that it is overly restrictive or prescriptive because it does not permit each item to have a different discrimination. A criticism specific to the use of multiple choice items in educational assessment is that there is no provision in the model for guessing because the left asymptote always approaches a zero probability in the Rasch model.
• Baker, F. (2001). The Basics of Item Response Theory. ERIC Clearinghouse on Assessment and Evaluation, University of Maryland, College Park, MD. Available free with software included from IRT at Edres.org
• Wu, M. & Adams, R. (2007). Applying the Rasch model to psycho-social measurement: A practical approach. Melbourne, Australia: Educational Measurement Solutions. Available free from Educational Measurement Solutions
• 
• 

8 more annotations...

• La mesure en psychologie (questionnaires, tests, modèles structuraux)
• - Comprendre les postulats, concepts et conditions d'application du modèle classique de la mesure (True Score Theory).
   - Comprendre les postulats, concepts et conditions d'application des modèle de la réponse à l'item (MRI), en particulier du modèle de Rasch.
   - Maîtriser procédures de base d'analyse classique et MRI, y compris l'interprétation des résultats.
   - Comprendre les postulats, concepts et conditions d'application de l'analyse factorielle y compris confirmatoire et des modèles structuraux d'équations dans les problèmes de mesure. Maîtriser les procédures de base de l'analyse factorielle et des modèles structuraux d'équations dans ces problèmes.

• Introduction au modèle de Rasch

   

  Traduction (partielle) du chapitre 16 de Messen und Testen, Rolf Steyer et Michael Eid. (2001). Berlin : Springer.