Trisha Gao's Bookmarks tagged statistics →  View Popular

This is purely arbitrary, but common social science practice uses a minimum cut-off of .3 or .35. Norman and Streiner (1994: 139) give this alternative formula for minimum loadings when the sample size, N, is 100 or more: Min FL = 5.152/[SQRT(N-2)]. Another arbitrary rule-of-thumb terms loadings as "weak" if less than .4, "strong" if more than .6, and otherwise as "moderate." These rules are arbitrary. The meaning of the factor loading magnitudes varies by research context. For instance, loadings of .45 might be considered "high" for dichotomous items but for Likert scales a .6 might be required to be considered "high."

In confirmatory factor analysis (CFA), a finding that indicators have high loadings on the predicted factors indicates convergent validity. In an oblique rotation, discriminant validity is demonstrated if the correlation between factors is not so high (ex., > ,85) as to lead one to think the two factors overlap conceptually.

• What is simple factor structure, and is the simpler, the better?
  
  
  A factor structure is simple to the extent that each variable loads heavily on one and only one factor. Usually rotation is necessary to achieve simple structure, if it can be achieved at all. Oblique rotation does lead to simpler structures in most cases, but it is more important to note that oblique rotations result in correlated factors, which are difficult to interpret. Simple structure is only one of several sometimes conflicting goals in factor analysis.

In confirmatory factor analysis (CFA), if theory suggests two factors are correlated, then this measurement model calls for oblique rotation. In exploratory factor analysis (EFA), the researcher does not have a theoretical basis for knowing how many factors there are or what they are, much less whether they are correlated. Researchers conducting EFA usually assume the measured variables are indicators of two or more different factors, a measurement model which implies orthogonal rotation. That EFA is far more common than CFA in social science is another reason why orthogonal rotation is far more common than oblique rotation.



When modeling, oblique rotation may be used as a filter. Data are first analyzed by oblique rotation and the factor correlation matrix is examined. If the factor correlations are small (ex., < .32, corresponding to 10% explained), then the researcher may feel warranted in assuming orthogonality in the model. If the correlations are larger, then covariance between factors should be assumed (ex., in structural equation modeling, one adds double-headed arrows between latents).



For purposes other than modeling, such as seeing if test items sort themselves out on factors as predicted, orthogonal rotation is almost universal.

Measured by the Kaiser-Meyer-Olkin (KMO) statistics, sampling adequacy predicts if data are likely to factor well, based on correlation and partial correlation. In the old days of manual factor analysis, this was extremely useful. KMO can still be used, however, to assess which variables to drop from the model because they are too multicollinear.



There is a KMO statistic for each individual variable, and their sum is the KMO overall statistic. KMO varies from 0 to 1.0 and KMO overall should be .60 or higher to proceed with factor analysis. If it is not, drop the indicator variables with the lowest individual KMO statistic values, until KMO overall rises above .60. (Some researchers use a more lenient .50 cut-off).


the
To compute KMO overall, the numerator is the sum of squared correlations of all variables in the analysis (except the 1.0 self-correlations of variables with themselves, of course). The denominator is this same sum plus the sum of squared partial correlations of each variable i with each variable j, controlling for others in the analysis. The concept is that the partial correlations should not be very large if one is to expect distinct factors to emerge from factor analysis. See Hutcheson and Sofroniou, 1999: 224.



In SPSS, KMO is found under Analyze - Statistics - Data Reduction - Factor - Variables (input variables) - Descriptives - Correlation Matrix - check KMO and Bartlett's test of sphericity and also check Anti-image - Continue - OK. The KMO output is KMO overall. The diagonal elements on the Anti-image correlation matrix are the KMO individual statistics for each variable.

Types of Factoring



There are different methods of extracting the factors from a set of data. The method chosen will matter more to the extent that the sample is small, the variables are few, and/or the communality estimates of the variables differ.








• Principal components analysis (PCA): By far the most common form of factor analysis, PCA seeks a linear combination of variables such that the maximum variance is extracted from the variables. It then removes this variance and seeks a second linear combination which explains the maximum proportion of the remaining variance, and so on. This is called the principal axis method and results in orthogonal (uncorrelated) factors. PCA analyzes total (common and unique) variance.
  

  
  

  
  
  • SPSS procedure: Select Analyze - Data Reduction - Factor - Variables (input variables) - Descriptives - Under Correlation Matrix, check KMO and Anti-image to get overall and individual KMO statistics - Extraction - Method (principal components) and Analyze (correlation matrix) and Display (Scree Plot) and Extract (eigenvalues over 1.0) - Continue - Rotation - under Method, choose Varimax - Continue - Scores - Save as variables - Continue - OK.
    

    
    
    

  • Canonical factor analysis , also called Rao's canonical factoring, is a different method of computing the same model as PCA, which uses the principal axis method. CFA seeks factors which have the highest canonical correlation with the observed variables. CFA is unaffected by arbitrary rescaling of the data.
    

    
    

  
  
  
• Common factor analysis, also called principal factor analysis (PFA) or principal axis factoring (PAF), Common factor analysis is a form of factor analysis which seeks the least number of factors which can account for the common variance (correlation) of a set of variables, whereas the more common principal components analysis (PCA) in its full form seeks the set of factors which can account for all the common and unique (specific plus error) variance in a set of variables. PFA uses a PCA strategy but applies it to a correlation matrix in which the diagonal elements are not 1's, as in PCA, but iteratively-derived estimates of the communalities (R² of a variable using all factors as predictors; see below).
  

  
  

  
  
  • Common factor analysis and SEM: Common factor analysis is preferred for purposes of modeling, as in structural equation modeling (SEM). Common factor analysius accounts for the covariation among variables, whereas PCA accounts for the total variance of variables. Because of this difference, in theory it is possible under common factor analysis but not under PCA to add variables to a model without affecting the factor loadings of the original variables in the model. Widaman (1993) notes, "principal component analysis should not be used if a researcher wishes to obtain parameters reflecting latent constructs or factors." However, when commonalities are similar under common factor analysis and PCA, then similar results will follow.
    

    
    

  • PCA vs. common factor analysis. For most datasets, PCA and common factor analysis will lead to similar substantive conclusions (Wilkinson, Blank, and Gruber, 1996). PCA is generally preferred for purposes of data reduction (translating variable space into optimal factor space), while common factor analysis is generally preferred when the research purpose is detecting data structure or causal modeling.

The selection of one technique over the other is based upon several criteria. First of
all, what is the objective of the analysis? Common factor analysis and principal component
analysis are similar in the sense that the purpose of both is to reduce the original
variables into fewer composite variables, called factors or principal components.
However, they are distinct in the sense that the obtained composite variables serve
different purposes. In common factor analysis, a small number of factors are extracted to
account for the intercorrelations among the observed variables--to identify the latent
dimensions that explain why the variables are correlated with each other. In principal
component analysis, the objective is to account for the maximum portion of the variance
present in the original set of variables with a minimum number of composite variables
called principal components.

Secondly, what are the assumptions about the variance in the original variables? If the
observed variables are measured relatively error free, (for example, age, years of
education, or number of family members), or if it is assumed that the error and specific
variance represent a small portion of the total variance in the original set of the
variables, then principal component analysis is appropriate. But if the observed variables
are only indicators of the latent constructs to be measured (such as test scores or
responses to attitude scales), or if the error (unique) variance represents a significant
portion of the total variance, then the appropriate technique to select is common factor
analysis. Since the two methods often yield similar results, only CFA will be illustrated
here.