Comparing BSEM and EUPD Estimates for Two-Group SB-MTMM Experiments

In 2018, two different approaches have been suggested to solve the estimation problems that have been detected during the analysis of the data of the two-group split ballot multi-trait multi-method (SB-MTMM experiments performed in many countries between 2002 and 2010 in the European Social Survey). One group suggested using the Bayesian estimation. The other group suggested a new estimation procedure (EUPD) that makes use of the pooled data across the different countries. In this note, we present a comparison of the results of the two approaches on the same generated data, which are comparable with the data that created problems in the ESS.     

Unbalanced 3-Group Split-Ballot Multitrait–Multimethod Design?

A common way of estimating measurement quality is the split-ballot multitrait–multimethod (SB-MTMM) approach. However, this approach leads often to non-convergence or improper solutions when using a 2-group design, whereas the 3-group design performs better. Nevertheless, the 3-group design is rarely implemented because it makes it complicated for applied researchers to use the data. Therefore, we propose to draw groups of unequal sample sizes: two larger groups and one third group as small as possible. Using Monte Carlo simulations and real data analyses, we investigate how well such a design works and which size is needed for the third group. Our results suggest that a 3-group SB-MTMM design with smaller size for the third group (reducing till 5–10%) leads to similar levels of accuracy and no large changes in the model or quality estimates.