IRT analysis – method notes #

The decisions behind the IRT analysis module – which threshold, estimator, refusal or fallback stands behind each card, and the numbers that validate them – on simulated item sets where the note says so, and otherwise on the example dataset (cafe.csv, its spec beside it). Sections follow the manual’s.

Requirements #

A Continuous column is taken as an item only at 3–10 distinct integer values. A polytomous model spends a threshold parameter on every category of every item, so a column with many distinct values is far more likely a score, a count or a measurement than a rating item, and a non-integer value is no response category at all; a column typed Ordinal is taken at its word, whatever its width, and two distinct values are a dichotomous item under any type. An item with more than half its responses missing is dropped before the classification, since its parameters would rest on a minority of the respondents; and the DIF grouping variable is removed from the items even when selected, since an item that defines the groups cannot also be tested across them.

Preliminary analysis #

The sample-size grade is a rule of thumb per model, read on the N the fit will see. The count is taken after listwise deletion when that is the missing-data setting, so the warning speaks of the same N the model will be fitted on. The bars rise with the parameters an item carries – 100 for any model, 200 once each item has its own discrimination (2PL, and Auto, which is graded at the 2PL’s bar), 500 for a lower or upper asymptote (3PL, 3PLu), which only respondents at the extremes of the trait inform, and 1000 for both (4PL). They are guidelines rather than power calculations – a smaller sample can fit, with less stable estimates – and they do not scale with the number of dimensions.

The unidimensionality eigenvalues come from one correlation estimator per item width. The items are split as psych’s mixedCor splits them: two categories tetrachoric, 3–8 polychoric, wider Pearson. polychoric() stops outright above 8 categories, and past about seven the categorical and continuous treatments converge anyway, so a wide item goes through Pearson rather than pushing the whole matrix there. With no item wider than 8 the whole matrix is polychoric(correct = 0), with no continuity correction for empty cells; with every item wider it is Pearson on the pairwise-complete cases; and a categorical fit that fails falls back to that Pearson matrix, with R’s message on the card.

Reckase’s criterion reads both quantities, and the ratio only when the second eigenvalue is positive. The first component’s share of the variance is its eigenvalue over the item count – the trace of a correlation matrix – and Reckase (1979) put the share a unidimensional calibration needs at 20%; the ratio of the first eigenvalue to the second, at 3 or more, asks in addition that no second dimension rivals the first. The two are applied together, so one met alone is reported as mixed evidence rather than a pass. A pairwise or smoothed correlation matrix is routinely indefinite, and a second eigenvalue at or below zero makes the ratio negative or infinite rather than large, so it is left undefined and only the variance criterion is judged.

Four of the subject-quality cutoffs are Tukey fences on this dataset’s own indices. The absolute values these indices are usually quoted with – longstring ≥ 5, IRV < 0.5, a person-total r < 0.1 – have no null distribution behind them: each index’s distribution is set by the item count and the number of response categories, and on clean data those values flag between a quarter and two-fifths of the respondents. The longstring, variability, correlation and reliability cutoffs are therefore 1.5 × IQR beyond the quartiles of the index’s own distribution – above the third for longstring, below the first for the other three – so a flag means “unusual in this sample”. An index whose IQR is zero or undefined gets no fence and flags nobody, shown as “–”. The high-missing bar (more than 50% of the items) and the Mahalanobis bar (p < .001) stay absolute, since each has a meaning of its own – a share of the scale, a χ² tail. Any fence flags the tail of every distribution by construction, so the card counts the respondents flagged by two or more indices, where the evidence converges.

The four fenced indices, as computed. Longstring is the longest run of equal values among the person’s answered items, in item order. IRV is the SD of the person’s answers divided by $(r/2)\sqrt{n/(n-1)}$ – the largest SD n answers can have on a scale whose observed range, over the whole item matrix, is r – so that scales of different widths compare; on an all-dichotomous scale it is a deterministic function of the sum score and carries nothing the others do not, so it is not screened there. The person-total correlation correlates the person’s answered items with the item means of everyone else, the person left out, and needs three answered items. Resampled individual reliability splits the items at random into two halves of ⌊k/2⌋, correlates the person’s two halves over the items answered in both – at least three – applies the Spearman–Brown correction, bounds the result to [−1, 1], and averages 30 such splits, drawn under the reproducibility seed; a single even–odd split depends on which split was drawn, which the resampling removes. Below six items no half holds three items, and the index is not computed.

Mahalanobis distances use the minimum covariance determinant when the sample allows it. The sample mean and covariance are inflated by the outliers they are meant to expose, so the centre and scatter come from MASS::cov.rob(method = "mcd"), under the reproducibility seed. MCD needs a margin of cases over items: with no more than twice as many complete cases as items, or when the robust fit fails or its covariance is singular – the norm on items with few response categories – the classical moments of the complete cases are used, and the card names which limitation applied. With no more complete cases than items plus one, no distance is computed. The squared distance is referred to χ² on as many df as items, and a respondent with a blank has none.

Mokken runs on the complete cases, recoded to consecutive integers. The mokken package rejects a missing response outright, so the whole section runs on the complete rows while the checks above keep every row. Each item’s values are recoded to 0, 1, 2, … in their sorted order before the call: mokken enumerates item steps over the full observed range, so an unendorsed intermediate category or a gapped code costs orders of magnitude of runtime for the same coefficients. With fewer than three complete cases, or an item that is constant among them, no coefficient is defined and the section stops.

H is graded in bands for the scale and by admissibility for an item. The scale’s H is read in Sijtsma and Molenaar’s (2002) bands – 0.3 weak, 0.4 moderate, 0.5 strong, unscalable below 0.3. An item’s Hi is not graded for strength: 0.3 is the lowerbound a Mokken scale’s definition sets for every item in it, so an item row reads Admissible at or above it and Inadmissible below. The coefficients are coefH(nice.output = FALSE), whose plain numeric matrices carry the standard errors separately.

Violations are graded by crit, in Molenaar and Sijtsma’s bands. Monotonicity, invariant item ordering and nonintersection are check.monotonicity, check.iio and check.restscore at the package’s defaults. Each summary counts the violations above the minimum size, the significant subset, the largest, and its test statistic – z, except under check.iio on polytomous items, which switches to a t-test – and crit combines them into one seriousness index; Molenaar and Sijtsma (2000) read crit below 40 as no serious violation, 40–80 as minor and above 80 as serious, and the card grades each check’s largest. Invariant item ordering is also summarised by the scale’s H_T, read at 0.3 (Ligtvoet et al. 2010).

AISP sweeps the lowerbound, with the sequential search stated. aisp() takes a vector of lowerbounds and returns one partition per value; reporting the sweep – 0 to 0.55 in steps of 0.05, plus the user’s c rounded to three decimals – is what tells a stable partition from an artefact of the chosen c. The search is search = "normal", Hemker, Sijtsma and Molenaar’s (1995) sequential procedure, set explicitly rather than left to the default; it is greedy, so it can settle in a local optimum rather than the best available partition, which is why neighbouring columns are read together. aisp drops any lowerbound above the largest item-pair H, since nothing would be scalable there, so the columns are the lowerbounds it actually searched, and a user’s c among the dropped ones has no column and no verdict. The field’s value is clamped to 0–0.99, and an empty or unreadable one is read as 0.3, since the input is in no form and the browser enforces neither bound.

Dimensionality #

Rotation reaches the loadings solution alone; θ stays in the fitted basis. mirt() has no rotation argument: an exploratory fit is rotated where it is read, by summary(rotate = …), which builds the loadings table, its heatmap and the factor correlations – and the choice is passed even when it is None (unrotated), since summary()'s own default is oblimin. fscores() defaults to oblimin as well, while expected.test(), testinfo() and probtrace() have no rotation hook at all, so θ is scored with rotate = "none" and every θ-derived output – person scores, reliability, expected scores, the Wright map, the information curves – shares the one basis the curves can be drawn in. A confirmatory model is never rotated: its structure is the one you fixed.

Oblimin and the Crawford–Ferguson pair take their parameter from the panel, defaulting to GPArotation’s. mirt hands every criterion but promax to GPArotation, and forwards summary()'s gam = and kappa = to it; the panel’s Oblimin γ and Crawford–Ferguson κ fields fill them. Each defaults to 0 – GPArotation’s own default – at which oblimin is quartimin and the Crawford–Ferguson family is quartimax in its orthogonal form and quartimin in its oblique one, so at the defaults the three duplicate other entries; the parameter is what makes each a family rather than a second name. Higher γ lets the factors correlate more. κ is bounded to [0, 1], where 1/p for p items is varimax and k/(2p) for k dimensions equamax (Browne 2001); a blank or out-of-range value is refused rather than passed on. No GPArotation criterion is Kaiser-normalized, varimax included (GPArotation::Varimax(normalize = FALSE)). Promax is mirt’s own: stats::varimax, which normalizes, followed by a target rotation at power 4.

Validation. On a simulated two-dimensional 2PL item set – twelve items, 500 respondents – at the defaults the oblimin and quartimin loadings agree exactly, and the Crawford–Ferguson T and quartimax loadings to 2 × 10⁻¹⁶, where geomin Q differs from oblimin by 0.005. Away from the defaults the solutions part: oblimin at γ 0.5 gives loadings distinct from quartimin’s, and on a two-dimensional fit the factor correlation rose from 0.13 to 0.83 as γ ran from −0.5 to 1.

Correlated factors are declared as a COV line in the model syntax. mirt’s COV = TRUE argument is silently ignored, so a correlated confirmatory model writes every factor pair into the syntax (COV = F1*F2, …), and the line is how edited syntax carries the setting back to the checkbox. A factor name must survive the same syntax: a space, =, , or * splits the generated F = 1, 2 line, a quote breaks the mirt.model('…') string, a . comes back from R rewritten to _, so the fitted rows no longer match the name, and a factor named after one of mirt’s statement words (COV, MEAN, CONSTRAIN, PRIOR, START, FIXED) turns its own line into that statement.

Model types #

The dropdown offers what mirt can fit in the chosen structure. The rating-scale models (Rasch, GRSM in both parameterizations, the Andrich rating scale model) carry equality constraints that mirt does not generalize to several dimensions; the unfolding models (ideal point, GGUM) are single-trait by construction; and the nonparametric curves are not supported multidimensionally – so all eight are unidimensional only. mirt refuses the partially compensatory models outside a confirmatory model (“Partially compensatory models can only be estimated within a confirmatory model”). A model written for one response format fails inside mirt on the other – the dichotomous ones with “requires exactly 2 unique categories”, GRSM inside its category-probability assignment – so the app refuses the run beforehand and names the items.

GGUM is fitted with L-BFGS-B unless another optimizer is chosen. mirt switches to nlminb whenever a parameter is bounded, and GGUM’s discrimination is; on that path the fit dies with “NA/NaN gradient evaluation” instead of converging. L-BFGS-B is the fastest optimizer that does fit the family, so the run supplies it unasked and the card says so; an optimizer picked under Advanced tuning replaces it.

GGUM starts each ideal point from a one-dimensional scaling of the items. mirt 1.46.1, the build the app loads, sets a GGUM item’s starting ideal point from a correspondence analysis of the category-expanded response matrix but reads it by item index from a table indexed by category column, so item i starts at the score of the i-th category column – a value unrelated to where the item sits. From those starts the fit settles in a local optimum: the item on one side of the scale drops to the discrimination bound, and every comparison with the graded model then goes the wrong way. The run therefore writes its own start into the b1 rows of a pars = "values" table – each item’s coordinate on a classical multidimensional scaling of 1 − Spearman r over the items, cmdscale(k = 1) scaled to SD 1 and oriented so that the column order reads as increasing – and passes it on every GGUM fit: the primary one, each Q3 bootstrap refit, rebuilt from the simulated data, and the DIF multiple-group fits. The discriminations and latitudes keep mirt’s starts. Several random starts reached the same optimum but multiply a fit that already takes about a minute.

Validation. On six 4-category statements generated from GGUM itself (discrimination 1.5, ideal points −1.5 to 1.5, n 500), mirt’s default starts ended at a log-likelihood of −3826 with one item at a = 0 and AIC preferring the graded model, where the scaling start reached −3663.4 – the optimum a start at the generating values also reaches, above the −3678 at those values – with every ideal point within 0.06 and in order. At n 200 eight of eight fits from it reached the good optimum, and AIC preferred GGUM over the graded model by 55 to 110; the grouped DIF fit reached −1473.05 from the start table against −1541 from mirt’s defaults.

Spline knots are placed by position, and the fit starts from a rising curve. mirt’s two defaults for the spline family cannot fit it. It places no interior knot, which fits a flat response curve carrying no information about the trait; and a knot count given as df is spread over mirt’s quadrature grid, a fixed −6 to +6 whatever the data, so most knots land in tails no response reaches. The run therefore passes 1–3 knots at even positions between −2 and +2. mirt also starts every spline coefficient at 1 – a flat curve, and a symmetric stationary point the EM cannot leave once the item difficulties sit near the trait mean – so the coefficients start on an increasing sequence from −1 to 1 instead.

Validation. On one item set the default start left θ with an SD of 3.6 × 10⁻¹⁴ – every respondent at the same score – where the monotone start gave 0.87.

The spline family is fitted without standard errors and reports no information. mirt cannot size a spline item’s block of the information matrix, so asking for standard errors aborts the fit itself (“logical subscript too long”). It also implements no θ-derivative for the class, so testinfo() and iteminfo() return exactly 0 at every θ rather than an error, and every quantity built on them – the marginal reliability integral, the information curves, the conditional SEM – would be a zero that reads as a measurement. Those outputs are withheld and the card says why; the fitted curves do not depend on the derivative and are reported.

Estimation method #

mirt’s per-method defaults stand unless Advanced tuning overrides them. The run passes only the arguments you changed, so each method keeps the settings mirt tunes for it – 500 EM cycles against MHRM’s 2000, among others – and a change in one method’s defaults between mirt versions reaches the app unchanged.

A multidimensional fit computes standard errors only where they are affordable. The information matrix is what makes an EM or QMCEM fit of several dimensions expensive, so standard errors are requested for every unidimensional fit, for a multidimensional one under MHRM – which estimates them along the way – and whenever an SE calculation other than the default is chosen; the card says when they were skipped, and whether for cost or because the family (spline) cannot carry them.

Above three dimensions the integration is quasi-Monte Carlo. A full quadrature grid has as many nodes as its points per dimension raised to the number of dimensions, so the fit, the M2 statistic and the θ scores all take mirt’s QMC = TRUE beyond three, as mirt recommends; the note under the control flags plain EM from three dimensions on, where the grid is already the cost.

Scoring method #

Empirical reliability takes the form the scoring method needs. mirt::empirical_rxx() offers two identities. The variance of maximum-likelihood estimates (MLE, WLE) already holds the measurement error, so reliability is $1-\overline{S{E}^2}/\mathrm{var}(\hat{\theta })$ (T_as_X = TRUE); Bayesian estimates (EAP, MAP) are shrunk, which takes the error variance out of them, so it is added back: $\mathrm{var}(\hat{\theta })/(\mathrm{var}(\hat{\theta })+\overline{S{E}^2})$. Applying one form to both would overstate one family’s reliability or understate the other’s, which is why a number compared against another package should be compared under the same scoring method.

Non-finite θ leaves the person-level outputs and stays in the model-level ones. Under MLE an all-minimum or all-maximum pattern scores ±∞. mirt’s model-level calls need a θ row for every respondent, so they keep the full matrix, while every person-level statistic – scores, fit, reliability, targeting – drops the respondents whose θ or SE is not finite, and the card counts them.

Advanced tuning #

A tuning choice the method or the structure cannot take is greyed out rather than refused. mirt refuses a non-default SE.type on its stochastic estimators – every type under MHRM and SEM (“SE.type not supported for stochastic method”), all but complete-data under MCEM – and refuses it even when no standard errors are requested, so the whole run would die rather than the standard errors; those choices are disabled for the method, and a selected one reverts to Default. The empirical-histogram density is defined for unidimensional EM alone. A quadrature grid has as many nodes as its points per dimension raised to the number of dimensions, so Very fine (121) at three dimensions would ask for about 1.8 million; the finer grids are unidimensional only, and mirt’s own count already falls as dimensions are added.

Differential item functioning (DIF) #

The baseline depends on whether anchors were designated. A constant anchor set means something only when the tested items are free to differ from it. With no anchor, every item is held equal across the groups and each tested item is released in turn (DIF(scheme = "drop")) – each item anchored on all the others, a baseline that a biased item contaminates when DIF is widespread. With anchors, only they are held equal, every other item is free in the baseline, and each tested item is constrained in turn (scheme = "add") – the constant-anchor design of Woods (2009). Either way the latent mean and variance are free in every group but the reference. Anchors known to be invariant give the strongest inference; purification is the fallback when none are known, and better than all-other-items anchoring.

Purification keeps a fifth of the items as anchors, at least four. It is Woods’ (2009) two-stage selection: a preliminary all-other-items pass ranks every item by its χ², the items with the smallest become the constant anchor set – round(k ⁄ 5) of the k items, no fewer than four and at most k − 1, so something is left to test – and the model is refitted on them before the rest are tested. Designated anchors are the user’s own answer to the same question, so the option applies only while none is selected.

The tested parameters are read off the fit, and the asymptotes stay equal. No parameter name is common to every family – a graded item has a1 and d1…dk, a Rasch item d alone, the Andrich rating scale item its location c – so a test contrasts whichever of an item’s slopes, intercepts and location the multiple-group model estimates, and df counts them. The guessing and upper asymptotes of 3PL, 3PLu and 4PL are not among them and stay equal across groups. The non-uniform test releases the slopes alone, and is run only when a model estimates slopes and something besides.

Each pair is tested on a model refitted to its two groups. mirt’s own pairwise = TRUE is vacuous under scheme = "drop": removing two of G parameters from an equality constraint leaves it non-binding, so every pair would return the omnibus test. A refit to the pair alone is also what reaches the effect sizes, since empirical_ES() supports two groups only – which is why the main table carries SIDS, UIDS and ESSD only for a two-group variable. Every item × pair test is adjusted as one family.

Cases with no group are dropped, and groups under 100 are warned. multipleGroup() needs a group for every row it fits, and a missing value would otherwise become a level of its own that sorts first – the reference group, whose latent mean and variance are fixed. Each group is fitted its own item parameters, so the per-group N drives the stability of the tests as the total N drives the model’s, and it is graded against the same 100 the sample-size check uses.

Negatively keyed items #

Detection starts from the first principal component and flips items until no rest correlation is negative. Each item is signed by its loading on the leading eigenvector of the pairwise-complete item correlations; then, repeatedly, the item with the most negative correlation with the sum of the others – every item multiplied by its current sign – is flipped, until none is negative. Each flip raises the rescored total’s variance, so the loop ends. An item whose final |r| is under 2/√n, n the smallest pairwise count, is too weakly related to the rest to place and is left unselected as unclassifiable. The majority keeps its direction; on a tie the group holding the first classifiable item does, and the line beside the button names it. The raw item-total correlation, the obvious alternative, fails exactly where keying is balanced: with as many items keyed each way the raw total cancels, and both its sign and a band on it misplace nearly every item. The rule is pure JavaScript shared with the Reliability module’s picker, so Detect needs no R round trip.

Validation. In a simulation of 200 samples per cell – keying 6+0, 5+1, 4+2, 3+3 and 5+5, trait loadings 0.2 to 0.8, acquiescence loadings 0, 0.3 and 0.5, n 50 to 500 – the rule misplaced an item in at most 1.5% of samples from n 200 at loadings of 0.4 and above with acquiescence at 0.3 or less, balanced keying included (at most 7% at n 50), and left fewer than half an item unclassifiable on average; the raw item-total sign put an item on the wrong side in every sample at 3+3, loading 0.8, n 200. The JavaScript routine matches the R reference on 60 generated datasets, missing data and constant items included; on the café items with slow unreversed it selects slow.

The list, not detection, decides, because acquiescence can hide a reversed item. When agreeing with everything loads on the responses at least as strongly as the trait does, a reversed item’s covariance with the rest, $-{\lambda }^2+{\gamma }^2$ for trait loading λ and acquiescence loading γ, is not negative, so no sign rule can find it – and a larger sample only makes the wrong answer more certain. The keying is a property of the questionnaire the user usually knows, so detection only proposes it. Every run also applies detection to the data as the analysis reads it, after the declared keying, and names any item still keyed against the rest – keying forgotten, doubled, or applied to only some pre-reversed items.

Validation. In the same simulation, with five items keyed one way and one the other, trait loading 0.4 and acquiescence 0.5, every rule misplaced the reversed item in 12% of samples at n 50 and 74% at n 500.

The careless-responding screen reads the answers as given. Longstring and IRV look for a pattern across the items as the respondent saw them: an attentive respondent who agrees with every positive statement and disagrees with the one reversed statement shows a broken run on the form and an unbroken one once the item is reversed. So the fit and the other diagnostics read the keyed items reversed while those two read the answers as given, and a file reversed before import is un-reversed for them when Already reversed in the data is ticked – which detection cannot infer, since a pre-reversed item looks positively keyed. Reversal mirrors around the scale’s range, declared or else the extreme observed across all the items, not each item’s own: longstring compares answers across items, and a per-item mirror would shift an item nobody answered at one end off the others’ scale and break the runs. Exploratory MIRT leaves signs to the rotation, and a confirmatory model reads the data as it is.

Validation. On the café items with slow unreversed and selected, the graded slopes are all positive and match a fit on data with slow reversed beforehand to 10⁻³, and members 13, 15, 69 and 165 – each flagged by longstring and IRV on the pre-reversed file – are no longer flagged by two indices; the pre-reversed file with Already reversed in the data ticked gives the same screen.

Reading results #

Model fit #

M2 is mirt’s M2*, named as such on polytomous items. M2() computes the limited-information statistic in its M2* form by default, which on dichotomous items is M2 itself; the row is headed M2* whenever any item is polytomous, so the statistic is reported under the name of what was computed. RMSEA with its 90% interval, SRMSR, TLI and CFI come from the same call. M2() refuses a model that leaves it no degrees of freedom and can fail on a multidimensional fit, so it runs in its own error handler: the card then prints R’s message and keeps AIC, BIC and the log-likelihood, which the fit itself carries.

Model comparison #

The comparison pair is fixed by the response format, and refits reuse the model where they can. Rasch is 2PL with the discriminations constrained equal, so that pair takes a likelihood-ratio test; GRM and GPCM model different response processes with the same number of parameters, and neither nests the other, so they are compared by AIC and BIC alone. A mixed scale is refitted twice with 2PL on its dichotomous items and the graded or partial-credit model on the rest, so that only the polytomous parameterisation differs between the two fits. A pair member that is the model already fitted is reused rather than refitted, and a fitted model outside the pair is added as its own row, with a test against 2PL only where it nests it – 3PL, 3PLu and 4PL free asymptotes that 2PL fixes.

Dimensionality comparison #

Each smaller dimensionality is a refit, tested against its neighbour. The rows below the chosen count are fitted afresh, under quasi-Monte Carlo integration above three dimensions as the main fit is, and each is tested against the next by anova(); a test that fails leaves its row with the information criteria alone, rather than losing the table.

Confirmatory vs exploratory fit #

The constraint test is omitted when the confirmatory model is not a restriction. A specification that loads every item on every factor and frees their covariances estimates at least as many parameters as the exploratory model it is compared against; anova() then returns a non-positive df with a p of NaN rather than refusing, so the app reads the df and, when it is not positive, drops the test and says the models are not nested.

Item statistics #

Location standard errors are carried to the IRT metric by the delta method. mirt reports standard errors only in its slope–intercept metric, while the table shows locations $b_k=-d_k/a$. Each location’s SE is the delta-method transform over its own $(a,d_k)$ block of vcov(), with gradient $(d_k/a^2,-1/a)$; a parameter the model fixed is absent from vcov() and contributes no variance. The asymptotes are estimated on the logit scale, and their standard errors are reported there under an SE(logit …) header rather than beside the probability-scale estimate unlabelled. The nominal model’s estimates are read in the IRT metric while its SE table is always in the slope–intercept one – different parameters under the same names – so no standard error is shown for it.

GRSM thresholds are read in the slope–intercept metric and shifted per item by hand. mirt stores both GRSM selections as its grsmIRT class, on which coef(IRTpars = TRUE) errors, so the coefficients are read with IRTpars = FALSE: a slope per item, one shared set of step parameters $b_k$ and an item shift $c_i$, with $P(X\ge k+1)=\mathrm{logistic}(a(\theta +b_k+c_i))$ – checked on the engine. Each item’s thresholds are therefore the θ-metric boundaries $-(b_k+c_i)$, the shared spacing moved to the item, with standard errors by the delta method over the $(b_k,c_i)$ block of vcov(); the shift is folded into the boundaries rather than printed as a column of its own, and the note under the table says the set is shared.

Discrimination is graded on Baker’s bands. Baker (2001) reads a logistic-metric slope as very low, low, moderate from 0.65, high from 1.35 and very high from 1.70; mirt reports slopes in that metric, without the 1.7 scaling constant, so the bands apply to the printed values as they are. The table folds them into three – red below 0.65, amber below 1.35, uncoloured from there – and the note and the citation appear only when the table grades one slope per item, a unidimensional fit with no nominal, unfolding, spline or monotonic polynomial item.

A family without a slope and a location gets its own columns. The generic layout names what it recognises and mislabels the rest – a nominal a1 is an overall scaling slope, not a discrimination, and a GGUM b1 is the ideal point, not a threshold – so a scale fitted with one such family throughout is laid out in that family’s own parameters, and the discrimination and difficulty bands, which describe quantities it does not estimate, give way to a note on the family. A scale mixing families keeps the generic layout.

MDIFF on a polytomous item is an approximation. Reckase’s multidimensional difficulty, $-d/\text{MDISC}$, is defined for an item with one intercept. A polytomous item has one per step, and the table takes their mean, which locates the item’s centre along its direction of best measurement but is no standard IRT quantity – so the column is headed Mean location whenever any item has several intercepts.

The mean-square band is a Rasch convention. The 0.5–1.5 band for infit and outfit, and the 2.0 above which an item degrades measurement, were set for Rasch-family models, where the mean squares’ distribution under the model is known; with slopes that vary between items the statistics are distributed differently, so under any other model the band is shown for reference and not as a verdict.

Local dependence #

Only Q3* is flagged, at 0.2 unless bootstrapped. Raw Q3 carries a negative bias of about −1/(J − 1) for J items under local independence, which on a short test reaches the cutoff by itself; Marais’ (2013) Q3* subtracts the mean off-diagonal Q3 to centre it, and the flag reads Q3* alone. The 0.2 cutoff is a rule of thumb that takes no account of test length, sample size or response format.

The bootstrapped cutoff is a critical value for the largest |Q3*|. Each replication simulates a dataset of the same size from the fitted model (simdata()), refits the model with the same specification and no standard errors, and records the largest |Q3*| over every pair; the cutoff is the (1 − α) quantile of those maxima over the replications that succeeded, so it holds the chance of any false flag across all pairs at α. On short scales it often sits well above 0.2, and pairs the rule of thumb flags can drop out.

LD-X² is flagged on the adjusted p, over every pair. residuals(type = "LD", df.p = TRUE) returns Chen and Thissen’s (1997) statistic in the lower triangle and its p in the upper. The adjustment family is every item pair, not only those flagged, and the flag reads the adjusted p against a fixed .05; with no correction set the adjustment is the identity.

Person fit #

Zh counts both tails against their chance rate. Zh is the standardized log-likelihood $l_z$ of personfit(): negative for aberrant patterns, positive for patterns more consistent than the model expects, so the two tails are counted apart. Under the model each tail beyond 2 holds Φ(−2) ≈ 2.3% of respondents by chance, the rate the note prints; computed on estimated rather than true θ its variance falls below 1, so it under-detects misfit. The infit and outfit cutoff of 1.5 is the Rasch mean-square convention, which is why the note calls Zh the better indicator under other models.

Reliability and separation #

Marginal reliability integrates against the model’s own latent variance. mirt::marginal_rxx() integrates $I/(I+1)$ against a standard normal, which matches the definition only when the model fixes the latent variance at 1; the Rasch family frees it. The app therefore integrates $I(\theta )/(I(\theta )+1/{\sigma }^2)$ against $N(0,{\sigma }^2)$ with the fitted σ², over ±10 SD of that distribution rather than the whole line: mirt returns NaN probabilities for the GGUM far out on θ, which left its integral undefined, and the tails beyond 10 SD carry no weight any other family’s value can show. It is unidimensional only, and N/A for the spline family, whose information mirt returns as 0.

Separation and strata follow Wright and Masters. Separation is $G=\sqrt{R/(1-R)}$ from the reliability, left undefined outside 0 < R < 1, and the number of strata is $(4G+1)/3$ (Wright and Masters 1982): the count of levels three standard errors apart that the spread can hold.

Item reliability rests on delta-method location standard errors. It is $1-\overline{S{E}^2}/\mathrm{var}(\text{location})$ over the items, each location $-\bar{d}/a$ with its SE by the delta method over the item’s vcov() block – which is why it needs the standard errors and reads N/A without them. The index is Rasch measurement’s, where one location is the whole item; with slopes free it describes the locations alone, and the row is labelled Rasch-style.

A multidimensional targeting is read along each dimension. MDIFF places an item along its direction of best measurement, a composite on a different scale from any one θ, so a dimension’s targeting takes each item’s location on that dimension alone, $-\bar{d}/a_k$, over the items whose largest absolute slope is on it.

Score conversion table #

Equating is read off a fine grid, and EAPsum is left to mirt. True-score equating finds, on a θ grid from −6 to 6 in steps of 0.01, the point whose expected total score is nearest the raw score, and takes its SE as $1/\sqrt{I(\theta )}$ there. The lowest and highest raw scores map to θ = ∓∞, outside the range any grid’s expected scores reach, so they read N/A rather than the grid’s endpoint. The EAPsum columns are fscores(method = "EAPsum", full.scores = FALSE) under the model’s Gaussian prior, which shrinks toward the mean where equating diverges; the call fails on families with no defined sum-score likelihood (nominal, unfolding), and the columns are then omitted.

Plots #

Every curve is drawn on one θ grid, widened to reach the items. The grid runs from −4 to 4 in steps of 0.05, extended to one unit past the lowest and highest item location so that no item’s curve is cut off before it turns; a model with no location parameter keeps ±4. The information figure and the conditional reliability section share the grid, so the two cannot disagree about where a curve starts and ends.

A multidimensional curve is a slice through θ = 0 on the other dimensions. An item’s response curve, information and expected score are computed along its primary dimension – the one with its largest absolute slope – with the other dimensions held at 0, and its information in that direction (iteminfo(degrees = …)); the test information, its standard error $1/\sqrt{I}$, the test characteristic curve and the conditional reliability are computed the same way for each dimension in turn. The curves of one figure therefore sit on different slices of the latent space while sharing the axis under them, which is what the note under the figure says.

The dichotomous curve is the higher response, and P = 0.5 is a landmark. The overlay draws the probability of the item’s higher response code, whatever the two codes are, since “correct” holds only for keyed 0/1 items. The dashed line marks P = 0.5 rather than the item location: under a guessing or upper asymptote the location sits where the curve reaches $(c+u)/2$, not 0.5.

The expected score curves are on the raw score scale. expected.item() counts categories from 0, so each item’s curve is shifted by its lowest observed value (min = …) onto the scale expected.test() reports the test characteristic curve on, and the item curves sum to it.

Conditional reliability is computed from the information and the model’s latent variance. The curve is $I(\theta )/(I(\theta )+1/{\sigma }_{\theta }^2)$, mirt’s definition: the same quantity as $1-SE{(\theta )}^2/{\sigma }_{\theta }^2$ with $SE=1/\sqrt{I}$, but read off the information function, so it does not depend on the scoring method’s point estimates. σ² is the fitted latent variance – 1 unless the model frees it, as the Rasch family does – as in the marginal reliability above, and on a multidimensional panel each dimension’s own. The reference line at 0.70 is where the shared reliability bands turn from Questionable to Acceptable. The standalone section is unidimensional only; a multidimensional fit draws its per-dimension curves as a panel of the information figure.

The Wright map places each item where the targeting does. On a unidimensional model an item’s point is its location – the difficulty, the mean of the thresholds on a polytomous item, $-d/a$ for the ideal-point model – and the threshold ticks restore the spread that mean hides. MDIFF places an item along its direction of best measurement, a composite on another scale from any one θ, so a multidimensional panel places each of its items at $-\bar{d}/a_k$ on the panel’s dimension instead, the location the multidimensional targeting reads.

Missing data #

Pairwise deletion fits every row on the items it answered. mirt’s marginal likelihood is a product over a person’s answered items, so a missing response drops out of that person’s likelihood rather than taking a value – the full-information treatment, which holds when responses are missing at random. Listwise deletion discards the answered items of every incomplete row as well. Imputed responses enter the fit as observed, so the standard errors and reliabilities read as though nothing had been missing.