Correlation analysis – method notes #

The decisions behind the correlation analysis module – which estimator produces each coefficient, where a correction or a cap comes from, what a run refuses or warns about before it starts, and the numbers that validate them, each computed on the example dataset (cafe.csv, its spec beside it). Sections follow the manual’s.

Setting up #

The variable lists offer only what the method can compute. Each method’s lists are filtered to the types its estimator accepts rather than left open, because an inapplicable pair does not fail loudly – it fills the matrix with error cells or, worse, with a number that means something else. The import types every numeric column continuous, indicator columns included, so Point-biserial, Biserial, Polyserial correlation and Blomqvist’s β take a strictly continuous side – a 0/1 column would otherwise claim the continuous slot, and on it β’s median-quadrant filter drops every row on one side of the median, so β is either computed on one arm or degenerates to φ. Cramér’s V takes categorical and binary variables but not ordinals in a single-method run, since against an ordered partner the order-respecting measures (γ, Somers’ D, Spearman) apply; against a nominal partner nothing can respect order, so the presets do route ordinal × categorical to it. Polychoric correlation takes a binary as a two-level ordinal, matching the presets’ dispatch. The left list under Correlation ratio (ε²) holds only grouping-capable variables, for the reason the next section gives.

Choosing a method #

The correlation ratio is reported as ε², the bias-corrected η², and falls back to Cramér’s V on two grouping sides. The raw sample ratio is inflated under independence by $(k-1)/(n-1)$: with many groups and a modest sample it can land in the strong band on pure noise, and it can sit outside its own confidence interval, which is built for the population quantity. The module therefore reports ${\epsilon }^2=1-(1-{\eta }^2)(n-1)/(n-k)$ and keeps the F test, the p-value and Cohen’s bands, which were written for the population variance explained; the one visible consequence is a value slightly below 0 when the groups genuinely do not differ, while the interval starts at 0. A grouping side is a categorical variable or a numeric one with at most 10 distinct values – the same threshold the information family uses to decide what to bin, single-sourced so the R helpers and the list filter cannot drift – and it is the grouping side whichever list it sits in, so the matrix is symmetric in input order. When both sides qualify, assigning arbitrary numeric codes to one of them would make η² depend on the labelling, so R substitutes Cramér’s V for the cell and the Method column says so; when neither qualifies there is no grouping factor and the estimator errors, which is why the left list is filtered rather than the matrix filling with error cells.

Cramér’s V is Bergsma’s bias-corrected estimator, and reads 0 under independence. The classical $V=\sqrt{{\chi }^2/(n\cdot \min (r-1,c-1))}$ is biased upward, because χ²/n has expectation $(r-1)(c-1)/(n-1)$ rather than 0 when the two variables are independent – worst at small n and in large tables, where a table of unrelated variables reads as a weak or moderate association. Bergsma’s (2013) estimator subtracts that expectation from φ² and shrinks the dimensions to $\tilde{r}=r-{(r-1)}^2/(n-1)$, $\tilde{c}=c-{(c-1)}^2/(n-1)$, so it reads exactly 0 on an independent table, converges to the classical value for a non-tiny sample and is never larger than it. The analytic interval and the equivalence test invert the noncentral χ² onto the same shrunk denominator, so the three agree on one scale; φ is the exception, since |φ| on a 2×2 is the classical V and its interval inverts on the unshrunk denominator instead (argued below). The comparison module’s Cramér’s V is the uncorrected one, which is why the two read differently on the same table. Cohen’s (1988) cutoffs, which the strength label reads, were written for large samples and no correction changes that, so the label carries a caveat.

Validation. On the café’s recommend × branch table the corrected V reads .07 here against .10 in the comparison module.

Hoeffding’s D carries no tie correction, and its test turns conservative on ties. Hmisc::hoeffd applies no tie correction and phoeffd’s Blum–Kiefer–Rosenblatt table assumes continuity, so ties attenuate the coefficient itself and leave the test conservative: measured, two identical 5-level ordinals give D = 0.571 rather than 1, and the type-I rate at α = 0.05 falls to 0.0125 on 5-level data and to 0 on binary data. The module keeps the coefficient, flags heavy ties in the suitability column with the only remedy D has – read it as a lower bound and prefer distance correlation or the information family, which handle discrete data natively – and does not offer it the permutation p-value the rank methods get, since a permutation p would not restore the attenuated coefficient. Its scale is its own – roughly −0.5 to 1, strong dependence rarely above 0.3 – so it gets its own strength bands rather than the information family’s, and a sample D can fall below 0 under independence: that is the noise floor around the null mean, not anti-dependence, so the table shows the raw value and only the unsigned visualizations clamp at 0.

Chatterjee’s ξ breaks ties in its first variable at random, under a fixed seed. XICOR::xicor ranks the first variable with ties.method = "random", as the reference definition prescribes, so on tied data ξ is a random variable given the data – three calls on one 5-level variable gave 0.267, 0.219 and 0.200. Both the estimate and its envelope SE run under the same fixed seed and consume the same first draw, so the value is reproducible from run to run and the SE describes the ordering the estimate used; the seed is fixed rather than the bootstrap seed because an unseeded run must still publish the same ξ twice. A caveat under the table fires whenever a cell’s first variable actually carries ties, and only that direction is flagged: ξ is asymmetric, and it is the predictor’s ties the random ranking perturbs – ties in the second variable take maximum ranks by definition, and the null variance behind the p-value is adjusted for them.

Distance correlation is the square root of the bias-corrected estimator, clamped at 0, on a 3,000-row subsample past that. energy::bcdcor estimates dCor² – Székely and Rizzo’s bias-corrected R*ₙ – which can dip below 0 under independence; the module clamps it at 0 and takes the square root, which puts the value on the same 0–1 scale as |r|, the scale the strength bands and the colour ramp assume, so the displayed dCor is never negative. The distance matrices grow with the square of n and exhaust the WASM heap – bcdcor succeeds at n = 4,000 and fails at 4,500 – so above 3,000 complete cases the cell is computed on a random subsample of that size, leaving headroom on a fragmented heap for the estimator and dcorT.test both; the subsample is drawn under its own fixed seed, so the bootstrap seed and unseeded runs are unaffected, the reported N is the subsample’s, and a caveat under the table names the cap. The cap is a heap limit, not a power threshold, and is deliberately separate from the large-n cutoff the Shapiro and OLS checks use.

Polychoric fits above 20 levels a side get an advisory, not a refusal. A polychoric estimate is a two-step maximum-likelihood fit over the pair’s thresholds, so its cost grows with the number of categories on each side and not with the row count: measured on the live engine, 2.3 s for a 7×7 table, 28 s for 20×20 and 50 s for 40×40 – a 0–100 score declared ordinal turns a modest matrix into tens of minutes. Before a run under Polychoric correlation or the latent preset, the module counts the pairs with more than 20 distinct levels on either side and, if there are any, names their number and the expected cost per fit; it is advisory because the fit is correct, only slow, and the run stays cancellable from the overlay. The remedy is the user’s: treat the wide variable as continuous, or collapse its categories first.

Neither preset ever picks a directional method. Mixed/auto chooses per pair from the two variables’ types, and Somers’ D, Theil’s U and Chatterjee’s ξ are never among its choices even where they apply, because a directional cell in an otherwise symmetric matrix would break the visual contract – the cell at [A, B] would no longer equal the cell at [B, A] while every neighbour did – so they need an explicit choice. The observed preset keeps the assumption-light measures (φ, point-biserial, Spearman) in the ordered and binary slots and the latent preset the hetcor family (tetrachoric, biserial, polyserial, polychoric); the categorical combinations resolve identically under both, since against a nominal variable no measure in the catalogue can respect an ordinal’s order, so ordinal × categorical stays in the χ² family with its neighbours.

Checking assumptions #

Every applicable check runs for every pair, and the verdict is advisory. The pass runs every check whose family covers a pair rather than only the selected method’s, so a failed primary shows at once which alternative would fit, and an explicit method and a Mixed/auto preset share one path – the selection only marks the primary per pair. Nothing is switched automatically, because per-pair switching would leave a matrix whose cells use different measures and cannot be compared with one another; a single method across the matrix with the strained cells flagged keeps comparability while still steering toward the alternative the data want – Spearman for a failed linearity or normality check, Kendall for influential points. Only a check that invalidates the coefficient drops a method; a check that invalidates the inference names the remedy the module ships – the bootstrap interval, the Monte-Carlo χ² p, permutation p-values – since the coefficient is sound and only its interval or p is not.

Grouped runs check within each group and never the pooled rows. Under Between groups or Equality across groups every check runs on the row subset the per-group analysis uses, because a pooled Mardia, RESET or Levene verdict can fail – or pass – purely from between-group mean shifts, certifying or condemning an analysis that never runs. The compare-across variable is constant within a group and so leaves the checked pairs, while the family assignment is read from the full data so every group section carries the same rows and columns.

Only linearity drops Pearson; normality, homoscedasticity and influence caveat it. Under curvature r is no longer the linear-association summary its label claims, so a failed RESET drops it. A rejected Mardia or Breusch–Pagan changes nothing about the coefficient, only Fisher-z coverage and the normal-theory p, and the module ships the remedy in the bootstrap interval, so both are advisories. Influential points sit between: a dominating observation does move r itself, so the advisory bucket’s “not the coefficient” wording would be false for it – but Cook’s D > 1 is a magnitude rule rather than a test, too crude to strike a method over, so it takes a caveat of its own naming the bootstrap interval and Kendall’s τ. τ is named unconditionally because it accepts every pair Pearson does; Blomqvist’s β resists influence as well but needs strictly continuous columns, so naming it could point at a method the pair cannot use. The size-adjusted $4/(n-p)$ cutoff and the 2p/n leverage cutoff are shown in the tooltip for context and never decide the verdict. RESET, Breusch–Pagan and Cook’s D are fitted in both regression directions and the pair fails if either does, since the regression has a direction and the correlation has none.

Above N = 5000 the rejection-based OLS checks stop counting against Pearson. At that size Mardia, RESET and Breusch–Pagan reject almost any real data while Pearson’s central-limit-backed inference is at its most robust, so a “failure” would mislead; the cap is the app-wide large-n cutoff the Shapiro–Wilk tests share, so the two cannot drift. Mardia’s column is forced to N/A past it, matching the marginal table; RESET and Breusch–Pagan keep running with their verdicts visible, discounted from the Pearson verdict but each with an advisory of its own – detected curvature is worth a look at any N, and heteroscedasticity, unlike normality, does not wash out asymptotically: it enters the fourth-moment term of r’s standard error, so its advisory prefers the bootstrap interval. Cook’s D is exempt, since one dominating observation is a problem at any sample size.

Rejections no more numerous than chance are not counted. Every level-α pair check fires once per assessed pair, so on data that satisfies every assumption some rejections are expected – at ten continuous variables, 45 pairs, “any failure demotes” becomes “always demotes”. The pass compares each check’s failing pairs against a binomial upper bound at its per-pair rate: $1-{(1-\alpha )}^2$ for Mardia, whose skewness and kurtosis are two statistics either of which can reject, and plain α for RESET, Breusch–Pagan, the latent goodness-of-fit and Levene – RESET and Breusch–Pagan run in two orientations, but both are fitted on the same rows and reject together, so a single test’s rate is the honest one. Rejections within the bound stay visible in their column and a note says they are not counted. Cook’s D is outside the accounting, being a magnitude rule with no rate at α, and so are OLS rows past the large-n cap, which never demote a method anyway. The same bound governs within-group normality: a rejecting group counts only below the small-sample floor, since the t and F the point-biserial and ε² inference rest on are robust to non-normality above it, and only when more such groups reject than the bound allows – a 10-level grouping variable runs ten Shapiro tests, and one rejection among them at α = .05 is what exact normality looks like.

The tie threshold is a judgment cutoff anchored on 1/k², and the remedy differs by method. The tie correction $\sum (t^3-t)/(n^3-n)$ is reported rather than the share of tied observations because it is the term that enters the rank statistics’ variances – Spearman’s tie-corrected SD, Kendall’s τ-b denominator – whereas the tied share sits near 1 for every ordinal variable and near 0 for every continuous one and separates nothing; with k roughly equal levels it is ≈ 1/k², so the default 0.10 flags three or fewer effective levels and any strongly piled-up binary or ordinal variable while a balanced 5-point scale (≈ 0.04) passes. Spearman’s tie-corrected t / AS89 approximation degrades and Kendall’s exact test is unavailable once ties exist, so both are pointed at permutation p-values, which stay valid under ties; Blomqvist’s β takes a caution because a pile-up at the median makes its quadrant counts ambiguous and it has no permutation route; Hoeffding’s D takes its own warning, argued above; γ and Somers’ D take none, being built for tied ordinal data with tie-aware standard errors.

A sparse χ² table points at Monte Carlo instead of dropping φ / V. Cochran’s rule – no expected count below 1, no more than a fifth of the cells below 5 – guards the χ² approximation behind the p-value, not the coefficient, so the module keeps φ / Cramér’s V listed and names the Monte-Carlo p-value it ships for exactly this case; the earlier advice, an exact test, was not something this module can run. The rule is read on the same complete-case sample the coefficient uses, and the pass also counts observed zero cells and runs over discrete × discrete latent pairs, since zeros there push a latent estimate toward the boundary for a different reason.

The information-theoretic grid is measured, not calculated, and carries no verdict. Equal-frequency binning keeps tied values in one bin, so a tie-heavy variable realizes fewer bins than the rule asked for – an nbins = 5 request on a 50/30/20 column lands 3 – and reading the requested count would fire the advisory on exactly the data it is meant to describe; the pre-flight therefore discretizes each pair for real and counts what came out, so it and the sparse-table caveat under the results are the same statistic on the same grid and cannot disagree. On the default all-continuous run at n = 200 the cube-root rule gives a 6×6 grid with a smallest expected count of 5.12 and the advisory stays silent; the same data under Rice gives 12×12 at 1.125 and both fire. It adds no Pass / Fail column because Cochran’s floor is borrowed from the χ² literature and has no standing as an adequacy rule for mutual information – there is no threshold at which the measure stops being valid, only a grid the user chose and a number saying what it cost.

The latent goodness-of-fit is the estimator’s own, a saturated table is N/A, and a failure never drops the method. The check reuses the likelihood-ratio χ² polycor computes when it fits the pair, so it tests the same fit the coefficient comes from rather than a second model. A 2×2 tetrachoric fit, or any table with zero residual degrees of freedom, fits perfectly by construction and has nothing left to test, so it reads N/A rather than a Pass it never earned. A failure biases the coefficient itself, unlike the rank or χ² assumptions, but the latent model is frequently the only one whose data types fit the pair – for two ordinal variables no assumption-free alternative estimates the same latent association – so dropping it would leave the cell empty, and the honest response is an advisory beside the estimate.

Empty cells in a 2×2 get a +0.5 correction; wider tables get none. A tetrachoric estimate on a 2×2 table with an empty cell but non-zero margins drifts to the boundary and returns |ρ| ≈ 1, so the module adds the standard +0.5 to a zero cell of a 2×2 table only – lavaan’s zero.add = c(0.5, 0.0) default, which adds 0.5 for a 2×2 and 0 for anything larger. On a wider r×c polychoric table the same fill biases ρ̂ badly downward – a 7×7 Likert pair at n = 60 with a true ρ of 0.85 estimates 0.86 uncorrected and 0.61 corrected – while an empty cell there does not pin the estimate the way it does in a 2×2. The correction is a no-op when no cell is empty, and the coefficient, its bootstrap interval and the goodness-of-fit check all read the same table, so they can never describe different fits; it mitigates the drift without removing it, which is why sparse or empty cells keep a boundary advisory of their own.

A boundary-pinned fit is seeded, not swapped. Polychoric and tetrachoric coefficients are estimated in two steps – thresholds first, then ρ – with a delta-method standard error, which is fast and is the source of the SE column and the analytic interval. On strongly associated tables that optimizer, started from ρ = 0, can pin at the boundary and return ρ ≈ 0.9999 with an infinite variance – a blank SE, CI and difference test beside a near-perfect coefficient. The module therefore seeds the fit with polycor’s own cheap two-step estimate, which lands the optimizer on the right value directly: a 44/6/6/44 table returns ρ = 0.9298 with SE 0.0376 rather than pinning, matching a full maximum-likelihood refit to five decimals. Only a fit that still comes back pinned or with a non-finite variance is refit by full maximum likelihood, as a last resort, so every cell carries the same estimator at lower cost than refitting everything.

The latent p tests ρ = 0 by a likelihood ratio, not a Wald test on the delta-method SE. Polychoric and tetrachoric coefficients take their p from a 1-degree-of-freedom likelihood-ratio test against independence – the fit’s deviance against the independence deviance on the same thresholds – and polyserial and biserial from cor.test on the continuous side against the numerically coded ordinal one, which tests the same null. Both routes are boundary-safe: a Wald test on the delta-method standard error breaks down as |ρ̂| approaches 1 – on a perfectly associated 2×2 table it reports p ≈ 0.43 for an association as strong as it can be – while these stay monotone in the association up to the boundary. The SE column and the analytic interval still come from the delta method, so a very strong association can show a tiny p beside a wide interval; that is the delta method’s limit near ±1, not a contradiction.

The variance check is Brown–Forsythe’s, and its failure names Somers’ D only where it applies. The mean-comparison check centres Levene’s distances on the group medians because a continuous variable within a group is often skewed or discrete, where the median-centred test holds its level and the mean-centred original does not. Unequal variances leave point-biserial and ε² as the coefficients they claim to be and weaken only the pooled-variance t and F behind their p and analytic interval, so the advisory names the bootstrap interval; it adds Somers’ D – the rank-biserial, Cliff’s δ – only when the grouping side is numeric enough for the module to compute it on the pair, so the advice never points at a method the pair cannot use.

The beyond-monotone screen tests residuals against a monotone null drawn by Monte Carlo. The residuals are tested rather than dCor compared with Spearman’s ρ because either number alone is ambiguous and reading them together still fails when both components are present – a moderate trend plus a strong oscillation gives a solid ρ and a solid dCor, and no comparison of the two reveals the extra structure; removing the monotone fit first isolates exactly what a monotone lens cannot see, however strong the trend on top of it. The fit is a weighted pool-adjacent-violators regression on mid-ranks – tie-group means with group-size weights, so it is a proper monotone function of x and deterministic under row reordering, where stats::isoreg would break ties by row order – and the residual statistic is the square root of energy::dcor2d(type = "U"), the bias-corrected estimator by the O(n log n) bivariate algorithm, clamped at 0 so it sits on the dCor scale the effect floor uses. Its p is not dcorT.test’s: the residuals of a rank-scale fit are not i.i.d. – their spread shrinks toward the ends of a strong trend – so an i.i.d. reference flags clean monotone data almost surely. The reference is instead a Gaussian copula at the pair’s own margins and ties, with the normal-scores correlation as the plug-in, re-run through the same rank → isotonic → dCor pipeline once per draw; the draw count is $\min (199,\max (49,\lceil 2/\alpha \rceil ))$, which puts the requested α inside the resolution $1/(\text{draws}+1)$, and an α finer than that floor reports N/A rather than a pass the screen could never have failed. Requiring the residual dCor to reach the lowest information band as well as significance keeps the flag off trivial departures in large samples, where the null resolves differences far too small to change which coefficient is appropriate. The probe’s 1,000-row subsample is a cost cap, not a heap one: the fast estimator is O(n log n), but the calibrated null re-runs the pipeline once per draw, so the cap bounds (draws + 1) passes – which is why it is separate from the 3,000-row heap cap on the reported coefficient. The advisory is withheld when the primary is already an omnibus measure – the information family, ξ, Hoeffding’s D, dCor – since those are consistent against exactly this dependence and the suggestion would be wrong; Blomqvist’s β stays flagged, being a monotone-type measure.

Collinearity is read pairwise and by VIF, on continuous variables only. A partial correlation conditioned on near-collinear controls is numerically ill-conditioned and swings on small data changes, so the check correlates the continuous controls and targets and flags pairs involving at least one control above |r| = 0.90 – a target–target pair does not destabilize the conditioned estimate and already shows in the main matrix. Under Kendall the table shows raw τ, so the cutoff is displayed on that scale, (2/π)·asin(0.9) ≈ 0.71, while the flag is raised on the r-equivalent. The pairwise read is blind to a control set that is jointly near-singular – three controls mutually correlated below the cutoff can still be – which is exactly what the matrix inverse behind the partial fails on, so each variable’s VIF against the control set sits beside it at the severe threshold rather than replacing it. Categorical controls are outside the diagnostic because collinearity is a property of numeric predictors: an arbitrary integer coding of a multi-level categorical would make the correlation depend on the labelling.

Controlling for covariates (partial & semi-partial correlation) #

A semi-partial cell reports the partial’s p and its own interval. The semi-partial’s null – the row variable adds nothing over the covariates – is the null the full partial tests, and the one a regression of the column variable on the row variable plus the covariates tests for that coefficient, so the three p-values are one number and the semi-partial cell prints the partial’s rather than a second test that could differ only by rounding. The coefficient and its interval are the semi-partial’s own, since its sampling distribution is not the partial’s: a Pearson semi-partial’s analytic interval and TOST p use the Aloe & Becker (2012) delta-method standard error, $\mathrm{var}(sr)=(R_f^4-2R_f^2+R_r^2-R_r^4+1)/n$ with $R_f^2$ the R² of the column variable on the row variable plus the covariates and $R_r^2=R_f^2-s{r}^2$, in place of the partial’s Fisher-z one. Spearman and Kendall semi-partials have no analytic variance – the partial-r Fisher standard error assumes both sides conditioned – so their interval is offered only under the bootstrap mode and reads – under Analytic, because a resampled interval slipped into a column of analytic ones would put two constructions under one header unannounced. Their TOST does run, on a bootstrap z-scale standard error computed on demand whenever a margin is set, since the equivalence verdict was explicitly asked for and has no other route; the resampling is spent on those cells alone. The mirrored cell of a semi-partial matrix shares the row’s p for adjustment, so each pair counts once, while the TOST p is direction-specific and counts twice.

A partial pair needs k + 3 complete cases, deleted pairwise over the pair and its covariates. The partial t-test has n − 2 − k degrees of freedom, so at k + 3 cases it has one and below that none: the coefficient could still be computed and its inference could not, and the cell is refused rather than printed with a blank interval. Deletion spans the pair and every covariate because a case missing a covariate cannot be residualized on it; the covariates are the same for every pair, so the per-pair samples differ only by the pair’s own missingness, as in the zero-order run, and the zero-order companion is computed on the partial’s complete cases so the two numbers describe one sample. The list offers numeric covariates only: ppcor regresses on its controls as numbers, and an integer coding of a multi-level categorical would make the residual, and so the coefficient, depend on the labelling – the reason the collinearity diagnostic excludes categorical controls as well.

Kendall’s partial is not residualized on the plot, and the rank partials are ppcor’s definitions. A Pearson partial has a residual representation that reproduces the displayed coefficient – the correlation of the two residual series is the partial r, the OLS line through them has the partial regression slope, and the band on n − 2 − k degrees of freedom is the partial’s own inference – and a Spearman partial has the same on ranks, so both scatterplots are drawn on residuals. Kendall’s partial τ is a closed form on the pairwise τ’s, and no residual scatter reproduces it: plotting residuals would show a picture the number does not describe, so the plot keeps the raw points and the smoother, labels the partial as conditioned and lists the zero-order τ₀ the picture actually shows. The rank partials follow ppcor – partial Spearman as a Pearson partial on rank-transformed data, partial Kendall as that closed form – because it is the implementation the module’s partial path is built on and the operational definition most tools share; the Kendall partial τ derived from concordant and discordant triples is a different quantity, which is why the manual says a partial τ from another tool may differ.

Partial-correlation network #

The network is fitted on one listwise-complete sample, and the run’s covariates are nodes. The Gaussian graphical model is one joint model over every node, estimated from the nodes’ correlation matrix and, for the unregularised estimator, its inverse; a matrix assembled from pairwise-complete cells need not be the correlation matrix of any dataset – it can fail to be positive definite – and for an estimator that inverts it that is a validity question, not a precision one, so rows missing anywhere in the selection are dropped and the card reports the cost. A control variable becomes a node for the same reason: every edge is already partialled on every other node, so once the controls are in the model there is no remaining set for a covariate to be, and holding them outside would condition the network twice on the same variables. Nothing here changes the main matrix, which keeps its pairwise sample and its covariates.

The unregularised network is screened for rank and degrees of freedom before the fit. ppcor::pcor pseudo-inverts a singular correlation matrix instead of raising an error, so a node that is a linear combination of others – a scale score beside its items – would return a confident-looking coefficient with no content; the matrix’s rank is checked first and a deficient one refuses the run. Its per-edge t-test needs n − p ≥ 1, so fewer complete cases than nodes plus one refuses it too. Both messages point at the regularised estimator, which handles a singular matrix and a small sample by construction. Between that floor and the saturated model’s $p(p+1)/2$ parameters the network is fitted with an advisory, because edge-by-edge estimates at that ratio are what regularisation exists to replace. Under Spearman and Kendall the screens run on the rank-transformed data the estimator sees.

The graphical lasso is an ADMM fit over a log-spaced penalty path scored by EBIC. The grid holds 100 penalties, log-spaced from the smallest λ that zeroes every edge down to 1% of it – the path qgraph::EBICglasso established for this estimator – and is descended from the sparsest end so each fit warm-starts from the one above it; each fit solves the penalised precision matrix by ADMM with the diagonal unpenalised, and each is scored by EBICγ = −2ℓ + |E| log n + 4|E| γ log p (Foygel & Drton 2010), ℓ the Gaussian log-likelihood at the penalised precision matrix and |E| its non-zero edges; the minimum picks the graph. γ = 0.5 is the conventional default, 0 the plain BIC and 1 the strongest preference for sparsity. When the minimum falls at either end of the grid the criterion wanted a graph beyond it, so the card says the edge count is the sparsest or densest the grid could express rather than presenting the grid’s end as a choice. The regularised path takes Pearson or Spearman only, because the penalised likelihood is written for a correlation matrix on a Gaussian scale, which Kendall’s τ matrix is not; a selected edge is a model-selection verdict, so the card prints no p-value beside it, and the graph gates on the coefficient rather than on a test that never ran.

Only strength and expected influence are reported, as raw sums over the estimated edges. Closeness and betweenness are path-based: on a weighted, nearly complete graph their values turn on the exact weights of the weakest edges, which is why the psychometric-network literature finds them unstable under resampling while strength and expected influence hold; the card offers the two that survive and none that would invite a reading the data cannot support. Both are sums over the edge list the estimator already returned, unstandardised, so nothing is refitted and nothing thresholded: under the unregularised estimator every edge enters at its coefficient, significant or not, because thresholding at α would make a centrality depend on the sample size through the test rather than through the network; under the lasso the sums run over the selected edges, an edge shrunk to zero contributing nothing. Expected influence is the one-step form – the signed sum of a node’s own edges – and a node with no edge scores 0 on both by construction, which the note distinguishes from a measured 0.

Communities are found by spin-glass optimisation of signed modularity, bridged across components and pinned to the reproducibility seed. igraph::cluster_spinglass(implementation = "neg") is the one community algorithm in igraph that takes negative weights – walktrap, Louvain and fast-greedy each refuse one outright rather than silently taking absolute values, which would discard the distinction expected influence exists to preserve – and the reported figure is the signed modularity of Gómez, Jensen and Arenas (2009), which igraph::modularity() cannot compute on negative weights and which reproduces spinglass’s own objective on the graph it optimised. Spinglass refuses a graph it cannot walk, and a regularised network routinely falls into parts, so a negligible link (10⁻¹²) joins each extra component to the next before the search: it adds nothing to the annealing energy or the modularity, and lets the search optimise over the whole graph under one null model rather than inside each component’s own – a different objective, and one that splits a weak clique it should keep whole. The partition is then intersected with the components, so the bridges let the search compare across parts and never merge them, and a node with no edge is left out of the search and given a community of its own. The search anneals from a random start, so it runs under the reproducibility seed rather than the bootstrap seed – it is not a resampling step – and an empty seed is disclosed on the card. A failed igraph load leaves the partition out, never the card: the community is an addition, not a precondition.

Testing differences between correlations #

A data-driven reference cell carries an optimism note; a pre-specified one does not. A reference chosen because it holds the strongest or weakest coefficient is chosen from the same data it is then compared against, so every difference from it is biased away from zero and its p-values read optimistically – the selection effect of testing the largest of many estimates against the rest. The card keeps both pickers but captions a pre-specified cell (pre-specified, …) with no warning and a data-driven one with a note naming the bias, and the data-driven pickers break ties lexically by (row, col) name so a pick is reproducible across runs rather than dependent on enumeration order. A pre-specified cell that is missing from the current matrix, errored or non-finite is named with which of the three it was, because an empty comparison table would say neither the cell nor the cause.

The single-anchor patterns are hidden where they would restate compound symmetry. The three patterns coincide whenever one axis of the matrix is contained in the other: with the row variables a subset of the column variables every column-wise constraint already follows from the row-wise pass through the shared cell (r_ac = r_ab = r_bc), and the mirror holds the other way, so on a square symmetric matrix all three are one χ² reached by three routes. Offering them would give three H₀ statements for one test, so a single-anchor option is hidden both when its pass forms no constraint on the current matrix and when its axis is contained in the other; asymmetric runs are exempt, since r(X→Y) ≠ r(Y→X) and the two passes constrain different cells.

The two-group across-groups test is Jennrich’s canonical 1970 statistic. Its χ² can differ from tools that implement a deviating variant, notably psych::cortest.jennrich: in simulation the canonical form holds its nominal α and is more powerful, while the variant over-rejects when the variables are strongly intercorrelated. With three or more groups the test is a Wald χ² on the Olkin–Siotani covariance evaluated at the pooled R̄, since the trace form is written for two groups.

Above 50 levels the grouping variable is questioned, not refused. A compare-across key with that many levels is a continuous column or an identifier picked by mistake whatever its declared type, and every level costs a per-group correlation batch – but a declared categorical with many genuine levels is legitimate, so the run warns and proceeds rather than stopping.

The closed form is canonical for Pearson and a plug-in for Spearman, Kendall and point-biserial; a partial shifts n by k. The normal-theory derivation behind the difference tests assumes bivariate normality, which only Pearson’s r satisfies; Spearman’s ρ and Kendall’s τ take their own Fisher-z variances (Bonett & Wright’s, and Brown–Benedetti’s per-cell or Fieller–Hartley–Pearson’s constant form) inside the same derivation, and the point-biserial takes Fisher’s $1/(n-3)$ unchanged although its dichotomous margin makes the sampling variance depend on the group split (Tate 1954), so the caveat under the table applies especially at extreme splits. A partial correlation has no closed form for its difference either: the effective sample size shrinks by the number of conditioned-on covariates, giving df = n − k − 3 (n − k − 4 in Kendall’s variance) – Wilcox (2009), §10.11, for the heuristic’s rationale.

Seven analytic-SE methods get a closed-form test between groups and none within a sample. Independent samples share no rows, so the variance of a difference is the sum of the two cells’ own atanh-scale Wald variances and $z=(r_1-r_2)/\sqrt{{\text{se}}_1^2+{\text{se}}_2^2}$ needs nothing beyond the per-cell delta-method SE the analytic intervals already carry – for the polychoric, tetrachoric, polyserial and biserial coefficients, Goodman & Kruskal’s γ and Somers’ D (each direction of its ordered grid separately), and Blomqvist’s β on its asymptotic variance $(1-{\beta }^2)/n$. The dependent tests are different: their Steiger, Olkin–Siotani and Jennrich covariances are derived for product-moment correlations and are invalid for latent-ML and ordinal-ASE estimates, and an asymmetric coefficient’s supplementary cross-correlations fall on invalid same-role pairs, so the same-sample comparisons of these methods have no closed form and route to the bootstrap.

The bootstrap path is opt-in, shares its draws within a sample, and its joint test runs about 8% at 100 replications. Resampling every cell is the card’s most expensive step, so it runs only when bootstrap intervals were asked for; the earlier silent fallback, which resampled unasked whenever a method lacked a closed form, was removed so that a run’s cost is always the user’s choice, and a coefficient without a closed-form test shows the switch instead. Single-sample structures draw one row-index set per iteration shared across every pair, which is what preserves the dependence between two same-sample coefficients; between-group draws are independent per group, matching the independence the between test assumes. The shared draw is also what lets the single-sample pattern-equality test survive on this path: the same constraints go into a Wald statistic whose covariance is estimated from the resampled Fisher-z estimates, referred to its own recentred bootstrap distribution rather than a χ² tail. That covariance is only as good as the number of draws behind it – against a nominal 5% the joint test rejects about 8% of the time at the default 100 replications and about 6% at 400 or more – which is why the manual says to raise the replication count before reading its p near a threshold. Semi-partial mode always takes this path because the closed-form Δr variances are derived for symmetric (zero-order or canonical-partial) coefficients.

The back-transformed average is the default, and the two dependent variants are closer than the literature suggests. Both dependent tests substitute one H₀-restricted r̄ for the two coefficients in the covariance and differ only in how it is formed – the raw mean (Williams’ T2, Steiger 1980) or the back-transformed mean of the two Fisher z’s, $\tanh ((\mathrm{atanh}{r}_1+\mathrm{atanh}{r}_2)/2)$ (Hittner, May & Silver 2003 for overlapping cells, Silver, Hittner & May 2004 for non-overlapping ones), which the literature reports as holding its Type I rate better under non-normality. Measured here it does not pull ahead: at n = 60 over 4,000 replicates under the null, Williams and Hittner reject at .0500 / .0503 on normal margins and .1052 / .1050 on a mildly skewed one, and at n = 20 with ρ = .8 and skew the classic pair is marginally better (.119 vs .136); under heavy skew both closed forms are badly liberal (about .23 at a lognormal margin), which is the bootstrap path’s territory. The choice is one paired control rather than four separate tests because the reference structure mixes overlapping and non-overlapping cells in one table, so the choice has to name a pair. One structural note: the covariance denominator can turn non-positive at high |ρ| under skew, and a cell that hits it declines to test rather than reporting a statistic built on a negative variance – under 0.1% of replicates.

A joint test denied an admissible covariance falls back to a Cauchy combination. The pattern-equality and across-groups tests need an invertible constraint covariance, and they can be denied one in more than one way: a small sample or sparse group sizes leave it rank-deficient, and the pattern-equality test evaluates its covariance at the H₀-restricted matrix – each constraint class collapsed to its mean – which can be inadmissible at any sample size. Under a bootstrap interval the single-sample test builds the covariance from the resampled coefficients instead, so there it fails only when too few replicates survive or the resampled constraints are collinear; the across-groups test has no resampled counterpart and goes straight to the fallback. The fallback is Liu & Xie’s (2020) Cauchy combination (ACAT) of the component tests – valid under arbitrary dependence between the components, at the cost of less power than the joint test when that is available – and the card names which ran and why. Across groups the components it combines are the per-cell homogeneity χ²s, in which each group enters a cell once; the redundant pairwise rows – each group appearing in one comparison per other group – are combined only on the bootstrap path, which has no homogeneity test.

The homogeneity and per-anchor equality tables protect the pairwise family, and an explicit bootstrap request never gets a closed-form answer. With k > 2 groups the C(k, 2) pairwise rows of a cell are an unprotected family, and the module’s only other omnibus – Jennrich’s – tests the whole matrix under a structure the user would have to run instead; the per-cell homogeneity χ² pools each cell’s per-group coefficients weighted by their own variances, so Kendall and the Wald-SE methods use their own standard errors and Pearson reduces to the textbook Fisher-z form. The C(q, 2) rows under one anchor of a row-wise or column-wise run are the same kind of family, so a Steiger pattern-equality χ² per anchor leads them, for anchors with at least two comparisons – with one it would restate the pairwise row. Each block is adjusted as its own family, since leaving $k(k-1)/2$ omnibus tests raw inside a card built for multiplicity control would defeat it. Both are closed-form and skipped under a bootstrap interval, because a run that asked for resampling must not receive a normal-theory test unannounced.

The dependent tests run at one n, and the per-cell intervals are the matrix’s own. Williams’ T2 and Steiger’s z are derived for one sample complete on every variable the comparison touches, so the test runs on those rows – under pairwise deletion, fewer than either cell’s own n – and the interval for the difference is built at the same n, so the interval and the p-value can never disagree about how much data stood behind them; that n appears nowhere else on the card, which is why the column exists. The per-cell intervals beside the coefficients are the main matrix’s – under a bootstrap interval the cell’s own pairwise-complete resample, not the difference test’s whole-frame one – so one quantity never appears twice on the card with two different bounds. The per-group N columns of a between-groups table exist for the same reason: the between test’s variance is $1/(n_1-3)+1/(n_2-3)$, and the grouped path renders no pooled matrix where those n’s would otherwise show.

The Δ interval is Zou’s MOVER analytically and a bias-corrected percentile interval under the bootstrap, whose p shares its recentring. Zou’s (2007) method combines each coefficient’s own Fisher-z interval with the correlation between the two estimates, so it exists for every closed-form pair, and its inputs are taken at the test’s own n rather than the cells’ pairwise counts. Under the bootstrap the difference’s interval takes bias-corrected percentile quantiles of the resampled Δr – the same median-bias recentring the per-cell bootstrap intervals apply – and the bootstrap p-value is computed with the same z₀, so p (Δ) < α exactly when the α-level interval excludes 0 and the two never disagree at the boundary; the closed-form paths carry their own statistic’s p. A pair whose two cells were computed with different effective methods – possible under ε², where a categorical × categorical pair falls back to Cramér’s V – is not tested at all, since an r²-scale number against an r-scale one is not a difference.

The comparisons’ p-values are a family of their own, and the per-cell p’s belong to the matrix’s. The matrix’s p-values and the comparison card’s answer different families of questions, so the global adjustment is applied to each separately, and inside the card all comparisons of one structure form one family. The p (A) / p (B) columns are the cells’ own significance and stay in the matrix’s family: under replacement they show the batch-adjusted value the matrix shows – in a between-groups comparison, adjusted within each group’s own correlation family, which matches what filtering to that group and running the matrix alone would give – and under addition they stay raw.

Display options #

The information-theoretic analytic p is a χ² on the plug-in MI, and the appended MI is the Miller–Madow one. The analytic option rests on the asymptotic result that 2n·MI ~ χ²((k_x − 1)(k_y − 1)) under independence, so its statistic is the raw plug-in mutual information of the discretized table and not the normalized coefficient in the cell – the test is correct for the independence hypothesis and says nothing about the size of NMI, AMI, coherence or Theil’s U. Append MI prints the Miller–Madow-corrected MI instead, because that is the value that reconciles with the appended entropies, which carry the same correction; the two are different numbers, and the manual says so. The χ² approximation degrades when the discretized joint table has expected counts below 5, so the expected-count check runs in both modes – a sparse table inflates the coefficient itself, a coefficient-level caveat whichever p-value mode is active, with the analytic-p caveat added on top on this route. AMI is the exception in the family: it is not a monotone transform of MI at fixed margins, so a χ² on MI is only ever an approximation to a test of AMI, which a caveat states whenever AMI runs analytically; the permutation option, which resamples the reported coefficient, is the exact test for it.

Permutation replicates are sized to α, floored by the bootstrap setting and inflated by √(pairs) under adjustment. About 20 exceedances below the threshold are needed to estimate a p reliably there, so the count is ⌈20/α⌉ – 400 at α = 0.05 – with the bootstrap-replications setting as a floor, so a user who asked for more resampling gets it; under p-value adjustment the effective threshold sharpens with the number of tests, and the count is multiplied by √(pairs) – a compromise across Bonferroni, Holm and BH that keeps resolution after correction without inflating to Bonferroni’s worst case. One count serves every permutation option and the Monte-Carlo χ², each cell under its own seed so a run repeats; the p is $(\text{exceedances}+1)/(\text{replicates}+1)$, so it is never 0 and its floor is the resolution $1/(\text{replicates}+1)$. Kendall’s τ is O(n²) per shuffle, so the view prices a permutation run – pairs, rows and replicates – in a toast before it starts, the one method where the count reaches the wall clock.

Bin rules are capped at the distinct count, and the grid is part of the estimand. The cube-root, Rice and Sturges rules are applied to the variable’s n and then capped at its number of distinct values, and a variable with ten or fewer distinct values is coded level for level rather than binned – the same threshold the correlation ratio uses to recognise a grouping side, single-sourced so the two cannot drift – since binning a variable with fewer levels than bins would only merge levels at random. The grid changes the number, not merely its precision: on one pair NMI measured 0.057 under the cube-root rule (a 6×6 grid), 0.117 under Rice (12×12) and 0.073 at a fixed 3 bins, none more correct than the others – a sparse joint table inflates mutual information itself, so a finer grid buys resolution with bias. That is why coefficients from runs at different bin settings are declared not comparable, why the caveat names the rule whenever it is not the default, and why the pre-flight advisory (argued above) and the bootstrap resamples run on exactly the grid the coefficients use, so an interval always brackets the estimate it belongs to.

Distance correlation defaults to permutation because the analytic t over-rejects, and its permutation is budgeted per cell. energy::dcorT.test’s t approximation is asymptotic in the dimension of the data rather than in n, so on the two-column pairs this module builds it over-rejects and the inflation does not shrink as the sample grows: measured rejection rates on independent data at α = 0.05 were 0.089 / 0.060 / 0.060 at n = 20 / 50 / 200, and at α = 0.01 0.045 / 0.026 / 0.034 – three to four times nominal in exactly the tail p-value adjustment pushes results into – against 0.045 at α = 0.05 for the permutation route (energy::dcov.test), which is correctly sized; the coefficient is identical under both. The permutation runs inside compiled code that polls nothing, so neither the interrupt nor Cancel reaches a cell once it has started; rather than let one run for minutes, the work is capped at 10⁹ replicate·n² units per cell – about 7·10⁷ units per second on the live engine (3.2 s at n = 1,000 with 400 replicates, 12.4 s at n = 3,000 with 100), a worst case of about 14 s – so the replicate count is $\min (\text{requested},\lfloor {10}^9/n^2\rfloor )$: the trade-down begins where n exceeds $\sqrt{{10}^9/R}$, 1,581 rows at the default 400 replicates, and a caveat reports the count used; where even the 199-replicate resolution floor overruns the budget, above 2,241 rows, the cell refuses and points at the analytic t. The cap is a time budget and is separate from the 3,000-row heap cap on the coefficient itself.

The rank family’s analytic p is cor.test’s, with an atanh Wald on the tie-aware ASE for γ and Somers’ D that is blind at ±1. Pearson and the rank methods take cor.test’s p – Pearson’s t, which leans on bivariate normality, Spearman’s tie-corrected t / AS 89 approximation and Kendall’s exact test, which is unavailable under ties – so heavy ties and a failed OLS diagnostic are the two conditions under which the analytic route degrades, and the tie-burden and normality checks point at permutation for exactly that reason. γ and Somers’ D take a Wald z on the atanh scale with the tie-aware asymptotic standard error, which is exactly 0 when the coefficient reaches ±1 – γ does on any table with no discordant pairs – so no p, SE or interval can be reported there; the permutation route is the only p-value for a perfect γ or D, and it is exact for independence whatever the ties, since the null distribution is built from the data’s own tie pattern.

Under covariates the permutation is Freedman–Lane, with the surrogate mapped back onto the outcome’s own values. A plain column shuffle has no partial-correlation meaning – it would break the pair’s link to the controls along with the link to each other – so the conditional null X ⊥ Y | Z is built by regressing the outcome on the controls (on the scale ppcor inverts: raw values for Pearson, ranks otherwise), shuffling only the residuals, adding the fitted values back and recomputing the coefficient from that surrogate through the same pcor.test the point estimate uses; in semi-partial mode the reference statistic stays the partial (pcor) estimate, matching the p the cell displays. The surrogate is tie-free even when the data is not, and Kendall’s τ_b normalizes by the tie pattern, so the observed statistic would sit on a different scale from its own null; the surrogate is therefore mapped back onto the outcome’s own multiset of values before the statistic is taken. Measured under the conditional null at n = 60, a partial Kendall on 4-level data rejects at .348 analytically and .104 under the plain surrogate, against .052 with the remap; on untied data the remap is a monotone relabelling of a rank statistic’s input – a no-op. Because the surrogate is rebuilt rather than relabelled the test is approximate rather than exact, which the caveat says.

φ and Cramér’s V take their p from the χ² test without continuity correction, and Monte Carlo resamples its null. The analytic route is chisq.test(correct = FALSE), reliable exactly when Cochran’s rule holds, which the assumptions card’s Adequate expected counts verdict reads on the same complete cases the coefficient uses. The Monte-Carlo route is chisq.test(simulate.p.value = TRUE) – Hope’s (1968) simulated tables at the observed margins – with the shared replicate count and a per-pair seed, valid however sparse the table at the cost of the simulation. The coefficient is byte-identical in both modes, which is why a sparse table points at this switch rather than dropping the method.

The analytic interval is each method’s own closed form, and four bounded measures invert their own test statistic. Pearson, Spearman and point-biserial take Fisher’s z – Spearman with the Bonett–Wright variance – and Kendall a data-dependent Brown–Benedetti τ_b ASE for zero-order cells, delta-method-transformed onto the z scale, with partials falling back to the Fieller–Hartley–Pearson constant. The latent methods, Goodman & Kruskal’s γ and Somers’ D take an atanh-transformed Wald interval on the delta-method SE, which keeps the bounds inside [−1, 1]. Three unsigned measures get a closed form built from the very statistic behind the coefficient: η² by inverting the noncentral F of its one-way ANOVA, Cramér’s V by inverting the noncentral χ² of its table onto the Bergsma-corrected scale – so the interval matches the bias-corrected coefficient and collapses toward 0 under independence – and Blomqvist’s β from a Wilson score interval on the concordant-quadrant proportion, mapped back through β = 2p − 1 so the bounds stay inside [−1, 1] even at β = ±1. The φ coefficient rides on the same noncentral-χ² inversion as V: on a 2×2 table |φ| is the classical V, so the interval inverts the χ² φ’s p comes from, on the unshrunk denominator that matches the raw φ, and is re-signed to match the coefficient – chosen because φ’s asymptotic SE depends on the 2×2 marginal proportions rather than on φ alone, so a Fisher-z interval would disagree with the χ² test.

ξ’s interval is a plain-scale Wald on a data-adaptive envelope SE, and covers about 92% at n = 200. Chatterjee’s ξ has no Fisher transform, so its interval is ξ ± z·SE clamped to [0, 1] – a low estimate with a wide SE floors at 0 rather than printing a negative bound – on the larger of two variances: Lin & Han’s (2023) consistent estimate under the alternative and Chatterjee’s (2021) independence-null variance, so the interval is never narrower than it would be under independence, never degenerates on a small sample, and widens where the dependence structure implies more variability. ξ is a downward-biased finite-sample estimator, so the analytic interval is a fast approximation: over 400 replicates at n = 200 it covered about 92% against a nominal 95%, and a caveat says so whenever a ξ cell shows one and points at the bootstrap, which refits per resample. The same envelope SE is the one the SE column prints for ξ and the one its equivalence test uses – not the null-independence SE behind its p, so ξ ÷ SE does not reproduce the printed p.

Bootstrap-only methods show – under Analytic, and two analytic intervals carry warnings. The information-theoretic family, Hoeffding’s D and distance correlation have no defensible closed-form interval, and the Spearman and Kendall semi-partials lose theirs because the partial-r Fisher SE assumes both sides conditioned (argued above); a resampled interval slipped into a column of analytic ones would put two constructions under one header, so those cells show –, the column is dropped when no cell produced an interval, and a caveat names the method and the bootstrap mode whenever any cell has no analytic route, a mixed run included. Two intervals that do exist are flagged: point-biserial’s dichotomous margin breaks the binormality the Fisher-z interval assumes and its sampling variance depends on the group split (Tate 1954), so a split more lopsided than about 20/80 gets a caveat pointing at the bootstrap; and the atanh Wald SE of the latent methods carries a $1/(1-{\rho }^2)$ factor, so an association near perfect back-transforms to bounds spanning almost the whole [−1, 1] however precisely ρ was estimated – a perfectly associated 2×2 gives ρ_tet = 1.000 with SE 0.002 and prints [−1.000, 1.000] beside p < 10⁻⁵⁰. That interval stays visible, because the delta method’s limit is the honest thing to show, and a caveat says an interval covering 0 beside a coefficient that strong is the degeneracy, not weak evidence. The two-step latent SE is also conditional on the estimated thresholds, so it runs slightly narrow as categories grow – negligible for a 2×2 tetrachoric, mildly anti-conservative for fine tables – and a pair with a many-category variable carries a nudge to the bootstrap, which refits per resample and captures the threshold uncertainty exactly.

The bootstrap interval is bias-corrected percentile, floored at 100 replications, gated at 80% usable resamples and disclosed below 1,000. The interval is the percentile interval recentred for median bias – the first of BCa’s two corrections without the acceleration term – because the bounded, non-negative measures pile up near zero and their resampled distributions are skewed, while a symmetric distribution collapses the correction back to the plain percentile interval, so the classical coefficients are unaffected. A percentile interval’s resolution is roughly 1/B: measured on Pearson at n = 40, ρ = 0.5, a nominal 95% interval covers about 91% at the shipped default of 100 replications and 93.5% at 1,000. The setting goes as low as 10, which no interval can survive, so the module floors it at 100 – a lower value is raised rather than published – and above the floor discloses the gap instead of overriding the user: whenever bootstrap intervals are on and the setting is below 1,000, a caveat states the measured under-coverage. Some resamples fail outright – a degenerate polychoric table, a rank partial that cannot be inverted – and a 2.5% quantile over a handful of survivors is not a 95% bound, so a cell renders its interval only when at least 80% of the requested replications produced a usable value, and a caveat covers the cells that thinned below the gate.

Every method gets an equivalence test, two-sided for the signed coefficients and one-sided for the unsigned measures. The signed family – Pearson, Spearman, Kendall, point-biserial, biserial, polyserial, polychoric, tetrachoric, Somers’ D, γ, β and φ – pits H₀ |ρ| ≥ Δ against H₁ |ρ| < Δ by two one-sided tests on the Fisher-z-transformed coefficient against −Δ and +Δ, reporting the larger p; a TOST at α corresponds to the (1 − 2α) interval lying inside ±Δ, not the (1 − α) interval printed beside it, so the caption spells that level out. The unsigned measures live on [0, 1], where a lower test against −Δ is vacuous, so they test the single null H₀ θ ≥ Δ, whose p corresponds to the upper end of the (1 − 2α) interval falling below Δ. Because one table can mix the two shapes – a Mixed/auto run, or an unsigned method falling back to another – the caption always states the margin used and each family’s null. Under the bootstrap interval mode the whole signed family switches to the resampled z-scale standard error, so the interval and the verdict on one card rest on one sampling distribution rather than a bootstrap CI beside a Fisher-z p. The p is adjusted as a family of its own, never pooled with the significance p-values, because it asks the opposite question.

The unsigned family’s one-sided p comes by three routes, and a standard error with no closed form is resampled on demand. η² and Cramér’s V invert their own noncentral distribution – the noncentral F of the one-way ANOVA and the noncentral χ² of the table – which is exact, needs no standard error and costs nothing extra, and makes the equivalence test and the analytic interval two views of one calculation: set Δ to the interval’s upper bound and p (Δ) returns exactly α. Chatterjee’s ξ uses the same envelope SE as its interval, on its plain [0, 1] scale. NMI, AMI, coherence, Theil’s U, Hoeffding’s D and distance correlation have no closed-form SE at all and buy a per-cell bootstrap SE on demand, as do φ – whose asymptotic SE depends on the 2×2 margins rather than on φ alone – and the Spearman and Kendall semi-partials, while Blomqvist’s β derives its SE from the concordant-quadrant proportion behind the coefficient, the proportion its Wilson interval uses. Setting a margin therefore triggers a resample for those cells even when intervals are off; that is deliberate, and it is the only extra cost.

p (Δ) is floored at 1/(B + 1) wherever its standard error was resampled. A standard error estimated from B replicates carries a few percent of relative error, and a normal-approximation p is exquisitely sensitive to it – a 7% wobble in the SE can move the p by tens of orders of magnitude, so an unfloored 3e-246 says far more than the resample can support. The p is therefore floored at the resample’s own resolution, $1/(B+1)$ – 0.0099 at the default 100 replications, 0.002 at 500 – for the six measures with no closed form, φ, the Spearman and Kendall semi-partials, and the whole signed family under the bootstrap interval; a cell sitting exactly at the floor is the strongest claim the replication count supports, not a marginal result, and a caveat names the affected symbols. The routes with a genuine closed-form SE, and η² and Cramér’s V, which invert an exact distribution, are not floored.

Δ is a raw margin, not a rescaled one. Whatever the method, Δ is compared against the coefficient as the table prints it, with no measure mapped onto a common scale first – the honest reading of “the largest coefficient you would still consider practically zero” – but one number then marks out different amounts of association from cell to cell: Δ = 0.3 against Pearson’s r is a moderate linear association, against ε² 30% of the variance explained, against an information measure 30% of the shared-information ceiling. The caption states this whenever any cell carries a p (Δ), and a second line warns when one run’s equivalence cells span more than one strength-band set, since their verdicts are then not comparable even at the same margin; “scale” is exactly the band set a coefficient reads on, so the coefficients the module already treats as reading alike are one scale.

Reading results #

The strength label reads τ, β and coherence on their r- and NMI-equivalents; γ and Somers’ D keep the r bands with a caveat. Under bivariate normality Kendall’s τ and Blomqvist’s β both follow (2/π)·arcsin ρ, so r = 0.5 corresponds to τ ≈ 0.33 and banding τ on the Cohen cutoffs written for r would read a band low for the same association; the module maps each back with sin(π·stat/2) before banding – exact under an elliptical reference – so the label reports the strength Pearson’s r would, and the displayed coefficient is untouched. Rajski’s coherence is MI/H(X,Y), which is exactly $\text{NMI}/(2-\text{NMI})$ – a full band low across the range, an NMI of 0.50 being a coherence of 0.333 – so it is mapped back with $2C/(1+C)$ before the normalized-mutual-information bands are applied; Theil’s U and AMI track NMI with no fixed offset and need no map. Goodman & Kruskal’s γ and Somers’ D run larger than r for the same association – they inflate over τ on tied ordinal data – with no closed-form map back, so rather than invent unsourced cutoffs the module bands them on the r cutoffs as they stand and marks the labels approximate in a caveat under the table.

φ reads on Cramér’s V’s bands, ε² on Cohen’s η² cutoffs, and every corrected estimate on cutoffs written for the population quantity. On a 2×2 table |φ| is Cohen’s $w=\sqrt{{\chi }^2/n}$, the quantity Cramér’s V generalizes, so the χ²-family cutoffs are the calibrated ones for it and the same 2×2 association reads the same strength whichever of the two is picked – φ keeping its signed positive / negative label, which V has no sign to carry. Cohen’s .01 / .06 / .14 for η² are population variance-explained cutoffs, so the bias-corrected ε² the module reports (argued above) is the quantity they were written for, and the raw ratio would read a band or two high with many groups and a modest sample. Cramér’s V is corrected the same way (argued above), but Cohen’s df = 1 cutoffs assume a large sample and no correction changes that, so V’s labels carry a caveat on small samples and large tables.

P-value adjustment #

The matrix family counts each pair once, and a directional pair twice only where its two p’s genuinely differ. The adjustment runs before the matrix is mirrored, so a symmetric method’s two cells are one test; an asymmetric run computes both orderings and both are in the family, except where the two carry the same p by construction – a semi-partial run, since ppcor::pcor.test is symmetric in x and y, and Theil’s U, whose p comes from the symmetric mutual information – where counting both would run p.adjust at n = 2m and make every adjusted p roughly twice as conservative, so the mirror row rides on the first one’s adjusted value. The equivalence p is exempt from the collapse: its standard error is direction-specific – Aloe & Becker’s for the semi-partial, a resampled one for Theil’s U – so the two orderings’ p (Δ) genuinely differ and are adjusted as their own family.

Visualizations #

Unsigned measures are coloured and sized on a per-method ramp to their own observed maximum, and a lone cell falls back to the absolute scale. Unsigned statistics live in compressed ranges in practice – a dCor matrix rarely reaches 0.5 – so scaling them to the theoretical 1 would render most matrices as uniformly pale, fat glyphs; normalizing to the strongest coefficient the method contributed keeps the within-matrix ranking readable, and colour and glyph size – ellipse eccentricity, edge thickness, force-link distance – take the same scale, so the darkest cell is also the largest glyph. The normalization is per method rather than per run because unsigned measures share no scale: an η² of 0.30 is a very large effect and a Cramér’s V of 0.30 a moderate one, so one ramp over a shared maximum would make identical colour mean two different things – hence one legend strip per method, each titled with its symbol, in an η² run falling back to V on its categorical pairs or in any Mixed/auto run, with the size-coupling caption emitted once below the last strip. A method contributing a single plottable cell is its own maximum and would paint at full colour with a saturated glyph however small the value, so its strip runs 0…1 instead – a lone V of 0.40 reads as 0.40, not as 1 – and the caption names the strip’s printed top rather than “the observed maximum”, since for a singleton strip the two are no longer the same number. Mixed/auto colours its unsigned pairs on a green ramp rather than the unipolar blue one because blue would read as “strong positive” beside the diverging scale’s blue end, and both unsigned ramps start at a visible tint rather than near white, since a significant cell outlines itself in its own fill and a white low end would draw a computed ≈ 0 as an invisible cell, indistinguishable from the – that means “not computed”. Signed methods are untouched: their size and colour encode |r| on the fixed −1…+1 scale.

Sign rides on a second channel, and showing the non-significant links rebuilds the layout but not the scales. Colour alone carries a sign badly – an export, a print or a colour-vision deficiency flattens it – so a negative link is dashed and a negative correlogram cell tilts the other way, mirroring each other; an unsigned method has no sign to encode and gets neither. The two network graphs derive node order, spacing and forces from the visible link set, so Show non-significant links (faded) rebuilds the diagram rather than restyling it: the picture with the faded links is the honest layout for that set, not the significant-only layout with lines added on top. The intensity scales are deliberately left out of that rebuild: colour and thickness are normalized against every computable pair in the matrix, significant or not, so a link keeps its colour and width in both states and the legend prints the full-matrix maximum. Normalizing against the visible set would paint the strongest significant link at full strength even when a stronger non-significant one exists – the two come apart as soon as pairwise deletion gives pairs different sample sizes, since only at a common n is the most significant link also the largest – and toggling the box would silently repaint every edge that stayed. The dash slot stays with the sign, so a non-significant negative pair still reads as negative through its fade, and on the force graph the fade is set as whole-element opacity rather than stroke opacity, since an arrowhead marker paints from its own fill and would otherwise stay at full strength on a faded line. A run with no significant pair keeps the card and the toggle, so the faded links can be shown instead of an empty card; with the toggle on nothing is filtered, so an empty set then means no plottable pair at all, and the empty-state message says so rather than blaming the significance filter.

A directional run gets bowed arcs with arrowheads on the force graph, and no edge bundling. Under an asymmetric method or a semi-partial run A → B and B → A are different estimates and both are kept, but they sit on the same geometry, so a straight line would hide one behind the other; each ordering is drawn as a shallow arc with a fixed sweep, so a mutual pair bows to opposite sides and stays separately readable, at a radius of twice the chord – shallow enough that the straight-line intersection with the target’s label pill is still where the arrowhead lands. The arrowhead is pulled back to the pill’s edge so the label does not bury it, drawn in user-space units so every arrowhead is the same size whatever the stroke width, and one marker is defined per link colour, since a marker does not inherit its line’s stroke. A symmetric run draws no arrowheads, so the directed rendering never appears where it would be meaningless. Edge bundling is withheld rather than hidden: its arcs run between two positions on a circle and read the same in either direction, and directing them would need an arc convention and a second layout pass, which reads poorly at any density – so the checkbox is disabled with a note pointing at the force graph.

The correlogram’s clustered order is average linkage on 1 − |r|, computed from the run’s own coefficients, and off by default. The distance is 1 − |r| rather than the signed 1 − r, so a strong negative relationship counts as strong agreement about the block and is not pushed to the far end of the axis; an unsigned method’s coefficients are first rescaled by their own observed maximum – on the correlogram’s own glyph rule, a lone value keeping the full 0–1 range – because raw MI runs past 1 and the widest-scaled method would otherwise decide a mixed run’s ordering on its own. A pair that errored or was never computed enters at the maximum distance, so an absent coefficient never reads as agreement, and a directional run’s two orderings of a pair average, since they are two readings of one association and the axis is single. The linkage is hclust(method = "average") on the dissimilarity the coefficients already give, so no second pass over the data is made and the order is the tree’s leaf order; both axes take it, which is why it needs the same variables on both – a rectangular run has two unrelated axes and keeps its selection order, with a note saying so. The default is off deliberately: a variable set picked in a meaningful order, subscale by subscale, should not have that order disturbed unless asked.

Each scatter overlay is the fit the coefficient names, with the LOWESS smoother as the shape check beside a straight line. Pearson’s r measures the OLS line, so that line is drawn, with the conditional-mean band on n − 2 (n − 2 − k under covariates) degrees of freedom at the global level under Analytic and a percentile envelope of OLS refits under the bootstrap mode – capped at 400 refits and floored at 100 whatever the replication setting, since the loop runs synchronously inside the draw with no progress or cancellation, and a band from a few hundred refits is visually indistinguishable from one at the user’s count, which still governs the coefficient’s own interval; the refits are seeded from the run’s bootstrap seed, offset per pair, so the band repeats when a seed is set. Curvature is the one thing a straight fit cannot report and exactly what r misses, so a dashed LOWESS smoother shadows the line once the pair has five distinct x values – below that (a dichotomous predictor) it would only retrace the line, so it is suppressed. The smoother is a tricube-weighted local linear fit over a window holding two thirds of the sample, refined by three bisquare robustness passes that down-weight whatever the previous pass left far from the curve – local-linear rather than a local mean, which carries the slope of the design density into the fit, worst at the two ends, where a skewed overlay is read hardest; above 128 rows the robustness residuals read a fit interpolated between evenly spaced anchors instead of one local fit per row, the delta shortcut of lowess itself, and the curve spans the observed x range alone, since a nonparametric fit has no support outside the data. Point-biserial and biserial draw the same OLS mean-difference line, but biserial is kept out of the analytic band: as a normal-latent rescaling it has no valid conditional-mean t band, so only the bootstrap envelope is drawn. Spearman, Kendall, the latent methods, γ, Somers’ D and the three dependence detectors get the smoother alone – no parametric line names their number, and a monotone or nonlinear structure is what they respond to. Blomqvist’s β is defined by sign agreement around the two medians, so median crosshairs are its overlay; η² is the between-level share of variance, so a tick at each level’s mean of the continuous side, joined by a dashed profile, is the reported quantity – the k-group form of the two-group means line – and needs a numeric-coded grouping side, since a string-valued one drops every row as non-numeric and the plot declares itself not applicable rather than short of data. Tetrachoric’s two binaries make a 2×2 grid of points where any fitted line or smoother would be meaningless, and φ, Cramér’s V and the information-theoretic family are categorical, where a continuous fit would mislead – so those draw the points alone.

Point-biserial and biserial put the dichotomy on X. Whichever list the variables came from, the binary variable goes on the horizontal axis and the continuous one on the vertical: with the binary on Y the OLS overlay would be a linear-probability line whose confidence band assumes homoscedastic normal residuals a 0/1 response cannot produce, while with it on X the identical fit is the two-group means line whose assumptions the mean-comparison checks already test. Polyserial keeps the continuous side on X, where its smoother needs the predictor; a pair with both sides of one type keeps the selection order, having nothing to order by.

Grouped scatterplots draw no bands and residualize within each group; anchor overlays residualize once and commit to one orientation. K overlapping bands are visually unreadable, so the grouped layout draws none and points at the comparison table for the formal Δr and its interval; each group’s coefficient is starred by its significance verdict – the adjusted-when-present p, matching every other gate – since no per-group p is printed and an insignificant r would otherwise read on a par with a significant one. Residualization under a partial runs on each group’s own subset, so the residual scatter reflects that group’s conditional relationship – a pooled residualization would smear exactly the group differences the comparison is about – while the columns are converted before slicing, so a 0/1 coding cannot differ between groups when one lacks a level. The anchor overlay residualizes per variable rather than per pair, because every comparator series shares one anchor residual on the common X axis and the anchor cannot be re-residualized per pair; in a semi-partial mode the residualized side is the cell’s row or column, and a reference-mode panel can mix the two, so the first comparator fixes the panel’s orientation – a per-series decision would put residual and raw data on one shared axis. A comparator sharing no variable with the reference cell is left out, since a shared-axis overlay without a shared variable is not honest, and its Δr is tested in the table regardless; the panel’s overlay takes the comparators’ shared method when they agree and the run’s method otherwise, so a Mixed/auto run never forces an OLS line onto a rank or categorical pair. The single-matrix displays – edge bundling, force graph, correlogram – are skipped in a grouped run because they have no per-group rendering, and the single-sample pattern-equality omnibus gets no comparison-aware scatter, its honest picture being the correlogram.