Parametric and non-parametric tests #

The test list in comparison analysis comes in pairs. Beside the t-test sits the Mann–Whitney test, beside one-way ANOVA the Kruskal–Wallis test, beside the paired t-test the Wilcoxon signed-rank test, and correlation analysis offers Spearman’s coefficient beside Pearson’s. One of each pair is parametric and the other non-parametric, and the advice that usually travels with them – “use the non-parametric one when your data is not normal” – is true often enough to be repeated and wrong often enough to deserve a page. This one explains what the two words mean, what a rank is and what a rank-based test actually asks, where the two families agree and where they part, the robust tests that sit between them, how each kind of test arrives at its p-value, and how to choose. The numbers come from the café data, prepared as that page describes, and every test named here is in the comparison or correlation module.

Two families of test #

Parametric test is a test that assumes the data – or the noise in it – follows a distribution described by a few parameters, nearly always the normal distribution with its mean and standard deviation, and that asks its question about those parameters: do two means differ, is a slope zero. The t-tests, ANOVA and its relatives, Pearson’s correlation and ordinary regression are all parametric. In the app’s role vocabulary, a parametric two-group comparison asks whether the mean of the dependent variable differs between the two levels of the grouping variable. On the café data, Welch’s t-test on spend_before by branch reads: Riverside mean 7.91 (SD 2.97), Station mean 9.52 (SD 4.05), t(179.5) = −3.20, p = 0.002, a difference of −1.61 with a 95% CI of [−2.60, −0.62]. The assumption is what buys the precision: the test uses every value’s exact distance from its group’s mean, and when the model holds nothing extracts more from the data. What “roughly normal” has to hold of is the values within each group – not the dependent variable pooled across groups – and the central limit theorem covers a good deal of departure once each group has a few dozen cases; the normality section says how much.

Non-parametric test is a test that makes no assumption about the shape of the distribution the values came from. Most of them work on the ranks of the values rather than the values themselves – the Mann–Whitney, Wilcoxon signed-rank, Kruskal–Wallis and Friedman tests, and Spearman’s and Kendall’s correlations. The same comparison under the Mann–Whitney test reads: Riverside mean rank 88.5, Station mean rank 112.7, U = 3788, p = 0.003 – the same verdict, reached without using a single spend in currency units. The name is slightly misleading in both directions. The tests are not free of assumptions: every one of them assumes the cases are independent, and reading one as a test of medians assumes something more (below). And not every non-parametric procedure ranks anything – the permutation and exact tests further down work on the values themselves. What the family shares is that its p-values do not depend on the population being normal, or any other named shape.

Distribution-free is the more exact name for that property: the distribution of the test statistic under the null hypothesis is known whatever the population looks like, because under the null every arrangement of the ranks – or of the group labels – is equally likely, and the statistic’s behaviour follows from counting arrangements rather than from a formula about the population. It is the reason a rank test’s p-value can be trusted on ten cases per group where a t-test’s rests on a normality that ten cases cannot confirm.

A nurse comparing days to recovery between two wards, a teacher comparing test scores between two classes and a marketer comparing time on page between two designs face the same choice, in the same words: the dependent variable’s mean by the grouping variable, or its ranks.

What a rank is #

Rank is a value’s position when the sample is sorted from smallest to largest: the smallest value has rank 1, the next rank 2, and the largest has the sample size for its rank. The longest wait in the café data, 19.9 minutes, has rank 185 of 185; the 16.3 behind it has rank 184. Ranking throws away how far apart values are and keeps only their order: the 19.9 would have rank 185 if it were 12 minutes or 120. That is what makes rank methods indifferent to outliers and to skew – the catering order’s 66.53 in spend_after has rank 200 of 200, exactly the rank it would have as a 20.09 one cent above the runner-up’s 20.08 – and it is also what they pay for that indifference: a test on ranks cannot tell a gap of one cent from a gap of 46. Data transformation writes a variable’s ranks as a new column under Average ranks, which is what a rank test does internally before it starts.

Ties are values that are equal and so cannot be put in order; each receives the average of the ranks the group of them would have occupied. Two members waited exactly 11.9 minutes; between them they hold ranks 182 and 183, and each is given 182.5. Ties are rare on a measurement – the 185 waits take 82 distinct values – and the rule on a rating scale: the 193 overall scores take only ten distinct values, and the 40 members who scored 7 occupy ranks 61 to 100 and every one of them is given 80.5. Ties matter for two reasons. They shrink the spread of the ranks, so every rank test applies a tie correction to the variance its p-value is built on, and when ties are heavy enough the approximation behind that p-value degrades – which is what the correlation module’s Acceptable ties verdict and the heavy-ties advisory threshold in Settings watch for (the tie term is 0.058 for taste, a five-point item whose levels are all used, under the default limit of 0.10; 0.027 for overall; 0.0003 for wait). And tied values close off the exact route to a p-value, so a test that would have enumerated falls back on the approximation (both are explained below).

Mean rank is the average of one group’s ranks in the pooled ranking, and the group summary a rank test compares where a t-test compares group means. If the grouping variable makes no difference, every group’s mean rank sits near the middle of the ranking, $(N+1)/2$ – 100.5 for 200 members. On spend_before, Riverside’s mean rank is 88.5 and Station’s 112.7: Station’s members sit higher in the pooled ranking on average, and the size of that gap is what U measures. The comparison module adds the mean ranks to its group descriptives when Mean rank (for rank-based tests) is checked under summary statistics. A mean rank is read within its own table only – 88.5 means nothing without the 200 it was ranked among – and the gap between two mean ranks is not an effect size; Cliff’s δ and the rank-biserial r are, and they are computed from the same ranks.

What a rank test actually tests #

The Mann–Whitney test is the place to be precise, because its usual one-line description – “a test of whether the medians differ” – is not what it does. It ranks the two groups together and its statistic U counts, over every pair made of one member from each group – 101 × 99 = 9,999 pairs here – how often the first group’s member has the larger value. Under the null hypothesis that the two groups’ values come from the same distribution, a member from either group is equally likely to be the larger of a pair, so U sits near half the pairs. On spend_before, the Riverside member out-spends the Station member in 3,788 of the 9,999 pairs – 38%, where “no difference” predicts 50% – and that shortfall is the p = 0.003. What the test is sensitive to, then, is one group’s values tending to be larger than the other’s: the probability that a random Station member out-spends a random Riverside member, 62% here, which the common language effect size reports directly and Cliff’s δ rescales to −0.24 on its −1 to 1 scale. What it does not ask is by how much.

Location shift is the extra assumption that turns “tends to be larger” into “the median is larger”: that the two groups’ distributions have the same shape and the same spread and differ only by being slid along the axis. Under it the mean, the median and every quantile of one group sit the same distance from the other’s, and the test is a test of that distance. When the shapes differ – one group more spread out than the other, or more skewed – the Mann–Whitney test can reject with the two medians equal, and its “difference in location” reading breaks down. That is the case the Brunner–Munzel test (below) is built for, and why the comparison module’s assumption check steers there when the spreads differ.

Hodges–Lehmann estimate is the rank family’s own answer to “by how much”: the median of all the pairwise differences between a value from one group and a value from the other – across the 9,999 Riverside-minus-Station spend pairs, −1.52, with a 95% CI of [−2.52, −0.53]. It is the number in the Difference column of a Mann–Whitney card, where a t-test card holds the difference of means (−1.61 [−2.60, −0.62] on the same data). For a paired design it is the median of the pairwise averages of the within-pair differences, called the pseudo-median, which descriptive statistics also offers for a single variable as the HL pseudomedian. It is what a rank test estimates when the shift assumption holds, and it comes with a confidence interval computed from the rank distribution, so a rank test’s result can be reported as an amount and not only as a p.

The paired tests ask a matching question of the within-pair differences. The Wilcoxon signed-rank test ranks the sizes of the differences, ignoring their signs, and asks whether the positive and the negative differences share the ranks evenly. On the 199 members’ change in spend (the catering order blanked), the mean change is +1.09 and the median +0.93; the paired t-test gives t(198) = 5.37, p < 0.001, the signed-rank test W = 13,850, p < 0.001, and the pseudo-median of the changes is +1.05 – three ways of saying the same thing on a variable that is well behaved. The signed-rank test’s assumption is that the differences are symmetric about their centre (their skewness is −0.13 here), because it uses their sizes; the sign test drops even that and counts only directions – 125 of the 197 members whose spend changed at all spent more and 72 less, p < 0.001 – which makes it valid for anything ordered and the least powerful of the three. The dependent-samples table lists all three.

When the two families part #

Where the data is close to normal, the two families agree, as they did above. The instructive cases are the ones where they do not, and the café has two of them.

visits_before is a count with a long right tail (skewness 2.07): most members visited a handful of times in the eight weeks, and a few regulars visited twenty to forty-two times – six of them at Station, none at Riverside, whose most frequent visitor came nineteen times. Welch’s t-test compares means of 6.14 and 7.84 visits and gives t(166.8) = −2.20, p = 0.029. The Mann–Whitney test gives U = 4341.5, p = 0.107: a random Station member out-visits a random Riverside member 57% of the time, and the Hodges–Lehmann shift is one visit. The t-test rejects and the rank test does not, and neither is wrong, because they measure different things. The mean difference of 1.7 visits is carried in good part by Station’s regulars – the 20% trimmed means are 5.5 and 6.7, a gap of 1.2, and Yuen’s test on them gives p = 0.10 – and the t-test, which counts every visit, sees it; the rank test, which counts a regular’s forty visits as one member above the median, sees a modest tendency. If the owner’s question is about the till – visits are revenue, and a regular’s visits count as much as anyone’s – the mean is the quantity and the t-test the test, and with a hundred members per group the central limit theorem carries it through a skewness of 2. If the question is about the typical member, the median, the shift or the trimmed mean is the quantity, and the honest answer is “about one visit, not distinguishable from none at this sample size”.

The change in visits after the menu is the same story in a paired design. The mean change is +0.64 visits and the paired t-test gives t(199) = 1.96, p = 0.051 – the result on the boundary that the hypothesis testing page works through. The signed-rank test gives p = 0.165, the sign test p = 0.173 (97 members visited more often, 78 less often, 25 exactly as often), and the median change is zero. The mean change is real arithmetic – the total number of visits went up – but it lives in a few members who added many visits, and the typical member’s change is nothing. Here the rank tests are the fairer summary of what happened to the members, and the t-test the fairer summary of what happened to the till.

The six survey items go the other way. On friendly, the item where the two tests fall on either side of 0.05, Welch’s t-test on the 1–5 scores gives p = 0.016 and the Mann–Whitney test p = 0.051; on quick, 0.013 against 0.031. On a five-point item nearly every value is tied with dozens of others, the rank test spends its power on the ties, and the t-test on the item means – with a hundred members per group – is not the wrong tool it is sometimes said to be. Which of the two to report is the question of ordinal data, below.

When a t-test and a rank test disagree, the disagreement is information: the difference between the groups lives in the tail of one of them. Look at the box plot by group, find the tail, and decide which quantity the question is about before deciding which test was right. A ward where a few patients stayed for months, a class where a few students scored zero, a page where a few visitors left it open overnight: the mean and the ranks will part, and the question decides.

The price of each #

The rank tests are cheaper than their reputation. When the data really is normal, the Mann–Whitney test needs about 5% more cases than the t-test for the same power – its efficiency relative to the t-test is 3 ⁄ π, about 0.95, and it never falls below about 0.86 whatever the shape. When the tails are heavy, the rank test is the more powerful of the two, because the extreme values that inflate a t-test’s standard error are just the top ranks to it. “Non-parametric tests need larger samples” is true by a twentieth on normal data and false on the data that made you consider them.

What the parametric family can do and the rank family cannot is broader than power. A parametric test estimates the effect in the variable’s own units with a confidence interval as a matter of course; a rank test’s estimate, the Hodges–Lehmann shift, exists only under the shift assumption. Parametric models take covariates – ANCOVA has no rank twin in the app – and separate main effects from interactions in a factorial design, where the rank family’s one entry, ART-ANOVA, manages it by aligning the data first. And the parametric family is the one regression, factor analysis and structural models are built on, so a result that continues into a model is more easily reported in the same terms. When the parametric test is honest, it is the better choice; the point of this page is knowing when it is.

Sample size cuts both ways. The folk rule “non-parametric for small samples” is half right: with ten cases per group normality cannot be checked and the t-test’s p rests on it, where the rank test’s exact p rests on nothing. But very small samples also expose the rank tests’ granularity – with three cases per group the smallest two-tailed p a Mann–Whitney test can produce is 0.10, and it cannot reject at all; the signed-rank test on five pairs bottoms out at 0.0625, and needs six to reach 0.05 – so a null result from a rank test on a handful of cases means very little. At large samples the balance tips the other way: the central limit theorem makes the t-test robust, while a normality test rejects on departures that no analysis would feel, so “the Shapiro–Wilk test was significant” at n = 200 is not a reason to abandon a test on means. What a large sample does not repair is the rank test’s shift assumption for a location reading, and that is what to check when the two families disagree.

Ordinal data is data whose values are ordered categories – strongly disagree to strongly agree, never to always, a 0–10 rating – where the order is meaningful but the distances between steps are not guaranteed to be equal. A mean of such values assumes that the step from 1 to 2 is worth the same as the step from 4 to 5; a rank test assumes only that 2 is more than 1. The six survey items and overall are ordinal, and the sensible practice is a compromise. For a single item with a few levels, report the rank test, or an ordinal regression: the mean of a five-point item is a convenient number that answers a question nobody asked. For a score made by summing or averaging several items – the food scale from taste, fresh and value – means and t-tests are the norm, because the sum of several ordered items behaves close enough to a measurement for the difference not to matter. For a 0–10 rating either reading is defensible, and the choice should follow the question rather than the p-value. The Variables dialog types a column ordinal; the comparison module requires that type only for its trend tests and otherwise treats an ordinal column as numeric, so the choice of test is yours.

Correlation on ranks #

Rank correlation measures how consistently two variables move together in order – whether the member who waited longer tends to be the one less satisfied – without asking whether the relationship is a straight line. Replace each variable by its ranks and correlate the ranks: any relationship that is monotonic, always rising or always falling however curved, gives a strong rank correlation, and one wild pair of values counts for no more than its rank. Pearson’s r measures the straight line and is pulled by the far-out pairs; the two rank coefficients below measure the order and are not.

Spearman’s rho is Pearson’s r computed on the ranks. Where the relationship is close to linear and the variables well behaved the two agree: overall against taste gives r = 0.55 and ρ = 0.54. Where the values run out into a tail they part: overall against wait gives r = −0.30 and ρ = −0.37, because the members who waited sixteen and twenty minutes pull Pearson’s line about and are simply the two highest ranks to Spearman. The catering order shows the same thing from the other side: the correlation between spend_before and spend_after is r = 0.69 with the order blanked and 0.60 with it in – one cell in two hundred costs the coefficient 0.09 – where ρ reads 0.68 either way.

Kendall’s tau counts pairs of cases instead of correlating ranks: a pair is concordant if the case higher on one variable is higher on the other, discordant if not, and τ is the concordant share minus the discordant share. overall against taste gives τ = 0.45, overall against wait τ = −0.27. It runs smaller in absolute value than ρ on the same data as a rule – a different scale, not a weaker relationship – and reads more plainly: pick two members at random, and the one who rated the taste higher is more likely to be the more satisfied overall by 45 percentage points. It handles ties and small samples better than ρ and is the usual choice for both; ρ is the more widely reported. The correlation manual sets the two beside Pearson’s and the measures that go further, and its p-value method option offers a permutation p-value where ties make the approximation doubtful.

A doctor asking whether dose tracks response, a teacher whether attendance tracks grades, an analyst whether price tracks demand: where the question is “does more of one go with more of the other” rather than “by how much per unit”, the rank coefficient is the one that answers it.

The middle ground: robust statistics #

Robust describes a statistic, or a test built on one, that a few extreme values cannot move far. The median is robust and the mean is not: the catering order moves the mean of spend_after by 0.28 and its median by 0.01. Full robustness has a price – the median ignores most of what the data says about the values it does not sit on – and the estimators between the mean and the median are built to keep most of that information while capping the influence any one value can have. A parametric test built on them keeps the family’s strengths – an estimate in the variable’s units, unequal spreads handled, more than two groups – without its fragility, which is why the comparison module offers a robust test beside the classical and the rank one.

Trimmed mean is the mean of the values that remain after a fixed share is removed from each end of the sorted sample. At the app’s default of 20% per tail, the 185 waits lose their 37 shortest and 37 longest and the mean of the 111 in the middle is 4.85 minutes – against a mean of 5.20 and a median of 4.9. It sits between the two by construction (a 0% trim is the mean, a 50% trim the median) and shares the median’s indifference to what happens in the tails: the catering order moves the 20% trimmed mean of spend_after from 9.52 to 9.55, where it moves the mean from 9.75 to 10.03. Descriptive statistics reports it under Trimmed mean at the percentage set beside it; Yuen’s test and t1way compare trimmed means between groups at the trim level.

Winsorizing replaces the values in each tail with the nearest value that is kept, instead of removing them: the 37 shortest waits are all set to the 38th shortest and the 37 longest to the 38th longest. The sample keeps its size, and every extreme case still counts – as “at least this much” rather than at its face value. The winsorized standard deviation is the spread that belongs with a trimmed mean, and the one Yuen’s test builds its standard error from: 1.75 minutes on the waits against an SD of 2.91, and 2.34 on spend_after whether the catering order is in or out, where the SD reads 5.45 and 3.70. Descriptive statistics reports it as Winsorized SD under dispersion; data transformation’s Winsorize rule writes the winsorized values as a new column.

Median absolute deviation is the median of the distances of the values from their own median – the robust twin of the standard deviation, built from medians the way the SD is built from means. Half the waits lie within 2.0 minutes of the median wait of 4.9. Multiplied by 1.4826 it estimates the SD a normal variable with the same middle would have – 2.97 against the waits’ SD of 2.91 – and that scaled form is the one it usually appears in: data transformation’s Robust z divides by it in place of the SD, and the comparison module measures an equivalence margin in it for the rank tests, so that the margin’s scale matches the test’s. Descriptive statistics reports it unscaled, under the header Median AD.

The robust tests in the comparison module follow from these. Yuen’s trimmed-means t-test compares two 20% trimmed means with a standard error from the winsorized variances and a Welch-style adjustment for unequal spreads: on spend_before by branch the trimmed means are 7.79 and 9.33, the difference −1.54 with a 95% CI of [−2.59, −0.49], p = 0.004 – between Welch’s −1.61 and the Hodges–Lehmann −1.52, as it should be on data where all three are honest. It is the test for the corner where heavy tails and unequal spreads arrive together, which neither the classical nor the rank family handles well; t1way is its omnibus for three or more groups, and both are listed with the independent-samples tests. The Brunner–Munzel test is the rank family’s own robust member: it drops the shift assumption and estimates the probability that a random member of one group exceeds a random member of the other – 0.62 that a Station member out-spends a Riverside one – with a standard error of its own, so it stays valid when the spreads differ. On spend_before it gives p = 0.003, indistinguishable from the Mann–Whitney test because the two branches’ shapes are alike; the assumption check recommends it over Mann–Whitney exactly when they are not.

Trimmed means and the MAD are the same estimators in every field that has outliers – reaction times in psychology, incomes in economics, assay readings in a laboratory – and a robust test is the standard answer wherever a few extreme cases are real and must not be deleted.

How a p-value is computed #

Both families reach their p-values by one of three routes, and the notes under a results card say which one ran. The route is a property of the computation, not of the test’s assumptions, and it is worth knowing because it explains why the same test prints slightly different p-values at different sample sizes and why some cards carry no test statistic at all.

Exact test computes the p-value by enumeration: every way the observed values could have been arranged under the null hypothesis, and the share of arrangements at least as extreme as the one observed. Fisher’s exact test lists every 2×2 table that the observed row and column totals allow – for recommend by branch, p = 0.175. The Mann–Whitney and signed-rank tests enumerate the rank arrangements at small samples (R does so below fifty cases per group, when there are no ties), and McNemar’s test uses the exact binomial distribution below 25 discordant pairs. An exact p-value is exact for the null; it does not make the test’s other assumptions go away, and on a very small sample its coarse steps are the granularity described above.

Asymptotic describes a p-value computed from the distribution the test statistic approaches as the sample grows – the normal for U and W, the χ² for a chi-square statistic – instead of from an enumeration that would be too large. At 200 cases there is nothing between the two routes; at twenty there is little: the signed-rank test on the first twenty members’ change in spend gives an exact p of 0.130 and an asymptotic 0.131. A statistic labelled “asymptotic” is a statement about how its p was reached, not a warning about the result, and the app’s notes say when a test switched from one route to the other and why.

Continuity correction is a half-step adjustment made when a statistic that can only move in whole-number steps – a count, a rank sum – is read against a smooth curve: it shifts the statistic half a step toward the null before the curve is read, so that the smooth approximation does not claim more precision than the discrete statistic has. On the chi-square test of recommend by branch the Yates correction takes χ² from 2.12 to 1.71 and p from 0.145 to 0.192; Fisher’s exact p of 0.175 sits between them and is the one to trust when the table is small. The app applies the correction by default, so a chi-square card matches R’s and SPSS’s; on tables with every cell at 25 or more it is widely regarded as too cautious, and the continuity correction section says how to turn it off and which tests the checkbox touches. On a rank test at this sample size it is invisible: the Mann–Whitney p on spend_before is 0.003 with it and without.

Permutation test shuffles the group labels among the cases, recomputes the statistic, and repeats – hundreds or thousands of times – and its p-value is the share of shuffles that produced a statistic at least as extreme as the observed one, counting the observed arrangement as one of them. It is the exact test’s idea applied when the full enumeration is too large: a random sample of the arrangements instead of all of them. Under the null hypothesis that the label does not matter, every reshuffle is as likely as the real assignment, so the test assumes nothing about the population’s shape – only that the cases are exchangeable under the null, which unequal spreads can break. Shuffling the branch labels on spend_before 999 times (seeded) produced a gap between the two means as large as the observed 1.61 in two of them, for p = (2 + 1) ⁄ (999 + 1) = 0.003 – the same answer the three tests above gave, from a fourth route. The app uses permutation for the tests that have no usable formula: the information-theoretic and energy tests, the Jonckheere–Terpstra test, the Brunner–Munzel test’s small-sample branch, the Anderson–Darling test with ties, the correlation module’s permutation p-values and the moderator tests in meta-analysis. The number of shuffles is the Permutation replications setting, and its reciprocal is the finest step the p-value can take; the bootstrap seed fixes the shuffles so that a run can be repeated.

Monte Carlo is the general name for a p-value estimated from random simulation, of which the permutation p-value is the commonest case; the app also simulates tables under the null for a chi-square test whose expected counts are too small for the χ² curve. Being random, it differs slightly from run to run unless the seed is set, and a Monte Carlo p that reads exactly $1/(\text{replications}+1)$ is a floor, not a value – no simulated arrangement was as extreme as the observed one, and the note under the card says so.

Choosing #

Decide the question first and the test second, in this order:

1. Which quantity answers the question – the mean, the typical case, or the tendency for one group to be higher? Revenue, totals and anything that will be summed want the mean; a member’s or a patient’s experience wants the median or the trimmed mean; “does the treatment help” with no unit in mind is the rank test’s probability.
2. What kind of variable is the outcome? A measurement – minutes, currency, a scale score – takes the parametric test by default, and Welch’s version of it, which the app already prefers. A single ordinal item takes the rank test.
3. What shape do the groups have? Look at the box plots by group before any test of normality. Roughly symmetric, or a few dozen cases per group with a moderate lean: the parametric test is honest. Heavy tails or far-out cases on a small sample: the rank test or the trimmed-means test.
4. Do the spreads differ as well? Welch’s tests already allow for it. If the tails are heavy too, Yuen’s test; if you wanted a rank test, Brunner–Munzel rather than Mann–Whitney.
5. Is the question about the whole distribution rather than its middle? The Kolmogorov–Smirnov and Anderson–Darling tests compare shapes; the location tests will miss a difference that is all spread.

The comparison module’s Check assumptions button runs the normality and variance checks within each group, reads them against the assumption test significance level, and marks each test in the list as recommended or demoted; its test recommendations are advice about which test is honest on this data, not a verdict on the question, which is still yours. And never choose between the two families by which p-value came out smaller: that is not a method, and the previous sections have shown that the smaller p can belong to either.

Reporting #

Name the test and, where it was not the obvious one, why, in a clause: “Welch’s t-test (the branches’ spreads differ)”, “the Mann–Whitney test, as waiting times are strongly skewed”, “Yuen’s test with 20% trimming”. Report the estimate that belongs to the test – a difference of means with its confidence interval, the Hodges–Lehmann shift with its interval, the difference of trimmed means – beside the statistic, the degrees of freedom where there are any, the p and an effect size from the same family. When you ran both a parametric and a rank test, say so and report both: “the mean number of visits was 1.7 higher at Station (Welch t(166.8) = −2.20, p = 0.029) but the typical member’s was not (Mann–Whitney U = 4341.5, p = 0.107; Hodges–Lehmann shift 1 visit)” is a finding, and reporting only the one that reached 0.05 is not.

Where this appears in DataSuite #

• Comparison analysis – every parametric test beside its rank-based and robust alternatives in the independent, dependent and one-sample tables; Check assumptions and its recommendations; mean ranks among the group descriptives; the continuity correction and trim level controls; equivalence margins on the MAD scale for the rank tests.
• Correlation analysis – Pearson’s r beside Spearman’s ρ and Kendall’s τ, the tie checks, and permutation p-values as an option.
• Descriptive statistics – the Parametric and Nonparametric presets choose between the two families of summary; the trimmed mean, HL pseudomedian, winsorized SD and Median AD are among the statistics.
• Distribution analysis – the normality tests and plots that inform the choice.
• Data transformation – rank scores, winsorizing and trimming, and the robust z-score.
• Regression analysis – robust linear regression, the same idea applied to a model’s coefficients.
• Time to event analysis – the same two words for a different object: a non-parametric survival curve assumes nothing about the shape of the hazard, a parametric regression commits to one.
• Settings – permutation replications, the bootstrap seed, the heavy-ties threshold and the assumption test significance level.