DataSuite 2 Documentation
English Русский
On this page

Outliers and missing data

Two kinds of cell are not what they look like. One holds a value that should not be taken at face value – a typo, a sentinel, a real amount that answers a different question – and one holds nothing at all. Both are met before any test is chosen, both change results more than most choices of test do, and both were written into the café file on purpose. This page says what an outlier is, how the app finds one, and how to decide what to do with it; then what a missing value is, how the app counts around it, and what filling one in costs. The numbers come from the café data with its three fixes applied, except where the point is to show what happens without them.

What an outlier is

Outlier is a case whose value on a variable lies far from the rest of the sample – far enough that it changes what the sample’s summary says. The question is the same on any dependent variable: does this case sit with the others or away from them, and does the answer move when it is left out? In the café file the largest spend_after is 66.53 and the next largest 20.08; the longest wait is 19.9 minutes against a median of 4.9. What makes a value an outlier is not the distance alone but its pull on the summary: that one spend moves the mean of spend_after from 9.75 to 10.03 and its standard deviation from 3.70 to 5.45, one row in two hundred – while the median moves from 9.39 to 9.40, which is why the median is called robust and the mean is not.

Outliers, as a row or column header, is the count of cases the rule named beside it flagged, or the check that reports whether any case was flagged at all. The rule is in the header or the caption – 1.5·IQR, a modified z-score, studentized residuals, Mahalanobis distance – and each is explained on this page; the flag says a case is more extreme than the sample explains, not that it is wrong.

An outlier is one of three things, and the three want different treatment. An error – a value that was never true: a typo, a unit slip, a sentinel like the 999 the café’s form used for “age not given”. It should not be in the analysis at all, and the fix is to make it missing. A true value that answers the wrong question – the catering order: a real amount a real member paid, averaged into their per-visit spend, which is not what “did the menu change what people spend per visit” means. The fix is a rule stated in one sentence and applied to the cell. A true value that belongs – the 19.9-minute wait is a real bad day at the café, and the seven members who visited more than 18 times in eight weeks are its regulars. They are the tail of the distribution, not a mistake in it, and removing them answers a question about a café without regulars. The rules below find all three alike; only looking at the case tells them apart.

A nurse’s patient with a 90-day stay on a ward where stays run five days, a teacher’s pupil who scored 0 on a test the class scored 60 to 95 on, a marketer’s session with ten thousand page views: each is a case far from its group on the dependent variable, and each could be an error, a different question or a real extreme. Whatever the field, the check is the same – find it, look at it, decide by a rule, report the decision.

Finding one variable’s outliers

Inlier range is the pair of cut-offs a rule draws around the bulk of a variable, written [lower, upper]: every value inside is an inlier and every value outside is flagged. Three rules draw it, from three different centres and spreads, and they agree on a gross outlier and disagree at the margin. The app prints the pair beside each count so that a case filter can keep the inliers – Between lower and upper – or isolate the outliers.

Tukey’s fences are the box plot’s rule: 1.5 times the IQR below the first quartile and above the third, with a wider pair at 3 IQR for the extreme outliers. For the 185 waiting times Q1 is 3.0 and Q3 6.9, so the IQR is 3.9 and the upper fence 6.9 + 1.5 × 3.9 = 12.75: two waits are beyond it, 16.3 and 19.9, and the 19.9 is also beyond the extreme fence at 18.6. The fences are built from quartiles, which an outlier cannot move, so the rule sees the same cut-offs whether the outlier is 19.9 or 199; it is the rule to reach for first, on any shape of distribution. On the raw spend_after the upper fence is 19.64 and it flags two values, the 20.08 and the 66.53; on spend_before, 17.97, three – 18.03, 18.53 and 18.94, real spends by real members, which is the margin the fences are indifferent about.

Z-score rule flags a value whose z-score – its distance from the mean in standard deviations – exceeds 3 in either direction. It is the textbook rule and the weakest of the three, because the mean and the SD it divides by are the two numbers an outlier moves most. With the five 999s still in age, the column’s mean is 62.2 and its SD 150.7; each 999 has a z of 6.2 and is flagged, but a member whose age had been typed as 150 would have a z of 0.58 and pass unnoticed – and the oldest real member, 68, has a z of 0.04. The same rule on the fixed column (mean 38.2, SD 10.0) gives 150 a z of 11.1. The rule works only once the outliers it is meant to find are already gone.

Masking is that failure: one outlier hiding another by inflating the spread they are both judged against. It is why the 20.08 in spend_after reads as 1.84 SDs from the mean with the catering order in and 2.80 with it out – the same value, half again as far, once the SD is no longer stretched by a neighbour. Rules built on the median and the median absolute deviation do not mask, since a single value cannot move either.

Modified z-score is the z-score rebuilt on those two,

M=0.6745×x−medianMAD

flagged when |M| exceeds 3.5 – the 0.6745 scales the MAD to the SD of a normal distribution, so that 3.5 means what it would for a z. With the 999s in, age has a median of 36 and a MAD of 6, so an age of 150 has M = 12.8 and each 999 M = 108 – nothing is hidden. The app’s Modified Z outliers (|M| > 3.5) count in descriptive statistics is this rule; the Robust z method in Data transformation is the same score – it divides by 1.4826 × MAD, which is the MAD ⁄ 0.6745 – as a column to keep.

The three rules disagree exactly where a judgement is needed. On age with the sentinels declared missing, the fences (upper cut-off 63.5) flag 65, 67 and 68; the z-score rule flags nothing, since the largest z is 2.96; the modified z-score flags the 68 alone, at M = 3.60. All three members are real people in their late sixties. On visits_before the fences flag seven members (19 visits and up), the modified z-score five (22 and up) and the z-score three (24 and up); within each branch the fences flag Riverside’s 17, 17 and 19 and Station’s 22, 23, 24, 25 and 42. None of the counts is the truth; each is one rule’s answer, and the report names the rule.

More than one variable at once

Multivariate outlier is a case that is ordinary on each variable taken alone and impossible in combination. Member 179 spent 17.41 per visit before the menu change – high, but inside the fence, which is 17.92 on the 199 members with both spends – and 6.35 after, which is below average and unremarkable. Together the two numbers make the member the one case in 199 that the relationship between the columns cannot account for: spend_before and spend_after correlate at 0.69, and a member spending 17.41 before “should” spend about 16.37 after, ten units more than they did. No single-variable rule sees this; only a distance measured across both columns at once does.

Mahalanobis distance is that distance: how far a case sits from the centre of the cloud of all cases, in units that allow for each variable’s spread and for the correlations between them, so that moving along the cloud’s long axis counts for less than moving across it. Its square is compared with a χ² distribution on as many degrees of freedom as there are variables; the app flags a case beyond the 0.999 point – 13.8 for two variables – so that a clean sample of a thousand cases flags about one. Member 179’s squared distance is 19.1 on the classical version and 31.8 on the robust one, and it is flagged either way; the second-furthest member, at 8.8, is not. The classical distance has a small defect of its own: the centre and spread it measures from are the mean and covariance the outliers themselves have pulled – masking in more than one dimension – and the app’s classical run is judged against an exact bound (13.4 here) rather than the χ² approximation.

Minimum covariance determinant (MCD) is the robust way round that: the centre and spread are estimated from the tightest-fitting half of the cases, so an outlier has no hand in the yardstick it is measured with, and the χ² threshold applies as stated. The MCD spreads for the spend pair are 3.10 and 3.45 against the classical 3.57 and 3.70, and its correlation 0.76 against 0.69 – the cloud without its stragglers is tighter, and member 179 stands further from it. The estimate is found by random search, so it depends on the reproducibility seed; the app names the estimator it used on the card and falls back to the classical one when the sample is too small for the robust fit. Structural equation modeling and confirmatory factor analysis, reliability analysis and IRT all screen their cases this way before a fit.

Careless responding

A questionnaire has an outlier of its own: the respondent who did not read the questions. Each of their answers is one the scale allows, so no fence on a single item sees them; what gives them away is the pattern across items meant to measure the same thing. The indices below each look for one such pattern, and they are read together – the IRT module’s subject screen computes all of them beside the Mahalanobis distance. The numbers are the café’s six satisfaction items, 1 to 5, as the members answered them, with slow picked under the module’s Negatively keyed items, as on the latent variables page: longstring and IRV read the answers as given, since a run of identical answers is a pattern on the form, and the other indices read slow reversed, as the model does.

Careless responding is answering a survey without attending to its content – the same answer clicked down the page, answers at random, or anything between – and it is common wherever answering costs effort and earns nothing, as in most online surveys. Its signs are patterns: a run of identical answers, a spread too flat or too erratic, answers that disagree with themselves. None of them is proof, since a member satisfied with everything gives the same answer to every item honestly, so the indices flag and the analyst looks.

Longstring is the length of the longest run of identical answers to consecutive items – the straight-liner’s signature. On six items it runs from 1, every answer different from the one before, to 6, one answer throughout; the café’s typical member has a longest run of 2. The app flags a run beyond the upper Tukey fence of the sample’s own longstrings: the quartiles are 2 and 3, the fence 4.5, and seven members have a run of 5. Four of them broke it only at the last item – members 13 and 69 answered 4 to the five positively worded items and 2 to slow, 15 and 165 gave five 5s and a 1 – and here the reverse-keyed item speaks for them: each disagreed that “the service was slow” while agreeing with everything else, so each read it, and the flag is only the price of being satisfied. Two others carried their run into it: member 54 answered 5 to the first five items and 4 to slow, agreeing both that the order came quickly and that the service was slow, and member 161 gave five 4s and a 3 – the pattern of someone who stopped reading, and the two worth a look. The seventh, member 57, answered 3 from the second item on. A reverse-keyed item is what lets a longstring tell a careless respondent from a contented one, and it does so only on the answers as given: in a file with slow reversed, the four contented members would read as one answer throughout and the two who carried on as a run broken at slow.

Intra-individual response variability (IRV) is the standard deviation of a person’s own answers across the items: how far they move around the scale. The app divides it by the largest spread the answer scale allows, so it reads from 0 to 1 whatever the number of categories. A straight-liner’s is 0, and a very low one is the flat pattern that longstring misses when the repeated answers are not consecutive. The café’s quartiles are 0.41 and 0.67 and the lower fence 0.01, so only a 0 would be flagged, and nobody gave one answer to all six: the lowest is 0.19, members 46, 54 and 161, and the four whose run breaks at slow read 0.37 and 0.75 – the reverse-keyed answer at the far end of the scale spreads them. Random answering gives an IRV that is high, not low; IRV catches the flat pattern, and the erratic one is the business of the indices below.

Person-total correlation is the correlation, across the items, between a person’s answers and the average answers of everyone else: whether they find the same items easy to agree with as the sample does. An attentive respondent’s is positive, a random one’s near zero, and one that runs against everyone’s negative. It needs a pattern in the item means to follow, and the café’s six items barely have one – their means run from 3.34 to 3.79 – so the correlations scatter: the median is 0.22, the quartiles −0.23 and 0.50, and the lower fence falls at −1.31, below the lowest value a correlation can take, so nobody is flagged. A straight-liner has no correlation at all – an answer that never varies correlates with nothing – which is why the flat patterns are left to longstring and IRV.

Resampled individual reliability is a person’s consistency with themselves: the items are split at random into two halves, the person’s answers on one half correlated with their answers on the other and corrected to full length as split-half reliability is, and the result averaged over 30 splits. A respondent who reads the items gives similar answers to similar items and scores high; a random one scores near zero. It needs at least three items a half, and the café’s six are that floor: a correlation over three pairs of answers is mostly noise, the café’s median is −0.14, and the lower fence at −0.90 sits below the lowest member’s −0.65, so nobody is flagged. The index earns its place on a long survey; on six items, like the person-total correlation, it has too wide a spread for any fence to reach.

Two flags are not always two pieces of evidence. The app counts the members who trip two or more as the ones worth reviewing, and on the answers as given the café has none: longstring’s seven are not flat enough for IRV. Had slow been reversed in the file, the four contented members would have tripped both – longstring and IRV, two views of one pattern, and a false alarm the form itself had already answered. The four the Mahalanobis distance flags are four others, whose squared distances on the MCD estimate, 22.6 to 29.6, pass the χ² bar for six items at 22.5: patterns that cut across the items’ correlations, such as member 145’s 5 for taste and 2 for freshness, or member 168’s 3, 3 and 5 on the food items and 1 for friendly and quick beside a 2 for slow – the lowest marks for the service, and a denial that it was slow. Only looking at the answers says which are careless and which are honest people with unusual views.

Outliers in a model

A model – a regression line, a set of group means – adds a second kind of unusual. A case can be unusual in its outcome, sitting far from where the model puts it; or unusual in its predictors, sitting far from the other cases along the x-axis; and the two together decide how much the case pulls the fit. The café’s line of spend_after on spend_before shows all three, first with the catering order in (200 members) and then with it blanked (199). The gap between a case and the line is its residual; everything below is built from that gap and the case’s position.

Studentized residual is a residual divided by an estimate of its own standard deviation – so that residuals can be compared across cases, and to a t distribution – with the case itself left out of that estimate, so that a large residual cannot inflate the yardstick it is measured against. The distinction matters exactly when it matters: with the catering order in, member 136’s raw residual is 47.63, the model’s residual SD is 4.37, and the residual studentized with the member in is 11.1; studentized with the member out, where the residual SD falls to 2.68, it is 18.2. A rule of thumb flags |t| > 3; the app’s Outlier test does better and refers the largest studentized residual to a t distribution with its p-value multiplied by n – a Bonferroni correction for having looked at every case to find the worst one – so that the flag rate stays at the significance level whatever the sample size, where a fixed cut-off flags a fixed fraction of every clean sample. Member 136’s adjusted p is 10⁻⁴¹. With the order blanked, the largest studentized residual belongs to member 179 at −3.80, adjusted p = .039 – the multivariate outlier again, now in the model’s terms. The comparison assumption card runs the same test on the residuals from the group means: on spend_after by branch it reports Station’s 99 members with one flagged, at a studentized residual of 15.4 and p < .001.

Leverage is how unusual a case is in its predictors, on a 0–1 scale that sums to the number of coefficients: the further a case sits from the centre of the x-values, the more the line must pass near it. The app flags leverage above 2p/n – 0.02 for two coefficients and 200 members. Member 133, with the highest spend_before in the file at 18.94, has the highest leverage – 0.045 with the catering order in, 0.047 without – and a studentized residual of −0.40: the case pulls the line towards itself, and the line was already going there. High leverage with a small residual is a well-placed case, not a problem, and the flag count – 17 members above 0.02 – is a list of where the x-axis is thin, not a list of faults.

Influence is what the two together produce: how much the fitted model changes when the case is removed. The slope of spend_after on spend_before is 0.90 with member 136 in and 0.72 without – one row moves the coefficient by a quarter of itself, and the R² from .36 to .48. Member 133, for all its leverage, moves nothing.

Influential points are the cases whose removal changes the conclusion – the combination the diagnostics below are built to find. The signature is a large residual and high leverage; either alone rarely does much, which is why an outlier in the outcome at a typical x-value (member 179, leverage 0.035) counts for less than the same residual out at the edge would.

Cook’s distance is the standard single measure of influence: how far every fitted value moves when the case is dropped, scaled by the model’s error variance. The textbook cut-off is 1; the app also prints a size-aware one, 4/(n−p) – 0.020 here – because in a large sample almost nothing reaches 1. Member 136 has a Cook’s distance of 2.71, the only case above 1 and one of two above 0.020. With the order blanked, the largest is member 179’s 0.25 – below 1, well above 0.020 – and twelve members cross the size-aware line where two did before: with the outlier in, its residual inflated the error variance that everyone else’s distance is scaled by, and hid them. Masking, in a model.

DFFITS is the change in a case’s own fitted value when the case is dropped, in standard-error units; the flag is 2p/n, 0.20 here. Member 136’s is 3.80, member 179’s −0.73 – the fitted value the model gives them moves by nearly four and nearly one standard error respectively once they no longer pull on it.

DFBETAS is the same idea for each coefficient in turn: how many standard errors the coefficient moves when the case is dropped, one number per coefficient per case, with 2/n (0.14) as the usual flag. Dropping member 136 moves the intercept by −2.79 standard errors and the slope by +3.56 – the case is mostly pulling the slope – where dropping member 179 moves them by 0.52 and −0.67. The regression module summarises influence with Cook’s distance and DFFITS; meta-analysis and time-to-event analysis print DFBETAS per case, since there the question is usually which estimate a study or a subject moves.

COVRATIO asks a different question: not whether the case moves the coefficients but whether it changes their precision. It is the ratio of the coefficients’ covariance without the case to their covariance with it, so a value near 1 means the case is neutral; the app’s band is 1 ± 3p/n, 0.97 to 1.03 here. Member 136’s COVRATIO is 0.147 – removing the case shrinks the coefficients’ uncertainty to a seventh, because its residual had inflated the error variance – and member 133’s is 1.05: the high-leverage case at the far end of the x-axis was tightening the slope, and removing it loosens the estimate. A COVRATIO far below 1 is the mark of an outlying outcome, far above 1 the mark of a lone case holding down the end of the line.

What to do with one

The rules find the case; none of them says what to do with it, and the answer depends on which of the three kinds it is.

Look at it first. Read the row. The catering order was found by the maximum of a column, and one look at the member’s four visits and the amount said what it was; the 999s were found the same way. A case that is an error is made missing, not corrected by guesswork – unless the source of the data can say what the value was.

Decide by a rule you can state, not by the result you want. “An after-window average above 50 is not a per-visit spend” is a rule; “the case that made p go over .05” is not. The rule is written before the test is run, stated in the report, and applied to every case it covers, including the ones that help.

Fix the cell, not the row. A Range recode with its output set to missing in Data transformation blanks the one value and keeps the rest of the member’s row, so that every analysis not on that column still has 200 cases; a case filter removes a good row to fix one cell. The Outliers methods under Standardize do the same by rule – Flag marks the cases in a new column, Trim blanks them, Winsorize pulls them in to the cut-off – with cut-offs in percentiles, SDs or MADs, of which the MAD is the one the outliers cannot move.

Or keep the case and change the method. A method that works on ranks or trims the tails is indifferent to how far out an outlier lies. The Mann–Whitney test of spend_after by branch gives p = .002 whether the catering order is in or out; Spearman’s rho between before and after is .68 either way, where Pearson’s r is .60 with the order in and .69 without; the 20% trimmed mean of the before-to-after change is 1.02 and 1.00. In a model, robust regression down-weights the case by how far it disagrees and reports the weight, which is a more defensible act than deleting the row. The price of every such method is a little power on clean data; the gain is that the conclusion no longer rests on one member.

Report the decision, and what it changed. A reader should be able to see the case, the rule and the difference it made.

Sensitivity analysis is running the analysis both ways – with the case and without, or with the rule and without – and reporting whether the conclusion survives. The paired change in spend is +1.32 per visit (SD 4.37), t(199) = 4.28, p < .001 with the catering order in, and +1.09 (SD 2.86), t(198) = 5.37, p < .001 without: the mean and the spread both move, the conclusion does not, and the report says so in one sentence. The branch difference in spend_after is 2.19 with the order in and 1.63 without, p = .005 and p = .002 by Welch’s test. When the two runs disagree, that is the finding: the result depends on one case, and the report has to say which and why the case was kept or not.

Missing data

Missing value is a cell with nothing in it – an unanswered question, a measurement not taken, a value declared not to count. The app reads an empty cell as missing and every other cell as a value, whatever the value means: to it the café’s 999s were five members aged 999 until a rule said otherwise. After the three fixes the file has 28 blank cells in 28 rows – 7 in overall, 15 in wait, 5 in age and the 1 in spend_after – and no member is missing more than one. A missing value is not zero and is never averaged in as one; it is a case the column does not have, and the first thing it changes is the sample size.

Missing is the count of cells in the column that hold no value, with its percentage of the rows beside it where the table gives one. Read it with N: the two sum to the number of rows, and a count that is a large share of the rows is a warning about every other number on the row, computed as they were from the cases that remain.

Missing data is the condition of a dataset with such cells, and the name of the setting that decides what every analysis does about them – pairwise deletion, listwise deletion or imputation – for the whole session at once. The choice changes the n under every result and, with imputation, the results themselves, so the report names it.

Missing codes are the values a form or an export uses to mean missing – 999, −1, N/A, a blank string – which the app cannot know are codes. Declare missing codes in Data transformation turns each into a real blank, so that it stops being averaged, correlated and plotted as a value; the café guide walks through the 999s. A codebook that names its codes is the reason to read one before anything else.

Complete cases are the rows with a value on every variable an analysis uses. Of the café’s 200 members, 172 are complete on all fifteen columns; 193 have overall, 185 have wait, and 178 have both. Every analysis runs on complete cases – the question is only complete on which variables, and that is what the two deletion rules decide.

Pairwise deletion uses, for each statistic, the rows complete on the variables that statistic needs: the mean of overall from its 193, the mean of wait from its 185, and their correlation from the 178 that have both. It keeps the most data, and it is the app’s default. Its cost is that one table can carry several sample sizes – the cells of a correlation matrix each on their own n – and that a matrix assembled from different subsets can describe no sample that actually exists, which is why the analyses that fit a whole matrix at once (factor analysis, cluster analysis) refuse it and use listwise deletion instead, and why reliability analysis, which accepts it, computes its scale-level statistics on the complete cases alone.

Listwise deletion uses only the rows complete on every selected variable, for everything: select all fifteen café columns and every statistic runs on the same 172 members, whether it needs one column or all of them. Every number in the output then describes one sample, which is its virtue; its cost is that one sparse variable in the selection – an optional question, a measure taken on a subset – throws away every row that skipped it, and the app warns when the pass removes half the cases or more. The remedy is usually to deselect the sparse variable, not to switch rules.

Whether deletion biases a result – rather than just shrinking the sample – depends on why the cells are blank, and there are three answers.

Missing completely at random (MCAR) means the blanks fell without regard to anything – a page skipped, a sample tube dropped. Deleting the incomplete rows then loses precision and nothing else: the complete cases are a random sample of the whole. The café’s blanks in wait and overall are MCAR by construction, since the file’s spec drew them at random. And yet the fifteen members who skipped the wait question spent 7.21 per visit before the change against 8.83 for the rest, p = .049 by Welch’s test. That is chance – it must be, because we know how the file was made – and it is the lesson: a small set of blanks can look related to something, and the data alone can never prove that missingness is random. The claim rests on knowing how the blanks arose.

Missing at random (MAR) means the chance of a blank depends on things that are in the data – older members skip the online survey more, one branch’s till drops more records – but not, given those, on the missing value itself. Deletion is then mildly biased, since the complete cases over-represent whoever answers; a method that uses the observed columns to allow for it – FIML below, or multiple imputation – is unbiased.

Missing not at random (MNAR) means the blank depends on the value that is missing: the members who waited longest are the ones who did not answer the wait question, the least satisfied did not return the survey. No method recovers this from the data, because the information is in the cells that are not there; the honest report says the mechanism is plausible and gives the result as a bound. The café’s waits do not have this problem – but if the 15 blanks had been the 15 longest waits, the median wait of 4.9 minutes would be reading a café that does not exist.

Imputation replaces a blank with a value so that the row can be used whole. The setting fills every blank in a column with one number – its mean, median or mode, or a constant – and the app then runs as if nothing had been missing. What that costs is visible on wait: filling its 15 blanks with the mean of 5.20 leaves the mean at 5.20, by construction, and shrinks the SD from 2.91 to 2.80 – fifteen values sitting exactly on the centre spread nothing – so the standard error of the mean falls from 0.214 to 0.198 on a precision the data did not earn. The correlation between overall and wait goes from −.30 on 178 members to −.29 on 193, pulled toward zero by fifteen points with no relationship in them. And the filled column has a spike: the 4–6 minute bin of its histogram holds 69 members where the real data has 54, fifteen of them identical. Every single-value fill does this; the median does it to the same degree, and a constant does it to a value of your choosing. It is the right tool when a later step needs a complete column – a scale score, an export, a lagged series – and the wrong tool for the analysis itself, where deletion at least tells the truth about n.

Imputation method is the choice among those fills: Mean and Median for a numeric column, Mode for any column, Constant for a value typed in. Fill missing cells in Data transformation offers the same and more – a fill computed within each group of a second variable (Riverside’s mean wait 4.84 and Station’s 5.57, rather than 5.20 for both), or the nearest earlier or later value down an ordered column – and writes the result into the data as a rule, so that the report can say exactly what was filled.

Multiple imputation fills each blank not once but many times, each time with a value drawn to reflect how uncertain the fill is, runs the analysis on every filled copy and pools the answers – so that the standard errors carry the uncertainty a single fill hides. It is the method of choice for MAR data in most fields, and the app does not offer it: its imputation is single-value, and this page is where that is said plainly.

FIML – full information maximum likelihood – is the alternative the model-fitting modules have. Rather than filling or deleting, the model is fitted to each row using the cells it has, so that a member with fourteen of fifteen columns contributes fourteen columns’ worth of information. Under MAR it is unbiased and it loses no cases; structural equation modeling and confirmatory factor analysis offer it in place of listwise deletion, and the model summary names the treatment that actually ran.

Reporting

Name every rule, and the count it produced. For outliers: the rule (1.5·IQR, modified z with |M| > 3.5, Cook’s distance above 4/(n−p), Mahalanobis distance at p < .001 on the MCD estimate), the cases it flagged, what was done with each and why, and whether the conclusion changed when they were left in – “one after-window spend was recoded to missing as a non-per-visit amount; the paired difference was +1.09 (SD 2.86) without it and +1.32 (SD 4.37) with it, significant either way”. For missing data: how many cells per variable, the deletion rule or the fill, the resulting n under each result, and what is known about why the cells are blank. An n that differs from the number of rows is never a detail to leave for the reader to notice.

Where this appears in DataSuite