Dr. Mark Gardener 


Providing training for:

Statistics – A guideThese pages are aimed at helping you learn about statistics. Why you need them, what they can do for you, which routines are suitable for your purposes and how to carry out a range of statistical analyses. On this page:
<= Back to Summarizing Data Introduction  Forward to Next Topic=> 

See also: 


Plan your statistical approach before you collect any data Planning: Saves time 
Choosing the right statistical analysisThere are many different types of statistical analysis. Choosing the correct analytical approach for your situation can be a daunting process. In this section you'll get an overview of the statistical procedures that are potentially available and under what circumstances they are used. You should plan your statistical approach at the start of your project, before you collect any data. Different statistical tests have different requirements and planning in advance has various benefits:
If you simply collect data and then look for a way to carry out an analysis you may find that you do not have quite what you need to answer your research question. Knowing what type of project you have and what sort of data you will collect can be useful in determining the best analytical approach. Read notes on types of project, types of data and recording your data next or jump direct to the key. The key should allow you to work out what statistical test is most appropriate for your project. At some future point I'll add notes about how to carry out some of the analyses. Types of Project  Types of Data  Recording data  Key to statistics 

Types of project: Differences Some disciplines have special type of project, e.g. in ecology: Population 
Types of projectKnowing what kind of project you are undertaking can be a big help in working your way towards the most appropriate statistical approach. The following list covers a range of possibilities:
There are some miscellaneous analyses concerned with properties of the data and statistical tests:
There are also various miscellaneous categories that crop up in certain disciplines. In ecology for example:
Most projects will fit into one (or more) of the preceding types. In the key you will be able to choose the most appropriate type for your situation and work your way towards an analytical approach (or perhaps several). 

Two sorts of variable: Response (dependent) Three kinds of data: Interval The kind of data affects the statistical approach 
Types of dataThere are two main sorts of data variable you will collect:
The response and predictor variables that you measure and record come in different forms. The form of data will affect the kinds of statistical approach you take.
The kind of data you collect affects the statistical approach. You may decide to use ordinal measurements to save time for example, but this will limit the kinds of analysis you will be able to conduct subsequently. You'll be able to see how the kind of data affects your analytical approach in the key. 

Use recording layout to maximize usefulness and effectiveness Each column represents a single variable (response and predictors) Each row represents a single record, that is the observations or replicates See Scientific Recording Format article in Writer's Bloc 
Recording dataHow you record your data is important. If your data are written down in a sensible arrangement you can make sense of them more easily and carry out any statistical analysis more easily and effectively. Having a good data recording system is an important aspect of any project. In general you want to use a scientific recording layout for storing your data. In this format you have a column for each variable. Each row represents a single observation (replicate). For example; here is a simple dataset recorded as two samples. The data show the length of jawbones (in mm) of golden jackal from male and female specimens.
In scientific recording format you would represent the data like so (only part of the dataset is shown):
The first column is the response variable, the length of the jawbone, which you think is affected by the predictor variable. The second column shows the predictor variable (sex), which shows two levels, male and female. Having this layout makes it easier for most statistical programs to deal with the data. It also helps you to manage your data. You can use pivot tables to rearrange the data and help you explore the dataset. See more about scientific recording format in the Writer's Bloc section. 

Main KEY to statistical tests 
Key to statistical analysisStart by selecting the most appropriate type of project or analysis from the list in the first column of the table below. The second column contains notes about options for the various project types to guide your decision. Use the hyperlinks to jump to sections containing new choices; keep going until you end up with an appropriate type of analysis. You may find that your project falls into more than one type or that you end up with several options at the end. You can use the back button on your browser to backup and hit the key link to return to the start point at any time. I am an ecologist so my examples will generally lie in that area. This is a work in progress; there are many omissions and alterations are inevitable, please be patient... Start Here
Identify the most appropriate starting point from the preceding table. Click the links to go to the relevant sections. You can return to the start using the key link. In future there will be links to pages giving more details about how to carry out some of the statistical tests, with examples and so on. Please be patient... 

Descriptive: Summarizing Samples 
DescriptiveThere are two sorts of descriptive project to consider:
You should always summarize your data. At least you should know what data distribution you are dealing with, as this will affect the kind of analytical approach you can take. Other kinds of descriptive project will be specific to your discipline. In ecology for example you may classify a community of plants or describe the "features" of a species. See Community classification for more details. 

Data distribution: Testing for normal distribution 
Data distributionThe shape of the data affects the kind of analyses you can carry out. The most common test is to compare to the normal distribution but you can also compare a distribution to another known distribution or to another sample. There are various choices:
In some kinds of statistical testing it is not the actual data that need to be normally distributed but the residuals (in ANOVA, Correlations and Regressions that use the properties of the normal distribution for example). You can check these residuals after conducting the test. If the residuals are nonparametric then you should attempt to transform data to improve the situation or find an alternative. 

Assumptions: Homogeneity of variance 
AssumptionsMost statistical analyses make assumptions about the data. Usually the shape of the data is the most important, and comparing the data distribution is an important element of the analysis. Other assumptions are concerned with measures of dispersion. The options depend on the distribution of the data:


Differences projects split data into sampling groups 
Differences between samplesFeatures of differences projects:
Now select the option that most corresponds to what you want to do.
Choose the most appropriate option from the preceding table. Click the links to go to the relevant sections. You can return to the start using the key link. 

You can compare a single numerical sample to a fixed value 
Onesample differencesIf you have a single sample of numerical values you can compare the average (mean or median) to a fixed value. Your data can be interval or ordinal but not categorical.
There are two main choices, which depend on the shape of the data (the data distribution). If you have ordinal data or the data are not normally distributed then you will use the 1sample Wilcoxon rank sum test. If you have normally distributed data you can use the onesample ttest. You can also use a Permutation test, which involves repeatedly resampling your dataset. 

1sample Wilcoxon rank sum test compares the median of one sample to a fixed value  Onesample Wilcoxon sign rank test for nonparametric dataIf your data are ordinal or are nonparametric (i.e. the data do not have a gaussian distribution) you can use a onesample Wilcoxon rank sum test. Essentially you "convert" the data sample values into ranks (from smallest to largest). The median value is then compared to a fixed value, which you specify, using the test. 

1sample ttest compares the mean of one sample to a fixed value  One sample ttest for parametric dataIf your data are normally distributed (they must be interval level data) you can use the onesample ttest to compare the mean of your sample to a fixed value. 

Two samples can be compared using the Utest or the ttest, depending on data distribution 
Twosamples differencesIf you have two samples of numerical values you can compare the average (mean or median) of the two samples. Your data can be interval or ordinal but not categorical. You can only have a single response variable and a single predictor variable. When you have two samples you effectively have a single predictor variable.
There are two main choices, which depend on the shape of the data (the data distribution). If you have ordinal data or the data are not normally distributed then you will use the Wilcoxon rank test (also called a Utest). If you have normally distributed data you can use the Student's ttest. There are versions of both tests for the situation where the samples are not independent but are in matched pairs. You can also use a Permutation test, which involves repeatedly resampling your dataset. 

The U test compares medians between two nonparametric samples Wilcoxon sign rank test is used if samples are in matched pairs 
Twosample Wilcoxon rank test (U test) for nonparametric dataWhen you have two samples that are ordinal or otherwise nonparametric you can compare their medians using a Utest. The original values are converted to ranks and these ranks are compared to assess the degree of overlap between the two samples. There are two main options for the Utest:
If the samples are independent you use the regular Utest. If samples are in matched pairs you can use a version called Wilcoxon sign rank test. Although the tests are independent of the shape of the data. The two samples should have a similar shape (see Data distribution and Assumptions). If you have multiple samples you might use several pairwise Utests, getting a pvalue for each one. If so then you should modify the pvalues to take into account the multiple tests. However, it may be better to use a multiple sample approach. 

The ttest compares means between two parametric samples Versions for: Matched pairs 
Two sample ttest (Student's ttest) for parametric dataIf your data are normally distributed (usually this will be interval data) you can use the properties of the normal distribution to assess the difference in means between the samples. Student's ttest is the tool for the job. There are three main versions of the twosample ttest:
The "classic" ttest assumes equal variance for both samples. If the variance is unequal the Welch twosample ttest should be used; this adjusts the degrees of freedom to compensate. There are tests for equality of variance (see Assumptions). If you have multiple samples you might use several pairwise ttests, getting a pvalue for each one. If so then you should modify the pvalues to take into account the multiple tests. However, it may be better to use a multiple sample approach. 

Matched pairs: Observations are not independent, one observation has a corresponding measurement in the other sample 
Matched pairsIf you have two samples that are "matched", you can use special versions of the ttest and the Utest. The test you use depends on the shape of the data. In matched pairs tests a single observation from one sample can be matched to a corresponding observation from the other sample. The samples are therefore not independent. Examples of matched pairs include:
If you can pick an observation from one sample and find a natural partner in the second observation, then you've got a matched pair design. Just because you collected an observation from a quadrat and labelled it "1" does not mean it matches with a random observation in another site that you also labelled "1". If you do have matched pairs the ttest is used for parametric data and the Utest for nonparametric data. See Twosample tests for more information. You can also use a Permutation test, which involves repeatedly resampling your dataset. 

If you carry out multiple tests you need to modify the pvalues to take the mulitiplicity into account 
Adjustments for multiple testingIf you have carried out several statistical tests you will have several pvalues, one for each test. However, the more tests you run the greater the likelihood that something will be statistically significant. In this case you should modify your pvalues to take into account the multiple tests. If for example you had several samples you could use the ttest to explore differences between the samples pairbypair. You would end up with a pvalue for each pairwize comparison. What you should do is to modify the pvalues. The most conservative method is the Bonferroni correction. You multiply the pvalues by the number of tests you carried out. There are other less conservative methods (e.g. Holm). There are usually alternatives to multiple tests, for example instead of ttests use ANOVA. Instead of Utests use KruskalWallis. These approaches test the overall situation and permit posthoc testing of the pairwise comparisons with a less stiff penalty on the pvalues. 

Multiple sample testing. Method depends on shape of data and number of predictor variables For normal data use ANOVA and posthoc tests For nonparametric data use: KruskalWallis Permutation tests are becoming popular as they are virtually independent of data shape 
Multiple samples differencesIf you have more than two samples of numerical data your approach depends on the data distribution and the number of predictor variables:
When you have more than two samples your options depend largely on the shape of your data. If you have normally distributed data (parametric) then analysis of variance is the way to go. You can carry out ANOVA when you have more than one predictor variable. regular ANOVA is sometimes called 1way ANOVA to indicate that you have a single predictor. If you have 2 predictors then you use 2way ANOVA and so on. Once you've carried out ANOVA you can use posthoc testing to "drill down" into specific pairs of samples. If your data are nonparametric your options are limited. The nonparametric equivalent to 1way ANOVA is the KruskalWallis test. Once you have an overall result you can carry out posthoc testing on pairs of samples. If you have two predictors then the Friedman and Quade tests can work, but only for unreplicated block designs. Posthoc testing using these tests is not available. If you have more than two predictors, or do not have the unreplicated block design, your options are limited. You may be able to transform data to make it more normal (using logarithms or some other transformation). You could convert all the values to their ranks and carry out regular ANOVA; this is a very conservative test. Permutation tests are becoming more popular as they are virtually independent of the shape of the data. However, they can be "computer intensive" as you need to randomly resample your data many times. 

Analysis of Variance ANOVA used when data are normally distributed MANOVA can be used with multiple response variables Posthoc testing takes place after the main ANOVA and conducts pairwise analyses 
Analysis of Variance – ANOVAAnalysis of variance is used when your data are normally distributed (but see Data distribution). The properties of the normal distribution are used to assess the differences between sample means. The simplest situation is where you have one response variable and one predictor variable with two levels, that is, you have two samples. In this case the result is equivalent to the ttest. Most often you use ANOVA when you have a predictor variable with more than 2 levels, i.e. you have >2 samples. When you have 1 predictor (regardless of how many levels) the analysis is usually called 1way ANOVA. If you have two predictor variables the process is known as 2way ANOVA, and so on. If you have more than one response variable there is a special version of ANOVA called MANOVA (multipleANOVA). Essentially this carries out ANOVA for each response variable, then adjusts the results to take into account the multiple tests. ANOVA is closely allied to linear modelling and in many cases the two methods are interchangeable. Usually your response variable is interval and the predictor variables are categorical when you run ANOVA. If you have a continuous variable as well, the approach is sometimes called analysis of covariance (ANCOVA). After the main "result" your ANOVA will show you the probability that means between samples were different. You can focus on specific pairs of samples using posthoc testing. Essentially posthoc tests are a modified version of the ttest, which take into account the fact that you have run multiple tests. 

The KruskalWallis Rank Sum test is the equivalent of 1way ANOVA for nonparametric data 
KruskalWallis Rank Sum testIf your data are not parametric and you have a single predictor variable you can use the KruskalWallis Rank Sum test. This is sometimes called a nonparametric 1way ANOVA. If your predictor variable has only two levels (i.e. you have 2 samples) use the Utest instead. Essentially the procedure converts the original values into ranks. The ranks are then assessed for the amount of overlap. You are comparing sample medians. After the main "result" your test will show you the probability that medians between samples were different. You can focus on specific pairs of samples using posthoc testing. Essentially posthoc tests are a modified version of the Utest, which take into account the fact that you have run multiple tests. 

For unreplicated block designs with nonparametric data: Friedman test 
Friedman and Quade testsWhen you have two predictor variables and the data are nonparametric, your options are limited. The Friedman and Quade tests compare medians for unreplicated block designs. In other words, you have only one value for each combination of predictor variables. The data should be arranged in groups and blocks. Each block will contain one observation from each group. The data are "converted" to ranks and these are compared across blocks. Differences between the groups are assessed. These methods do not have a sensible posthoc methodology. It is possible to look at pairwise comparisons but the power of the tests is so small that differentiation is all but impossible. 

Permutation tests: Resample the data many times and estimate pvalues from the results. Can be carried out for virtually any differences analysis 
Permutation testsPermutation tests are virtually independent of the shape of the data. You can carry out differences tests using permutation procedures for any of the differences scenarios. Sometimes tests are called bootstrapping, randomization or Monte Carlo. Essentially you take your samples and examine the differences in the average (you can use the mean or median, as seems most appropriate). Then you use permutation to shuffle the samples. You carry on many times (generally 1000 or so). After the permutations you end up with many "reruns" of the original result. You can get approximate pvalues by looking at how many times the permuted values were larger than the original. You can run permutation tests on virtually any data that you would explore for differences in sample average. Such tests are becoming more common as computers become more powerful. 

Compare proportions: Proportion test These use count (frequency) data so are allied more to goodness of fit tests than differences 
ProportionsIf you have proportions you really have count data, that is you have a count of "success" and number of "trials". Proportion data fall under the categorical data heading, tests of proportions are more similar to tests of goodness of fit, rather than differences. There are two main options:
See also the Links section (Association tests). 

Power tests allow you to determine discriminatory power of your tests under different conditions 
Discriminatory powerPower tests are designed to help you plan your data collection. The most common use for power tests is to determine the sample size(s) required to achieve a certain level of discriminatory power. There are power tests for the following:


Links between variables or samples: Correlation 
Links between variablesWhen you are looking at links you are not looking for differences between sampling units but are generally interested in finding links between variables. There are several approaches:
There is some crossover between approaches – I will keep correlation, regression and association in the Links heading and deal with classification and patterns separately. In the following table you can see some examples of project for the various approaches.
Use the links in the Type column of the preceding table to jump to the most appropriate section or follow links in the Notes column to jump direct to specific topics. 

Correlation assesses the strength of the link between two variables 
CorrelationIn correlation you are looking for the strength (and direction) of the link between two continuous variables (but see Mantel tests). The variables can be interval or ordinal. There are three main options:
Generally speaking you will have your data arranged so that each row is a single observation and each column is a variable. You can also perform a correlation between two matrix objects, as long they have the same dimensions. 

Parametric correlation Pearson's product moment 
Parametric correlationIf your data variables are normally distributed you can use parametric correlation. The Pearson Product Moment correlation uses the properties of the normal distribution to determine the relationship. This method is similar to linear modelling (regression) except that you only get the strength of the link between the variables. The coefficient varies between 1 and 1. The closer to unity the stronger the relationship. A positive correlation implies that as the value of one variable increases so does the other. Just because you have a statistically significant correlation does not mean that there is direct cause and effect. 

Nonparametric correlation: Spearman's Rho 
Nonparametric correlationIf your data variables are ordinal or otherwize nonparametric you cannot use the properties of the normal distribution. Methods of correlating variables in these instances convert the original data to ranks (i.e. a form of ordinal data) and assess the strength of the relationship using the "matchup" of ranks. There are two general correlation coefficients for nonparametric data:
The coefficient varies between 1 and 1. The closer to unity the stronger the relationship. A positive correlation implies that as the value of one variable increases so does the other. Just because you have a statistically significant correlation does not mean that there is direct cause and effect. Although the tests are independent of the shape of the data. The two samples should have a similar shape (see Data distribution and Assumptions). 

Matrix correlation: Mantel tests 
Correlation between matricesIf you have two matrix objects, with the same number of rows and columns, you can carry out correlation to compare their relationship using Mantel tests. You can think of the Mantel test as a multivariate correlation. The Mantel tests use regular correlation coefficients but applied to the matrices under test. So you can specify:
Mantel tests use permutation to determine the statistical significance of the relationship between the two matrices. 

Regression describes the links between variables as a mathematical model Linear modelling 
Regression and modellingThere are several forms of regression but you can think of it as an extension of correlation where not only do you look at the strength of the relationship between variables but also describe the mathematical properties of the link(s). There are forms of regression that can handle multiple predictor variables and data that are interval, ordinal or categorical. In the following table you'll see the various sorts of regression with some notes about when each can be used.
Use the preceding table to guide you to the most appropriate kind of regression for your situation. Use the hyperlinks in the Type column to go to the appropriate section. 

Linear regression (linear modelling) is closely allied to ANOVA 
Linear regression (multiple regression) and linear modellingLinear regression (called variously, multiple regression or linear modelling) is a method closely allied to ANOVA. There is a general expectation of a relationship between the variables in the form y = mx+c (linear) and with parametric residuals. Once you have carried out the regression you should check the normality of the residuals. Your response variables will usually be interval data but the predictor variables can be interval, ordinal or categorical. The "classic" form of multiple regression uses interval data (i.e. numbers on a continuous scale). When predictors are categorical the regression is most like ANOVA. When there is a mixture the method is sometimes called analysis of covariance (ANCOVA). You an also carry out linear modelling when you have several response variables. Linear modelling is often used to assess the relative strengths (importance) of the various predictor variables on the response. In modelbuilding you aim to add predictor variables one at a time, based on their importance, until you have incorporated only the statistically significant components. 

Generalized Linear Modelling is linear regression using alternative data distribution 
Generalized Linear Modelling (GLM)Generalized linear modelling (GLM) is very similar to linear regression. In GLM you use a modelfitting approach but the data do not have to be normally distributed. The general relationship between variables is still assumed to be y = mx+c but a range of data distribution types can be assessed; these include: poisson and gamma. If you use a gaussian distribution this is identical to linear regression. You can also use a binomial response variable with GLM. This tends to be known as Logistic regression. 

Logistic regression is a binomial GLM 
Logistic regressionIf your response variable can take one of two forms you have binomial data (e.g. presence or absence, 0 or 1). In this case you use GLM, which is known as logistic regression. Your predictor variables can be interval, ordinal or categorical. Essentially you are assessing how likely it is that given values of your predictors will result in the response being one form or the other. 

Nonlinear modelling is used when y = mx+c is not the expected relationship 
Nonlinear modellingIn nonlinear model fitting there is not an expectation of a y = mx+c relationship. The method essentially reshuffles parameters and continuously refits the model until the "best" set of parameters are achieved. 

Association tests use categorical (count or frequency) data 
AssociationIn association tests you have categorical data, that is count or frequency data. There are several options:


Patterns and classificationRuns tests, nearest neighbour, clustering, kmeans, ordination. 

OrdinationIn ordination you generally have many response variables. In ecology for example this is usually abundance of species. There is some crossover with methods of Patterns and Classification. 

Timerelated dataThere are several approaches when you have timerelated data.


PopulationIf you have populations from two samples (or more) or from the same population at 2 (or more) times, you can proceed as if you were comparing samples. 

Community classificationNVC, Ellenberg and miscellaneous methods. 

DiversityYou can compare diversity from one or more samples using a range of approaches. 

This page is still under construction... please come back soon.


Navigate:  <= Summarizing Data Introduction  Next Topic=> Excel Tips & Tricks  Tips & Tricks for R  Learn R  MonogRaphs 

See my Publications about statistics and data analysis. Courses in data analysis, data management and statistics. 


Follow me... 

Top  DataAnalytics Home  Contact  GardenersOwn Homepage 