Everyone should consult a statistician at the research planning stage.

Why?

Sample size and statistical analysis are hugely important considerations if your research is to produce conclusions that are reliable, and also to recruit in a timely manner. You need to calculate the chances of coming to a wrong positive conclusion, and a wrong negative conclusion (not the same). These are known as the statistical significance level and the false-negative rate. Typically projects are designed to give a 5% chance of making a wrong positive conclusion. A project with a 20% chance of making a wrong negative conclusion has a power of 80%, which is often acceptable.

Advice from the RNOH statistical consultants

The statistical consultants used by RNOH, who also provide workshops (see information at the bottom of the page) have kindly provided a statistical advice and checklist explaining some important aspects of statistics for clinical research. The advice includes a checklist of considerations and a table for guidance on what statistics tests to use when.

Sample size and ethics

It may not be immediately apparent that neglecting careful statistical consideration could be unethical, but possible knock-on effects of poor planning are described below.

The following passages of text have been taken from online medical statistics pages adapted by Ed Juszczak, based on an idea by Mike Bradburn and Sharon Love on the Oxford Radcliffe NHS Trust website.

When planning a clinical trial, it is very important to consider how many participants you will need to reliably answer the clinical question. Too many participants is a needless waste of resources (and possibly lives), which could result in a beneficial treatment being denied to patients unnecessarily. Too few participants will not produce a precise, reliable and definitive answer, which can also be considered unethical. Under this common scenario, patients might be denied a useful treatment because the trials were frequently underpowered (i.e. too small to detect a treatment effect) - this can also result in further studies being cancelled without good reason. Choosing a sample size is a combination of logistical and pragmatic considerations. These include (a) the number of participants you can reasonably expect to recruit in the given time period within available resources, and (b) mathematical calculation. The final figure calculated indicates the minimum number of participants required to reliably answer the research question.

A tip from the RNOH statistical consultant:
"Whenever a sample size is calculated, it has to be remembered that the value will probably be an underestimate due to non-response, patient withdrawals, etc. The calculated sample size should thus be appropriately inflated to allow for this."

Points to remember
Sample size increases:

• When a small treatment difference is expected rather than a large one, for example, when comparing two active treatments rather than an active treatment versus placebo
• With higher power - the higher the power, the more likely, on average, you are to detect a treatment effect if it exists, for example, more participants will be needed for a trial with 90% power rather than 80%
• With a lower significance level - the lower the significance level, for example, 1% (or a=0.01) rather than the typical 5%, the less likely you are to get a chance (but spurious) treatment effect i.e. a false-positive result
• When measurements are highly variable - natural variability can be thought of as noise and makes the signal more difficult to hear, for example, measurements such as blood pressure and peak flow are highly variable. A simple way to alleviate this problem is to take a few repeated measurements and use the average (or the maximum for the latter)

Statistics consultancy workshops

Statistics consultancy workshops at the RNOH can be booked through Anita on 5425 or email anita.ownsworth@rnoh.nhs.uk.

Information from the consultancy service

The statistical consultancy service at the RNOH is a chance to get some input to study design and/or analysis to improve the quality and reliability of your research. Appointments are half an hour or one hour long; please ensure you book enough time and arrive punctually.

We encourage everyone to consult a statistician at the research planning stage. You will be advised on a design which enables you to answer a scientific question reliably. Number of patients required, organisation of data for statistical analysis and analysis plans can be advised.

If it is already too late to take advice on study design and planning then we can still advise on the analysis. Stats clinic is a consultancy service. You will be advised on how to do the analysis, so please do not arrive with a set of data and expect the answers to come out of a statistics black box!

Come along expecting to have your analysis plan changed and scientific question altered. This will often take the form of simplification such as collecting less information on more of patients. You cannot determine causes of a condition from a handful of patients whose data were collected routinely. Except in unusual cases you will need statistical software to analyse your data. The RNOH and UCL both endorse the use of SPSS (Statistical Package for Social Sciences), a package which is trusted by medical journals to provide reliable output.

If you would like to do better research, a recommended text is:
Petrie and Sabin (2000) Medical Statistics at a Glance, Blackwell Science. Topics cover basic statistics in detail and give an indication of the underlying concepts of, and methods used for, more extensive analyses. Each topic is displayed on a two- or three-page spread and includes examples in medicine or dentistry. There is an accompanying website (www.medstatsaag.com).

Other reading material is also suggested at the bottom of these online medical statistics pages adapted by Ed Juszczak, based on an idea by Mike Bradburn and Sharon Love on the Oxford Radcliffe NHS Trust website.