Where student research fails
Prof. Ronán Conroy offers 11 tips for student research projects.
Over the years, when reviewing student research proposals, I have come to realise that there are a number of key features that distinguish proposals that are likely to result in a useful, publishable piece of research from those that are likely to founder, or not to be of publishable quality. There were originally ten tips, but I am indebted to my colleagues, especially those on the Research Ethics Committee, for adding their wisdom to the
original ten and contributing an eleventh.
1. Know the current state of knowledge
Research involves adding to knowledge. In order to do this, you must know the current state of knowledge, the current theoretical approaches and current best practice in terms of measurement.
Many student research proposals show an inadequate grasp of the literature, or failure to identify appropriate validated methods. Hence, they risk either answering a question that is already answered, or carrying out research whose findings cannot be integrated with current knowledge. If you can show how your research question arises from the current state of knowledge, then your research holds out the promise of some scientific value.
2. Be able to state a single, clear research question that will add to our current knowledge, and make this the title of your research
Your research question should be clearly linked to a gap in our current knowledge. You should be able to state the research question clearly, in simple language, and in one sentence. If you can do this, you will be able to work out what data are needed to answer the question, and to identify a suitable study methodology to gather these data.
Without that initial step in place, however, there is no way of deciding on an appropriate study methodology. It is worth spending time trying to phrase the question exactly right. It is the single most important step in your research. When you come up with the exact question you want to ask, make that the title of your research project. Remember that a research question is a question. ‘I’m interested in patient litigation’ is not a question, nor is ‘We have data on 120 patients on our deliberate self-harm register’.
Tip: Write the introduction section to your paper before you finalise the methodology. This should have three sections: what we already know, what we don’t know, and what you have decided to do. If these are clear in your head – and properly referenced – all is well.
3. Have one research question as your study aim
A student study should ask one question and answer it well. If you have several research questions, try to identify the most basic question and to research it first. Studies with multiple research questions pose considerable problems through the sheer volume of data they collect and the complexity of writing them up.
4. Translate your research aim into between three and five objectives
The objectives of your study specify what you will have to achieve in order to answer your research question. Having fewer than three objectives suggests that your study is too narrow and more than five strongly suggests that it is too complex. Both are warning signs that you may have to think again about what you are proposing. There may be studies that break this rule, but there should be a very good reason for breaking it.
5. Adopt/adapt methodology – do not invent it
Try to adopt or adapt methodology used in other studies. Avoid any research that involves developing an untried methodology. Failure to use already established methods makes it harder to integrate your study findings into the literature. Do-it-yourself questionnaires and scales require validation, which adds greatly to the complexity of the research.
Tip: Authors usually answer emails and will often provide questionnaires and pdfs.
6. Gather only the necessary data
You should gather the minimum data needed to answer a single, clear research question. There is a tendency to gather data without a clear purpose in mind – especially demographic and background data. You should draft your results section before starting the study to ensure that you gather all the data you will need, but only the data you will need. The data must be gathered only at the level of detail required for your final report.
No one should start analysing data without a plan. You become overwhelmed by the thousands of results that a computer package is capable of generating. Read papers in your area and see how others have analysed and presented their results.
Tip: Write your results section before you finalise your methodology, making blank tables to hold your results. This will ensure that you know what data you will need, and the detail required.
7. Learn by replicating the work of others
If you are learning to cook, you begin by following a recipe. This gives you a framework to guide you, and often a picture of the intended outcome. Following a published methodology allows you to learn how to translate a protocol into an actual exercise in data gathering, analysis and writing up, and the published paper acts like the picture in the cookbook. Inevitably, just like cookery, the results are less perfect than you hoped. But this experience will build research skills while providing a framework to guide your study conduct, and a template for writing it up. Replication is important in extending research findings into new populations and settings, and overcoming weaknesses of the original design or conduct of research. In addition, researchers are generally willing to help students by providing protocols, questionnaires etc.
Even if you cannot find a study to replicate, try to assemble your protocol using methods taken from other studies. The more ‘pre-tested’ components your study has, the less scope for disaster.
8. Learn to cook an egg before you try to cook a chicken
Students sometimes have ideas for really interesting, useful and original studies. The best advice is to wait until you are a more experienced researcher before tackling these. The purpose of student research is to teach you the basics, so that when you undertake a really valuable research project you will be capable of doing it well.
9. Have a time plan
Divide your time into quarters. Use the first 25% for developing ideas. Use the middle 50% of your project for data collection. Plan to finish data gathering 75% of the way through your project at the latest, and to finish data analysis shortly after that.
Here is an outline, based on a one-year schedule:
- January: Develop three ideas, read up on background, start building
- February: Pick best idea of the three, identify methods to be used to
recruit participants and measures that will be used.
- March: Draw up study protocol, submit ethics application, write introduction and methods section for report/paper/thesis.
- April: Pilot data collection, try to shorten everything, draft blank results
- May: Main data collection starts, make formal data analysis plan, learn analysis methods you will need.
- June-August: Data collection: cleaning and consistency checking.
Preliminary analysis of data – save all analysis scripts/do-files for
re-running when all data are available.
- September: Final data analysis and cleaning. All data collection ends at the end of September.
- October: All analysis completed by the end of October.
- November-December: Write up final report.
For projects longer than one year, multiply up.
10. Be aware of sample size limitations
You should be aware at the outset of the amount of data that you can reasonably hope to capture, and how this will limit your ability to draw conclusions. You should think about the smallest subgroup of interest in your data and work out how much data will be available. Planning an analysis of a subgroup that will only contain a handful of cases is a waste of time.
11. Engage your peers
Being able to discuss research ideas with your peers and supervisors is a valuable learning resource. The exchange of ideas, experience and criticism will help to focus research proposals and to recognise pitfalls before they occur. Don’t work alone – that’s not how science is done.
Professor Ronán M. Conroy is an Associate Professor, Division of Population Health Sciences, RCSI.