What is a meta-analysis and how is it used in research?

What type of research is a meta-analysis, and what are its phases of development?

Let us imagine that we intend to generate a new type of therapy to treat a disorder, for example borderline personality disorder. To do so, the first step to follow is to collect and assess the effectiveness of the different pre-existing treatments and methodologies for that disorder: to look for the effectiveness of the treatments in previous studies. But a simple primary study would not be valid for us, since it may be biased and may even reflect results that contradict those of others.

Thus, we need to resort to studies that have analyzed the behavior of what we want to observe across different studies, and for this the most reliable methodology or document is the use of meta-analysis.

What is a meta-analysis? Let's see it throughout this article.

What is a meta-analysis?

We can define a meta-analysis as the result of carrying out a systematic and exhaustive investigation on a very specific topic or question to be answered,** combining and evaluating the results obtained by a large number of studies on that aspect or topic**. It is an in-depth review that makes it possible to see very clearly the way in which the studies reviewed have been selected and evaluated, in addition to analyzing their results and quantitatively assessing them by means of various statistical procedures, among which the evaluation of the effect size and the control of possible publication bias stand out.

It has the great advantage that it allows us to work with large amounts of information from a large number of studies, in a highly organized manner and allows us to work with the scientific method with large samples in a relatively simple way, in addition to assessing differential aspects between studies and sources of heterogeneity that the basic studies may not have taken into account.

However, meta-analysis is not the only existing methodology for systematizing and synthesizing the results of research on a specific topic.There is also the concept of systematic review, which, like meta-analysis, reviews in an objective and systematized way, reflecting its methodology in order to be able to replicate the existing data in the scientific literature so far.

It is necessary to bear in mind that, technically speaking, any meta-analysis is a systematic review that incorporates the statistical analysis of the results. statistical analysis of the results is incorporated. And this is fundamental and allows us a much greater precision in our research, being able for example to assess from the statistical analysis which treatments for a certain disorder or even symptom are more effective in a consistent way in the various studies evaluated.

When should they be used?

The use or performance of meta-analyses can be of great help in a large number of areas, but they are much more relevant when we are faced with the need to investigate aspects in which previous studies show a certain degree of controversy and contradiction between them, or when we want to investigate the degree of relationship between different variables or try to assess the effect size of a program or treatment with respect to its effectiveness. Thus, in fields such as psychology or medicine, this type of review is generally used to explore, investigate, prioritize and apply evidence-based treatments.

However, it should be borne in mind that its application is only viable when the primary studies that are chosen have a quantitative perspective and with a homogeneous or very similar sample and conditions, and obviously, it is not possible to apply it to all the studies. or very similar to each other, and obviously starting from at least the same objective and main question. It would also be of little relevance if what we are looking for are cause-consequence relationships.

Main phases in performing meta-analyses

Conducting a meta-analysis is not simple and requires the systematic follow-up of a series of steps or phases, which are detailed below.

1. Formulation of the research problem/question

The first step, probably the most important but also the most obvious, is to formulate or formulate the problem, topic, and research question. formulating or posing the problem, theme or aspect that we want to investigate and analyze.. It is not a matter of posing a question at random, but of defining it as clearly and concisely as possible and taking into account the various variables that we will have to evaluate.

Based on this first question, the objectives of the research will be constructed and the meta-analysis will begin to be structured, and even the criteria for inclusion and exclusion of studies will be generated.

2. Bibliographic search and selection of studies

The second step, probably one of the longest, involves an active search of the bibliography that we are going to analyze together in order to make a good review of our research topic.

To do this, it is not enough to simply pick up every study we see: it is first of all necessary to have generated inclusion and exclusion criteria that allow us to select from among those that exist those that meet the requirements to be evaluated in the meta-analysis.

The criteria should take into account the type of design of each study, how the results were measured and the main characteristics of the samples used or the experimental methodologies. Thus, if we consider, for example, the effects of a drug on the menstrual cycle, we will not take studies in which the sample is male. Or if age is a factor to be taken into account in our research, we will limit ourselves to studies whose sample includes only subjects with the age that is relevant to us, or at least has separated the groups by age.

In addition, other factors should also be taken into account when making the selection: the language in which they are written (if this is relevant) or the date of publication (it is recommended not to use data from studies older than ten years), among others.

For the search it is usually advisable to use indexed databases such as SCOPUS or PubMed, at the computerized level, although it is also recommended not to use data from studies older than ten years.although other types of documents or systems can also be used.

3. Coding of studies

Once the studies to be included in the meta-analysis have been selected, it is necessary to prepare a coding manual in which the reliability of the data and the variables taken into account in the model used are recorded: these include sample, extrinsic, methodological and contextual variables.. In general, any moderating variable that may influence the results should be included, as well as indicating how they have been assessed so that other researchers can arrive at the same results if they use the same methods.

4. Statistical analysis and interpretation

The fourth step, in which we are already dealing with a meta-analysis, is the statistical analysis of the results. In this phase the implications or interpretations of the results are evaluated and then discussed..

As a general rule, this statistical analysis usually assesses the effect sizes of the different treatments or studies compared, assesses whether or not heterogeneity exists (if necessary using strategies to reduce the effect that this may have on the interpretation of the data) and, if it does exist, assesses which variables can explain this heterogeneity based on tests such as ANOVA.

5. Publication

Once the interpretation of the results has been completed, the last step to perform the meta-analysis is to write it up and publish it, having at least the following sections: introduction (which includes objectives and hypotheses as well as the theoretical framework), method (which should be very clear and detailed so that other authors can replicate it and should include how and where we searched for the studies, the coding manual and the type of analysis performed), results and discussion (where the results are interpreted based on the data).

The problem of publication bias

A possible problem to take into account when assessing the results of research and even meta-analyses is that that publication bias may existThis is understood as the deviation in the results that can be caused by the tendency of the scientific literature to search for and favor studies that show statistically significant relationships between the variables used and ignore those that do not.

That is: studies in which relationships between variables are seen are published, but those in which the results do not show a significant relationship tend to be less published or ignored. Fortunately, this effect can be visualized and assessed in meta-analyses, although it is a difficult problem to solve.