Flowchart for selecting RCTs for a systematic review
Despite significant advances in the treatment of gastric and duodenal ulcers, many aspects of its complicated course remain unresolved within the framework of evidence-based medicine [1].
The purpose of the study is to obtain the most reliable information on the surgical treatment of ulcerative pyloroduodenal stenosis based on the methodology of evidence-based medicine.
Material and methods
The work was carried out in a systematic review design and represents the so-called secondary scientific research. Unlike a literature review, a systematic review, thanks to the methods of Evidence Based Medicine and Critical Appraisal Tools, allows us to identify and summarize the results of the most reliable primary studies in a particular branch of medical science.
The inclusion criteria for the systematic analysis were randomized clinical trials (RCTs) studying the surgical treatment of ulcerative pyloroduodenal stenosis in patients over 18 years of age.
Methodologically, the work was carried out taking into account the recommendations of the Prism group [2] and the mechanisms of critical analysis from the Oxford Center for Evidence-Based Medicine, used to assess the likelihood of systematic errors in selected primary scientific works [3]. To determine the degree of reliability of the information obtained, the Oxford gradation of evidence, edited by J. Howick, was used [4]. According to this hierarchy of evidence, reliable information can be obtained from studies that correspond to the first level of evidence, which include RCTs. To determine whether primary scientific studies meet the stated highest level of evidence, they were scored based on the possible risks of major systematic errors, as recommended by Cochrane experts [5].
Electronic search of evidence was carried out in Russian and English. From Russian databases, the elibrary electronic library was used; from English-language ones, the Cochrane library and the PubMed database of medical and biological publications were used. Search keywords: peptic ulcer disease, pyloric stenosis, gastric outlet obstruction, vagotomy, antrectomy, randomized controlled trials, meta-analyses ( metaanalysis).
The primary search for evidence is limited to 2014–2018. However, due to the fact that currently there is practically no research on the topic of surgical treatment of ulcerative pyloroduodenal stenosis, in order to give the systematic review the necessary power, we were forced to expand the recommended time frame to the full possible depth in these databases. In addition, references to literature sources from all scientific papers selected for this systematic review were carefully reviewed.
Diagnosis of pyloroduodenal stenosis
Differential diagnosis is carried out, as a rule, with pyloric stenosis of blastomatous origin, tuberculosis and syphilis of the stomach, and a tumor of the head of the pancreas.
Instrumental diagnostic methods:
- X-ray examination: Stage I - the stomach is of normal size or slightly dilated, peristalsis is increased. The pyloroduodenal canal is narrowed. Evacuation delay up to 6–12 hours; Stage II - the stomach is dilated, fluid is detected on an empty stomach, peristalsis is weakened. The pyloroduodenal canal is sharply narrowed. Evacuation delay for 12–24 hours; Stage III - the stomach is sharply distended, a large amount of contents is detected on an empty stomach, peristalsis is sharply weakened. Evacuation delay of more than 24 hours.
- Endoscopic examination: stage I - severe cicatricial and ulcerative deformation of the pyloroduodenal canal with a narrowing of its lumen to 1–0.5 cm; Stage II - the stomach is distended, the pyloroduodenal canal is narrowed to 1–0.3 cm due to sharp cicatricial deformation. Peristaltic activity of the stomach is reduced; Stage III - a huge stomach. Atrophy of the mucous membrane. Marked narrowing of the pyloroduodenal canal.
Laboratory diagnostics: signs of progressive dehydration, hypoproteinemia, hypokalemia, severe alkalosis.
results
A chronological analysis showed that before 1964 there was no evidence-based work on the treatment of peptic ulcers and pyloroduodenal stenosis. In the period from 1964 to 1974, only 13 studies of the first level of evidence were carried out, in which the issues of treatment of complicated peptic ulcer disease were practically not considered. From 1974 to 1994, more than 150 randomized studies were conducted to evaluate the effectiveness of various vagotomy options and compare organ-sparing and resection interventions, including in patients with ulcerative stenosis. The period from 1994 to the present is characterized by a lack of interest on the part of evidence-based medicine in the problem of surgical treatment of ulcerative pyloroduodenal stenosis. A few randomized studies are being published that evaluate the long-term (10-20-year) results of traditional surgical treatment, and the possibilities of endoluminal flexible endoscopy are beginning to be studied. A separate unpleasant point is that all Russian studies carried out over more than half a century in this field of surgery [6, 7] due to the lack of instructions on the design and methodology of the studies cannot be selected for a systematic analysis corresponding to the first level of evidence.
For the final analysis in the framework of the presented systematic review, only 20 (9%) publications were selected from 215 randomized studies examining various aspects of laparotomic interventions in the treatment of peptic ulcer disease and its complications. This was due to the fact that in these studies one of the criteria for inclusion in the study was the presence of varying degrees of ulcerative stenosis in patients, usually excluding decompensated forms. Half of these studies were conducted in the USA [8–17]. Among them, it is necessary to highlight a series of 6 scientific works carried out under the leadership of Paul H. Jourdan Jr. During these studies, during 1970-1994. More than 1000 vagotomies were performed, the long-term results of which were monitored by the authors for 20 years or more. Five studies with the first level of evidence were carried out in the UK [18–22]. Another 5 studies were conducted in Switzerland [23], China and the UK (study in Hong Kong [24]), Sweden [25] and Chile [26, 27]. The results of the selection of primary scientific studies for the systematic review are presented according to the recommendations of the Prisma group (Prisma flow-chat) (see diagram).
The total sample size based on the evaluation of 20 RCTs was 1794 patients. However, due to the multidirectional nature of these studies, which leads to significant heterogeneity of clinical material, a formal mathematical generalization within the framework of a meta-analysis is not possible, and relevant conclusions on the treatment of ulcerative pyloroduodenal stenosis can only be presented after discussion and critical analysis of individual RCTs.
Discussion
In assessing the validity and reliability of the findings and practice recommendations selected for this systematic review of RCTs, the primary consideration was external validity, internal validity, and the likelihood of false positives and false negatives. ) conclusions. In the presented systematic review, the assessment of external validity indicates the possibility of extrapolating the findings to the general world population of patients with ulcerative pyloroduodenal stenosis. This is due to the wide geography of primary studies (scientific work was carried out in various countries on several continents), the possibility of performing these surgical interventions by surgeons with different levels of training, the duration of observation of patients (in some studies this period was 20 years or more), and also with the total number of patients in the selected RCTs ( n
=1794) with the required minimum for a systematic review (without special mathematical calculations) [28], equal to 1591 study participants. All these facts indicate that the presented systematic review is also sufficiently powered to avoid false negative conclusions that are possible with an insufficient number of patients.
Internal validity is determined by the number of systematic errors made by the authors of specific studies. The main errors have been known for several decades and, according to Cochrane experts, the mechanisms that reduce the likelihood of these errors in scientific papers are defined as “adequate”, “unclear” and “inadequate”, which give the corresponding score for a detailed assessment of specific scientific papers. works
Since most of the primary studies selected for the systematic review were carried out at the end of the twentieth century, during a critical analysis we assessed the main possible systematic errors [5], which made it possible to score the rating of scientific works and, if necessary (>3 points), reduce the level of reliability of the presented data. conclusions [5, 29, 30].
The first systematic error that reduces the reliability of RCTs is selection bias. It is associated with the generation of allocation sequence. If randomization is not used in the selection of participants, the probability of error is high; if the authors did not indicate the method of randomization or the method of randomization was incorrect, then an error is likely; If a recommended method of randomization (such as a table of random numbers or a program that generates random numbers) was used, then the likelihood of selection bias is low.
At the entrance to the study, it is also necessary to evaluate the nature of the distribution of patients into groups (allocation concealment). If the distribution of patients becomes known, then there is an error. If the method of hidden allocation of patients is not specified, then a system error is likely. An adequate method of hidden allocation guaranteed by the authors (a centralized approach or sealed envelopes with numbers obtained when generating the allocation sequence) allows one to avoid this error in the study.
The next 2 errors are related to the conduct of the study itself (performance, detection bias) and the use of a blind method. If the blind method is not used, there is an error; if the blind method is not described in detail, an error is likely. The use of a double-blind method, in which neither the patients nor those authors from the research group involved in the statistical assessment of the results of surgical interventions know the extent of the surgical intervention, guarantees the absence of errors during the study and in identifying treatment outcomes.
When determining treatment outcomes (detection bias), systematic error due to interruption of observation (follow-up) should also be assessed. The likelihood of its presence is determined based on the principle of compliance with the study protocol, attrition bias, as well as the impossibility of assessing the results of treatment of individual patients outside the framework of the group of patients determined during randomization (intention-to-treat). If the analysis of the results is done in violation of the planned course of scientific work, then the probability of error in the results is high. If the analysis of the nature of the results is not indicated, then an error is likely. The likelihood of bias is low if treatment outcomes were studied in accordance with the study protocol.
We did not assess the risks of other systematic errors, such as profit bias, which were included by Cochrane experts in the evaluation sheets after the completion and publication of most randomized trials on the surgical treatment of ulcerative pyloroduodenal stenosis (see table).
Risk assessment of systematic errors in RCTs, points
The practical implementation of critical analysis mechanisms showed that 7 (35%) of 20 RCTs correspond to the highest level of evidence (received 3 points, rating 1). The remaining 13 (65%) have a greater number of systematic errors, which requires a downgrade of their rating (rating 1-) and significantly reduces the reliability of the presented conclusions.
In addition to assessing validity, we examined the selected RCTs for the potential for false-positive and false-negative findings. The presence of false-positive conclusions is associated with the level of α-risk and error of the first type, which is determined by the quality of the statistical calculations carried out by the authors, false negatives - with β-risk, error of the second type and the power of the study, which directly depends on the number of patients in each specific randomized trial. A detailed assessment of the selected RCTs for statistical accuracy is beyond the scope of this systematic review. It should only be noted that in some of the primary scientific works it is not possible to assess the reliability of calculations of the significance level (p). From which we can assume that the α-risk in these works exceeds 5%, therefore, there is a certain probability of false-positive conclusions.
To exclude a type II error, before conducting the study, the required number of patients should be calculated so that the power of the study is at least 80% and the β-risk does not exceed 20%. In 16 (80%) of the selected studies on ulcerative pyloroduodenal stenosis, the likelihood of type II error was minimal, and these studies did not contain false negative findings.
Incidence (per 100,000 people)
Men | Women | |||||||||||||
Age, years | 0-1 | 1-3 | 3-14 | 14-25 | 25-40 | 40-60 | 60 + | 0-1 | 1-3 | 3-14 | 14-25 | 25-40 | 40-60 | 60 + |
Number of sick people | 0 | 0 | 28 | 43.6 | 43.6 | 28 | 20 | 0 | 0 | 16 | 18 | 18 | 13 | 8 |