Sample

A request to complete the survey was sent to 352 individuals, including WAME members and editorial board members of WAME journals. A list of journals with WAME member editors (n = 560) is posted on the WAME website.¹² However, it should be borne in mind that not all these journals updated websites, which may be related to broken links or discontinued titles.

Methods

The survey instrument, along with the related link, was generated through the Webankieta website (https://www.webankieta.pl/). The survey was conducted independently of WAME. The association was not involved in the conceptualization and creation of the survey, analysis of the results, or drafting this manuscript. As per WAME policy, the questionnaire was assessed by the WAME Scientific Committee to ensure it was appropriate for its members before providing permission to have it circulated. The study protocol was approved by the Bioethics Committee of the Medical University of Warsaw (AKBE No. 274/2023). The first 4 questions of the survey concerned the role held in the journal by the respondent, their continent of origin, and duration of scientific experience. The next questions asked about the percentage of accepted manuscripts with a correct statistical analysis, the overall percentage of accepted manuscripts in a WAME journal, and the annual number of submissions. The respondents also assessed their level of statistical knowledge on a scale of 1 to 10. The following questions (8–19) dealt with various aspects related to the functioning of the journal with respect to statistical analysis of the submitted manuscripts, while the last set of questions (20–25) concerned the most important recommendations that should be implemented to improve statistical reporting. To ensure full anonymity, following advice from the WAME Scientific Committee, the respondents were not asked to specify the name of the journal they are affiliated with. The questionnaire was completed voluntarily. Prior to its completion, each person was familiarized with the survey and its purpose. No personal identifiable information was collected from the respondents.

Survey procedure

Following approval from WAME and the Bioethics Committee of the Medical University of Warsaw, the questionnaire was distributed to WAME members. The link to the survey was also sent via email to the editorial board members of WAME journals (editor‑in‑chef, managing editor) or to a general email address using contact data found on the journal website. A request to complete the survey was re‑sent 1 month after its launch. The survey was available for completion from November 2 until December 15, 2023.

Statistical analysis

Statistical analysis was performed using the SPSS25 statistical package (IBM SPSS Statistics, Armonk, New York, United States). When analyzing relationships between variables measured on a nominal scale, the χ² test was used. Effect sizes for bicategorical variables were measured using the Φ coefficient, while multicategorical variables were measured using the Cramer V coefficient. The values of the Cramer V and the Φ coefficient range from 0 to 1, where 0 indicates no association and 1 indicates a perfect association. Generally, values below 0.1 suggest a very weak relationship, from 0.1 to 0.3 indicate a weak relationship, from 0.3 to 0.5 suggest a moderate relationship, and those above 0.5 indicate a strong relationship. To test whether there were significant differences between more than 2 groups of respondents, the Kruskal–Wallis test was used. When significant differences were present, the Dunn post‑hoc test was employed to examine them in more depth. The relationship between variables measured on quantitative and ordinal scales was analyzed using the Spearman association analysis. The Spearman rank correlation coefficient ranges from –1 to 1, with values from 0.1 to 0.3 indicating a weak correlation, 0.3 to 0.5 indicating a moderate correlation, 0.5 to 0.7 indicating a strong correlation, and 0.7 to 1 indicating a very strong correlation. A P value below 0.05 was assumed as significant.

Sample characteristics

Among the 352 questionnaires sent out to WAME members and the editorial staff of the journals they are affiliated with, responses were obtained from 141 individuals (response rate, 40%), of whom 52 (36.9%) were editors‑in‑chief, 51 (36.2%) were co‑editors, 11 (7.8%) were deputy editors, and 10 (7.1%) were editorial board members. The remaining 17 responders were assistant editors (n = 9; 6.4%), managing editors (n = 7; 5%), and a section editor (n = 1; 0.7%). Most respondents were from Europe and North America. The length of their scientific experience was mainly between 11 and 20 years. A majority of the respondents evaluated the overall percentage of accepted manuscripts to fall in the range of 11%–30% (Table 1).

Evaluated percentage of accepted submissions with correctly performed statistical analysis and self‑assessed statistical knowledge

Most respondents (40%) reported that only 31%–50% of the accepted manuscripts are statistically correct (Figure 1).

The respondents were asked to assess their level of statistical knowledge by declaring how strongly they agreed with the statement “I have the level of statistical knowledge necessary for the daily work of my journal” (on a scale of 1–10; with 1 – strongly disagree; 5 – neither agree nor disagree; 10 – strongly agree). The median (interquartile range) response was 7 (5–8).

A weak relationship was observed between the respondents’ self‑assessed level of statistical knowledge and their estimated percentage of accepted manuscripts that are statistically correct (r_s = –0.28; P = 0.001). The higher the respondents’ assessment of their statistical knowledge, the lower the declared percentage of accepted manuscripts with a correct statistical analysis. To thoroughly evaluate the impact of these 2 variables on each other, additional analyses were performed. The Kruskal–Wallis test showed a significant difference in this regard (H(3) = 10.43; P = 0.02). The Dunn post hoc test showed that the respondents who believed that more than 70% of accepted manuscripts are statistically correct assessed their statistical skills to be worse than in the case of the respondents rating this percentage at 11%–30% (P = 0.02) (Figure 2).

Statistical reviews

About a half of the respondents stated that the editorial board of the journal they work for includes a statistical editor. According to approximately 40% of the respondents, statistical reviewers are sometimes selected from among the editorial board members. Between 1% and 10% of articles submitted to the journals are reviewed for statistical correctness, as declared by 39.7% of the respondents. In the group of respondents who stated that statistical peer review is normally used in their journal, as many as 90% expressed the opinion that there are problems with finding a reviewer. A total of 70 respondents declared that the journal offers no remuneration for performing statistical reviews. A major proportion of respondents (86.5%) stated that there are situations involving receiving divergent opinions on statistical analysis evaluated in a single submission. Most of them also believed that statistical reviews greatly improve the quality of published manuscripts. The percentage of manuscripts rejected after a statistical review was evaluated by a majority of respondents to range from 11% to 30%. Slightly fewer respondents assessed this percentage as 1%–10%; however, more than one‑fifth of them did not know the answer to this question (Table 2).

Question 14 of the survey pertained to additional benefits associated with performing a statistical review. The benefits mentioned by the respondents included receiving acknowledgment, being mentioned in the journal impressum, gaining educational credit, opportunity to publish an article for free, and receiving a certificate for their work.

Noteworthy is the significant relationship between the presence of a statistical editor on a journal editorial board and the respondents’ opinion on the extent to which statistical reviews improve the quality of published manuscripts (χ²(2) = 18.13; P <⁠0.001; Vcr = 0.37). In the group of respondents reporting that the editorial board of their journal does not include a statistical editor, 49% believed that statistical reviews improve the quality of published manuscripts. The respondents who stated that such a person is part of the editorial board evaluated this percentage as high as 84% (Figure 3). The effect size, as measured by the Cramer V coefficient, indicates a moderate relationship.

Among the respondents working for journals with a statistical editor on the editorial board, 42% declared the percentage of manuscripts rejected after a statistical review to range between 11% and 30%. On the other hand, among the respondents who asserted the absence of a statistical editor, 33% admitted to being unaware of the quantity of manuscripts rejected for this reason, while 28% declared this percentage to be lower (1%–10%). The Spearman correlation analysis showed that the greater the volume of statistical reviews, the more respondents believed that such reviews improve the quality of published manuscripts (r_s = 0.21; P = 0.02); however, this is a weak association. The same holds true for opinions on how many manuscripts are rejected after statistical peer review—the more often statistical reviews are conducted, the more manuscripts are rejected as a result of this activity (r_s = 0.38; P <⁠0.001).

Problems with conducting statistical reviews

A challenge in conducting statistical reviews that was identified by the highest proportion of respondents (60%) was difficulty in finding individuals with the appropriate expertise. More than a half of the respondents stated that a lack of adequate funding for people conducting specialist statistical reviews was a problem. Approximately 40% also highlighted an issue related to the substantial volume of articles submitted to journals, coupled with a limited number of available reviewers. A similar percentage of respondents pointed out a lack of uniform statistical recommendations. The fewest respondents indicated a lack of meetings and international events for editorial board members at which various statistical issues are discussed, as well as a lack of statistical recommendations for authors posted on a journal website (Figure 4).

Other issues identified by the respondents were insufficient knowledge of the editors on conducting statistical reviews, lack of guidance for reviewers on determining which manuscripts should be assessed, uncertainty about the time allotted for statistical reviews, and widespread low level of statistical knowledge in many countries.

Statistical recommendations

A small proportion of respondents (23%) declared that statistical guidelines for authors are available on the website of the journal they work for. A similar percentage confirmed publication of articles including statistical recommendations in the journal. Of note, approximately 90% of the respondents stated they would like to be provided with uniform statistical recommendations for biomedical journals. Only 20% of the respondents declared that their workplace organizes online meetings for editorial board members to discuss key statistical aspects. More than 90% would like to attend such meetings (Figure 5).

Unfortunately, approximately 70% of the respondents are not familiar with the Statistical Analyses and Methods in the Published Literature (SAMPL) guidelines. A total of 27% stated they were familiar with these guidelines, but the journal does not use them (Figure 6).

Noteworthy is the relationship between the respondents’ familiarity with the SAMPL recommendations and their declaration on whether the journal seeks external statistical reviewers (χ²(1) = 5.86; P = 0.02; Φ = 0.2). However, the effect size measured using the Φ coefficient indicates that it is a weak relationship. In the group of respondents who declared they are not familiar with the SAMPL recommendations, 63% confirmed that the journal seeks external statistical reviewers. On the contrary, among the respondents who declared familiarity with such recommendations but stated that the journal does not implement them, 61% declared that specialist statistical reviews are not solicited.

Another noteworthy association was observed between the presence of a statistical editor on the journal editorial board and the availability of statistical guidelines for authors on the journal website (χ²(1) = 9.76; P = 0.002; Φ = 0.26). In the group of respondents who stated that such guidelines are available, 73% also declared that the editorial board includes a statistical editor. In contrast, in the group of respondents who stated that there are no such guidelines, the percentage of respondents confirming the presence of a statistical editor was lower (42%). A significant relationship was also found between the presence of a statistical editor and publication of statistical recommendations in the journal (χ²(1) = 12.28; P <⁠0.001; Φ = 0.3). The effect size measured using the Φ coefficient indicates that it is a moderate relationship. Among the respondents who believed that such recommendations have been published, 79% also declared that the journal editorial board includes a statistical editor. In contrast, in the group who believed that such recommendations have not been published, the percentage of respondents confirming the presence of a statistical editor was much lower (42%).

Limitations

A limitation of the present survey is its low response rate (40%), although it is similar to that obtained in a survey conducted in 2020 by Hardwicke et al⁸ (35%). The reason for a lack of response despite the reminder may be that minor journals do not use statistical reviews. In the future, it may be worth carrying out a similar survey at an organized event, such as the WCRI or the International Congress on Peer Review and Scientific Publication. An additional limitation is related to the fact that responses “0” and “not applicable” to the survey questions concerning self‑assessment of statistical knowledge and the frequency of conducting statistical reviews, respectively, were not included in the analysis.

Conclusions

Nowadays, members of editorial boards encounter significant challenges in conducting statistical reviews in their journals, in part due to difficulties in finding reviewers with appropriate expertise in this area. In light of this observation, more emphasis should be placed on implementing statistical recommendations rather than just on publishing them.