The scientific production of institutions of higher education, as well as the dissemination and use of this published work by peer institutions, can be assessed by means of quantitative and qualitative measurements. This type of analysis can also serve as the basis of further academic actions. Variables such as the type of evaluation, the number of faculty members and the decision to include or exclude researchers who are not professors are difficult to measure when comparing different schools and institutions.
OBJECTIVESThe purpose of this study was to assess the scientific production of tenured faculty from the Universidade de São Paulo, Faculdade de Medicina performed from 2001 to 2006.
METHODSMedline/PubMed database was considered and the Impact factors (IFs - Journal Citation Report, 2006) and the number of generated citations (Web of Science/ISI Thomson) were also evaluated.
RESULTSThe analysis of the scientific production of 66 full professors (level MS-6) revealed 1,960 scientific articles published in 630 scientific journals, of which 31.3% were Brazilian and 68.7% were from international sources. Among these, 47% of the articles were published in 62.9% of the journals with IFs above 10, although 16.4% of the journals did not have assigned IF values. We verified that 45% of the published articles received 9,335 citations (average of 11 + 17), with the majority of these (8,968 citations) appearing in international scientific journals.
CONCLUSIONSOur results indicate that it is possible to analyze the scientific production of a learning institution by the number of papers published by full professors, taking into account not only their academic position and influence, but also the fact that publication is an opportunity to stimulate joint projects with other members of the same institution.
The scientific activity of a group of researchers is manifested through published papers, which are validated and legitimized by their peers. These papers can also serve as indicators of the degree to which their scientific knowledge in their area of research has been developed.
The formal means of releasing scientific publications, which include the results of research conducted at medical education institutions, is the collection of national and international scientific journals that have been indexed into databases representing various areas of knowledge.1–4
There are several major international databases that contain publications from scientific journals in the area of health science: Medline/PubMed, which is organized by the National Library of Medicine and the National Institutes of Health; the Web of Science, which is maintained by the Institute of Scientific Information (ISI)/Thomson Reuters; Embase, which is an European database; and Current Contents, which is also maintained by ISI. In Brazil, the main databases are LILACS (Latin American and Caribbean Health Sciences) and SciELO (Scientific Electronic Library Online), which is a database allowing free access to full texts of Brazilian, Latin American and Caribbean scientific literature.5,6 Among these databases, Medline/PubMed (http://www.pubmed.gov), which offers free access and has more than 5,100 titles, is the most often searched.
Another important tool in the recovery of available indexed articles in databases is the controlled vocabulary known as Medical Subject Headings (MESH),7 which allows researchers to retrieve information using a more exact, standardized description of the document in question.8–10
International and national databases make it possible to search for an article using keywords as well as the authors’ last names, without considering the order of their authorship. In some cases, however, the recovery of articles and citations is compounded by the different ways that the names of authors or institutions can be entered.11–13 Compound last names, names with suffixes (e.g., Neto, Jr.) or names which include “de” and “da” are often included in publications in a non-standardized way.14 In addition, different researchers from the same group frequently list their department and institution names in a non-standardized way. Small details like these make it difficult for researchers to find desired articles and for institutional evaluations to be made by the government or even by their peers.12,13
The journals selected for publication also reflect the level of productivity of a research group and, consequently, the institution to which they belong. Journals indexed in databases often receive impact factors (IFs). These values are calculated from the number of citations of the articles published by the journals over a two-year period.15,16 Journals with high IFs are considered to be more relevant to, or influential on, the scientific field in question.17
Few Brazilian journals, even those that are indexed in databases, have IFs. However, this number has recently increased, from 23 in 2006 to 30 in 2007, despite the fact that the demand for the publication of English-language articles has favored a more visible role for them within the international community.18
In addition to the absolute number of publications, institutions are also qualitatively evaluated according to certain bibliometric indicators. One such indicator is the assessment of the number of times that the scientific articles generated in a particular institution are cited by their peers.19
The evaluation of scientific production from a university is rather complex, because it involves different areas of knowledge, each one with its own peculiarities. Analysis of smaller units, like university departments, disciplines or services, can be made more easily. When the number of researchers is significantly high, a raffle system is often used to choose representatives from different functional categories, including professors and non-professors.
In this current investigation, our purpose was to provide a panoramic view of publications from the “Universidade de São Paulo, Faculdade de Medicina”, during the period of 2001 to 2006, by analyzing the scientific production of full professors (level MS-6) at this institution and subsequently verifying their impact.
METHODSResearch into the number of publications in the Medline/PubMed database by full professors of the Faculdade de Medicina da Universidade de São Paulo (FMUSP) who were working during 2001–2006 was performed using a list of referred professors. The FMUSP “Assistência Técnica para Assuntos Administrativos” (ATAAD, http://fmadm.fm.usp.br/attaad)20 provided the following data, in addition to names: job position, registration, department, discipline, kind of work and dates of admission and dismissal. The inclusion criteria of the study were full professorship and employment by FMUSP during the entire established period of time. Some full professors were excluded because they were dismissed during this period, mainly because of retirement, and others because they were hired through recruiting held during the same period.
The survey of the number of publications in Medline/PubMed was first done by individually searching for the name of each author. Once published articles for that author had been identified, an Excel spreadsheet was created to record relevant data: the kind of authorship (sole author, main author, co-author); the bibliographical references of each retrieved article according to the year of the search; the IF of each retrieved journal; and the citations generated by each of the published homonyms that had been used in publication, which were determined by a search of the Lattes Curriculum records for each full professor. In addition, complete articles that were retrieved on the internet were confirmed.
After compiling the list of articles by each professor that have been indexed in the Medline/PubMed database, the IF of each scientific journal was determined, using the Journal Citation Report (JCR, 2006 edition). Finally, the number of citations by the authors’ peers that have been generated by these published articles was calculated using the March, 2009 update of the Web of Science database.
RESULTSThere were 66 full professors at the FMUSP during the period between 2001 and 2006, of whom 57 (86.4%) were males and only 9 (13.6%) were females. We verified that 36.4% of these professors were teaching full-time (40 hours), that 60.6% of them were working intermediate shifts (24 hours), and only 3% were working partial shifts (12 hours).
Our survey of the journal databases revealed 1,960 bibliographic records that were published in 630 international and national scientific journals. The number of published articles for each year that were retrieved on Medline/PubMed are provided in Table 1, with 31.3% overall being published in national journals and 68.7% in international journals. There was a gradual growth in the scientific production of the researchers during these years, with an average publication rate of five articles per year by each professor increasing from 3.8 in 2001 to 5.9 in 2006.
Number of articles published by full professors (MS-6) in national and international journals, indexed in the Med-line/PubMed database, during the period of 2001 to 2006
| Year | National Level | International Level | Total | ||||
|---|---|---|---|---|---|---|---|
| Number of articles | Number of journals | Number of articles | Number of journals | Number of articles | % | Number of journals | |
| 2001 | 71 | 13 | 183 | 147 | 254 | 13 | 160 |
| 2002 | 76 | 13 | 194 | 257 | 270 | 13.8 | 270 |
| 2003 | 96 | 13 | 212 | 169 | 308 | 15.7 | 182 |
| 2004 | 108 | 17 | 218 | 168 | 326 | 16.6 | 185 |
| 2005 | 131 | 18 | 281 | 204 | 412 | 21 | 222 |
| 2006 | 131 | 19 | 259 | 197 | 390 | 19.9 | 216 |
| Total | 613 (31.3%) | - | 1,347 (68.7%) | - | 1,960 | - | |
With regard to the authorship of the articles, only 2% (31) of the published articles had the professors as sole authors, and 5% (103) had them as first authors. The FMUSP professors were co-authors for the majority of publications (93%), which included partnerships with other researchers from the institution. Of the total number of articles that were published with other collaborators, 62.3% had 4 to 7 different authors.
Of the 630 journals in which the articles were published, only 103 (16.4%) did not have IFs listed in the 2006 edition of the JCR. Just considering those articles published in international journals (1,347), 21.8% were in journals without IFs. Taking into account all of the articles, 47.7% (935) were printed in journals with IFs between 1 and 5. These journals (396) represented 62.9% of the total and were mostly international. In addition, 6.4% of the articles were published in journals with IFs between 5 and 10, which represented 8.2% of the journals, and 1.5% of the articles were published in journals with IFs between 10 and 52, representing 1.7% of the journals (Table 2).
Total list of published articles by journal impact factor
| 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | Total | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Art | J | Art | J | Art | J | Art | J | Art | J | Art | J | Art | J | |
| IF | 80 | 28 | 73 | 25 | 94 | 24 | 103 | 33 | 134 | 38 | 137 | 37 | 621 (31.7%) | 103 (16.4%) |
| <1 | 35 | 26 | 39 | 22 | 46 | 22 | 45 | 20 | 51 | 25 | 32 | 20 | 248 (12.7%) | 68 (10.8%) |
| ≥1 to <5 | 113 | 87 | 124 | 99 | 141 | 114 | 157 | 114 | 203 | 138 | 197 | 139 | 935 (47.7%) | 396 (62.9%) |
| ≥5 to <10 | 20 | 14 | 25 | 17 | 21 | 16 | 20 | 16 | 22 | 18 | 18 | 15 | 126 (6.4%) | 52 (8.2%) |
| ≥ 10 to <20 | 1 | 1 | 4 | 3 | 5 | 3 | - | - | 2 | 1 | 4 | 3 | 16 (0.8%) | 7 (1.1%) |
| ≥ 20 | 5 | 3 | 5 | 2 | 1 | 1 | 1 | 1 | - | - | 2 | 1 | 14 (0.7%) | 4 (0.6%) |
| Total | 254 | 274 | 308 | 326 | 412 | 390 | 1,960 | 630 | ||||||
IF: impact factor; Art.: articles; J.: journals
With regard to those articles published in Brazilian journals (613), 127 (20%) were published in four journals with assigned IFs: Brazilian Journal Medical Biological Research (IF = 1.075), Arquivos de Neuropsiquiatria (IF = 0.400), Memórias do Instituto Oswaldo Cruz (IF = 1.209) and Revista de Saúde Pública (IF = 0.343) (Table 3).
List of Brazilian journals indexed in the Medline/PubMed database and respective percentages of published articles during the period from 2001 to 2006, including IF (JCR, 2006)
| Journal | 2001 | 2002 | 2003 | 2004 | 2005 | 2006 | Total | IF |
|---|---|---|---|---|---|---|---|---|
| Acta Cirúrgica Brasileira* | - | - | - | - | 1 | 1 | 2 | - |
| Arquivos Brasileiros de Cardiologia** | 11 | 10 | 15 | 14 | 15 | 25 | 90 | - |
| Arquivos Brasileiros de Endocrinologia e Metabologia** | - | - | - | 4 | 6 | - | 10 | - |
| Arquivos Brasileiros de Oftalmologia*** | - | - | - | - | 2 | 2 | 4 | - |
| Arquivos de Gastroenterologia* | 4 | 3 | 10 | 8 | 4 | 3 | 32 | - |
| Arquivos de Neuro-Psiquiatria* | 3 | 7 | 8 | 11 | 11 | 4 | 44 | 0.400 |
| Brazilian Journal of Infectious Diseases* | 3 | 1 | - | 2 | 2 | 3 | 11 | - |
| Brazilian Journal of Medical Biology Research* | 7 | 5 | 7 | 13 | 10 | 12 | 54 | 1.075 |
| Brazilian Journal of Urology | - | - | - | - | - | 1 | 1 | - |
| Cadernos de Saúde Pública*** | 1 | 2 | - | 2 | - | 5 | 10 | - |
| Clinics* | - | - | - | - | 31 | 35 | 66 | - |
| International Brazilian Journal of Urology** | - | 3 | 12 | 11 | 14 | 12 | 52 | - |
| Jornal Brasileiro de Pneumologia+ | - | - | - | - | - | 5 | 5 | - |
| Jornal de Pediatria (Rio J)** | 6 | 4 | 1 | 1 | 1 | 2 | 15 | - |
| Memórias do Instituto Oswaldo Cruz* | - | - | 2 | 1 | - | - | 3 | 1.209 |
| Pesquisa Odontológica Brasileira* | - | - | 1 | - | - | - | 1 | - |
| Pro Fono** | - | - | - | 1 | 3 | 2 | 6 | - |
| Revista da Associação Médica Brasileira*** | 8 | 4 | 11 | 8 | 8 | 7 | 46 | - |
| Revista Brasileira de Otorrinolaringologia (Engl Ed)* | - | - | - | - | 3 | - | 3 | - |
| Revista do Hospital das Clínicas da Fac. Med Sao Paulo** | 12 | 18 | 19 | 16 | - | - | 65 | - |
| Revista do Instituto de Medicina Tropical de Sao Paulo* | 5 | 6 | 1 | 4 | 6 | 1 | 23 | - |
| Revista de Saúde Pública*** | 3 | 8 | 4 | 4 | 1 | 6 | 26 | 0.343 |
| Revista da Sociedade Brasileira de Medicina Tropical*** | 1 | - | - | 2 | 2 | 3 | 8 | - |
| Sao Paulo Medical Journal* | 7 | 5 | 5 | 6 | 11 | 2 | 36 | - |
| Total | 71 | 76 | 96 | 108 | 131 | 131 | 613 | - |
+ Portuguese.
IF: Impact Factor (JCR 2006)
Until 2004, the journal Arquivos Brasileiros de Cardiologia accounted for the greatest number of published articles (90/613), followed by Revista do Hospital das Clínicas da Faculdade de Medicina USP (65/613). Starting in 2005, the journal Revista do Hospital das Clínicas da Faculdade de Medicina USP, which had its title changed to Clinics, accounted for 21.4% of the articles published in Brazil (66/613, Table 3).
The qualitative analysis, which was based upon the number of times that an article was cited by peers, was performed using the Web of Science database. Almost half of the total published articles (45%) were cited, which amounted to a total of 9,335 citations. These citations were not only found in articles published in Brazilian journals, in which 9% of the FMUSP articles generated 367 citations, but also for articles published in international journals, in which 91% of the FMUSP articles generated 8,968 citations. We verified that 27 of the 1,960 published articles (1.4%) were cited more than 50 times by their peers, as (Table 4).
Articles published in international journals that received more than 50 citations
| Journals | Year | Vol. | Fasc. | First page | Final page | IF/06 | IF/08 | Citations |
|---|---|---|---|---|---|---|---|---|
| American Journal of Obstetrics and Gynecology | 2003 | 188 | 2 | 419 | 424 | 2.805 | 3.453 | 140 |
| American Journal of Respiratory and Critical Care Medicine | 2004 | 170 | 8 | 857 | 862 | 9.091 | 9.792 | 50 |
| American Journal of Respiratory and Critical Care Medicine | 2002 | 165 | 12 | 1610 | 1617 | 9.091 | 9.792 | 83 |
| Archives of Internal Medicine | 2001 | 161 | 2 | 242 | 247 | 7.920 | 9.110 | 82 |
| British Journal of Cancer | 2002 | 86 | 5 | 705 | 711 | 4.459 | 4.846 | 71 |
| Chest | 2001 | 119 | 3 | 801 | 806 | 3.924 | 5.154 | 76 |
| EMBO Journal | 2002 | 21 | 13 | 3317 | 3326 | 10.086 | 8.295 | 141 |
| EMBO Journal | 2002 | 21 | 13 | 3307 | 3316 | 10.086 | 8.295 | 145 |
| Environmental Health Perspectives | 2002 | 110 | 12 | 1191 | 1197 | 5.861 | 6.123 | 58 |
| FEBS Letters | 2002 | 512 | 1 | 25 | 28 | 3.157 | 3.264 | 78 |
| Free Radical Biology and Medicine | 2001 | 30 | 10 | 1137 | 1144 | 5440 | 5.399 | 99 |
| Journal of the American College Cardiology | 2004 | 43 | 10 | 1743 | 1751 | 9.701 | 11.438 | 87 |
| Journal of Biological Chemistry | 2001 | 276 | 52 | 49400 | 49409 | 5.808 | 5.520 | 84 |
| Journal of Clinical Investigation | 2001 | 107 | 4 | 449 | 455 | 15.754 | 16.559 | 126 |
| Journal of Experimental Medicine | 2003 | 197 | 11 | 1501 | 1510 | 14.484 | 15.219 | 53 |
| Journal of Infectious Diseases | 2002 | 185 | 3 | 324 | 331 | 5.363 | 5.682 | 81 |
| Journal of the National Cancer Institute | 2002 | 94 | 21 | 1604 | 1613 | 15.271 | 14.933 | 75 |
| Journal of Neurochemistry | 2001 | 79 | 1 | 79 | 87 | 4.260 | 4.500 | 60 |
| Journal of Thoracic Cardiovascular Surgery | 2002 | 124 | 6 | 1216 | 1224 | 3.560 | 3.037 | 59 |
| Journal of Vascular Surgery | 2004 | 39 | 5 | 967 | 976 | 2.505 | 3.770 | 130 |
| Kidney International | 2003 | 63 | 1 | 209 | 216 | 4.773 | 6.418 | 73 |
| New England Journal of Medicine | 2002 | 346 | 15 | 1105 | 1112 | 51.296 | 50.017 | 168 |
| Nature Genetics | 2004 | 36 | 3 | 228 | 230 | 24.176 | 30.259 | 85 |
| Pediatrics | 2002 | 109 | 2 | 200 | 209 | 5.012 | 4.789 | 60 |
| Obstetrics and Gynecology | 2002 | 99 | 3 | 389 | 394 | 3.813 | 4.397 | 88 |
| Proceedings of the National Academy of Sciences of the United States of America | 2001 | 98 | 21 | 12103 | 12108 | 9.643 | 9.380 | 75 |
| Social Psychiatry and Psychiatric Epidemiology | 2002 | 37 | 7 | 316 | 325 | 1.577 | 1.959 | 58 |
IF/06: Impact Factor JCR/2006; IF/08: Impact Factor JCR/2008
The present investigation aimed at providing an overview of the scientific production of the Faculdade de Medicina da Universidade de São Paulo, with a focus on a specific group of faculty members with high-level positions at the university, who were the full professors (level MS-6). The use of only one group of professors in this study provides results that can later be confirmed for the other research professors and non-professors of the university. This group was selected to be representative of the other professors, taking into consideration the level of scientific maturity achieved and especially the possibility of collaboration with other members of the same institution, as well as other national and international institutions.
Analysis of the publications by the 66 full professors on staff during the period of 2001 to 2006 revealed a total of 1,960 published articles. This number represents 0.04% of worldwide scientific publications, 2.4% of national publications, 4.3% of the all-time total publications of the FMUSP and 12.33% of the total publications of the FMUSP during the same period.21,22
Of these articles, 68.7% were published in international journals, and 31.3% were published in Brazilian journals. The obligatory use of the English language in a number of the Brazilian journals that published the professors’ articles has led to a better visibility and accessibility of research results. Few Brazilian journals have IFs available, and only 4 of the 24 journals represented here had IFs. This number, however, represents 17.39% of the total number of Brazilian journals that have assigned IFs (2006 JCR).
There is, unfortunately, little data from the FMUSP that would enable a comparison of the results of this study. In a paper published in 2005, the scientific production that was generated by the Laboratório de Investigação Médica do Complexo Hospital das Clínicas da Faculdade de Medicina da Universidade de São Paulo was evaluated. The doctors, not all full professors, who represented the 62 laboratories in the hospital complex at that time published a total of 399 articles in 2004.23 A comparison of the number of publications by full professors during the same period indicates that, although the number is small (326), it can serve as an estimate of the number of publications of the institution over a certain period. Although the estimation of scientific productivity was possible for full professors, it is interesting to note that these estimates can be lower than those that were actually generated by the institution of interest. This type of data should be evaluated using different categories in order to make comparisons with other groups of professors.
During this time period, this group of full professors had an average publication rate of five published articles per year per professor. Records found in the “Anuário Estatístico da Universidade de São Paulo, 2007”22 indicate that the productivity of professors and researchers from the FMUSP during the period of 2003 to 2007 was 3.1 articles per teacher per year. If we consider that the number of publications per year per teacher has remained almost constant, our study shows a higher number of publications per year per professor than what was expected from the total group of teachers at the institution.
Approximately 89% of the articles were published in international journals with assigned IFs, which provides high worldwide visibility. However, the analysis of these IFs should take specific research areas into account, in addition to the kind of publication, because some areas have a greater number of researchers in comparison to others. Basic disciplines have a great advantage over specialized ones, in that the citation practices and publications are more focused.24
Considered as a whole, the articles counted in this study were published in journals with IFs that varied from 0.063 to 51.296. For 20.7% (406) of the articles, the IFs varied from 3.015 (Am J Cardiol) to 51.296 (New Engl J Med). These values vary depending on the specific area of research. For example, research in the dentistry area carried out in 2006 revealed that a large portion of dentistry publications occurred in journals with IFs varying from 0.692 to 1.569.13
With regard to citations of these published articles, 31.3% of the citations occurred in Brazilian journals and 68.7% in international ones, which resulted in a total of 9,335 citations. It is noteworthy that 20 of the evaluated articles received more than 70 citations. In a recent publication of “Faculdade de Medicina da Universidade de São Paulo,” the collective publications of the FMUSP were reported to be among the 100 most cited in the world.25,26
Despite some limitations, the bibliographic record data in this study can be useful in analyzing the scientific production of a group of professors from the FMUSP and for establishing a basis for comparison with other institutions in Brazil. In addition, this data could possibly be used to evaluate the output of scientific production in the entire field of health.
LIM-40 - Hospital das Clinicas da Faculdade de Medicina da Universidade de São Paulo.