10.4185/RLCS-2014-1030en | ISSN 1138 - 5820 | RLCS # 69 | 2014 | |
Analysis of the productivity, impact, and collective h-index of the communication research carried out in Spain based on the information shared by researchers in their individual Google Scholar profiles
M Túñez López [CV] [ORCID] [GS] Professor at the Department of Communication Sciences - Universidad de Santiago de Compostela (USC)– email@example.com
Introduction. This article examines the productivity, collective and individual h and h5 indexes, dissemination platforms (mainly books and journals), and visibility of the communication research carried out by Spanish scholars in the last four decades, based on the information shared by the 683 members of Spain’s largest communication research association (AE-IC) in Google Scholar Metrics (GSM). Method. The study is based on the analysis of the information shared in GSM by 683 researchers, of which 142 have a public profiles and together have more than 7,000 publications. We analysed the dissemination platform, year of publication, number of citations, and title of the nearly 2,300 works that had received at least one citation. Results. The visibility of the area of communication is low. The average h-index is 4, while the global h-index for the community of communication researchers is 56, with an h5-index of 34: h-index of 44 and h5-index of 34 for articles, and h-index of 34 and h5-index of 13 for books. Four of every ten researchers had h-index of 0 while two of every three published works did not manage to receive a single citation. Individually, the highest h-index is 26 and the highest h5-index is 18. Meanwhile, the importance and impact of books and journals as dissemination platforms has been inverted. Before 1980, of each ten citations six were given to books and three to articles, but after 2010, of each ten citations three are given to books and seven to articles. In terms of differences across gender, female researchers have lower impact values than their male counterparts.
Keywords: H-index; Google Scholar; Google Scholar Metrics; research productivity; impact; communication research.
Translation of de CA Martínez Arcos, Ph.D. (Universidad Autónoma de Tamaulipas)
The democratisation of the possibility of becoming a transmitter with the potential to reach mass audiences has meant a change in the communication roles played by individuals in real and virtual environments. We have moved away from product-focused planning to customer-focused actions and then to the actions focused on the human spirit, which is characteristic of the Marketing 3.0 that proliferated in the early 21st century (Kotler, 2010 and 2013). Receivers are also transmitters. Consumers have disappeared because in the Internet age they have become content prosumers, producers and consumers, all at the same time.
Relationships and their management have been transformed to such an extent that the communication management has led to the development of communication formulas that empower people, humanise the organisation, develop an identity committed to real social values and concerns, and bet on all-to-all types of relationships. Audiences have assumed the role of watchdog in the communication in organisations: they organise, judge, evaluate, share opinions, and give sentences. And this has motivated organisations to bet on the integrity of their image as the basis for their reputation and to champion themselves as promoters of the social values of their targets audiences.
Communication has gone from speech to dialogue, has abandoned the one-way sender-message-receiver models and developed circular and two-way communication models; has abandoned the task of narrator to focus on choral communications; and has ceased to plan on rigid formats that are limited to their dissemination platform in order to focus on liquid communication formats and contents, i.e., those that can adapt to new workflows and become part of the dynamics of the transmedia communication that characterises the actions and stories that take place on the Internet.
The dissemination of research results cannot be planned in an isolated bubble that only allows us to maintain traditional forms of relations in an ever changing context. Researchers must neither neglect the social commitment of research, particularly when it is financed with public funds and infrastructure. Online journals, and journals available in both paper and online platforms, are multiplying and play the role of prosumers, as proactive generators of content.
This change in roles coincides with the debate on the incorporation of new ways of measuring the visibility and impact of research. The movements in favour of the dissemination of content in open-access platforms and even in platforms governed by user-based impact indicators (downloads, visits, comments, etc.) coincides with the debate on the complementarity between journals’ impact factor and researchers’h-index and researchers’ dissemination platforms.
1.2. H-index and impact factor
The measurement of the research activity has been traditionally based on the journals’ impact factor, but in recent years the h-index has been gaining ground as a bibliometric indicator and has been adopted by Thomson Reuters and Elsevier. Proposed in 2005 by Jorge Hirsch, the h-index represents both the number (h) of publications for an individual that have been cited at least h times. On the other hand, the origin of the impact factor goes back to 1963, when Eugene Garfield and Irving Sher proposed to evaluate the influence of scientific journals through based on the number of citations that journals have received in other quality journals in a given period of time, initially the two years prior to the measurement of impact factor.
The most novel aspect of the h-index in comparison to the impact factor is that h takes into account a researcher’s productivity and peer appreciation. Among the negative considerations towards the h-index are that it does not consider the citations received by articles with a number of citations exceeding h; that it does not differentiate between emerging and decaying careers in research; and that it promotes mass producers of articles to the detriment of selective producers of few highly-cited texts.
To help correct these distortions so-called context indicators have been developed. The most commonly used context indicators, in addition to the median, are: the h5-index, which refers to the citations received by a researcher’s articles in the past five full calendar years, and the i10-index, which refers to a scholar’s number of articles that have received, each, at least ten citations.
1.3. Profiles in Google Scholar
This study quantitatively examines the dissemination and impact of the activity of communication researchers in the past four decades, based on the number of works they have published and the number of citations they have received, according to the information they have shared in their public profiles in Google Scholar Metrics (GSM). Our universe of study is the largest communication researchers association in Spain: the Asociación Española de Investigación de la Comunicación (hence, AE-IC, according to its initials in Spanish). The use of this universe of study derived from the refusal of the Spanish Ministry of Education to provide us with a list of communication professors working at Spanish universities, even when it should be transparent about the number of professors working in public schools.
The relevance of the object of study of this research is reinforced by the fact that all the scholars requesting funding from the Spanish National R&D Programme (focused on the “Challenges of Society” programme promoted by the Ministry of Economy and Competitiveness in September 2014) are asked to provide their h-index in their applications; and by the inclusion of Jorge Hirsch as speaker at the 11th International Forum on the Evaluation of the Quality of Research and Higher Education (FECIES) carried out in July 2014 in Bilbao by the Universities of Deusto and Granada, the National Agency for Quality Assessment and Accreditation of Spain (ANECA), the Spanish Foundation for Science and Technology (FECYT), the National Commission the Evaluation of Research Activity (CNEAI), the Conference of Chancellors of Spanish Universities (CRUE), Thomson Reuters and Elsevier, among other collaborators.
Google Scholar was launched in April 2012 and offered h5-index values and rankings of journals by language. In November 2012, Google Scholar fixed some bugs and offered rankings by thematic area and discipline for journals in English, and then updated its data in mid-2013 and 2014. Google Scholar also allows researchers to create individual and group profiles, where they can sort and group together publications, improve the visibility and impact of the works, manage and update their curriculum vitae, develop individual bibliometric indicators, find out who cites who, receive alerts, and follow other researchers. Researchers can edit the title and authors of the articles but cannot modify or edit citation. Google Scholar, however, does not exclude self-citations when calculating the h-index.
GSM profiles allow scholars to collect all of the academic production available online with direct links to these documents and to manually add references of publications that are not available on the Internet. To create a profile in GSM scholars have to identify the research areas to which they are freely attached. By selecting an area the scholar becomes part of a small network and can see all other scholars (arranged in decreasing order according to their number of citations) who have also selected the same areas of interest.
A GSM profile works as self-managed individual repository since it connects directly with documents that cite a publication listed in it, adds citations when Google’s web crawlers locate them, and allows automatic and previously-authorised content updates. It also allows users to set up alerts about the impact of their new own articles or articles created by the scholars the follow.
In terms of statistic indicators, the GSM profile provides users with his/her h-index, the number of received citations, and global i10-index. In another column, the profile provides the same information, but limited to the past five full calendar years. It also provides graphs of the annual evolution of the scholar’s impact, which since the 2014 update cannot show before the year 2000.
This mid-2014 update adjusted the aesthetics and format of the profile in order to facilitating its viewing and managing in mobile devices, but did not introduce significant changes to its capabilities. The value of the Google Scholar profile as a visibility platform, as well as collector of the academic and scientific production, lies in the fact that it is the first option that appears in Google Scholar’s search box whenever someone searches for the name of a researcher who has created a profile on GSM.
Google’s proposal would just be another search service aiming to create different groups of users, in this case those belonging to the international community of university researchers, if it were not for the fact it offers individual bibliometric indicators and breaks in this way in a field that seemed to be reserved exclusively worldwide to Thomson Reuters and Elsevier (two multinationals that have bibliometric data at the centre of their business). Google Scholar is also novel because, besides breaking in full in the field of bibliometrics with open and free indicators, it bets on the h-index and challenges the dominance of the impact factor by creating and publishing its own rankings of journals based on their h-index and classified by language of the edition and by areas of knowledge.
On its short life, Google Scholar has been welcomed with controversy because it does not reveals the criteria it uses to create the lists of areas and disciplines to assemble journals; because it does not reveal in how many areas a journal has been included; because it does not take into account the platform in which the citation is made; and because it validates all the citations that its engines detect.GSMis “an immature product, which presents several short-coming in its current configuration for evaluating of scientific journals” but also “a prodigious mine of scientific information [...] especially useful for scientists from the Humanities and the Social sciences because it controls like nobody else has done it before non-English literature and literature transmitted by means other than scientific journals, which is the literature worst controlled by the dominant information systems in the academic world” (Delgado et alt., 2012: 4). Qualified as “low cost” tools in bibliometrics (Martin-Martin et alt., 2014), Google resources are not only the basis for the EC3 (Evaluation of Science and Scientific Communication) group to measure the impact of scientific journals in Social Sciences and Law, but also constitutes an interesting centre of analysis to determine the usefulness of the volume of documents it contains (Orduña-Malea et alt., 2014).
1.4. H-index in the field of Communication
The interest on and knowledge of the h-index in the area of communication in Spain are still scarce. The interest in this index increased significantly in 2012 when Google Scholar Metrics was launched and became a direct competitor for Web of Knowledge (WoK, Thomson Reuters) and Scopus (Elsevier) as it offered its own bibliometric measures for journals and offered scholars the possibility to create their profiles and calculate their individual h indicators, as a way of personal bibliometrics.
In scientific forums of the area of communication (with the exception of the conferences organised by journal editors) the first reference to Google Scholar Metrics was made in December 2012 in the 4th International Conference of the Latina Social Communication (“IV Congreso Internacional Latina de Comunicación Social”), in the communication paper written by Sánchez Pita, titled “Update of the h-index for journals of communication, according to Google Scholar Metrics” (Actualización del Índice H de las revistas de Comunicación, según Google Scholar Metrics). The first discussions on the h-index in scientific meetings of communication did not occur until May 2013, in the round table on Research Methods of the “2nd Conference of the Spanish Association of communication research (AE-IC)”, carried out by the University of Valladolid in early May 2013 (Túñez and De Pablos, 2013).
The h-index is not a valid indicator to make comparisons between different areas of knowledge, but it is considered viable to calculate a reference index, or hR (Imperial and Rodríguez-Navarro, 2007). Taking as a basis the impact factor of all journals included in the Impact Index of the Spanish Journals of Social Sciences(known as In-recs) in 2011, the h-index of referencein the area of communication in Spain would be 17 (Túñez, 2013). In GSM’s own statistics for 2014, the highest h5-index among the journals listed in the communication “subcategory” of the area of “Humanities, Literature and Arts” in English is 45 and is held by the New Media & Society journal (2009-2013), followed by the Journal of Communication which has an h5-index of 38. The impact indicators are considerably lower among the journals of communication included in the top 100 scientific publications in Spanish in 2014. The highest impact indicator is held by Comunicar, which occupies the 4th place with an h5-index of 19, followed by El Profesional de la Información, in 9th place with an h5-index of18; Revista Latina de Comunicación Social, in 25th place with an h5-index of 15; Pixel-Bit.Revista de Medios y Educación, in 41st place with an h5-index of13; and Telos, in 98th place with an h5-index of10.
These h indicators match those offered by the Journal Ranking published by the EC3 group of the University of Granada in 2014, in an updated list of 413 journals from around the world, including four journals in Spanish in the first quartile: Comunicar (48th place with an h5-index of19), El Profesional de la Información (56th place with an h5-index of 18), Revista Latina de Comunicación Social (70th place with an h5-index of 15), and Pixel-Bit.Revista de Medios y Educación (91st place with an h5-index of 13) (Repiso and Delgado-López Cózar, 2014). According to the ranking produced by the EC3 of the Spanish scientific journals in the 2009-2013, based on the h5-index, in the area of Communication (Ayllon et al., 2014) the highest index of the journals is 18.
When proposing the formula to calculate the hR, Imperial and Rodriguez-Navarro (2007) warned that h does not apply, except for a few exceptions, to the Social Sciences and Humanities, because these areas lacked sufficiently extensive journals databases and because of the great weight books have in publication in these areas. However, after 6 years the Social Sciences and Humanities are characterised by an increase in the number of scientific publications, by a growing trend to use articles as references, and by the inclusion of article references as criteria in the admission of articles in some of the major journals in the area.
The change in platforms, the number of researchers and the dynamics of dissemination of research results are reflected also in recent studies on the productivity in the area of communication. The content of communication research has been well analysed by Martínez and Saperas (2011) based on a sample of 235 articles published in the journals Anàlisi, Comunicación y Sociedad, Estudios sobre el Mensaje Periodístico and Zer in the 1998-2007 period. These scholars conclude that communication research in Spain tends to focus on journalism and that in fact half of the sample of articles addresses this subject matter. The other half of the sample is addresses “the media and professional areas in which the activity communicative takes place: advertising, marketing, audiovisual communication, public relations, corporate communication and communication based on the new platforms based on digitisation (excluding journalism)” (Martínez and Saperas, 2011: 23).
In the analysis of communication research carried out in Spain from 1980 to 2010, Fernandez-Quijada and Masip confirmed the impact of the so-called ANECA effect, since in the 2004-2008 period production increased twice as much (2013, p 18). Fernandez-Quijada and Masip also pointed out that, although the majority of articles that are disseminated in national media are signed by a single scholar, in 2010 the average number of articles per article stood at 1.46, although in articles disseminated in international platforms the average was 3.23. In the analysed period, 43 journals of communication emerged in Spain while five disappeared.
In 2009, Nicolas Martínez warned us that ”just as it happened with the evolution of the communicative system in the second half of the past decade in the institutional context in which the Spanish scientific community of communication researchers operates, there is also an exacerbation of the trends initiated in the 1980s”. Nicolas Martínez quotes Moragas (2005) to contextualise his claim with quantitative data: “we have gone from that proliferation of schools to the current inflation of universities that offer communication studies. By the mid-1990s there were 20 universities, 20,000 students and 2000 professors, and ten years later there were 44 Spanish universities attending no less than 113 communication degrees (Moragas, 2005: 1)” (Martínez, 2009: 8).
Data provided by the Dissemination and Publishing Quality of the Spanish Journals of Humanities and Social Sciences and Law (DICE), which are used by the National Agency for Quality Assessment and Accreditation (ANECA) of Spain as indicators of quality of the Spanish publications in its evaluation of professors in 2014, included 52 publications of Communication of which 5 have disappeared and 6 have an online and print editions, which left us with a sample of 41 active journals. Our review of the literature on the use of books as platform for the dissemination of the communication research did not reveal any relevant studies.
The change in schools, the publication platforms, and the number of researchers coincides with the transformation of the modes of relating and communicating and with a shift in the use of impact indicators. The acceptance of the h-index as an indicator of quality of dissemination is based on its combination of productivity and impact. It also involves a reinterpretation of the value of the articles according to the publication platform: the impact factor is collective because it is an indicator of the journal and it is projected onto the articles that the journal includes; the h-index is individual and collective since it measures the acceptance of each published work through the citations it receives and it is collectivised when the article is evaluated as part of its author’s set of publications or as part of a journal (Túñez, 2013).
Given that the impact factor depends on the citations received by the articles of a publication, it would be logical for journals to undertake actions that promote the visibility of their contents among the scientific community of its area of knowledge; in the same way, given that the h-index is an indicator that applies to people and journals (or research groups), it would also be logical to undertake actions to promote the visibility of the published research to face what has been referred to as an incipient research marketing 2.0 (Túñez, 2012) because it responds to the need to plan new strategies of visibility that allow dealing with the dissemination of research results as an integral task of communication management. What varies is the attitude of the researcher.
This research study does not address qualitative or content-related aspects, although it takes into account the areas researchers are assigned to. The study has a quantitative approach that aims to examine communication researchers’ profiles to establish how many research articles are published, in what platforms is research published, and the impact of that dissemination in terms of citations. Our interest derives not only from the increasing protagonist role of the h-index but also from 1) our desire to establish whether proactive and prosumer researchers (those who have created a profile on Google Scholar) are only those who arrived to the area of communication as a result of the increase in communication schools and researchers in the last two decades and 2) our desire to make a transversal journey through the collective activity in communication research.
Since collective profiles can be created, it is realistic to explore the possibility of projecting a possible profile of the area in which the publications of its researchers are integrated.
The examination of the profiles of communication researchers in Google Scholar Metrics (GSM) was based on a methodological triangulation and quantitative and qualitative approaches. In addition to compilation of bibliography to establish the state of the art on h-index in the area of communication, we performed an analytical review of the web indicators of WoS, Scopus and GSM; an exploration of GSM’s statistics in 2012, 2013 and 2014 and identified and evaluated the open GSM profiles created by Spanish communication researchers. The review of the profiles was carried out in two stages: first in 2013, it was based on a sample of 273 individuals and, subsequently, in 2014, on the whole available universe of study: 683 researchers.
2.1. Technical data
For the development of that first stage of research, in early 2013 we requested the Ministry of Education, Culture and Sports a list of university professors working in the areas of Journalism, Audiovisual Communication and Advertising in order to identify and have a universe (N) of reference of researchers in the field of communication. The denied our request on the basis that they could only provide “information about university professors when this information is requested by the vice-chancellorship of the universities for the purposes of formation of evaluating committees”. In view of this response we opted to use the list of members of the Spanish Association of Communication Research (AE-IC), the largest of the current associations with 649 members according to the list published on its website in September 2013. Three duplicate entries were rejected and as a result the universe of study was composed of 646 researchers. A random sample was selected with a sampling error of 5% and a level of confidence of 97%, which was set on (n) 273 researchers (see table 2). The sample was selected by selecting from the list of members, in alphabetical order, one member every four positions. The frequency of four was randomly chosen to coincide with the number of letters included in the acronym for the Association (results in Túñez, 2014).
From these preliminary data and due to the low number of GSM profiles detected in the sample, in the first half of 2014 we repeated the content analysis but considering all the 683 members of the Association as the universe of study and by searching for researchers’ profiles by introducing several combination of their names and surnames. A total of 144 profiles were detected, but two profiles were discarded because they refer to people completely unrelated to the areas of communication. The sample became the totality of individuals with a GSM profile, but if we projected the 142 individuals with valid profile over the total number of individuals, the reliability of the sample would be maintained, with a heterogeneity of p=q=50, at a confidence level of 95% and a margin of error of 7%.
The sample of this research, comprised of 683 subjects, does not respond, therefore, to a random selection but to the totality of the available universe. The amplitude of this set of sample subjects reinforces the validity of the results which, although strictly speaking should not be considered projectable to the totality of communication researchers, can be considered as a framework of study as sufficiently broad and valid as to accept the representativeness of the results as descriptive indicators with acceptable levels of reliability. This study does not aim to provide exhaustive description of the activity in the area of communication, but in the absence of similar studies, it aims to provide a rigorous analysis of the dissemination declared by the 142 researchers and the bibliometric indicators provided in their GSMprofiles.
2.2. Research questions
The research design ruled out any premises derived from the first stage of the study to avoid bias resulting from the low number of profiles detected in this first approximation and was not guided by hypotheses. Instead, the descriptive study was guided five objectives and their corresponding research questions:
2.3. Data collection and analysis
Data was collected between April and June 2014, by means of an analysis sheet that collected the following data from the detected profiles: gender, areas of identification, h, h5, i10 and i10/5 indexes, the global number of citations, number of citations in the past five years, total number of publications for each scholar, year of the first publication of each scholar, editing of the profile’s sections, incorporation of photo, and number of followers (no longer offered in the new version of GSM).
The analysis of results was carried out between June and September 2014. During this period a database was created with all the works included in the 142 profiles, which contained a total of 7,118 works. The 2,359 works that had not received at least one citation were catalogued by identifying their title, dissemination platform, year of publication and number of citations received. During the detailed analysis we excluded several works because they were duplicated or had been erroneously assigned to the owner of the profile. The impact indicators of the journals of communication and the products that identify the individual h indicators by areas were available (such as the beta version of the ranking of researchers developed by the EC3 and available at http://hindexscholar.com). For the first time, we outlined a natural profile of the area to obtain collective indicators of the group of researchers, not of the dissemination platforms. As Hirsch argued in 2005 when proposing the h-index, if an indicator is applicable to the parts of a whole it applies to the whole.
The valid sample stood at 2,298 publications with citations that were addressed globally to characterise the collective profile of communication research and obtain the impact indicators h-index and h5-index. In this group of publications with citations we distinguished between publications in books and publications in journals or in the form of articles (the categories of journals and articles, which are treated separately by Google, were grouped together). Subsequently, we ordered the works by year of publication, and segmented the results by gender and in four categories: publications before 1980, 1980-1989, 1990-1999, 2000-2009 and 2010-2013. The publications listed as produced in 2014 were discarded because they belonged to an ongoing period.
3. Results and discussion
Based on the search for the profiles of more than 683 subjects considered as the universe, it can be concluded that 142 communication scholars have created a profile in GSM, so that the trajectory of two of every ten researchers (20.79%) are available, although only 7.04% of them has carried out the activities requested by the profile, i.e. to upload a picture, to check for errors in titles and names of authors, to unify the criteria for the identification of the name of the scholar in each work and the use of uppercase and lowercase letters.
A significant finding is that in terms of areas of dissemination by experts in communication, one out of three scholars (37.3%) do not include a personal picture in their profile or use an avatar (2.11%).
There are some scholars who erroneously included in their profile scientific works that have not been authored by them (but have cited their publications). This error can be caused by 1) the importing of groups of files (one of the options given by the profile) as Google encompasses all the works with references to the name of the profile’s owner, and b) by the activation the automatic update, recommended by GSM, which incorporates to the profile the works found by Google web crawlers and may include works whose author has the same as the profile’s owner.
3.2. What are the h and i10 indexes of communication researchers?
The analysis of data extracted from the 142 profiles reflect an average h-index of 4 (4.81), with extreme values of 0 and 26. The mode and the median both stood at 4.
Of the researchers, 66.2% are men. Their average h-index is 5 (5.26), their mode is 4 and 5 and their median is 5. Women represent 33.8% of the scholars with a public profile in GSM. Their average h-index is 3 (3.5), their mode is 3 and 4, and their median is 3.
Table 1. H-index and h5-index across gender groups
Nearly five in ten (46.5%) of those scholars with cited works have an i10-index of 0, i.e., none of their publications has received ten citations or more. As this indicator increases the percentage of scholars who achieve it progressively decreases: 17.6% has an i10-index of 1; 11.2% an i10-index of 2; and 9.9% an i10-index of 3. The analysis across gender groups generates very similar results, except in the group of scholars who have failed to achieve an i10-index, which constitutes half of the female researchers and three tenths of the male researchers.
Figure 1. Concentration of h-index according to the year in which the research career started
3.3. What is the h-index of the communication research of the last five years?
The analysis of the recent research activity reflects that the impacts of the past five years (2009-2013) are concentrated in works that have obtained between one and four citations, a ratio where the h5-index outperforms the h-index.
Table 2. i10 and i10/5 indexes of the past five years, across gender groups
The results obtained for both indicators are not statistically different, which seems to reveal a concentration of the dissemination activity and of the impact that accumulates progressively as the research career progresses and that coincides with the period of multiplication of the dissemination platforms, with the increase in scientific journals of communication and with the consequent increase in the use of the article as a vehicle for dissemination, in consequence, also as an impact platform.
Table 3. H-index expressed in ranges of impact (cumulative percentages)
Overall, four of every ten profiles (42.25%) have an h-index equal to or less than 3. In eight out of every ten (80.23%) profiles the h-index did not exceed 6. Researchers with an h-index greater than 9 constitute only 9% of the sample. The situation is similar across gender groups, although the difference during the first range is significant. An h-index equal to or less than 3 is achieved by 36.17% of men and 54.17% of women.
Figure 2. Concentration of h5-index according to the year in which the research career started
As detailed in table 2, almost five out of ten researchers (46.9%) with cited works have an i10-index of 0 for the 2009-2013 period. This indicator reflects a concentration of citations in recent years since the most common i10-index is of 1, in 22.5% of researchers, with similar impact on men and women.
Table 4. Detailed summary of the h5 and i10 indexes, across gender groups
Only one of every ten researchers has an i10-index greater than 5 for the last five years. The figures for h5 represent a 5-point increase both in the number of scholars who have failed to achieve it (i10-index= 0) and in the number of scholars who have an i10-index of 1, with respect to the non-temporal measurement reflected by the h-index (see table 4).
The results provided by GSM in the 142 public profiles are presented in table 4, which presents the h-index and h5-index of 0 to 9 and groups together those indicators that are equal to or exceed 10, because the h indicators greater than 9 and, especially in the case of h5-index and i10-index, are very scattered. Figure 3 presents the values achieved for each indicator and shows the concentration of dissemination and citations in the last five years (red area) over the overall academic career of each researcher (blue area).
Figure 3. Overlapping of the h and h5 indexes of the sample of communication researchers
3.4. Can a global h-index be calculated for communication research?
The creation of a database with more than 2300 articles included in the 142 profiles detected in GSM allows us to calculate a joint h indicator for the totality of the publications and, as it is explained in the analysis of the production in books and journals, even quantify the h impact of publications across platforms.
The cited articles that were compiled by way of group profile reflect the production and the overall impact of communication research, beyond the individual indicators of researchers or journals. The results must be interpreted with caution because the profiles can include publications erroneously assigned to the profile’s owner and because GSM’s indicators do not allow the exclusion of self-citations.
In any case, this group profile constitutes a new look at the descriptive parameters of activity of the group of communication researchers, in which the h indicators are calculated based on the most cited works of each scholar so that, when considered as a whole, the communication publications reach an h-index of 56. Taking into account only those publications published in the 2009-2013 period, the h5-index would be 34.
Table 5. H-index and h5-index to create a holistic general profile for communication researchers
3.3. How many publications are cited?
Figure 4. Cited publications included in the GSM profiles
To examine the impact of the works included in the individual profiles we summed up the number of citations and created a new database that incorporated all the cited works’ titles, number of citations, dissemination platform, and the category with which their authors present them. In general terms there is greater dissemination of research results in the article format than in books. However, books are the platform that accumulates the highest number of citations.
This detailed analysis of the cited works reveals a striking fact: two of every three texts included by researchers in their public profiles (66.86%) have not been cited at all. With regards to this situation in relation to the whole of the communication area, there is a proliferation of texts that do not get any endorsement from other members of the community of communication researchers despite GSM accepts all citations regardless of their origin.
3.6. What is the average number of citations received per publication?
The distribution of citations across the nearly 7,200 analysed texts reflects a pyramidal type of impact. As mentioned 66.86%, of the texts in the sample have not received a single citation (4.759 works), while 21.2% has received between 1 and 5 citations; 5.29% between 6 and 10 citations; 2.37% between 11 and 15 citations. The percentages of works that have received larger numbers of citations are few small (table 6).
Table 6. Number of citations received by the publications listed in the GSM profiles
Focusing on the cited publications included in the profiles, 22.11% of them have only received one citation, while 64.05% have received between one and five references; 16% between six a 10 citations; 7.1% between 11 and 15 citations; and only 2.8% between 16 and 20 citations.
From there, citations are concentrated on a small number of very-referenced works. Eight out of ten works do not have more than ten citations, and only one of every ten articles, books or conference papers have more than twenty citations. The number of works with more than 100 citations represents only 0.56% of the sample. Only 0.14% of all the works (books, articles, conference paper, etc.) included in the profiles has received more than 300 citations.
Table 7. Distribution of cited publications by number of received citations
Examining the production of all profiles as a whole, only one of every ten publications (9.92%) achieved an impact greater than 20 citations, above the maximum h-index attributed to the journal of communication best positioned on Google’s listings (Repiso and Delgado-López-Cózar, 2014; Ayllón et alt., 2014).
3.7. What platform has the widest dissemination and greatest impact? Is it books or articles?
The most common dissemination platform among the cited works listed by scholars in their profiles is the article (67.12%); followed by the book (25.8%), and the conference papers (4.26%). Doctoral theses and other platforms are hardly listed.
Table 8. Dissemination platform of the cited works
Quantitatively, articles beat books in number of products and number of citations received. When related the quantity of works produced in each platform with the number of received citations the ratio is more favourable for journals than books
The h-index of the communication area would be 34 according only to the citations achieved by works disseminated in the books platform, and 44 according to the citations received by articles. The h5-index of the communication area would be 13 if it was based only on the books included in the GSM profiles of communication researchers, and 20 if it was based only on the journals.
The comparison of both values reflects that the production of the last five years has been greater and that the citations received by articles outweigh those received by books. The average number of citations received per text is 3.31 among the works included in the profiles. On the other hand, the average number of citations received by the publications in the 2009-2013 period reflects again the recent increase in production and impact: 5.96 citations per work.
If we only take into account those works that have been published between 2009 and 2013, the average number of citations received per work was 4.83 among books and 5.08 among articles. In those five years, the percentage of works that received a single citation stood at 29.38% among and at 24.56% among articles.
Table 9. Citation averages for books and articles
3.8. How has the use of books and journals as dissemination platforms changed in recent decades?
More than 2,300 entries were grouped in decades and examined in order to see how the profiles reflected the evolution over time of each dissemination platform and their impact. In absolute terms, the analysis confirms that the book is currently a live platform that has had a significant growth over the last 40 years: the number of books included in the profiles and published in the 2000-2009 decade is almost four times larger than the number of books published in the previous 10 years, from 1900 to 1999. The trend seems to continue in the current decade according to the data on books published from 2010 to 2013.
Regarding the use of journals, the multiplication of their presence as a dissemination platform is much greater. The number of articles published in the 2000-2009 decade is four times larger than that published in the previous decade. Moreover, the number of articles published in from 2010 to 2013 already constitutes half of the number of articles published throughout the previous decade.
Table 10. Evolution of the dissemination platforms that have received citations
In any case, it is very enlightening to see how the disseminated of research works in books and journals changes over time. Before 1980, of every ten cited publications six were a book (63.63%) and three an article (36.36%). After 2010, of every ten cited publications three are books (22.86%) and seven are articles (71.72%). It should be noted that, as already pointed out by numerous studies, between these two periods there was a decade during which there was a proliferation of communication schools accompanied by an increase in the number of communication professors-researchers subjected to the ANECA effect.
3.9. Has the use of books and journals as cited sources changed in recent decades?
In order to determine whether changes in the use of dissemination platforms only involved variations in productivity across platforms or also changes in the impact of platforms, we analysed the books, articles, conference papers and theses that had been included in the profiles and had received citations. The analysis shows that, as mentioned, two of every three published texts fail to receive a single citation but also that those works that do receive citations maintain a progressive accumulation of citations over time.
The analysis by periods of times shows a clear downward trend in the citation of books and an upward trend in the citations of articles. In any case, the data on the impact of books are very significant if we take into account that the criteria adopted by public institutions to evaluate the work of researchers tend to favour articles in scientific journals.
Table 11. Evolution of citations across dissemination platforms
Table 9 and figure 5 show how the book has been losing prominence as a source of reference cited by other scholars, which is the opposite of what is happening with the scientific works published in journals. The analysis of more than 30 years shows that conference papers have slightly increased their presence as sources of reference cited in other works, but also that this presence is almost symbolic as it does not even reach 2% while the increase in twenty years is almost imperceptible (0.5%).
Something similar happens with doctoral theses, which do not have a significant impact in the area of communication. They only receive one of every 300 citations (0.32%) and this is a situation that has changed very little over more than three decades, although their percentage of impact has doubled, going from 0.15 in the 1990s to 0.32% in the current decade.
Figure 5. Comparative evolution of the impact (citations) reached across dissemination platforms (percentage)
However, if we look at the accumulation of impact over time, the works that concentrate the greatest number of citations are books. In order to understand not only the distribution of citations but also the concentration of citations, we isolated the works that had received one hundred or more citations from other researchers: eight out of ten works (84.6%) are books.
Table 12. Distribution of the dissemination platforms of the works that have accumulated one hundred or more citations each
Figure 6. Distribution of the dissemination platforms of the works that have accumulated one hundred or more citations each
3.10. How has the area of communication research evolved since it began?
To answer this question, in each profile we examined the year of the first publication and the gender of the scholar in order to understand the evolution of these variables throughout the more than three decades covered in this study. The result is a constant and increasing incorporation of researchers, although in all periods of time the number of men with a public profile is larger than that of women. This analysis also confirms the increase in the number of researchers that began their activity in the 1990s and the first decade of the 21st century.
Table 13. Evolution of impact across gender groups
Research production has been on the rise over the years. According to the information included in the profiles, the publications published before 1980 represent 9.15% of all the scientific and academic production. Of all the published research, 9.86% was published from 1980 to 1989; 31.69% was published in the 1990-1999 period; and 40.14% between 2000 and 2009. The works published from 2010 to 2013 constitute the remaining 7.04%. The publication year is not stated for 2.11% of the works listed in the profiles.
The results of this research are based in the publications incorporated by researchers to their GSM profiles (although they could include erroneously assigned works) and the impact indicators offered by Google (which include citations made in any platform and do not exclude self-citations). In an exercise of rigour, it must be stressed that the data provided must interpreted with a downward trend that does not diminish the results because they are presented as broad descriptions and indicators of trends on the dynamics of dissemination and the levels of impact of the research carried out in the field of communication.
The analysis of the sample indicates a low presence of communication researchers in Google Scholar Metrics, since 79.2% of the 683 researchers did not seem to have a public profile in the most-visited scientific network in the world. Moreover, almost 93% of the 142 profiles found in GSM (20.79% of the total) were not properly completed in the sections requesting the basic details of the scholar and in the sections requesting the identification data of their works (e.g. they included duplicate citations and unlinked entries of the same article). There is a desire to be part of GSM and publicise the research activity and increase its impact, but the image of the researcher is not properly and the information of his/her works is not properly managed in the profiles, which is striking in the area of communication. In fact, only 7.04% of the public profiles were fully completed.
In the profiles under review, the h-index of researchers ranges from 0 to 26, with an average of 4 (4.81) and mode of 4 and 5. The averageh5-index stands at 4 (4.01) and the mode at 4. Almost half of the profiles with citations have an i10-index of 0. The collective h-index of the totality of cited research works included in the 142 profiles is 56. Meanwhile, the h5-index forthe 2009-2013 period is 34. In terms of differences across gender groups, among male researchers the highest h-index is 26 and the highest h5-index is 18, while among their female counterparts the highest h-index and h5 index are 13 and 11, respectively.
The analysis of the works published in books and journals shows that there is a preference in favour of the article. In profiles with publications made only in books the h-index is 34 and the h5-index is 13, while in profiles with publications made only in journals the h-index is 44 and the h5-index is 20. Data are not directly comparable because in this study the totality of researchers is taken as an homogeneous group (Hirsch has already explained in 2005 that if the h-index is applicable to the part of a whole it is also applicable to the whole) but the only contextualising references of the collective indicators are the rankings produced by GSM or the EC3 based on data provided by Google, although both analyse research production by platform/journal.
The referential impact in the area of communication is 17 according to the formula developed by Imperial and Navarro (2007) based on constants corrected according to the impact factor of the major journals (Túñez, 2013). GSM’s own statistics indicate that the highest h5-index among journals of communication in English, in the 2009-2013 period, is 45 and belongs to New Media & Society. Among the communication journals included in the 2014 top 100 publications in Spanish the highest h-index is 19. This h-index coincides with that of the Journal Ranking developed by EC3 with 413 journals from around the world, including four in Spanish in the first quartile: Comunicar, El Profesional de la Informacion, Revista Latina de Comunicación Social and Pixel-Bit.Revista de Medios y Educación (Repiso and Delgado-López Cózar, 2014).
Of the analysed sample of profiles, the scholars with an h-index equal to or greater than 3 only constitute 42.25%, while those with an h-index equal to or greater than 9 represent only 9.1%. The comparison of the h5 and h indexes shows that the citations have been accumulated in recent years. The average values for both indexes are similar (h-index=4.8 and h5-index=4.01), which suggests an impact concentrated on the recent dissemination activity, since the h5-index only takes into account the citations received in the last five full calendar years.
The overall results indicate that there are three very different groups: a multitude of works with few citations, an intermediate group of works without hardly any citations, and a group of few works that concentrate a large number of citations, most of them received in recent years (figures 1 and 2). The results suggests a very diversified production and impact in which only one of every 10 works has an h-index higher than 10 and only three of every 100 publications have 20 or more citations, which places them above the highest indicators among scientific journals of communication.
The interpretation of these observations about the publications that have received citations must take into consideration that, in the area of communication, two of every three published works, regardless of whether they adopt the format of books, articles, theses or conference papers, do not receive recognition from the rest of researchers. In other words, they receive zero citations. It is striking that the published works that have received no citations are twice as many as those that do receive citations, even though there are not referential studies from other areas to be able to properly contextualise these data. In any case, GSM’s statistics indicate a high volume of production and a low level of impact despite the fact that Google web crawlers take into account all citations regardless of the platform in which they are made.
The representativeness of the descriptive results of this study is reinforced by the large size of the sample (683 subjects) and the alignment of these results with those generated by studies on the evolution of the research activity in communication in recent years, which reflect a significant increase in the number of researchers and dissemination actions in the 1990s and the first decade of the 21st century.
In terms of differences across dissemination platforms, the use of articles as dissemination format has experienced a significant growth, although the book has also slightly increased its presence as a dissemination vehicle. The analysis of more than forty years of research production reveals a clear reversal of roles in the impact of books and journals as dissemination platforms: before 1980, of each five citations four (80.5%) were given to books and one (19.5%) to journals, but after 2010, of each five citations almost four (73.9%) are given to journals and one (22.78%) to books
The average number of citations is 11.67 for books and 9.45 for articles. However, if we focus on the works published between 2009 and 2013 the average number of citations for book is 4.83 and 5.08 for articles, which is slightly higher. In these five years, the percentage of works that have received just one citation is 29.38% among books and 24.56% among articles.
These average values are less indicative of the weight of each platform than the h values. Grouped together as a single profile, the works published in journals would have an h-index of44 and an h5-index of 20. On the other hand, grouped together as a single profile, the works published in books would have an h-index of 34 and an h5-index of 13.
Scientific networks, applications, or online dissemination platforms are creating new spaces of interrelation for researchers and promote new opportunities to increase the visibility of the disseminated research as part of the management of results and research projects that goes beyond the traditional stages of development of a project and includes the management of the communication of the published work as new responsibility shared by scholars and publishers.
The use of the h and h5 indexes as bibliometric indicators in GSM profiles individualises the indicators and makes visible the scientific production. The public profile is the first option in the list of links offered by Google Scholar after someone searches for the name of a scholar. The article or the book cease to be ultimate goal (dissemination) to become products that need to be visible in order to increase their chances to be cited (management of disseminated works). In other words, the ultimate goal is no longer to publish but to manage the visibility of the published products in order to reach the target audiences to increase their chances to be valued in the incipient research marketing.
DE Acuna, S Allesina and KP Kording (2012): “Future impact: Predicting scientific success”. Nature, vol. 489, p. 201-202 doi: 10.1038/489201a.
R Aleixandre-Benavent, JC Valderrama-Zurián and G González-Alcaide (2012): “Journal Scholar: Una alternativa internacional, gratuita y de libre acceso para medir el impacto de las revistas de Arte, Humanidades y Ciencias Sociales”, EC3 Working Papers, 5.
JM Ayllón, A Martín-Martín, E Orduña-Malea, R Ruiz Pérez and E Delgado López-Cózar (2014): “Índice H de las revistas científicas españolas según Google Scholar Metrics (2009-2013)”. EC3 Reports, 17. Granada, 28 July 2014.
L Bornmann and HD Daniel (2007): “What do we know about the h Index?”. En Journal of the American Society for Information Science and Technology, vol. 58 (9), p. 1381-1385.
R Cotas y M Bordons (2007): “Una visión crítica del índice h: algunas consideraciones derivadas de su aplicación práctica”. El profesional de la información, vol. 16 (5), p. 427-432. http://www.elprofesionaldelainformacion.com/contenidos/2007/septiembre/04.pdf
M de Moragas i Spa (2005): “Investigación de la comunicación y política científica en España”. Ponencia presentada en la Reunión Científica de la Sociedad Española de Periodística (SEP). Santiago de Compostela, 27 y 28 de mayo de 2005.
JM de Pablos Coello, C Mateos Martín and M Túñez López (2013): “Google cambia el paradigma de la métrica científica”. Historia y Comunicación Social, vol. 18. Nº Especial diciembre, pp. 225-235.
L Egghe (2006): “Theory and practise of the g-index”. Scientometrics, vol. 69 (1), p. 131-162.
D Fernández-Quijada and P Masip (2013): “Tres décadas de investigación española en comunicación: hacia la mayoría de edad”. Comunicar nº41, v. XXI, p. 15-24. http://dx.doi.org/10.3916/C41-2013-01
JE Hirsch (2005): “An index to quantify an individual’s scientific output”. En Proceedings of the National Academy of Sciences of the United States of America, vol.102, p. 16569- 16572.
JE Hirsch (2012): “An index to quantify an individual’s scientific research output that takes into account the effect of multiple coauthorship”. En Scientometrics, vol. 85, p. 741-754.
J Imperial and A Rodríguez-Navarro (2005): “Utilidad del índice h de Hirsch para evaluar la investigación en España”. http://www.bit.etsia.upm.es/Imperial_Rodriguez-Navarro.pdf
J Imperial and A Rodríguez-Navarro (2007): “Usefulness of hirsch’s h-index to evaluate scientific research in Spain”. En Scientometrics, vol. 71 (2), p. 271-282.
P Kotler, H Kartajava and I Setiawan (2010): Marketing 3.0. Madrid: LID editorial. Tercera edición, 2013.
A Martín-Martín, JM Ayllón, E Orduña-Malea and E Delgado López-Cózar, E. (2014): “Google Scholar Metrics 2014: una herramienta bibliométrica de bajo coste”. Granada: EC3 Working Papers.
M Martínez Nicolás (2009): "La investigación sobre comunicación en España. Evolución histórica y retos actuales". Revista Latina de Comunicación Social, 64, pp. 1 a 14. La Laguna (Tenerife): Universidad de La Laguna.
M Martínez-Nicolás and E Saperas-Lapiedra (2011): "La investigación sobre Comunicación en España (1998-2007): Análisis de los artículos publicados en revistas científicas". Revista Latina de Comunicación Social, 66. La Laguna (Tenerife): Universidad de La Laguna, páginas 101. 129
E Orduña-Malea, JM Ayllón, A Martín-Martín and E Delgado López-Cózar (2014): “About the size of Google Scholar: playing the numbers”. Granada: EC3 Working Papers.
R Repiso Caballero and E Delgado López-Cozar (2013): “H Index Communication Journals according to Google Scholar Metrics (2008-2012)”. EC3 Reports.
A Rodríguez Navarro and J Imperial Ródenas (2007). Guía para la evaluación española en ciencia y tecnología utilizando el índice h. Consejería de Educación, Comunidad de Madrid: Madrid. http://www.madridiario.es/imagenesPieza/libor_madridmasd.pdf
F Sánchez Pita (2012): “Actualización del Índice H de las revistas de Comunicación, según Google Scholar Metrics”. Actas IV Congreso Internacional Latina de Comunicación Social: La Laguna, http://www.revistalatinacs.org/12SLCS/2012_actas/209_Pita.pdf [04-09-2013]
LC Silva Ayçaguer (2012): “El Índice-h y Google académico: una simbiosis cienciométrica inclusiva”. Acimed, vol. 23 (3), p. 308-322.
D Torres-Salinas, E Ruiz-Pérez and E Delgado López-Cozar (2009): “Google Scholar como herramienta para la evaluación científica”. El profesional de la información, vol. 18, p. 501-510. http://ec3.ugr.es/publicaciones/d700h04j123154rr.pdf
M Túñez (2014): “Perfiles de comunicación en Google Scholar Metrics, índice h y nuevas estrategias de difusión de la investigación”. Historia y Comunicación Social. Vol. 19. Especial marzo, pp. 15-26.
M Túñez López (2013): “El ‘índice h’ de la investigación en Comunicación en España, Portugal y Latinoamérica: Web of Knowledge (Wok), Scopus y Google Scholar”. En Communication&Society/Comunicación y Sociedad, Vol. 26 (4), p. 53-75.
M Túñez López (2012): La gestión de la comunicación en las organizaciones. Sevilla/Zamora: Comunicación Social Ediciones.
M Túñez y JM de Pablos (2013): “El ‘índice h’ en las estrategias de visibilidad, posicionamiento y medición de impacto de artículos y revistas de investigación”. Actas del II Congreso Nacional sobre Metodología de la Investigación en Comunicación y del Simposio Internacional sobre Política Científica en Comunicación. Universidad de Valladolid: Segovia, pp. 133-150, http://uvadoc.uva.es/handle/10324/2996
M Túñez López and Y Martínez Solana (2013): “El nuevo mapa de los estudios de Documentación y Comunicación en España: oferta de grados, índices de matrícula, tasas de empleo y perfiles de contratación”. Anales de Documentación, 2013, vol. 16 (2). http://dx.doi.org/10.6018/analesdoc.16.2.168081
M Túñez-López, K Valarezo-González and I Marín-Gutiérrez (2014): “Impacto de la investigación y de los investigadores en comunicación en Latinoamérica: el índice h de las revistas científicas”. Palabra Clave 17 (3), 895-919.
B Velasco, JM Eiros, JM Pinilla and JA San Román (2012): “La utilización de los indicadores bibliométricos para evaluar la actividad investigadora”. Aula Abierta, vol. 2, p. 75-84.
How to cite this article in bibliographies / References
M Túñez López, MY Martínez Solana, KP Valarezo González (2014): “Analysis of the productivity, impact, and collective h-index of the communication research carried out in Spain based on the information shared by researchers in their individual Google Scholar profiles”. Revista Latina de Comunicación Social, 69, pp. 684 to 709.
Article received on 28 October 2014. Accepted on 30 October. Published on 23 December 2014.