research paper ads

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Behav Sci (Basel)

Consumer Behaviour to Be Considered in Advertising: A Systematic Analysis and Future Agenda

Ahmed h. alsharif.

1 Azman Hashim International Business School, Universiti Teknologi Malaysia, Skudai 81310, Johor, Malaysia

Nor Zafir Md Salleh

Shaymah ahmed al-zahrani.

2 Department of Economic & Finance, College of Business Administration, Taif University, Taif 21944, Saudi Arabia

Ahmad Khraiwish

3 Department of Marketing, Faculty of Business, Applied Science Private University (ASU), Amman 11931, Jordan

Associated Data

Not applicable.

In the past decade, neurophysiological and physiological tools have been used to explore consumer behaviour toward advertising. The studies into brain processes (e.g., emotions, motivation, reward, attention, perception, and memory) toward advertising are scant, and remain unclear in the academic literature. To fill the gap in the literature, this study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) protocol to extract relevant articles. It extracted and analysed 76 empirical articles from the Web of Science (WoS) database from 2009–2020. The findings revealed that the inferior frontal gyrus was associated with pleasure, while the middle temporal gyrus correlated with displeasure of advertising. Meanwhile, the right superior-temporal is related to high arousal and the right middle-frontal-gyrus is linked to low arousal toward advertisement campaigns. The right prefrontal-cortex (PFC) is correlated with withdrawal behaviour, and the left PFC is linked to approach behaviour. For the reward system, the ventral striatum has a main role in the reward system. It has also been found that perception is connected to the orbitofrontal cortex (OFC) and ventromedial (Vm) PFC. The study’s findings provide a profound overview of the importance of brain processes such as emotional processes, reward, motivation, cognitive processes, and perception in advertising campaigns such as commercial, social initiative, and public health.

1. Introduction

Self-report has been adopted in marketing activities to evaluate and identify consumer responses toward stimuli in the marketing sector, such as advertising practices. According to Carrington, et al. [ 1 ], the self-report methods reflect/measure the conscious responses (e.g., perception, approach/withdrawal attitudes, and thoughts) toward advertising. In fact, self-report cannot measure the unconscious or subconscious responses, which represent the majority of consumer responses, such as decision-making. Thus, researchers and marketers have adopted neuroscience tools such as electroencephalography (EEG) in the marketing field, to better understand the unconscious responses of consumers [ 2 , 3 , 4 ], which has led to an emerging new field, the so-called “Neuromarketing”. Smidts [ 5 ] was the first business researcher who coined the term “neuromarketing”, in 2002. Neuromarketing is a hybrid field containing numerous areas/fields such as psychology, marketing, and neuroscience [ 6 ]. According to Fortunato, et al. [ 7 ], the thanks for spreading this term was given to the Bright House Company when it created the first fMRI laboratory for marketing research.

In the hyper-competitive environment, neuromarketing is a mainstay in advertising because it has an opportunity to gauge consumers’ neural responses as emotional responses toward advertising; thereby, it is a revolutionary field for a better understanding of the subconscious and unconscious consumer responses. According to the literature, the rapid technological progress in neuroscience technology led to a better understanding of consumers’ behaviour in several contexts, such as, but not limited to, advertising [ 8 , 9 , 10 , 11 ]. Therefore, marketing and advertising leaders have used this technology to boost innovation and success in marketing and advertising, by controlling and minimizing task conflict [ 12 ]. According to Ramsoy [ 13 ], neuromarketing tools have been divided into four clusters, as follows: (1) neuroimaging tools such as electroencephalography (EEG), magnetoencephalography (MEG), functional magnetic resonance imaging (fMRI), positron emission tomography (PET), and transcranial magnetic stimulation (TMS); (2) physiological tools such as galvanic skin responses (GSR), eye-tracking (ET), electrocardiogram (ECG), and electromyography (EMG); (3) self-report methods such as surveys, observations, focus groups, and interviews; (4) behavioural measurements, such as the implicit association test (IAT). For instance, neuroimaging tools have been used to gauge emotions, attention, and memory regarding advertising [ 14 , 15 ]. At the same time, physiological tools have been used to gauge the physiological responses of consumers, such as, but not limited to, visual fixation in-store at the purchasing point, thereby providing valuable and fruitful insights into the attitudes of consumers (i.e., approach, withdrawal) [ 16 , 17 , 18 ]. Behavioural measurements are used to measure the reaction time of consumers toward stimuli, and self-reporting is used to measure the conscious behaviour of consumers toward stimuli such as approach/withdrawal attitudes [ 18 ].

According to the literature, the first fMRI investigation in neuromarketing was carried out by McClure, et al. [ 19 ], and largely contributed to the practical studies of neuromarketing [ 18 ]. Therefore, neuromarketing research has received attention from both academia and the industrial world as a means of filling the gap in traditional marketing methods and overcoming the limitations by reducing consumer social-bias (e.g., consumer choices can be affected by others) [ 7 ]. However, understanding the global trends in advertising research within the neuromarketing field is still unclear in academic studies. In addition, to date, no investigation has determined the current neurophysiological and physiological techniques that have been used in studying the unconscious/subconscious responses of consumers toward advertising such as YouTube video scenes, TV ads, public health ads (antismoking), initiative ads (encouraging the use of seat belts in cars). To sum that, this study tries to analyse the extracted articles in depth, to provide a precise and concise conclusion. The contributions of this work are summarized as follows:

• Provides a profound evaluation of the current advertising research that has been used to investigate unconscious and subconscious consumer behaviour, such as emotional dimensions, perceptions, reward processes, and approach/withdrawal motivation toward advertising.
• Provides an overview of the current neurophysiological and physiological tools that were used in advertising within the neuromarketing context between 2009 and 2020.

In this vein, the current paper provides an inclusive overview of neuromarketing research and the current research objectives. Section two presents the methodology and data-collection process. Section three presents the growth of the publication, topics of interest and a thematic analysis. A discussion of the study’s findings is presented in Section four. Section five presents a concise conclusion and the implications of our work. Finally, Section six presents the limitations and future directions.

2. Materials and Methods

This review study is designed to select empirical articles from the Web of Science (WoS) database in advertising within the neuromarketing context, to fill the gap in the literature. The reason behind choosing the WoS database over Scopus is the availability of cleaner data, which means reducing the duplications as compared to the Scopus database; additionally, WoS includes publications in top-tier journals [ 20 ]. The first step was to follow the instructions of the PRISMA protocol to select the empirical articles which used neuroimaging and physiological tools to investigate consumer responses to advertising research within neuromarketing [ 21 ]. The reason behind the use of the PRISMA protocol to select the relevant articles for this study were that it has been widely used in social science and business to extract and select articles related to the study, for example, online learning [ 22 ], neuromarketing [ 23 ], and service and healthcare [ 24 , 25 ].

For the second step, we used the content analysis of selected empirical articles for this study. The above processes will provide us with a profound insight into the advancement in advertising research by identifying and analysing the general and specific areas. Additionally, providing us with a better understanding of advertising research that used neuroimaging and physiological tools/methods and which can be considered when we are conducting further research into advertising research. Therefore, the findings of this study provide a guide for new scholars who are interested in advertising research within a neuromarketing field.

Relevant empirical articles have been selected by using the following query applied to the title, abstract, and keywords: (((“neuromarketing” OR “consumer neuroscience”) AND (“advertising” OR “advertisement”) AND (neuroimaging OR physiological))). This study extracted 76 empirical articles relevant to this review paper from January 2009 to December 2020. This study focused on empirical journal-articles, in comparison to conferences and book chapters, which generally undergo a much more rigorous review process and therefore improve the credibility of published research in journals [ 26 ]. Figure 1 demonstrates the selection of articles processed for this study, as follows:

• Methods: neuroimaging and physiological tools;
• Publication year: January 2009 to December 2020;
• Language: English;
• Document type: original articles (chapters of books, articles from conferences, reviews, and proceedings books were excluded).

PRISMA protocol to extract empirical articles for this systematic study.

3.1. Growth of the Publication

Seventy-six articles were extracted from the WOS which used neuromarketing tools. According to the literature, McClure, et al. [ 19 ] published the first neuromarketing study in 2004, when they investigated the neural correlates of consumer behaviour (e.g., choice, decision-making) toward two brands (Coca-Cola vs. Pepsi Cola). However, the first investigation into advertising was carried out by Morris, et al. [ 27 ]. They found the gyri regions of the brain were highly related to pleasure/displeasure and high/low arousal. From January 2009 to December 2020, there was a fluctuation in the number of published articles in advertising. In 2020, it was the highest number of annual publications with thirteen articles, as depicted in Figure 2 .

By exploring the relevant articles to develop this review article, it was possible to classify the global academic-research trends and advancement in advertising and neuromarketing, as follows: (i) neuroimaging and physiological tools used in advertising; and (ii) consumers’ brain processes to be considered in advertising, such as emotions, motivation, reward process, attentions, perception, and memory. By reviewing the selected articles, we can enrich our understanding, to achieve the objectives of this study. Table 1 shows the summary of the neuroimaging and physiological tools that were used to investigate the brain processes of consumers toward advertising research.

Classifications of neuroimaging and physiological tools used in advertising research.

3.2. Topics of Interest and Thematic Analysis

3.2.1. neuroimaging and physiological tools used in advertising.

We found that the neural-response recording-tool most used in advertising was EEG. EEG is an electrical and also a non-invasive technique to gauge the unconscious/subconscious responses of consumers toward ads by recording the voltage changes of frequencies at scalp regions [ 28 , 29 , 30 , 31 ]. According to [ 32 ], there are five frequency bands (e.g., delta, theta, alpha, beta, and gamma). The EEG tool uses a 10–20 global system, which is used to describe the electrode locations on the volunteer’s scalp, for example, (Fp), (F), (P), (O), (C), and (T) describe the prefrontal, frontal, parietal, occipital, central, and temporal, respectively. Moreover, it uses the same number of electrodes on both side of the volunteer’s head (i.e., right and left side) [ 33 , 34 ]. In addition, it has an excellent temporal accuracy (estimated in milliseconds (ms)) and a poor spatial accuracy (estimated at 1 cm 3 at the scalp regions) [ 35 , 36 , 37 ]. In addition, it is not expensive or noisy [ 38 ]. fMRI and fNIRS are non-invasive, metabolic tools. Both are used to record oxygenated and deoxygenated haemoglobin in the brain [ 39 , 40 ]. fMRI has a superior spatial accuracy (estimated at 1–10 mm 3 of the deep structure of the brain) compared with fNIRS, which has poor spatial accuracy (estimated at 4 cm of cortical-activity regions) [ 41 ]. Meanwhile, both have acceptable temporal accuracy (estimated in seconds) [ 42 , 43 ]. fMRI and fNIRS have been used in marketing research to record the subconscious/unconscious responses of consumers (e.g., preference, perceptions, purchase decisions, choices) toward marketing stimuli [ 41 , 43 , 44 ]. fNIRS is a portable, novel, promising, and silent neuroimaging technique, which is also cheaper than fMRI [ 39 , 45 , 46 ].

ET is a portable technique that is used to gauge physiological reactions such as eye movements, pupil dilation, saccade, and fixation toward the stimuli of marketing [ 18 ]. According to Hoffman [ 47 ], it is used for measuring eye movements and the attention of consumers, which is beneficial for psychology and neurological research. Eye fixations last between 200 and 350 ms while reading a text and watching video scenes, respectively, while 200 ms refers to the duration of saccadic eye-movements [ 48 ]. The set of fixations and saccades is named the scan route, and analyses visual perception and cognitive purpose [ 49 ]. Pupil dilation with a longer blink-period gives us better information about processing [ 18 ]. The GSR and ECG tools are used to gauge the emotional reactions of consumers toward ads [ 50 ]. For example, sweating level is recorded by the GSR tool and the heart rate/heartbeat is measured by the ECG tool [ 50 , 51 , 52 ]. In addition, both of them can measure the autonomic nervous system and evaluate the internal emotional status of consumers [ 53 ]. Therefore, GSR and ECG are convenient and reliable techniques for measuring the emotional status of consumers and changes in skin conductance, respectively [ 54 ]. IAT is able to identify the customers’ attitudes toward marketing stimuli such as brands or ads (e.g., like/dislike) by recording the reaction time of customers [ 18 ]. EMG is used to measure the reactions shown on individuals’ faces (e.g., pleasure/displeasure, …, etc.) toward advertising [ 55 ], because facial-expression analysis is significant for marketers and advertisers, because faces can provide beneficial information about perceptions of customers toward ads in terms of measuring visible and invisible facial-muscle reactions [ 29 ].

3.2.2. Brain Processes to Be Considered in Advertising

Emotion and feelings.

The feeling is a relatively conscious aspect of emotional status [ 56 ], which derives from individuals’ judgments such as level of pleasure or unpleasure toward advertising [ 31 ]; it is likely the best way to understand and explain the physiological responses of the consumer toward ads [ 31 , 57 ]. Many research studies have affirmed that ad-elicited feelings are strong indexes of consumers’ response toward advertising [ 58 ]. On the opposite side of the spectrum, emotion is an unconscious aspect of emotional status which correlates to an automatic somatic response such as increased heart beat in some conditions (fright, anger) [ 56 , 59 ], which is crucial for making decisions, learning, and solving problems [ 60 ]. Advertisers and marketers can use both in advertising to captivate consumers’ attention, thereby enhancing purchase intention. Emotions are accompanied with changes in the autonomic nervous system (i.e., zygomatic facial muscles, corrugator facial muscles, and heart-rate), which can provide rich information about the emotional status of consumers. Therefore, the study of emotions has attracted many researchers and advertisers [ 61 ].

Emotion is constructed from a neural network in the brain, which performs basic psychological activities/functions (e.g., memory, perception, salience detection) [ 62 ]. Therefore, emotion is defined as the set of changes in the individual’s physiological and subconscious and unconscious responses, based on the individual’s experiences [ 63 , 64 ]. In addition, emotion is a relationship between humans and the environment, including multiple subcomponents (e.g., physiological, behavioural, appraisal, and expression [ 65 ]. The cognitive and neurological frameworks of the role of emotion in decision-making has been investigated more through Damasio’s theory, which is also known as the somatic marker hypothesis [ 66 , 67 ]. Consequently, researchers have agreed on two dimensions for measuring emotional responses toward stimuli: (i) valence/balance (i.e., pleasure or displeasure, depression or excitement), (ii) arousal (e.g., high or low) ( Figure 3 ) [ 28 , 68 , 69 ]. Figure 3 shows that the valence indicates either positive or negative emotional-status which is evoked by external stimuli such as advertising. In addition, valence is measured from the positive to the negative side. On the other side of the spectrum, arousal indicates the level of excitement; whether high or low, it is measured from high to low levels [ 68 , 70 , 71 ].

Dimensions model of emotions [ 68 ].

Table 2 shows several methods to measure emotions and feelings toward marketing stimuli. Previous studies have used, for example, self-report and physiological methods to analyse the emotional responses of consumers toward advertising [ 72 ]. For example, the EMG and self-report investigation of Lajante, et al. [ 73 ] gauge the consumer’s pleasure/displeasure toward advertisements. The findings of the experiment revealed that pleasure/displeasure had positively impacted the attitudes of customers toward advertisements. The ECG, EDA, and questionnaire study of Baraybar-Fernández, et al. [ 50 ] found that audio and visual ads with sad messages have more influence on participants. Barquero-Pérez, et al. [ 53 ] used the ECG, EDA and questionnaire in their experiment. They found that each type of ad generated a different emotion, such as surprise, anger, and so forth. In addition, physiological tools have been used to investigate the effectiveness of online advertising (e.g., YouTube) [ 51 ]. For example, Guixeres, et al. [ 51 ] used brain response, ECG, and ET to investigate the relationship between ad effectiveness and the number of views on YouTube. They found that there is a solid relationship between ad effectiveness and the number of views on YouTube. Herrador, et al. [ 74 ] conducted an EDA experiment and the findings revealed that both groups of participants (male and female) indicated a strong initial activation; moreover, they noticed a reduced activation during the most significant part of video material in the male group. Venkatraman, et al. [ 28 ] used several neuromarketing tools to evaluate participant’ responses to a 30-second TV advertisement. The findings revealed that the activity in the ventral striatum could be the predictor of response to advertising. Grigaliunaite and Pileliene [ 75 ] conducted an experiment by using ET and they found that emotional and rational advertising appeals influence consumers’ responses in various ways. The IAT and ET investigation of Pileliene and Grigaliunaite [ 76 ] found that warm-colour temperature attracts more visual attention to the advertisement; thereby generating a positive implicit attitude and inducing the buying intentions toward the advertised product, compared with cool-colour temperature advertisements, whether the spokesperson is a female or male celebrity. Similarly, Grigaliunaite and Pileliene [ 77 ] found that negative smoking images reflected a negative implicit attitude/behaviour of individuals toward those images and smoking behaviour, increasing the influence on individuals’ intention to whether to quit or not to start smoking. The ERP, ET, and questionnaire investigation of Pileliene and Grigaliunaite [ 78 ] found that a well-known female spokesperson has a significant impact on the effectiveness of fast-moving consumer-goods advertising.

Neuromarketing tools to measure emotions and feelings.

There has been an interesting growth in understanding the non-verbal responses of emotional status toward advertising by using neuroscience methods such as fMRI, EEG, fNIRS [ 31 , 51 , 79 , 80 , 81 , 82 ]. For example, Plichta, et al. [ 45 ] conducted an fNIRS experiment to investigate the detection of sensory activity by measuring emotional signals in the auditory field. The findings revealed that pleasurable/unpleasurable sounds increased the activity in the auditory cortex, compared to neutral sounds. Gier, et al. [ 41 ] conducted the fNIRS experiment to explore whether the fNIRS tool had the ability to predict the success of point-of-sale elements by measuring the neural signals of brain regions such as the dlPFC. The findings revealed that the fNIRS has the ability to predict the success elements at the point of sale, relying on the cortical-activity effect.

The EEG investigations of Vecchiato, et al. [ 83 ], and Vecchiato, et al. [ 84 ] found that activity in the right frontal alpha is associated with pleasure/liking ads, while the left frontal alpha correlated to displeasure/disliking ads. Additionally, Vecchiato, et al. [ 85 ] found that there were gender (i.e., male and female) differences in interest toward commercial categories and scenes in two ads. The EEG experiment of Harris, et al. [ 86 ] found that emotional advertisements are more effective than rational advertisements, which leads to a positive change in decision-making, inducing donation, and liking. The findings of Chen, et al. [ 87 ] revealed that e-cigarette ads increased the smoking desire; additionally, e-cigarette ads increased activity in the left middle-frontal-gyrus, the right medial-frontal-gyrus, the right parahippocampus, the left insula, the left lingual gyrus/fusiform gyrus, the right inferior-parietal-lobule, the left posterior-cingulate, and the left angular-gyrus. Wang, et al. [ 88 ] and Royo González, et al. [ 89 ] found that the narrative approach of ads and exposure to branding products have a favourable influence on the consumers’ preferences and excitement. The fMRI investigations of Morris, et al. [ 27 ] and Shen and Morris [ 90 ] found that pleasure and displeasure are correlated with more activity in the inferior frontal- and middle temporal-gyrus, respectively, while low and high arousal is associated with the right superior-temporal- and right middle-frontal-gyrus, consecutively. Leanza [ 91 ] used EEG and found that some of the emotive features of the virtual reality (VR) experience significantly influenced consumers’ preferences. Ramsoy, et al. [ 92 ] found that arousal and cognitive load were highly connected to subsequently stated travel-preferences; moreover, consumers’ subconscious emotional and cognitive responses are not identical to subjective travel-preference. Shestyuk, et al. [ 93 ] found that the EEG is a convenient tool to predict the success of TV programs and determine cognitive processes. Wang, et al. [ 94 ] conducted an experiment to propose a generative-design method using EEG measurements. The findings revealed that the product image that was generated with preference EEG-signals had more preference than the product image generated without preference EEG-signals. Kim, et al. [ 95 ] conducted an experiment to identify the effect of visual art (e.g., Mondrian’s and Kandinsky’s artworks) on consumers’ preferences, by using EEG. The findings showed that the visual effects induced high emotional arousal, which might promote heuristic decision-making. Mengual-Recuerda, et al. [ 96 ] found that food served by a chef positively influences emotions, while dishes with special presentations attract more attention than traditional dishes. The EEG study of Eijlers, et al. [ 31 ] found that arousal is positively connected to prominent ads in the wider population and negatively to consumer attitudes toward these ads.

According to Lang and Bradley [ 97 ], emotions and motivation processes are highly intersected and correlated. Chiew and Braver [ 98 ] and Pessoa [ 99 ] found that cognition and consumers’ behaviours are highly affected by motivational processes. For example, positive motivational stimuli will urge individuals toward achieving goals (e.g., obtain or predict a reward by performing a task correctly) [ 100 ], while negative motivational stimuli can lead to distraction [ 101 ].

Pessoa [ 102 ] and Raymond [ 103 ] suggested that motivational processes are a compass of consumers’ attitudes toward external stimuli to engage with the environment and achieve goals. Higgins [ 104 ] suggested two dimensions for measuring motivational processes such as withdrawal and approach attitudes. Researchers and practitioners attempted to investigate the neural responses of motivational processes to better understand consumers’ behaviours toward advertisements and products [ 83 ]. For example, Cherubino, et al. [ 105 ] carried out an experiment using EEG to investigate the relationship between the PFC and motivational dimensions. The findings revealed that the PFC is related to motivational dimensions, wherein the right PFC correlated to withdrawal attitudes and the left PFC related to approach attitudes. The EEG investigations of Pozharliev, et al. [ 106 ] and Zhang, et al. [ 107 ] recorded the brain responses of subjects toward luxury products (motivations). The findings showed that social motivations have a vital role in influencing the purchase of luxury products in order to satisfy social goals (at least one goal). The EEG investigation of Bosshard, et al. [ 108 ] found that liked brands reflect more motivational aspects and activity signals in the right parietal-cortices than disliked brands.

Therefore, there is a strong relationship between the activation of the PFC and motivational dimensions toward marketing stimuli such as advertisements [ 109 ]. Therefore, marketing researchers and practitioners have to focus on the motivational processes of consumers to orient the marketing mix (e.g., target-appropriate audiences and markets, increasing the effectiveness of ads and products) [ 110 ]. According to previous studies, NM research has used the approach/withdrawal attitude to evaluate TV ads [ 111 ]. Therefore, approach/withdrawal motivational attitudes are highly significant for marketing and advertising research.

Reward Processing

According to the literature, it is highly significant for researchers and practitioners to study and know the neural responses that are responsible for reward processing, such as money, food, and social activities [ 112 , 113 , 114 , 115 ]. This is because the positive rewards such as gaining money, foods, or other types of rewards, enhance the accuracy and cognitive task [ 116 , 117 , 118 ] through modifying the early attentional process. Anderson, et al. [ 101 ] demonstrated that visual features (e.g., product design) that are correlated to reward, will make the consumer prioritize, therefore attracting the consumer’s attention automatically. For example, the design preference of a product or brand can increase the activity in the regions which are responsible for reward processing, thereby, causing more activation in regions of motivations that might impact consumers’ purchase decisions [ 29 ]. Many studies concentrated on the individual’s response toward a monetary reward by studying the approach/avoidance attitude [ 112 , 113 ]. For example, Bechara, et al. [ 119 ] carried out an experiment named the “Iowa Gambling Task” by using GSR to investigate the influence of reward on decision-making. They divided participants into two groups, the healthy group and the group with lesions in the vmPFC. The findings revealed that healthy participants sweated more, which led them to infer that participants had a negative emotional experience toward picking up cards from a monetary losing deck; meanwhile, the lesion group picked up cards regardless of whether they were cards with monetary wins or losses. Consequently, reward highly influenced decision-making [ 112 , 120 , 121 ].

Many researchers have confirmed that the importance of the striatum activity in reward processing, wherein the components of the striatum such as the caudate nucleus, nucleus accumbens (NAcc), and putamen play a central role in expectation and evaluation of reward [ 115 , 122 , 123 ]. For example, Galvan [ 124 ] and Geier, et al. [ 125 ] carried out an experiment to investigate the relationship between reward processing and the striatum. Their findings revealed that the ventral striatum (VS) has a key role in the prediction of reward. Jung, et al. [ 126 ] found that the rewards, memory, semantics, and attention regions in the brain were lit up when viewing a combination of a celebrity face and a car, compared with viewing a combination of a non-celebrity face and a car. In addition, car favourableness correlated positively with activation in the left anterior-insula, left OFC, and left higher-order visual cortex in the OL. Padmanabhan, et al. [ 127 ] investigated the influence of the reward system on attention processes. Their findings showed that incentives facilitate cognitive control. Previous neuroimaging studies demonstrated that rewards activate the ventral medial prefrontal cortex (vmPFC) and ventral striatum [ 128 , 129 , 130 ]. The ventral striatum has been discussed before as a part of the reward system [ 131 ]. Therefore, findings suggest that neurodevelopmental changes in the striatum systems may contribute to changes in how reward influences attentional processes [ 56 ].

Attention is defined as the way “people tend to seek, accept and consume the messages that meet their interests, beliefs, values, expectations and ideas, and overlook the messages that are incompatible with this system” [ 132 ]. It has also been defined as selective perception [ 133 ]. Selective perception is associated with filtering out information and concentrating on significant information (e.g., different aspects of stimulus or different stimuli) [ 134 ]. For instance, consumers are exposed to nearly 10 million bits of visual information (e.g., ads, images, sound, video, and colour) per second through their senses (e.g., eyes, ears, skin) daily. Most input information goes by unnoticed, with consumers able to process almost 40 bits of input information per second [ 29 , 135 ]. This lead us to infer that attention has a strong influence on consumer behaviour in how consumers represent, perceive and process information and thus select and prioritize information. Attentional and emotional processes are relatively intersected/connected, and emotion is considered a reliable and effective source for attracting consumers ’ attention [ 136 , 137 ]. For example, the activation in the amygdala (AMY) and cingulate cortex (CC) in the brain is related to emotional stimuli.

Attention is a significant brain process, which has a central role in measuring the effectiveness of advertising campaigns; thereby, it is an indicator of consumer’s behaviour and the effectiveness of advertising [ 138 ]. According to the literature, the majority of researchers have agreed on two systems to measure attention toward advertising: (i) bottom-up, and (ii) top-down, systems [ 28 , 139 , 140 ]. Bottom-up (visual saliency/exogenous/ involuntary) attention is the type of attentional system which is initiated by external stimuli such as colour, voice, promotion, faces, text, novelty, brightness, and so forth, which lead to a process in which information in external stimuli is received automatically. Top-down (goal-driven/endogenous/voluntary) attention is the other type of attentional system, which is initiated by internal and external goals and expectation; thereby, it is necessary to focus all your mental power toward the goal that you are looking to achieve, thereby filtering goals to achieve your goals ( Figure 4 ) [ 2 , 140 , 141 ].

The bottom-up and top-down attention processes [ 59 ].

For this reason, the underlying brain reactions of attention and visual processing are highly interesting for advertising. Moreover, the anterior cingulate cortex (ACC) is highly related to the function of top-down and bottom-up attentional systems [ 142 , 143 ]. For example, Smith and Gevins [ 144 ] found that the occipital lobe (OL) is associated with attentional processes toward TV advertisements. The EEG investigation of Kong, et al. [ 145 ] found that variation in activity in the cerebral hemisphere related to the cognitive task can help to determine the success or lack of success of the advertisement. A recent fMRI investigation by Casado-Aranda, et al. [ 146 ] found that the correspondence between advertising and gender voice (male, female) induces attention regions in the brain. Ananos [ 147 ] carried out experiment using ET to investigate the attention level and processing of information in advertising (content recognition) among elderly and young people groups. Their findings revealed that the attention level among both groups was the same, but recognition by the young-people group was higher than that of the elderly-people group for untraditional advertising. Guixeres, et al. [ 51 ] conducted an experiment to investigate the effectiveness of an ad (e.g., a recall ad) and the number of views on YouTube channels, using neural networks and neuroscience-based metrics (e.g., brain response, ECG, and ET). Their findings suggest an important relationship between neuroscience metrics and self-report of ad effectiveness (e.g., recall ad) and the number of views on YouTube. Cuesta-Cambra, et al. [ 148 ] investigated how information is processed and learned and how visual attention takes place. Their findings indicated that the visual activity of men has different patterns from women, and does not influence subsequent recall, wherein recall relies on the emotional value of ads and simplicity, while complex ads need more visual fixation and are therefore hard to remember. In addition, the importance of the playful component of memory and low-involvement processing were confirmed by EEG. Treleaven-Hassard, et al. [ 149 ] examined the engagement of the consumer with interactive TV ads with a particular brand’s logo compared with non-interactive TV ads. The findings revealed that brands linked with interactive ads attract more automatic attention. Boscolo, et al. [ 81 ] conducted an experiment using ET and questionnaires to investigate differences in the visual attention between genders (male and female), toward print ads. Their findings revealed that there is difference in visual attention in the case of male, while no differences were noticed in the case of females.

According to Simson [ 150 ], studies into the perception of value and how it is formed reflect what is known in marketing theory, wherein the marketing-mix elements can be changed to influence the perceived value of a product. However, studies on how attention systems impact consumers ’ perception and actions have been limited to consumer report and behavioural studies, which depend on a rational report; this is not enough to explain attention processes, wherein there are two attentional systems influencing consumers’ perceptions (e.g., top-down and bottom-up attention system) ( Figure 5 ) [ 59 ]. Consumer perception is the first step in engagement with marketing stimuli or any other stimuli in the environment [ 151 ]. Hogg, et al. [ 152 ] defined perception as “the process by which marketing stimuli are selected, organised, and interpreted”. Therefore, individuals add meaning and interpret it in a certain way, which leads to the perceptions of the individual’s findings for each one. As stated by Belch and Belch [ 153 ] perception processing is extremely reliant on internal processes such as prior knowledge (experiences), current goals, beliefs, expectations, needs, and moods, and also external stimuli such as colour, orientation, intensity, and movement [ 59 ]. Although this explains the process of how consumer perceptions are formed, the exact the part concerning the explanation of sensations and the internal and unique assignation of meaning to sensations is what lies concealed, and remains unexplained in detail in the current consumer-behaviour literature. However, it is widely believed that this process is driven by the unconscious.

Two attentional systems impact the consumers’ perceptions.

Cartocci, et al. [ 154 ] and Modica, et al. [ 155 ] conducted experiments to estimate the accuracy measurement of the cerebral and emotional perception of social advertising campaigns (i.e., antismoking) using several methods such as EEG, GSR, and ECG. The findings showed that the antismoking campaign which was characterized by a symbolic communication style gained the highest approach-values, as evaluated by the approach/withdrawal index. Meanwhile, an image based on “fear-arousing appeal” and with a narrative style reported the highest and lowest effort-values index, respectively. The fMRI investigation of Falk, et al. [ 156 ] predicted the out-of-sample (population) effectiveness of quit-smoking ads. The findings revealed that activity in the prior mPFC was largely predictive of the success of various advertising campaigns in the real world. Plassmann, et al. [ 157 ] carried out an experiment to study the perception of pleasantness in the taste of wines, using the fMRI tool. Their findings showed a stronger activation in the medial OFC (mOFC) regions in the brain when subjects believed they are drinking expensive wine, showing that the mOFC is responsible for experiencing pleasantness. This led to infer that the pleasantness report was correlated with perceived value and price of product more than taste itself. Neuroscientists have found that the OFC and ventromedial prefrontal cortex (vmPFC) are involved in decision-making, through the perceived value of products [ 158 ]. Nuñez-Gomez, et al. [ 159 ] carried out an experiment using EEG to examine how advertising material is perceived by two groups (e.g., healthy group and group with Asperger syndrome). The findings revealed that there are large difference between these groups in their perception of emotion and their attention variables. Gong, et al. [ 160 ] carried out an experiment to identify the influence of sales promotion (e.g., gift-giving, discount) on the perception of consumers and purchase decisions by using EEG/ERP. The findings revealed that discount promotions have more impact on purchase decisions than gift-giving sales promotions.

Memory is defined as an ongoing learning-process, which has input and output functions in the brain [ 161 , 162 ]. The input function encodes information, while the output function retrieves information, and this is very important for advertising research [ 137 , 163 ]. For example, recall and recognition advertising-information is a retrieving function [ 28 ]. Myers and DeWall [ 162 ] and Atkinson and Shiffrin [ 163 ] presented the multistore model of memory, which includes three steps, as follows: (i) a sensory memory, (ii) short-term memory (STM), and (iii) long-term memory (LTM) ( Figure 6 ) [ 164 ]. Brain processes related to memory have revealed certain valuable outcomes, as to the factors which influence the consumers’ behaviour ,such as recall- and recognition-advertising [ 165 ]. Input and output functions in the memory are highly important for marketers and advertisers, due to each function having an awareness and unawareness aspect in the brain [ 137 , 166 ]. Memory and emotion are highly connected to each other. For example, previous studies confirmed that the emotional events are usually remembered more than neutral events, and especially if emotions correspond to events at that moment [ 167 ].

The information phases in the memory’s stages [ 164 ].

The memory process has been widely studied, and it has concluded that the hippocampus (HC), located in the temporal lobe (TL), plays a major role in generating and processing memories [ 165 ]. Additionally, activation of the HC has a strong relationship with LTM and STM, which highly impacts consumers’ purchase decisions [ 168 , 169 ]. In addition, the AMY is located next and close to the HC, which is significant for the memory system [ 165 ]. For example, the EEG investigation by Rossiter, et al. [ 170 ] found that the left hemisphere is correlated with encoding in the LTM. Similarly, the EEG investigations by Astolfi, et al. [ 171 ] used EEG to determine the brain regions that were triggered by the successful memory-encoding of TV ads. They found stronger activity in the cortical regions. Morey [ 172 ] investigated the impact of advertising message on recognition memory. The findings revealed stronger activity in the gamma band, which directly affected memory. The fMRI investigation by Bakalash and Riemer [ 173 ] and Seelig, et al. [ 174 ] measured the brain regions of memory ads. The findings revealed that stronger activity in the amygdala (AMY) and frontotemporal regions is associated with memorable and unmemorable ads. Similarly, [ 175 ] carried out experiments to investigate the content of ads and the activity of frontal regions and memory. The findings showed that the content of ads increased the activity in the frontal regions and the input function (encoding) of memory.

The study of these mental processes such as emotion and feelings, attention, memory, reward processing, motivation, and perception are highly important considerations for advertising research.

4. Discussion

A total of 76 articles were extracted and analysed, wherein the content analysis of the relevant articles revealed that the annual and the accumulative number of publications has been growing since 2009, reaching its peak in 2020 with twelve empirical articles that used neuroimaging, physiological, and self-report techniques to study the consumers’ brain processes such as, but not limited to, emotions toward the stimuli of marketing such as advertising. We followed the PRISMA protocol to select the relevant empirical articles for this study as brain processes such as emotions, feelings, motivation, reward, attention, and memory need to be considered in advertising research. The findings of the study revealed that the neuroimaging tool that is used most in studying the brain processes of consumers is the EEG, with 38 empirical articles, followed by the fMRI with 20 articles; it was also noticed that only four articles used the fNIRS tool in neuromarketing research. In addition, for physiological tools, it was observed that five techniques were used in neuromarketing studies to investigate consumer responses toward the stimuli of marketing such as advertising. The ET was used in 14 articles alongside neuroimaging tools such as EEG, wherein ET is the most used tool, followed by GSR with 12 articles; it was also used alongside other physiological tools such as ECG and EMG. Finally, self-report (i.e., surveys, interviews, focus groups, and observation) was used in seven articles.

This study found that the brain processes to be considered most in advertising research are emotions, feelings, attention, memory, perception, approach/withdrawal motivation, and reward processing. The findings demonstrated that the strongest activity in the inferior-frontal- and middle-temporal-gyri are associated with pleasure and displeasure, while the activity in the right superior-temporal and the right middle-frontal-gyrus correlated with high or low arousal [ 90 ]. As well as this, we found the OL connected with the attention system [ 144 ], and the HC, located in the temporal lobe (TL), plays a major role in generating and processing memories [ 165 ]. In addition, the VS, which is located in the basal ganglia plays a central role in the reward system; for example, the key functions of VS (i.e., control movement and planning) have a vital role in the reward system, while the components of VS such as the putamen, caudate nucleus, and nucleus accumbens (NAcc) have a central role in the assessment of consumer expectations, compared to the actual reward received [ 123 ]. In addition, the ventral tegmental region is considered a part of the reward system, which passes the neurotransmitter dopamine to other brain areas, thereby affecting goal-seeking behaviour [ 123 ]. For motivation, it was found that the anterior cerebral hemispheres play a central role in withdrawal and approach motivation; for example, the increase in activation in the right PFC is linked to withdrawal behaviour, while the increase in activation in the left PFC is associated with approach behaviour [ 105 , 176 ]. Finally, in accordance with the literature, it was found that the OFC and vmPFC regions play vital roles in perception (i.e., perceived value) [ 158 ].

5. Conclusions and Implications

Implication of the research findings for theory and practice: Theoretically, the current findings can be divided into three areas, as follows: firstly, neuroscientific techniques and methods enable the capture/measurement of the activity signals of the brain and body relating to consumers’ responses (e.g., emotions and feelings, attention, memory, perception, reward processing, and motivation) toward advertising campaigns. For example, neuroimaging tools (e.g., fMRI, EEG/ERP, fNIRS) enable the recording of the neural signals of the mental responses (e.g., pleasure/displeasure, low/high arousal, advertising recall and recognition) toward advertising, which can be beneficial for advertisers and marketers in creating more effective advertising campaigns to attract, captivate, and impact consumers. Meanwhile, physiological tools (e.g., ET, GSR, EMG, and ECG) enable researchers to gauge the physiological reactions of the consumer, such as pupil dilation, fixation, eye movements, saccade, heart rate, blood pressure, sweating level, and reaction time toward advertising. Secondly, neuroimaging and physiological tools will help advertisers and scholars to identify the weak elements in advertising which lead to withdrawal behaviour and to address these, besides identifying the strengths which lead to approach behaviour, and to enhance them. Thirdly, many articles have concentrated on detecting the neural and physiological responses of emotions, feelings, attention, memory, reward processing, motivation and perception toward advertising such as the presenter’s features (i.e., celebrity), because these processes play a key role in the decision-making of consumers (i.e., purchasing decisions). Additionally, some advertising research concentrated on gender voice (i.e., male, female), ads appeal, faces of celebrity, social campaigns (i.e., using seat belts in the car), and public health (i.e., anti-smoking campaigns). These areas can provide a reasonable explanation of the neural and physiological correlates of emotions and feelings (e.g., pleasure/displeasure, high/low arousal), attention (e.g., top-down, bottom-up), memory (e.g., encoding, retrieving), motivation (e.g., approach/withdrawal), reward processing, and perception (e.g., perceived value of ads) to be considered in advertising research. Thus, an application of this research perhaps offers reasonable explanations of how advertising works in consumers’ minds, and the relationship between the neural correlates of consumers’ brain and physiological responses toward advertising, thereby better understanding consumers’ behaviour, which leads to the creation of more attractive advertising for political, social and business sectors.

General Conclusion : Neuromarketing is a promising field, not only to study and solve the commercial issues such as the weaknesses of advertising campaigns and to reduce the wastage of advertising budgets, but also to create more effective advertising campaigns in social, political, and public-health sectors, in order to increase human awareness. In today’s hyper-competitive environments among advertising agencies, each agency seeks to find the most beneficial methods to beat competitors and be the first priorities in the consumer’s mind. Thus, advertisers have adopted neuroscientific methods in their research to study, analyse, and predict the neural and physiological responses of consumers toward the stimuli of marketing (i.e., advertising), thereby identifying the most important mental and physiological responses to be considered in advertising research to raise advertising effectiveness. Most studies in advertising research have determined the main mental processes of interest for advertising research, such as emotions and feelings, attention, memory, reward processing, motivation, and perception.

The findings of the study suggest that neuroscientific methods and techniques are significant to gauge the brain and physiological reactions of consumers toward the stimuli of marketing, such as advertising research. For example, neuroimaging tools are able to gauge the neural-activity signals of the consumer’s brain. At the same time, physiological tools can gauge physiological reactions such as eye movements, sweating level, and fixation.

6. Limitations and Future Directions

This paper tried to minimize the limitations in methodology; however, some limitations occurred and provided several directions for further research. This research concentrated on the English articles that were published in open-access journals from 2009 to 2020 and were listed in the WOS database. Therefore, this paper overlooked non-English articles, non-open-access articles, and other documents such as books, review papers, conference proceedings, and so forth. Thus, this paper is not free of bias. For future directions, we hope to overcome the obstacles in the future, which include the cost of research and techniques, lack of labs and facilities, use of time (e.g., data interpretation, recruiting participants, and so forth), and increased investment and funding in neuromarketing research and technique [ 177 ]. We encourage researchers and marketers from emerging countries to enter this embryonic field and leave their footprint by publishing articles for future works. In addition, we suggest that researchers and scholars identify the influence of advertising on consumers persuasion, engagement, and excitement, as well as the contributions of neuromarketing research to various domains (e.g., social sciences, public health, politics, and stock exchanges). We believe that this review study provides a profound overview of the global academic-trends in neuromarketing research, using the neuroimaging and physiological studies in advertising to study the brain processes of consumers. Thus, it provides valuable and reliable insights into the appropriate brain processes to be considered in future research.

Acknowledgments

The authors would like to thank Universiti Teknologi Malaysia (UTM), Azman Hashim International Business School (AHIBS); Taif University, Department of Economic & Finance, College of Business Administration; and Applied Science Private University (ASU), Department of Marketing for supporting this study.

Funding Statement

This research received no external funding.

Author Contributions

A.H.A., conceptualization, methodology, writing—original draft preparation, result discussion, and data curation; N.Z.M.S., supervision, review and editing; S.A.A.-Z., review, editing, and methodology; A.K., review and results discussion. All authors have read and agreed to the published version of the manuscript.

Institutional Review Board Statement

Informed consent statement, data availability statement, conflicts of interest.

The authors declare no conflict of interest.

Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

• Research Priorities
• Advanced search

Advanced Search

Open Access

The following papers are available open access:

How Does Retargeting Work For Different Gen Z Mobile Users? Customer Expectations and Evaluations of Retargeting via the Expectancy-Theory Lens Volume 63, Issue 4, 2023 Yllka Azemi, Indiana University, Northwest, [email protected] Wilson Ozuem, Anglia Ruskin University,  [email protected]

Why the Experiential View Is Vital To Marketing Communications Research Now: An Enhanced Framework for Examining the Effects of Contemporary Marketing Volume 63, Issue 2, 2023 Jean-Luc Herrmann, University of Lorraine, [email protected] John B. Ford, Old Dominion University,  [email protected]

When Brands Go Dark: A Replication and Extension – Examining Market Share of Brands That Stop Advertising for a Year or Longer Volume 63, Issue 2, 2023 Peilin Phua, Ehrenberg-Bass Institute for Marketing Science, University of South Australia, [email protected] Nicole Hartnett, Ehrenberg-Bass Institute for Marketing Science, University of South Australia, [email protected] Virginia Beal, Ehrenberg-Bass Institute for Marketing Science, University of South Australia, [email protected] Giang Trinh, Ehrenberg-Bass Institute for Marketing Science, University of South Australia, [email protected] Rachel Kennedy, Ehrenberg-Bass Institute for Marketing Science, University of South Australia,  [email protected]

When Brands Go Dark: Examining Sales Trends when Brands Stop Broad-Reach Advertising for Long Periods Volume 60, Issue 3, 2021 Nicole Hartnett, Ehrenberg-Bass Institute for Marketing Science, University of South Australia, [email protected] Adam Gelzinis, Endeavour Group,  [email protected] Virginia Beal, Ehrenberg-Bass Institute for Marketing Science, University of South Australia, [email protected] Rachel Kennedy, Ehrenberg-Bass Institute for Marketing Science, University of South Australia,  [email protected] Byron Sharp, Ehrenberg-Bass Institute for Marketing Science, University of South Australia,  [email protected]

Do Billboard Advertisements Drive Customer Retention? Expanding the “AIDA” Model to “AIDAR” Volume 60, Issue 2, 2020 John L. Fortenberry Jr., Louisiana State University Shreveport, Willis–Knighton Health System, [email protected] Peter J. McGoldrick, University of Manchester,  [email protected]

It is possible for papers submitted to the journal to be made open access. For more information on this process please email our editorial team at [email protected] .

• Join the AMA
• Find learning by topic
• Free learning resources for members
• Credentialed Learning
• Training for teams
• Why learn with the AMA?
• Marketing News
• Academic Journals
• Guides & eBooks
• Marketing Job Board
• Academic Job Board
• AMA Foundation
• Diversity, Equity and Inclusion
• Collegiate Resources
• Awards and Scholarships
• Sponsorship Opportunities
• Strategic Partnerships

We noticed that you are using Internet Explorer 11 or older that is not support any longer. Please consider using an alternative such as Microsoft Edge, Chrome, or Firefox.

Advertising Effectiveness

By Peter J. Danaher

The internet has enabled many business developments, but it has turned media allocation and planning on its head. In traditional mass media like television, advertisers can purchase a commercial slot and expect large audiences.

However, many of those reached are not interested in the advertised product or service, so a large percentage of those exposed to advertising do not respond to the message. In digital advertising, websites containing specialized content (e.g., model airplanes) allow advertisers to display their products to loyal and attentive audiences. In the social media space, Facebook delivers ad content to ideal target audiences by examining the web activity of users and their networks. Paid search advertising sends firms customers who are already “in the market” for their products, as indicated by their keyword use.

Over the past 15 years, television channels have grown in number. But the more significant change has been the exponential growth in websites supporting themselves with advertising, not to mention the rapid uptake of paid search advertising.

Advertisers have moved to new digital media outlets not only because of their ability to target customers, but also their lower cost compared to traditional media. Furthermore, digital media allows firms to connect ad exposures and search clicks to downstream sales, a feature Danaher and Dagger (2013) suggest eludes traditional media. Sethuraman, Tellis, and Briesch (2011) show the most convincing way for firms to demonstrate advertising’s effectiveness is by linking the effort to sales. In turn, researchers can use two methods to assess advertising effectiveness: field experiments and econometric models.

Field Experiments

Targeting and retargeting customers who are more likely to respond to offers, an increasingly common practice, makes advertising appear more effective than it is. Lambrecht and Tucker (2013) , in an award-winning Journal of Marketing Research paper, reported a comparison of advertising response between customers exposed to standard banner ads and retargeted banner ads showed the ads displaying products previously viewed were six times more effective at generating sales. However, the consumers receiving retargeted ads had already demonstrated product predilection. The researchers therefore randomly assigned consumers to a treatment group seeing retargeted, product-specific ads and a control seeing generic product category ads. They found the retargeted ads were less effective than the generic ads, as the customers were in different stages of the purchase funnel, and while retargeted ads work well near purchase, they are not effective for the larger group of customers embarking on their search.

The use of field experiments to determine ad effectiveness has subsequently blossomed, with studies using “ghost ads” on Google ( Johnson, Lewis, and Nubbemeyer 2017 ) and Facebook ( Gordon et al 2019) to create randomized control groups. For example, Sahni (2016) used a field experiment to show digital ads for one restaurant increased sales at competing restaurants offering similar cuisine.

In every case, these field experiments have shown that advertising effects are often difficult to detect. For example, the study of Facebook ads by Gordon and colleagues (2019) examined 15 campaigns and found that only eight produced a statistically significant lift in sales.

Econometric Models

The studies by Johnson, Lewis, and Nubbemeyer and Gordon and colleagues also highlight the challenges of designing an experiment to assess digital ad effectiveness. Individual customers use the internet in different ways, and providers deliver digital ads via unique online auction processes. Econometric models therefore provide a versatile approach to gauging advertising effectiveness. And while field experiment studies have been limited to examining one medium at a time, econometric models allow researchers to compare effectiveness across several media.

Researchers can use econometric models to examine time series data, such as weekly or monthly advertising and sales records. Dinner, van Heerde, and Neslin (2014) studied traditional and digital advertising’s effects on in-store and online sales for an upscale clothing retailer across 103 weeks. The retailer made about 85% of its sales in-store, and the researchers examined three media: traditional (i.e., total spend on newspapers, magazines, radio, television, and billboards), online banner advertising, and paid search. They found online display and paid search were more effective than traditional advertising. Although firms might expect digital advertising to influence only online sales, the researchers found it also influenced in-store sales.

Researchers can also use econometric models to examine single-source data linking individual-level ad exposure to sales, the strategy employed by Danaher and Dagger in 2013. They examined 10 media types employed by a large retailer: television, radio, newspaper, magazines, online display ads, paid search, social media, catalogs, direct mail, and email. The researchers found traditional media and paid search effectively generated sales, while online display and social media advertising did not.

Multimedia, Multichannel, and Multibrand Advertising

Danaher and colleagues (2020) also used single-source data but extended it to multiple retailer-brands, two purchase channels, and three media (email, catalogs, and paid search). They collected the data from a North American specialty retailer selling mostly apparel, where 80% of sales were in-store. The parent retailer owned three relatively distinct brands operating independently. They collected customer data in a combined database, giving them information on sales for each retailer-brand over a two-year period.

The researchers found emails and catalogs from one retailer-brand negatively influenced competing retailer-brands in the category. Paid search influenced only the focal retailer-brand. However, competitor catalogs often positively influenced focal retailer-brand sales among omni-channel customers. The researchers also segmented customers by retailer-brand and channel usage, revealing customers shopping across multiple retailer-brands and both purchase channels were the most responsive group to multimedia advertising.

In the contemporary business environment of ever-increasing media channels but static advertising budgets, firms must be able to measure advertising effectiveness. Many businesses have shifted their advertising expenditure toward digital media, but multiple studies show traditional media remain effective.

How do marketing managers decide what is best for their companies? Digital media firms like Google and Facebook offer in-house field experiment methods of examining advertising effectiveness. For multimedia studies, analysts can apply econometric models in any setting where time series or single-source data are available.

Peter Danaher is Professor of Marketing and Econometrics and Department Chair at Monash Business School in Melbourne, Australia. He was recently appointed a co-editor of the Journal of Marketing Research .

Danaher, Peter J. (2021), “Advertising Effectiveness,” Impact at JMR , (January), Available at: https://www.ama.org/2021/01/26/advertising-effectiveness/

Danaher, Peter J., and Tracey S. Dagger (2013), “Comparing the Relative Effectiveness of Advertising Channels: A Case Study of a Multimedia Blitz Campaign,” Journal of Marketing Research , 50(4): 517-534. https://doi.org/10.1509/jmr.12.0241

Danaher, Peter J., Tracey S. Danaher, Michael S. Smith, and Ruben Laoizo-Maya (2020), “Advertising Effectiveness for Multiple Retailer-Brands in a Multimedia and Multichannel Environment,” Journal of Marketing Research , 57(3): 445-467. https://doi.org/10.1177/0022243720910104

Dinner, Isaac, Harald J. van Heerde, and Scott A. Neslin (2014), “Driving Online and Offline Sales: The Cross-channel Effects of Traditional, Online Display, and Paid Search Advertising,” Journal of Marketing Research , 51(5): 527-545. https://doi.org/10.1509/jmr.11.0466

Gordon, Brett R., Florian Zettelmeyer, Neha Bhargava, and Dan Chapsky (2019), “A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook,” Marketing Science , 38(2): 193-225. https://doi.org/10.1287/mksc.2018.1135

Johnson, Garrett A., Randall A. Lewis, and Elmar I. Nubbemeyer (2017), “Ghost Ads: Improving the Economics of Measuring Online Ad Effectiveness,”  Journal of Marketing Research , 54(6): 867-84. https://doi.org/10.1509/jmr.15.0297

Lambrecht, Anja, and Catherine Tucker (2013), “When Does Retargeting Work? Information Specificity in Online Advertising,” Journal of Marketing Research , 50 (October): 561-576. https://doi.org/10.1509/jmr.11.0503

Sahni, Navdeep S. (2016), “Advertising Spillovers: Evidence from Online Field Experiments and Implications for Returns on Advertising,” Journal of Marketing Research , 53(4): 459-78. https://doi.org/10.1509/jmr.14.0274

Sethuraman, Raj, Gerard J Tellis, and Richard A. Briesch (2011), “How Well Does Advertising Work? Generalizations from Meta-Analysis of Brand Advertising Elasticities,” Journal of Marketing Research , 48 (June): 457-471. https://doi.org/10.1509/jmkr.48.3.457

By continuing to use this site, you accept the use of cookies, pixels and other technology that allows us to understand our users better and offer you tailored content. You can learn more about our privacy policy here

Impact as equalizer: the demise of gender-related differences in anti-doping research

• Open access
• Published: 29 June 2024

Cite this article

You have full access to this open access article

• Anna Kiss 1 , 2 , 3 ,
• Sándor Soós 1 , 2 , 3 &
• Andrea Petróczi 2 , 4 , 5 , 6  

64 Accesses

Explore all metrics

In general, the presence and performance of women in science have increased significantly in recent decades. However, gender-related differences persist and remain a global phenomenon. Women make a greater contribution to multidisciplinary research, which renders anti-doping research a compelling area for investigating the gendered aspects of academic research. The research design was based on the overall research aim to investigate whether gender in a specific field (ADS) has an effect on different aspects of research impact, including (1) the size of citation impact obtained by the research output, (2) the impact on the development of the knowledge base of ADS, expressed as the capacity of integrating knowledge from different research areas, and (3) the (expected) type of research impact targeting either societal or scientific developments (or both). We used a previously compiled dataset of 1341 scientific outputs. Using regression analysis, we explored the role of authors’ gender in citations and the effect of authorship features on scientific impact. We employed network analysis and developed a novel indicator (LinkScore) to quantify gendered authors’ knowledge integration capacity. We carried out a content analysis on a subsample of 210 outputs to explore gender differences in research goal orientation as related to gender patterns. Women’s representation has been considerably extended in the domain of ADS throughout the last two decades. On average, outputs with female corresponding authors yield a higher average citation score. Regarding women's knowledge integration roles, we can infer that no substantial gender differences can be detected. Dominantly female papers were overrepresented among publications classified as aimed at scientific progress, while the share of male-authored papers was higher in publications classified as aimed at societal progress. Although no significant gender difference was observed in knowledge integration roles, in anti-doping women appear to be more interdisciplinary than men.

Similar content being viewed by others

How to design bibliometric research: an overview and a framework proposal

Literature reviews as independent studies: guidelines for academic practice

The journal coverage of Web of Science, Scopus and Dimensions: A comparative analysis

Avoid common mistakes on your manuscript.

Introduction

Since the 1970s, there has been a growing focus on gender issues (Zuckerman & Cole, 1975 ), accompanied by significant efforts to foster gender equality at both international and national levels (UNESCO, 2024 ). Beyond policymakers, researchers, and various organizations have also become increasingly interested and dedicated to narrowing the gender gap. Extensive evidence in scientific literature indicates that, generally, male academic productivity surpasses that of females, despite notable improvements in the representation and performance of women in science overall (Larivière et al., 2013 ; Chan & Torgler, 2020 ; Elsevier, 2021 ; OECD, 2021 ). Gender disparities in academic productivity/success have been thoroughly examined through the application of standard bibliometric indicators in science studies. However, the results of these studies are not consistent. The gender gaps in productivity, citation, and promotion have been evolving, albeit at a gradual pace, as evidenced by recent reviews of gender disparities in science (Kwiek & Roszka, 2021 ). The increasing participation of women in academic science alters the framework within which gender disparities in terms of productivity, citation impact, research areas/topics, and international research collaboration are currently examined. Recent bibliometric studies are increasingly employing diverse methods to determine gender among authors and authorships (Halevi, 2019), while comprehensive research on gender disparities in science is being conducted (e.g., Chan & Torgler, 2020 ; Thelwall et al., 2019 ), shedding light on the extent of ongoing transformations. In Table 1 , we summarized the findings of some recent large-scale studies on the gender gap in science. This purposive sample only aims to demonstrate that the prevailing significance of gender differences has also recently been challenged in various approaches (and by no means considered a comprehensive review).

A high number of bibliometric analyses have been conducted in recent years on gender disparities in different fields of sciences, such as aspects of gender differences and gender disparity in research/scientific productivity, women's contribution to science in life sciences (DesRoches et al., 2010 ), medicine (Henderson et al., 2014 ; Pashkova et al., 2013 ), economy (Maske et al., 2003 ), astronomy, immunology and oceanography (Leta and Levinson, 2003 ), psychology (D’Amico et al., 2011 ), and criminal justice and criminology (Snell et al., 2009 ). From the results and conclusions of academic research, an increasing body of evidence is drawing on women's participation in science.

Gender disparities vary by research field and country (Chan & Torgler, 2020 ; Larivière et al., 2013 ). Bibliometric evaluation of gender imbalances in different research areas is especially important hence bibliometric indicators are considered important features of research excellence. Showing the degree of gender disparities in different fields of science opens a lot of doors towards informing policymakers, creating more equal opportunities for women scientists, improving good governance, and also can have an impact on research priorities as well as funding allocations.

The selected subject of the present research is the area of anti-doping research (ADS), a relatively young and emerging specialty. The main reason for selecting ADS for investigating gender differences was that though specific in scope, this interdisciplinary domain is made up of STEM and SSH reference fields often characterized by different gender-related patterns. ADS as a „melting pot” emerging from interdisciplinary interactions can therefore be assumed as a case where these dynamics potentially interfere (e.g. knowledge integration, also studied in our work). Anti-doping science intersects with various fields due to its multidisciplinary nature. It encompasses sports science, medicine, pharmacology, toxicology, analytical chemistry, but also law, ethics, public health, psychology. Together, these interdisciplinary perspectives shape the foundation of anti-doping science, driving research, policy development, and enforcement efforts to address the complex challenges associated with doping in sports. This characteristic renders this domain of interest for a case study with valuable contributions to the more general topic of gender-related issues in science.

Related work on gender-related trends in sport science and anti-doping research

Gender-related differences in sport sciences were examined several bibliometric studies in terms of academic productivity, authorship position (Chang-Yeon et al., 2019 ; Dynako et al., 2020 ; Loder et al., 2021 ; Mujika & Taipale, 2019 ; Ryan et al., 2020 ) editorial leadership position (Martínez-Rosales et al., 2021 ), academic positions, women scientists’ career development or international collaboration. Analyzing the trends of gender distribution in sport sciences there is an increasing body of evidence that the presence, role, and contribution of women have increased significantly in recent decades, but the gender gap remains a global phenomenon in the field.

For instance, Mujika and Taipale ( 2019 ) investigated gender differences in authorship within the International Journal of Sports Physiology and Performance. They found that only 13% of the authors were women in the first 5 issues of the journal published in 2019. In the realm of sport-related medical science, Chang-Yeon et al. ( 2019 ) analyzed orthopedic sports medicine literature from 1972 to 2018 to observe changes in the proportions of female authors across various authorship positions. Although only 16.6% of the authors were female, there was a significant increase in female authorship from 2.6 to 14.7% over the 46 years. Ryan et al. ( 2020 ) evaluated articles in the Sports Health journal from 2009 to 2018, noting an increase in publications with at least one female author from 52 to 64% over the study period. They highlighted Sports Health's comparatively high rates of female authorship compared to other journals. Loder et al. ( 2021 ) conducted a bibliometric analysis of English musculoskeletal literature over 30 years, revealing gender differences in first and corresponding author positions. The percentage of female first authors increased from 10.8 to 23.7% between 1985–1987 and 2015–2016, while corresponding authorship increased from 8.9 to 18.9% over the same period. Dynako et al. ( 2020 ) examined trends in two American sports medicine journals over 30 years, finding an average of 13.3% and 8.1% female first authors for the American Journal of Sports Medicine and Arthroscopy, respectively.

Specific to anti-doping science only a few studies provide bibliometric analysis. The study by Agulló-Calatayud et al. ( 2008 ) identified key research centres and authors of scientific articles on anabolic steroids. A working paper by Engelberg & Moston (n.d.) focuses on doping-related papers published in sport management journals.

The present research aims to provide an in-depth analysis of gender-related patterns that go beyond the simple identification of the gender gap. It is also a follow-up to our previous study, completing the picture of gender-related patterns in ADS. In our previous study (Kiss et al., 2022 ) we aimed to examine the contribution of women to anti-doping research and provide a picture of the relational structure of gender aspects of country-related, authorial, and topical features via bibliometric analysis. To our knowledge, our study was the first original article in anti-doping science that specifically studied the presence and role of women based on a bibliometric analysis of international academic literature.

Aims and research questions

The research design was based on the overall research aim to investigate whether gender in a specific field (ADS) has an effect on different aspects of research impact, including (1) the size of citation impact obtained by the research output, (2) the impact on the development of the knowledge base of ADS, expressed as the capacity of integrating knowledge from different research areas, and (3) the (expected) type of research impact targeting either societal or scientific developments (or both). This latter aspect (social vs. scientific impact) is not directly investigated, so it is not social/scientific impact itself that we subjected to analysis, but we used a proxy that can be expected to predict the type of impact: research aim or, to put it differently, research orientation. In particular, it is a plausible hypothesis that the outcomes of research with e.g. „societal” orientation/aim will be valorized in societal progress, as a type of (social) impact. We capitalized on this expectation by mapping the relationship between gender and research aim – which is assumed to be causally related to the type of subsequent impact – within the field.

Accordingly, our three research questions were the following:

RQ1. How do women’s authorship features (first, last, corresponding, position in the author network) affect scientific impact within ADS?

RQ2. Are there gendered differences in knowledge integration between different areas behind ADS?

RQ3. Is there a difference between dominantly female vs. dominantly male authored papers in research goal orientation (in terms of contributing to societal progress orientation vs. scientific progress/orientation?)?

For RQ1, we employ a form of regression analysis to disentangle the effects of several potential factors on citation impact, including female authorship and its patterns in the anti-doping literature. Via this tool, we aim to capture and quantify the true, individual effect of female authorship on research impact (if any) and separate it from the contribution of other factors. On the other hand, it was a natural way to test whether any “gender gap” existed in the field(s) of ADS, being well documented in earlier studies for many cases and research areas.

In the study of knowledge integration (RQ2), we introduce a new metric to uncover gender differences in authors’ capacity to connect the diverse reference field behind ADS.

For RQ3 we applied qualitative content analysis to investigate the difference between dominantly female vs. dominantly male authored papers in research goal orientation. Our focus (in this part of the research) was the identification of research orientation which can be expected to predict the type of impact (scientific progress, societal progress) for the paper. However, the concept of impact is not directly targeted in this approach.

The study concept is depicted in Fig.  1 .

The logical framework of investigations

Data collection

To delineate the anti-doping research field, bibliometrics-aided retrieval was used in line with the methodological paper of Gal et al. ( 2015 ). Initially, a core dataset was established using a keyword profile based on a core publication (Acute Topics In Anti-Doping) for ADS (Core set, C). Subsequently, a broader dataset of publications and research outputs was assembled using a more expansive search profile (Broad search term set, B). Lastly, citation-based similarity measures were employed among the document sets to gain a final dataset with a high degree of precision.

For data collection and field delineation, the Web of Science Core Collection databases were used (including the Science Citation Index, the Social Science Citation Index, and the Arts&Humanities Citation Index). The Core set was based on the WoS database with a search query of “anti-doping* OR antidoping*” OR “anti-doping”*. The search terms were applied to the title only in WoS. No limitations were placed on the dates of the searches; the final date of data collection was October 20, 2021. The search resulted in a core set of 572 publications.

For the Broad search term set, B, we identified relevant anti-doping-specific search terms in sport science journals, systematic reviews, highly cited research papers, and WADA documents. We created a broad search term using the following query: “doping control*” OR”doping prevention*” OR”doping-free sport”* OR”clean sport*” OR”drug-free sport*” OR”anti doping in sport*” OR”fair sport*” OR “WADA*” OR “fair play*” OR “anti-doping*” OR “antidoping*” OR “anti doping*” in (Topic) in WoS database. The broad search term set included 2390 research papers.

In the next step, citation-based similarity measures were applied to integrate the core dataset and the broad search term set, resulting in two further document groups. Group I included papers in the broad search term set citing the core dataset at least twice (N = 889). To group II. belonged papers in a broad search term set with at least 10% references among the pooled references of the core dataset (N = 1594). In the final set, publications from the core dataset (C) and publications belonging to group I or II.) were included, thus ensuring the inclusion of relevant topics only. After eliminating duplications, the final set included 1802 publications, ranging between 1998 and 2021. After excluding studies to which genders cannot be assigned to authors 1341 studies were included in the analyses. Figure  2 illustrates the identification workflow of relevant research papers.

Source: Kiss et al. ( 2022 )

Flowchart of database construction.

Sex/gender assignment

To identify authors' genders, we first standardized author names by removing abbreviations, and symbols, and abbreviating first names to full names. We retrieved full author names from bibliographic data obtained from WoS databases. Using the Gender API (available from https://gender-api.com/en/ ), a platform analyzing names to ascertain gender, we assigned genders to authors based on their full names. We also noted the country assigned to each author from the publications, which increased result accuracy. The final dataset included 3628 author names, of which 2415 full author names were identified. Among these, genders were assigned to 2346 authors (97%) with an average accuracy of 93.83%. A detailed description of sex identification is provided by Kiss et al. ( 2022 ).

Data analysis

The effect of gender differences in authorship on scholarly impact.

To construct an explanatory (regression) model of citation impact in anti-doping research, two sets of explanatory variables were defined and calculated for our sample. The unit of analysis was the individual publication; therefore, factors of citedness were also defined at this level. Most importantly, first, the indicators that accounted for female authorship were developed. This set was then complemented with the group of those factors that have been demonstrated to have a significant effect on citation counts. In the subsequent section, we outline the gender-related indicators, followed by the list and definition of variables that constituted the second group we refer to as “main citation factors”.

Indicators conveying female authorship

The operationalization of the concept of female authorship at the publication level is not an unproblematic task. The main difficulty comes from the fact that papers are typically multiauthor with a mixed gender composition, so the direct attribution of a paper to any gender category is mostly unfeasible. In related studies, a heuristic is most often applied, in which the gender of the 1st (or 1st and last, or the corresponding) author is used as the decisive rule to classify a paper as a female- or male-authored publication (Zhang et al., 2021 ). Although this heuristic is a reasonable choice on the common assumption regarding distinguished roles within author listings, we attempted to take a step further and formulate women’s contributions from further, different angles. To that end, the following indicators were employed (short names for each variable are indicated in parentheses; the measures are defined for individual papers).

Gender of the corresponding author (c.au) . A binary variable indicating whether the corresponding author of the paper is male or female (0 = male, 1 = female).

The proportion of female authors (weight). The ratio of the number of female authors to the number of all authors of a paper. The measure represents the weight of female authorship at the paper level, and its value ranges between 0 and 1 (for single-authored papers, this measure, therefore, takes its maximum or minimum value depending on the gender of the author).

Relative position (rel. position). The indicator is a generalization of those measures that account for the (assumed) role of female authors based on the byline, i.e. the order of authors. It is defined as the rank of the first appearance of a female author in the author list, normalized to the number of authors. For example, if a paper has n = 4 authors, of which the 2nd and 4th one is a female, then the value of this indicator equals 2/4 = 0.5. It can be conceived as the maximum (relative) rank of female authors in the author list as well. The 1st author position then becomes a special case, which is the maximal relative rank for a single paper, and its value is increasing with the size of the author group (technically, the value is decreasing in this case: a consequence of this definition is that the maximal value is the closest to 0, while the minimal value is always 1. For single-authored papers, a correction was applied, setting the value to 0 or 1 for female- and male-authored papers, respectively).

Variables for the main factors of citation impact. The remaining variables introduced in the explanatory model of citation impact all played the role of controlling for those factors that have been shown to exert a considerable effect on citation counts (Tahamtan et al., 2016 ). Since these factors constitute a relatively wide spectrum, we enrolled the most important ones based on the bibliometrics research tradition. This consideration left us with the following indicators.

Research field and area . As long evidenced in bibliometrics, research fields, and areas show differing citation behavior so that the impact of papers from different fields are not directly comparable. To account for this ingredient of citedness, control variables were applied conveying the research area to which a paper belongs. To support a more robust model, we used a relatively high-level taxonomy distinguishing between six broad categories. Papers were preassigned to these categories in the data source (Clarivate InCites Analytics©, providing the so-called GIPP research categorization scheme). The six categories included were referred to as (1) Clinical, Preclinical and Health , (2) Life sciences , (3) Physical sciences , (4) Engineering and technology , (5) Social sciences and (6) Arts and Humanities . These areas entered the model as a dummy variable each, with values {0,1}. For example, a paper took on the value 1 in the Life sciences variable if it belonged to that area and 0 otherwise. Note that multidisciplinarity, which is clearly present in ADS, was allowed to be represented this way since any paper could belong to more than one category.

Journal Quality . Since publications in top-tier journals have been shown to attract more citations than lower-ranked journals, a journal metric is applied in the model as an explanatory variable. For this role, the well-known Journal Impact Factor (JIF) has been chosen. JIF values for our dataset were retrieved from the InCites database (as provided in Clarivate’s Journal Citation Reports).

Further indicators of authorship . The author's composition of papers exhibits several characteristics that influence the extent of their recognition in the scientific community. Given this insight, we included further variables to complement those of female authorship, thereby disentangling the effect of different author-related characteristics of papers. In particular, three indicators were implemented: (1) the number of authors for a paper (co.au) , (2) Author productivity (au.prod), and (3) Country of affiliation (cou.freq) . The motivation for co.au is that the number of co-authors and citedness typically show a positive correlation. Author productivity for paper P is the total number of papers within our sample that pertain to the authors of P. This serves as a proxy for the visibility of the authors of a paper within the area of ADS. Similarly, Country of affiliation accounts for another aspect of author visibility and reputation by summing for a paper P how many times the affiliated countries (of the authors) of P appear within the affiliation data in our sample. In other words, this variable is a score of country-level affiliations for a paper, where each country contributes with its own weight (frequency) in the sample.

Type of document. A distinct set of factors in relation to citation impact is the so-called document type . It is also a well-confirmed hypothesis that, in general, research (or target) articles, reviews, and other bibliographic types, such as editorials, proceeding papers, book chapters, etc. tend to have differing potentials to attract recognition and hence citation counts. To control for this, we used a categorical variable with three levels: (1) Article, (2) Review, and (3) Other.

Open Access. Related to the aspect of document type is the open access status of a paper. There is a vast amount of literature confirming the so-called “OA citation advantage” assumption. that a publication available to the reader free of charge, that is, published under some form of OA, can collect more citations than the same publication behind the paywall. To address the intricacies of OA publishing, we introduced a two-level dummy variable into our model that simply recorded whether a paper is available only through subscription (non-OA, value = 0) or whether it is published through any open access channel (OA, value = 1). The latter case encompassed all the versatile types of OA publishing (“gold”, “green”, “hybrid”, etc.). Data were retrieved from the Web of Science databases.

The base regression model

Beyond the explanatory variables outlined above, an appropriate measure for citation impact had to be selected for the target variable. Instead of pure citation counts, we have chosen a measure that is designed to yield directly comparable impact scores between different types of publications. In particular, the so-called field-normalized citation score (FNCS or NCS) was applied as the outcome variable, which is defined for a publication as the ratio between the number of citations to the average citation number within the same research field and in the same publication cohort (i.e. papers with identical publication year). The FNCS indicator hence normalizes citation counts in two ways, to the field of research and the age of the paper. Although we control for these factors already in the model, as research fields and the year of publication are featured among our explanatory variables, model performance can still be improved by controlling for these on “both sides” of the equation. The main reason for this choice, however, is that this citation measure can be applied as a continuous outcome variable suited to a simple linear regression model (as opposed to raw citation counts): its values are real numbers. In particular, its value is FNCS  = 1 (< 1 or 1 <) if a publication reaches (remains below or levels beyond) the citation average within the field: in fact, its value conveys how many times the field’s average a paper is being cited.

In sum, our base regression model was a multivariate linear regression of the following form.

\(G(i,\dots ,n)\) is the series of n variables on gender and female authorship ( n  =  3 ),

\(F(i,\dots ,m)\) is the group of m variables controlling for the research field ( m  =  6 field variables),

\(C\left(i,\dots ,u\right)\) is the set of u variables accounting for the main factors on citation impact ( journal quality, further authorship patterns, document type, year of publication, open access status).

FNCS , the normalized impact measure, is included with the regular logarithmic transformation (log-normalization). This was motivated by FNCS being a variable with a highly skewed distribution (as is characteristic of citation distributions), so a transformation was needed to achieve a normally distributed target variable for the linear regression model. Regarding data collection FNCS values have been obtained from the InCites service of Clarivate, measured at the Subject Category level.

Women’s impact on knowledge integration

The other main research question of ours concerned a special but equally important aspect of the scientific impact of women in ADS. This aspect can be termed the impact on knowledge integration. The methodology of measuring to what extent female authors as “mediators” are positioned between different fields was as follows. In the first step, we constructed a network representation of our publication sample. Given the pre-existing assignment of papers to the six research areas of the GIPP category system, we extracted the author—paper and the paper—area relations from the data, out of which the interconnections between authors and their research areas (within the document sample) were recorded. Based on these interconnections, we built a bipartite graph or network with authors appearing as one type of network node and the six broad research areas as the other type. The edges of the graph, connecting an author to one or more research areas each, conveyed the relation that the author has been active in the related field (i.e. at least one of her publications belonged to those areas). The network was also a weighted graph in that edges were assigned a numeric value equal to the number of papers of the author in the corresponding research area. This representation allowed us to investigate to what extent authors – especially female authors – participate in “integrative” research that connects different areas. For this purpose, we introduced a positional network measure based on the author-field graph outlined above. We refer to this as LinkScore, alluding to the idea of quantifying the capacity of an author’s work in linking research fields. The LinkScore was calculated for each author in the following steps:

Counting the number of connected fields per publication. In the first step, each publication was given a raw score by counting the number of research areas assigned to it. Hence, for “monodisciplinary” papers, the score equaled 1; for papers sorted into two GIPP research categories, it was 2, and so on.

Weighting connections based on the distance between fields. Another consideration for the LinkScore measure was that linking distant research areas represents a larger extent of knowledge integration than linking closer areas. Therefore, we scored higher in the cases of “long-distance” interdisciplinarity over “short-distance” interdisciplinarity (Larivière et al., 2015 ). In particular, the raw LinkScores for publications were weighted according to the following scheme: a weight of 3 was applied to raw LinkScores if there was a minimum of 1 “ natural science ” field plus Social Sciences and/or Humanities, whereas LinkScores within “natural” or within “social” science fields were not amplified (weight = 1). As a consequence, a paper linking, e.g., the Life Sciences and the Social Sciences, will have a LinkScore  = 6 (2 fields with a link weight = 3), while the same paper would gain a LinkScore  = 2 if it was assigned, e.g., to the Life Sciences and the Clinical Sciences (2 fields again, but with a link weight = 1). The reason for tripling the weight for the “social + natural sciences” combination was to generate sufficient discriminatory power for the measure, since lower values would not allow us to distinguish between “long-distance” and “short-distance” interdisciplinarity in all cases (e.g., the LinkScore for a combination of the four hard science fields would be the same as that of a “social + natural science” combination).

Aggregating publication-level scores for authors . The final step was to relate these publication scores to authors, i.e. to aggregate the author’s LinkScores of their papers. Since we intended to create a size-independent measure (so that the number of publications should not influence the score), the maximal score by author was taken. In particular, the LinkScore for an author with n papers was calculated as follows: \(\text{max}\left({\sum }_{i,\dots ,n}{\text{LinkScore}}\left(i\right)\right).\) With this measurement specified in the previous three steps, we calculated the LinkScore values for all authors in our sample and used the results to characterize female authors’ contribution to knowledge integration as well as to compare female and male authors along this scale.

Research goal orientation of dominantly female vs. dominantly male papers

To detect potential differences in the research goal orientation of dominantly female- vs. non-female-authored papers, content analysis was carried out. For the analysis of whether dominant research goals are associated with gendered authorship, we had to sort publications into at least two categories: one that is dominated by female authorship and the other that is characterized by male authorship. We referred to these two as “dominantly female publications” and “dominantly male publications”, respectively. In the definition of the two categories, we relied on our indicators of female authorship, namely, c.au , recording the gender of the corresponding author; weight , the share of female authors among the author group per paper; and rel.position , the rank of female authors within author lists. In particular, we applied the following rules:

Dominantly female publications. For a publication to be primarily female-authored, at least one of the following conditions were to be met: (1) the corresponding author is a female, (2) at least half of the authors of the paper are female, and (3) female authors (at least partially) appear in the first 50% of the author list. Formally, ( c.au  = 1) ∨ ( weight female  ≥ 0.5) ∨ ( rel.position female  ≤ 0.5).

Dominantly male publications. Consequently, for a publication to be deemed primarily male-authored, each of the following conditions were to be met: (1) the corresponding author is a male, (2) at least half of the authors of the paper are male, and (3) male authors (at least partially) appear in the first 50% of the author list. Formally, ( c.au  = 0) & ( weight male  > 0.5) & ( rel.position male  < 0.5).

Although logically sound, the above definitions imply some asymmetry between the two groups: at the cost of accounting for various roles of female authors in the “female-dominated group”, the “male-dominated” set becomes more restrictive. To counterbalance this effect, we relaxed the second definition (to retain the sensitivity of the first) and considered publications that met at least two criteria out of the three (corresponding authorship, weight and rank of male authors) as eligible for the male-dominated category. Based on this consideration, a certain set of papers was reassigned to the male-dominated category in a post hoc manner.

In our attempt to capture gender dominance based on the authorship patterns of papers, it would be natural to incorporate the existing evidence on author credit allocation practices. However, in doing so, a basic difficulty would be that the ADS domain under study has a broad multidisciplinary scope, incorporating social sciences, health/medical sciences, natural sciences, engineering, social sciences and humanities as well, Since these disciplines substantially differ in their practices of allocating author credit, we decided to overcome this difficulty by using the most general considerations on author credit indication (taking the first, the corresponding author and the gender ratio in our case) and use them in tandem so that they can compensate each other’s biases when applied to the diverse subfields within ADS.

Coding criteria

After categorizing the papers according to gender, a random sample of abstracts of 105 dominantly female and 105 dominantly male publications were selected from the final set of anti-doping publications. This dataset included the WOS ID, title, abstracts, and authors' keywords of the randomly selected research paper. The reading and coding of the selected abstracts were performed by two of the authors. Following the coding of abstracts, the authors discussed their differences until a consensus was reached. They conducted the evaluation simultaneously but independently to avoid influencing each other's judgment on scoring, ensuring that the evaluation was free from biases. When coding the selected publications, only the content of the abstract was considered for categorization, as the content of the article is often not discernible from the publication title alone. During the coding of the abstracts, the two authors adhered to the definitions determined in the working paper of Zhang et al. ( 2021 ). To classify publications by their aims they introduced a new distinction instead of the widely used basic/applied distinction by differentiating between three variations of the aims of the research: aiming at scientific progress, aiming at societal progress, or both. The criteria for classifying publications into three categories are shown in Table  2 .

It should be pointed out that the operationalization of the notions of „basic science” and „scientific orientation” vs. „applied science” and „social/societal orientation” is context-dependent, i.e., it cannot be detached from the domain of investigation in our research. In our case, where the context is the domain of ADS, „applied” is considered to cover the approaches aimed at directly contributing to Anti-Doping applications (regardless of this being the medical, social or ethical aspect of AD), while „basic” or „scientific” is considered as a more general research aim. Therefore, although e.g. Clinical Studies are usually considered as applied research (not being Basic medicine), in this context these would also be (and have been) categorized as „basic” or with „scientific aim” on an individual basis, if their subject went beyond direct applications, that is, external utilization of knowledge was addressed, and hence aiming at contributing the clinical knowledge base beyond ADS.

Gender differences in the proportion of authors and authorship position

Female authorship throughout the timespan of the publication records in anti-doping is best described by the dynamics of three gender-related indicators. Figures  3 and 4 depict trends in these indicators – corresponding authorship, mean relative position of female authors on papers, and the share of female authors in author lists (mean weight)—for the last one and a half decades, covering approximately 17 years between 2005 and 2021.

The annual values of the number of papers with female corresponding authors and the fractional count of papers with regard to female authors (weight.total) between 2005 and 2021

The yearly averages of the relative position of female authors on papers and the share of female authors in author lists (mean weight) between 2005 and 2021

The first indicator captures whether the role of the corresponding author for a paper is taken by a male (0) or female (1) researcher (c.au). We used the annual number of papers with a female corresponding author (no. papers with c.au = 1). Striking from Fig.  3 is that this quantity has been increasing since 2007 in a linear and mostly monotonic fashion (with a few exceptional years of “recession”). The curve is also rather steep, as the number of such papers doubled in approximately every five years during this period. This tendency clearly signals a growing representation of women in ADS.

A similar tendency is also present in the dynamics of the other two indicators. For the proportion of female authors per single paper (weight), we used the annual sum of this quantity (Fig.  4 ), which is the total weight of female authors in a particular year. In bibliometric terms, this is identical to the “fractional counting” of papers with regard to authors, where each publication is counted according to the weight of its female authors (meaning that a paper with 40% women on its author list was counted as 0.4 paper instead of 1). In this way, we can calculate the total amount of outputs (per year) that could be assigned to female authors only. Just as with the number of papers with female first authors, the total weight of women-authored papers has also increased since 2005, in fact, in the very same manner as the extent of first authorship. The two curves proceed in close proximity (almost overlap), so the same steep slope and overall monotonicity are demonstrated by the annual share of female authors (in papers), approximately doubling every five years.

The annual values of the size-independent version of the same indicator, the weight of female authorship within author lists, are depicted in Fig.  4 . In this case, the yearly averages of the shares of female authors are reported (mean weight). For the period under study, this share has been slowly but clearly (monotonically) growing: it has increased from 5% (2005) to more than 20% (2021) on average. This trend has been strengthened by the positional changes within the order of authors, as witnessed by the development of the third indicator, that of the relative position. In Fig.  4 , just as for the weight indicator, the mean values of the relative position indicator are indicated for each publication year (mean rel. position). Note that it is a’tricky’ measure to interpret because lower values represent higher ranks; therefore, the slightly descending curve actually signals improving positions for women. More specifically, from 2007 on, the mean position has gradually been improving from approximately 0.6–0.5 to 0.4, meaning that female authors are moving forward from the “first 60%” to the first 40% on author ranks (i.e. within individual papers’ author lists). Although in certain research areas, being the last author is also a distinguished position, in such a multidisciplinary domain as ADS, higher ranks (being the 1st or among the first authors) can still be expected to trigger broader recognition. Altogether, each of the three measures on female authorship mutually confirms that women’s representation has been considerably extended in the domain of ADS throughout the last two decades. Next, we turn to our main question on the relationship between this elevated representation and research impact.

Effects of women’s authorship features on scientific impact

In the regression analysis seeking the relationship between female authorship and citation impact, we followed the strategy of finding the best available model. Regarding the procedure of variable selection, that is, in our attempt to find the relevant factors influencing citation scores (out of the indicator set described above), we preferred to choose the “theory-based” method. That is, over automatic selection algorithms that seek the best fitting model (in purely quantitative terms), we prioritized theoretical considerations in setting up our collection of alternative models or indicator sets. In particular, we decided to include in each alternative model the variable group C(i,..,u), which we called “the main factors of citation impact”, as well as F(i,…,m), i.e., the control variables for the research area since the effect of these variables is vastly demonstrated in the literature. The only source of variation between our models was the set G(i,…n), that is, the variables of female authorship, the effect of which we wanted to test in all possible configurations. This left us with seven models for testing. Of these, models 1–3 involved only one of the three indicators (c.au, rel.position, weight), models 4–6 involved two of them, accounting for all such combinations, and model 7 involved each of the three variables. The most informative results are reported in Table  3 , comparing the effect size of each explanatory variable across the seven alternative models using the standardized regression coefficients (plus indicating the model fit measures).

As reflected by the model fit measures in Table  3 , the seven models performed almost equally in terms of the goodness of fit. The AIC (Akaike Information Criterion) value is practically the same across all models, meaning that varying the set of gender-related variables does not affect how well the model fits the data. Also equal is the R 2 value, according to which these models account for approximately 15–20% of the variation in citation scores, which makes them a week but (regarding such complex relationships) acceptable explanations. In fact, we also ran automatic model selection algorithms to complement the theory-based selection, but the performance of the best “automatic” model did not differ from the ones below. Moreover, the effect sizes of individual factors also remained quite stable throughout the models.

First and foremost, indicators of female authorship appear to have a weak effect on citation scores. More precisely, the corresponding author being a female (c.au) is weakly but positively correlated with citation scores (the β -values ranging from 0.04 to 0.06 across models). The relative position of female authors (rel.position) technically shows a mostly negative, weak correlation with impact (the β -values ranging from − 0.04 to 0.01). However, recall that, by the definition of relative position, lower values mean a higher rank of female authors in the author list. Hence, we can say that this negative correlation signals a positive relationship between female author rank and citation scores so that the higher the rank of female authors is, the greater the expected impact of the paper. On the other hand, the proportion of female authors (weight) came out as slightly negatively affecting citation values (the β -values ranging from − 0.05 to 0.01).

Regarding the pattern of effect sizes in relation to the other variables, our models confirmed the previous findings regarding the relevant factors of citation impact. In the “main factors of citedness” set, the greatest and invariable positive effect is attributed to journal quality ( Journal.Impact. Factor ), which is moderate ( β -value = 0.16) but the highest among all variables with a positive influence. The open access citation advantage has also been confirmed ( β -value = 0.13). The next such indicator is author productivity (au.prod) , ( β -value = 0.10), emphasizing the value of the extent of authors’ representation within the professional community. Interestingly, the number of co-authors ( co.au ) and authors’ nationality ( cou.freq ) appeared to have a small – but positive – effect on impact. Finally, a weak negative influence was detected on the part of publications that did not belong to the “Article” or “Review” category ( DT  =  Other, ( β -value = − 0.19), and of the year of publication ( PY, β -value = 0.14), the latter warning that even age-normalized citation measures as our target variable cannot fully omit the effect of the citation window (so that older publications still get higher citation scores than more recent ones).

To gain a deeper insight into the structure of effects exerted by individual variables, especially the gender-related ones, we looked into the composition of a selected model out of our seven variants. Since all the models tended to be equivalent regarding their performance, we have chosen the one that was most equipped with the indicators of female authorship: in Model 7, all three such variables were included. The detailed regression results for this model are reported in Table  4 (primarily including the raw regression coefficients and the significance values).

According to the detailed analysis, the effect of female authorship variables is not statistically significant. Significance (at the level of p ≤ 1%) was detected for the same factors that showed higher effect sizes (as discussed above): Journal Impact. Factor , the year of publication ( PY ), author productivity ( au.prod ), OA status and document category of “Other” ( DT  =  Other ). Additionally, publishing papers in the Physical Sciences or, to a lesser extent, in the Clinical, Preclinical & Health area appear to have a positive and significant influence on citation scores.

Although not statistically significant, it is still worth considering the raw estimates of coefficients in the case of gender-related variables, more specifically, that of c.au , the corresponding author being a female researcher. The reason for this is the simple fact that our “sample” can even be conceived as the population itself, since our data were retrieved by delineating the field of ADS and using “all available” publications. This value is B(c.au)  = 0.36, meaning that female corresponding authors (in the sample) trigger a 43% higher citation score ( exp (0.36) = 1.43) on average than male corresponding authors (other factors held constant). Since an impact value above FNCS  = 1 already means a recognition level above the world average (within the field), this can be viewed as a considerable increment.

The overall structure of knowledge integration present in our data is first reported via visualization of the network of authors and research areas. Recall from the Methods section that this bipartite graph represents the relations between authors and their research fields (given the GIPP categorization scheme) by connecting each author to the research categories assigned to her/his papers. In the visualization (Fig.  5 ), the nodes corresponding to the six research areas are indicated by squares, while nodes corresponding to authors are shaped as small dots. To support the interpretation and increase visibility, two further modifications were applied. First, the edges of the network (connecting authors to fields each) were colored according to the research field involved in the relation (similarly, the borders of the research area nodes were colored the same way). Colors, therefore, were specific to research areas as follows: red edges signal connections to the Physical Sciences (PS), blue edges to the Life Sciences (LS), (3) green edges to the Clinical, Preclinical and Health Sciences (CP&H), orange edges to the Social Sciences (SS), yellow edges to the Arts and Humanities (A&H), and gray edges to Engineering and Technology (E&T). Additionally, as the second modification, the graph has been simplified (or “purified”) for a better structural view: only authors attached to at least two fields are included in the visualization; hence, the reported subgraph shows the pattern of interdisciplinarity within the data.

The bipartite graph represents the relationships between authors and research areas (according to the GIPP categorization scheme). Colors are specific to research areas as follows: red edges signal connections to the Physical Sciences (PS), blue edges to the Life Sciences (LS), green edges to the Clinical, Pre-Clinical and Health Sciences (CP&H), orange edges to the Social Sciences (SS), yellow edges to the Arts and Humanities (A&H), and grey edges to Engineering and Technology (E&T). The network is being presented with two layouts: the joint distribution of authors and research areas is represented (using the Fruchterman-Reingold layout algorithm), while the second visualization captures the basic structure of the network, where author nodes in the same position (related to the same set of fields) are being collapsed into a single node by overlaying one another (using a Multidimensional Scaling-based layout).

This pattern of interdisciplinarity, as observable through the graph, is strikingly clear. The strongest connection emerges between the Physical Sciences (PS) and the Life Sciences (LS), as signalled, on the one hand, by the close proximity of these two nodes at the centre of the network (the graph layout being sensitive to the number of connections, positioning densely related nodes close to each other) and, on the other hand, by the large number of authors (dots) with blue and red edges. This part of the network is joined with the rest through the Clinical, Preclinical and Health sciences (CP&H), which is in a typical knowledge broker position: many authors relate it to either the Physical or the Life Sciences or both, as witnessed by many authors (dots) with green, blue, and/or red edges. However, the area of CP&H also plugs in the social sciences (SS) into the system, as the latter is attached to the network through its interconnections with the CP&H area. This becomes visible through a distinct set of authors (dots) “on the North part” of the network, possessing green and orange edges. Finally, both the Arts and Humanities (A&H) and Engineering and Technology (E&T) behave as satellite areas. The humanities are pulled in through some connections with the social sciences , while engineering is mainly connected through the physical sciences . (In fact, as we shall later see, in some papers, the latter fields were present in broader combinations, but – as a further simplification – for this overview, we only kept the “backbone” of our graph where stronger links with at least two related publications per author were preserved.)

The analysis of knowledge integration scores (the LinkScore values) among authors yielded the distribution shown in Table  5 , which is a cross-tabulation of two variables, namely, gender and LinkScore. The latter was interpreted here as a categorical variable, for which some explanation is in order. Since, conceptually, the integration score is an author-level maximum over publication scores, the application of this measure to our sample resulted in five discrete values. More specifically, as expected from its design, the measurement resulted in.

1, for authors who have been active in a single field,

2 or 3, for those who have typically been connecting two or three more closely related areas of “hard science”,

6, for those who have typically linked a “hard science” field with either the social sciences or the humanities ,

9, for those who have been linking more than two fields, including the social sciences and/or the humanities .

The level of knowledge integration increases with the score. The test for the association of gender and LinkScore did not support that the author’s gender has a statistically significant relationship with the role in knowledge integration in general (χ 2  = 18.26, p = 0.14), and the association of these variables was also weak (Cramer’s V = 0.05).

Considering the distribution of LinkScores among female and male authors reported in Table  5 , it is indeed striking that the percentage of female authors is quite close to that of male authors in almost every category. The majority in both gender groups (45–50% of the authors) is a “specialist” in that these researchers have been active only in one field. A fair amount, however, belongs to the “short-distance interdisciplinary” category (approximately 20–30%), connecting 2 or 3 “hard science” fields. (Nonetheless, we should note that interconnections within the “hard sciences” category can also vary in the level of knowledge integration, as the Physical Sciences—Life sciences combination, which is rather frequent in our data as could be observed through the network visualization, potentially covers a greater distance than the Life Sciences—Clinical Sciences combination.) Authors with a value of 6, that is, connecting an SSH field with a hard science area, are an order of magnitude lesser, approximately 4–5% for both genders. Finally, the share of the most integrative authors with a value of 9, linking SSH fields with multiple hard science fields, is approximately 0.8–0.9% among female and male researchers as well. The distribution of female and male authors according to LinkScore categories is presented in Fig.  6 .

Comparison of LinkScore categories by gender

Based on our results, we also compiled a list of the most integrative researchers in the ADS field. This list is reported in Table  6 , where authors on outputs with a LinkScore of 9 and 6 – that is, all authors participating in research that embodies “long-distance interdisciplinarity” – are being collected. Within the table, researchers are ordered according to their gender so that the group of female researchers can easily be separated from the other gender group. Because LinkScore relates to a given output and because the measure is not size-dependent, it means that authors listed in any LinkScore categories in Table  6 are those who have at least one paper that fits the criteria for that category. In other words, the Linkscore measure captures the potential of an author to connect fields, not the overall performance in connecting fields.

Gender differences in research goal orientation

To explore gender differences in research goal orientation, 210 randomly selected publication abstracts were coded for three variations of the aims of the research: advancing science, societal impact or both. The key determinants of the coding were the primary purpose of the research and the consideration of external knowledge utilization. The coding criteria for the three categories of the aims of the research are presented in Table  2 . The results of the two coding exercises are compared in a confusion matrix (Table  7 ), based on which the Cohen’s Kappa statistic of the agreement between the two codings was calculated. Results show a considerable agreement of the two coding rounds (disagreement could detected on the Orientation type = 1: „scientific aim” category) with a Kappa value of 0.84 (usually interpreted as an indication of a „fairly good” agreement).

The general share of female- and male-authored papers was made equal. Half of the abstracts (n = 103, 49.0%) were coded for scientific progress as the main aim. The other half was split between aiming for societal progress (n = 44, 21.1%) and having dual aims (n = 63, 30.1%). Dominantly female papers were overrepresented among publications classified as aimed at scientific progress, while the share of male-authored papers was higher in publications classified as aimed at societal progress. Among publications classified as having both aims, the share of male-authored papers is higher by 8%. Contrary to other studies, female authors are engaged more in research we have classified as aimed at scientific progress, while male authors often value research we have classified as aimed at societal progress and research having both aims. Figure  7 represents the gender differences in the research goal orientation of dominantly female vs. dominantly male papers (χ 2  = 12.7, p = 0.002).

Gender differences in research goal orientation of dominantly female vs. dominantly male papers

We have included research areas in the analysis and displayed the distribution of subject areas. Among the coded papers, the subject areas of the anti-doping-related articles are mainly distributed in Clinical Sciences (n = 65), followed by the field of Social Sciences with 51 publications and Life Sciences, then Physical Sciences (n = 25). The share of dominantly female vs. non-female papers varies by the areas of research (Table  8 ). The largest imbalances in publications of different research areas are in Social Sciences (19 dominantly female papers vs 32 dominantly male papers) and Clinical Sciences (36 dominantly female papers vs 29 dominantly male papers).

Table 9 shows the distribution of all three categories of research goal orientation according to research areas. Within the 102 publications classified as aimed at scientific progress, anti-doping-related articles mainly occur in Clinical Sciences with 34 publications, Social Sciences with 17 and Physical Sciences; Life Sciences with 16 papers. The largest shares in publications classified as aimed at societal progress or having both aims appear also in the field of Clinical Sciences (n = 14; n = 17) and Social Sciences (n = 16; n = 18).

Figure  8 summarizes the analyzed abstracts according to research areas, gender differences, and research goal orientation. The largest shares of dominantly female papers that aimed at scientific impact were in Clinical Sciences, Social Sciences and Physical Sciences. In the field of engineering and technology, predominantly male papers have been published in all three research goal orientation categories.

Gender differences in research goal orientation by scientific fields

The primary aim of this work was to provide a novel approach for the analysis of research impact and identified actors within the anti-doping research landscape who play the role of connecting different research fields in their work. Specifically, we explored the characteristics and factors of the scientific impact of women’s contribution to ADS and detected potential gender differences in research goal orientation. In addition, we were interested in women’s contribution to the interconnections and combinations of different areas and fields behind anti-doping that would lead to effective interdisciplinarity, which is a requirement to address such complex and multi-faceted problems as doping. To realize these aims, we used a mixed method approach including regression models, network measures, and qualitative content analysis.

Academic impact

The results suggest that the domain of anti-doping exhibits fairplay conditions or a “healthy” ecosystem, where no gender bias against female authors can be detected. Furthermore, in terms of citation scores, the contribution of female authors with research excellence is even more impactful. This is reflected in the study outcomes that, on the one hand, most gender-related variables showed a small positive, albeit not statistically significant, effect. On the other hand, the higher rank of female authors in the paper’s author list (such as corresponding authorship or being the first author or being “among the first couple of authors”) still appeared to trigger a somewhat larger impact if we consider our data as the “population” of research output within ADS. What seemed to matter more to impact was the set of publication-related variables (as opposed to author-related ones), such as journal quality or the Open Access status. One exception to this is author productivity, which cannot be underestimated in scientific recognition. These results are very much in accord with recent studies on the relation between female authorship and academic impact (Thelwall, 2020b ). Furthermore, Hang et al. (2022) found no substantial or statistical significance of gender in shaping citation impact but identified gender-related author rank (namely, female author as first author) as the only factor that exerts some effect on citation scores. Previously, collaboration between two female researchers or with a female researcher was found to be more transient (Shen et al., 2022 ) than collaboration between male researchers, but collaboration pairs with a female partner yielded more citations.

Impact via knowledge transfer and integration

Regarding women's knowledge integration roles, the general structure of the female author group was similar to that of the male author group, and we can infer that no substantial gender differences can be detected here. However, small differences could still be observed in particular categories. First, female authors tend to be somewhat more interdisciplinary in that fewer female researchers were shown to be active in just one area (45% compared to 51% in the male gender group). Second, this increased interdisciplinarity is manifested mostly within the hard sciences: a higher share of female authors can be found in the LinkScore category of 2 and 3 than of male authors (28% vs. 24%, and 23% vs. 19%, respectively), meaning that it is slightly more common among women to work at the intersection of two or three areas within ADS. Finally, in the most integrative category ( LinkScore  = 6), where social sciences and humanities research fields are connected with more of the hard sciences, it is female authors again whose percentage is slightly higher (0.9% vs. 0.8%). Although these differences are not conclusive, they still reflect a tendency towards gender equity or slight superiority (on the part of female authors) regarding the knowledge integration aspect of scholarly impact. Our findings are consistent with those found by Elsevier's gender report ( 2021 ), indicating that in general, women's scholarly output includes a slightly higher proportion of highly interdisciplinary research compared to men's.

Research orientation

The content analysis of 210 randomly selected abstracts of anti-doping papers showed significant differences in the aims of research between dominantly female and dominantly male papers. Dominantly female papers contributed more to research aimed at scientific progress, while dominantly male papers are overrepresented among publications classified as aimed at societal progress or as having both aims. This finding is surprising and novel to the anti-doping literature. Zhang et al. find that female first authors are relatively more involved in research categorized as aimed at societal progress, while male researchers more often engage in research mainly aimed at scientific progress in different research fields. The results of Shang et al. are consistent with the findings of Zhang et al., indicating that female researchers more frequently participate in research aimed at societal progress. The authors concluded that female first authors are more likely to be involved in research related to Sustainable Development Goals.

However, our results suggest that focusing on specific multidisciplinary specialties (other than SDG) may reveal different patterns of gender involvement as is the case with our result.

This may be due to anti-doping being a relatively young and still emerging research field where established researchers, men, and women, took on the dual role of advancing the field (academic progress) and conducting research aiming for societal progress. Perhaps women have a stronger affinity to adapt to the (perceived) demands as well as the standards of the field. These results may reflect differing research priorities or interests between genders within the anti-doping community, however, Thelwall et al. ( 2019 ) demonstrated that gender disparities in research fields and topics cannot be solely attributed to inherent differences in people/thing interests or inclinations.

Relation of the study outcomes to the existing literature

Since the results of the present study apparently show a mixed relationship to the existing results, it is worth briefly summarizing its position – and considering its potential explanation – in the landscape of bibliometric research focusing on the relationship between gender and research impact.

For RQ1 (gender and citation impact), the purposive sample in Table  1 shows that although our results diverge from those of (Zhang et al., 2021 ), which served as the direct motivation for our study, several other studies did not find a significant effect of female authorship on citation impact, similar to our case. Among these studies were research reports on multidisciplinary and large samples collected on relatively wide time intervals. In our account, this „contradictory picture” is only seemingly contradictory: the contextual differences (time period, areas of research/levels of aggregation, maturity, and traditions of research domains) may vastly influence the gender-related patterns discerned from the study. In our case, the previously emphasized peculiarities of ADS (also among the reasons for its selection as the subject area for this research) can amplify this context-dependence of these findings (even more illustrative in this respect is the case for RQ3, see below).

Beyond contextual differences such as the scope and the subject matter analyzed in these studies, it is natural to assume that methodological differences also play an equally important role in accounting for discrepancies in respective study results. The methodological variability in question ranges from the whole research design to the choice of instruments for measuring academic impact, as we shall see below. We can gain sufficient insight into the role of methodology by invoking a previous study on the potential role of female authorship on academic impact in psychology in the US (Thelwall, 2020c ). Thelwall applied a very similar methodology to ours: they used a multivariate regression model to measure the effect of female authorship in various author positions on normalized citation counts. Notably, they argued for the application of the citation measure that we adopted here, the log-normalized citation score (LNCS), as providing valid (hence fair) comparisons between papers in terms of their recognition. The results of their study showed no (either practically or statistically) significant effect of gender on citation impact, except for a small positive effect of female first authorship, which is then termed „female citation advantage” as it could be reproduced in a series of further studies (cf. Thelwall, 2020b ). What makes this study exceptionally useful for our purposes is that the authors provided a very short, still, in a sense, comprehensive list of methodology-related reasons that may have led other bibliometric approaches to opposing results. As the authors claim, „ It is possible that misinterpretations of citation counts, such as use of the h-index, career citations, or un-normalised citation counts compared between fields, have incorrectly led to males gaining greater recognition. ” (Thelwall, 2020c , p. 693.) In other words, the authors deem the instruments used to measure citation impact as responsible for the differences, and also misleading, for example, the h-index, apart from not being an impact measure per se, is known to be unnormalized for many impact-unrelated factors affecting citation counts. Moreover, many of those measures are author-level indices – such as the h-index – aggregating paper-level citation counts to gauge the impact of the author. Though it is a tempting option, since authors can much more straightforwardly be sorted into gender categories (male, female), this step introduces many potential biases as well. In the first place, by summing over individual papers, the opportunity to control for the diverse paper-level factors behind citation impact (journal prestige, co-authorship, international vs. domestic collaboration, etc.) is mostly lost. This methodological difference implies a different research design, whereby (gendered) authors are compared on their impact, instead of the impact of papers being tested for the potential effect of authorship features. This design was applied in different research on psychology (González‐Álvarez & Cervera‐Crespo, 2019 ), along with paper-level comparisons within groups homogenized for several citation-related factors (number of authors, scope of collaboration), as a solution for controlling these factors even at aggregate levels. For the measurement of impact, however, raw citation counts were used and averaged over the female- and male- (first- and last-) authored papers. Even this setting resulted in very small differences between male and female papers in terms of effect size (according to the ANOVA test), though favoring male papers in this case. On Thelwall’s abovementioned account, however, raw citation counts for such a broad field as psychology (encompassing „soft” and „hard” areas) and sheer averages for highly skewed citation distributions profoundly distort the results – as demonstrated by his analysis of psychology on subfield-level and using a robust paper-level citation measure (LNCS) to preserve commensurability. In any case, this discourse conveys the sensitivity of the results to the selected methodology – in our case, we based our choice on Thelwall’s arguments.

For RQ3 (gender and research aim), the divergence in research orientation patterns found in (Zhang et al., 2021 ) is a more subtle issue. As an aspect of context-dependence, a specific feature in our method might have led to this difference: in the present context (ADS domain) we used an adapted version of the coding criteria in (Zhang et al., 2021 ) to categorize papers, but we argued for a modification specific to the research domain (ADS). Namely, most research in clinical medicine has been categorized here as having a scientific aim, the reason for which is discussed in detail in the description of coding. This modification, considering the high share of such studies in the sample, is one potential factor of the differences from the previous study. This alteration, however, was not arbitrary, but arguably domain- or field-related, reflecting a more adequate conceptualization of „societal aim” within the confines of the specific context (ADS). In sum, a detailed comparison dissolves the picture that the present results directly contradict existing evidence (as we may speak of differences, but not contradiction per se).

As is the case with the sensitivity of authorship effects on citation impact (RQ1) to the selected methodology, further literature also suggests a similar sensitivity on the relation of gender and research orientation (related to RQ3). Although this literature is very scarce, a specific study could be identified as directly reflecting our methodology. Santos et al. ( 2022 ) provided an approach to the relationship between academics’ research agendas and their preferences for basic research, applied research, or experimental development. In contrast to our qualitative approach, they used a questionnaire-based correlational design, a previously validated inventory for measuring the components of research agendas, and analyzed their effect on the weight of basic/applied research in academic practice. The latter was also measured via an instrument recording the proportion of time allocated to each, which served as a tool for cluttering them into “applied” and “basic” categories (similarly to our dichotomy on “social/societal” vs. “scientific” aims). Their sample consisted of 1160 respondents collected from various universities (no further specification was given, concerning e.g. the research fields sampled). Regarding their results, the authors explicitly pointed out that “ we do not find different research focus preferences between male and female academics, which is inconsistent with other studies indicating that male academics lean toward the basic sciences and female academics lean toward the applied sciences ( Zhang et al., 2021 ) ” (Santos et al., 2022 , p. 4212). In any case, we can suspect that alterations of the methodology lead to differing results: the quantitative methodology just described was associated with no difference, the qualitative methodology applied in Zhang et al. and our study showed an existing difference but with opposite signs, which we partially attributed to a different context-based modification of the coding criteria. Naturally, it is difficult to assess the individual contribution of the context, subject matter characteristics – e.g. research fields – or the methodological choice to the differences in the results, which is beyond the scope of the present study but would be worth further investigations. As we basically adopted the qualitative approach in a strict manner, we can assume, as discussed above, that the inconsistency of our study with the previous one on research aims and gender can be attributed – beyond the moderate alteration of the method – primarily to the context under study.

Limitations and future directions

This paper provides insight into the correlation between gender differences and research impact on anti-doping research. During the analysis, we faced several challenges; thus, our study is limited by several factors. One, for example, is that the pool of publications we used for our research was limited due to gender identification. Although we have chosen different methods to increase the precision of gender assignment, there are still a few authors' names we were unable to identify by full author names. Thus, the number of papers we could include in the analysis was restricted by a deficiency in gender assignment. The identification of the remaining author names would allow us to further increase the set of anti-doping publications suitable for analysis.

Another limitation is related to the sample of publication abstracts for examining research goal orientation. To select the abstracts, we used stratified sampling, in which we partitioned the publications based on the gendered characteristics of the papers. Then, we selected 105 abstracts independently from dominantly female papers and dominantly male papers. Despite demonstrated gender differences in research goal orientation by scientific fields in the results, the analyzed sample cannot be considered representative of the research field. Focusing on the research field in the sampling would give a more robust result regarding gender differences in research goal orientation in subfields of anti-doping research.

We should note that our study did not incorporate funding information on anti-doping scholarly output as a variable among the main factors of citation impact. Funding has been studied empirically as an explanatory variable affecting the citation impact of individual publications elsewhere (e.g., Roshani et al., 2021 ; Yan et al., 2018 ). Having funding supporting the research can be attributable to the research impact of scholarly communication, and funded projects can contribute to the advancement of research cooperation and collaborative publishing. However, it was also noted that a direct link between research funding and output cannot be drawn because research outcomes from grant-funded work often generate multiple outputs that are tailored to different audiences with different foci (Rigby, 2011 ). Furthermore, funding is essential in some subfields of anti-doping (e.g., experimental research), while applying for funding within the subfield of social sciences may not be vital in all cases. Since an assessment of the impact obtained via the output of individual projects on the advancement of anti-doping research can be analysed through the measurement of citation impact, mapping such an impact of funded projects by major organizations operating in the anti-doping research area would be recommended in future studies. Future research could also look into the impact of gender in the peer-review process of grant applications and publications (Bianchini et al., 2022 ; Chubb & Derrick, 2020 ; Squazzoni et al., 2021 ; Tricco et al., 2017 ).

Contributing to advancing the field from a bibliometric viewpoint, we developed a new score for the assessment of authors whose work links different research fields. The LinkScore allows detecting to what extent authors participate in “integrative” research that connects different areas as well as quantifying the capacity of the author’s work in linking research fields. Applying LinkScore in future scientometric analyses is a feasible approach to detect gender differences in knowledge integration in other research areas that appear to be highly interdisciplinary.

Additional future research questions revolve around the influence of male dominance in ADS on our understanding of the doping problem and proposed solutions to date. The results presented in this paper are not able to answer this question, but future research could build on the results we presented and employ citation path analysis to follow the influence of the ‘star’ men and women researchers identified in our previous study (Kiss et al., 2022 ) and via LinkScore in the current study.

This study provides a novel approach for the analysis of research impact and research goal orientation within the anti-doping research landscape. Specifically, we explored the role of women’s contribution to the scientific impact of ADS and detected potential gender differences in research goal orientation. The large-scale, data-driven bibliometric models we used in our study, can account for the structure, dynamics, and drives of both the global system of sciences and such specific, complex, and multidisciplinary (that is, otherwise hardly characterizable) constellations as the field of anti-doping sciences.

The results from this study showed that the gender gap observed in anti-doping scientific outputs wanes for research impact. The characteristics of female authorship (e.g., gender of the corresponding author) do not have an effect on citation impact except for gender-related author rank (female first author), which has a positive influence on citations for women scientists. The trend-defying picture in ADS may not be because women’s ADS are different, but it could be the consequence of the field being relatively young and still emerging. Future research is needed to further elucidate the underlying factors of women’s success in ADS. Regarding research goal orientation, dominantly female papers were overrepresented among publications classified as aimed at scientific progress, while the share of male-authored papers was higher in publications classified as aimed at societal progress. A better understanding of the dynamics of women’s research priorities or interests will offer valuable insight into how anti-doping scientific advances. Anti-doping research is in a unique position due to its multidisciplinary nature, in essence, scholars can truly grasp the richness and complexity of this research field only by applying a wide range of methodologies.

Agulló-Calatayud, V., González-Alcaide, G., Valderrama-Zurián, J. C., & Aleixandre-Benavent, R. (2008). Consumption of anabolic steroids in sport, physical activity and as a drug of abuse: An analysis of the scientific literature and areas of research. British Journal of Sports Medicine, 42 (2), 103–109. https://doi.org/10.1136/bjsm.2007.036228

Article   Google Scholar  

Bianchini, S., Llerena, P., Öcalan-Özel, S., & Özel, E. (2022). Gender diversity of research consortia contributes to funding decisions in a multi-stage grant peer-review process. Humanities and Social Sciences Communications, 9 , 195. https://doi.org/10.1057/s41599-022-01204-6

Chan, H. F., & Torgler, B. (2020). Gender differences in performance of top cited scientists by field and country. Scientometrics, 125 , 2421–2447. https://doi.org/10.1007/s11192-020-03733-w

Chang-Yeon, K., Sivasundaram, L., Trivedi, N., Gilmore, A., Gillespie, R., Salata, M., Liu, R. W., & Voos, J. E. (2019). A 46-year analysis of gender trends in academic authorship in orthopaedic sports medicine. JAAOS, 27 , 13. https://doi.org/10.5435/JAAOS-D-18-00669

Chubb, J., & Derrick, G. E. (2020). The impact a-gender: gendered orientations towards research impact and its evaluation. Palgrave Communications . https://doi.org/10.1057/s41599-020-0438-z

D’Amico, R., Vermigli, P., & Canetto, S. S. (2011). Publication productivity and career advancement by female and male psychology faculty: The case of Italy. J Divers High Educ., 4 (3), 175–184. https://doi.org/10.1037/a0022570

DesRoches, C. M., Zinner, D. E., Rao, S. R., Iezzoni, L. I., & Campbell, E. G. (2010). Activities, productivity, and compensation of men and women in the life sciences. Academic Medicine, 85 (4), 631–639. https://doi.org/10.1097/ACM.0b013e3181d2b095

Dynako, J., Owens, G. W., Loder, R. T., Frimpong, T., Gerena, R. G., Hasnain, F., Snyder, D., Freiman, S., Hart, K., Kacena, M. A., & Whipple, E. C. (2020). Bibliometric and authorship trends over a 30 year publication history in two representative US sports medicine journals. Heliyon, 6 , 3. https://doi.org/10.1016/j.heliyon.2020.e03698

Elsevier. (2021).  Gender in the Global Research Landscape . Retrieved January 25, 2022, from  https://www.elsevier.com/__data/assets/pdf_file/0008/265661/ElsevierGenderReport_final_for-web.pdf

Engelberg, T., & Moston, S. Somebody else’s problem. A decade of doping research in leading sport management journals. Sport Management Review , 19 (1), 10–1016.

Gal, D., Sipido, K., & Glänzel, W. (2015). Using bibliometrics-aided retrieval to delineate the field of cardiovascular research. In ISSI.

González-Álvarez, J., & Cervera-Crespo, T. (2019). Contemporary psychology and women: A gender analysis of the scientific production. International Journal of Psychology, 54 (1), 135–143.

Henderson, M. T., Fijalkowski, N., Wang, S. K., Maltenfort, M., Zheng, L. L., Ratliff, J., et al. (2014). Gender differences in compensation in academic medicine: The results from four neurological specialties within the University of California Healthcare System. Scientometrics, 100 (1), 297–306. https://doi.org/10.1007/s11192-014-1266-y

Kiss, A., Lakner, Z., Soós, S., & Petróczi, A. (2022). Women’s footprint in anti-doping sciences: A bibliometric approach to research impact. Frontiers in Sports and Active Living, 4 , 866648. https://doi.org/10.3389/fspor.2022.866648

Kwiek, M., & Roszka, W. (2021). Gender disparities in international research collaboration: A study of 25,000 university professors. Journal of Economic Surveys, 35 (5), 1344–1380.

Larivière, V., Haustein, S., & Börner, K. (2015). Long-distance interdisciplinarity leads to higher scientific impact. PLoS ONE, 10 (3), e0122565. https://doi.org/10.1371/journal.pone.0122565

Larivière, V., Ni, C., Gingras, Y., Cronin, B., & Sugimoto, C. R. (2013). Bibliometrics: Global gender disparities in science. Nature, 504 (7479), 211–213. https://doi.org/10.1038/504211a

Leta, J., & Lewison, G. (2003). The contribution of women in Brazilian science: A case study in astronomy, immunology and oceanography. Scientometrics, 57 (3), 339–353. https://doi.org/10.1023/A:1025000600840

Leydesdorff, L., Carley, S., & Rafols, I. (2013). Global maps of science based on the new Web-of-Science categories. Scientometrics, 94 , 589–593.

Loder, R. T., Kacena, M. A., Ogbemudia, B., Ngwe, H. N., Aasar, A., Ninad, N., Mufti, O., Gunderson, Z., & Whipple, E. C. (2021). Bibliometric analysis of the English musculoskeletal literature over the last 30 years. The Scientific World Journal . https://doi.org/10.1155/2021/5548481

Martínez-Rosales, E., Hernández-Martínez, A., Sola-Rodríguez, S., Esteban-Cornejo, I., & Soriano-Maldonado, A. (2021). Representation of women in sport sciences research, publications, and editorial leadership positions: Are we moving forward? Journal of Science and Medicine in Sport, 24 (11), 1093–1097. https://doi.org/10.1016/j.jsams.2021.04.010

Maske, K. L., Durden, G. C., & Gaynor, P. E. (2003). Determinants of scholarly productivity among male and female economists. Economic Inquiry, 41 (4), 555–564. https://doi.org/10.1093/ei/cbg027

Mujika, I., & Taipale, R. S. (2019). Sport science on women, women in sport science. International Journal of Sports Physiology and Performance, 14 (8), 1013–1014. https://doi.org/10.1123/ijspp.2019-051410.1123/ijspp.2019-0514

UNESCO. (2024). Call for action “Closing the gender gap in science”. from https://www.unesco.org/en/articles/closing-gender-gap-science-accelerating-action

OECD. (2021). Women are under-represented in the population of scientific authors . Retrieved October 8, 2022, from https://www.oecd.org/gender/data/it-is-time-to-close-the-gender-gap-in-research.htm

Pashkova, A. A., Svider, P. F., Chang, C. Y., Diaz, L., Eloy, J. A., & Eloy, J. D. (2013). Gender disparity among US anesthesiologists: Are women underrepresented in academic ranks and scholarly productivity? Acta Anaesthesiologica Scandinavica, 57 (8), 1058–1064. https://doi.org/10.1111/aas.12141

Pulido, C. M., Redondo-Sama, G., Sordé-Martí, T., & Flecha, R. (2018). Social impact in ocial media: A new method to evaluate the social impact of research. PLoS ONE, 13 (8), e0203117.

Rigby, J. (2011). Systematic grant and funding body acknowledgement data for publications: New dimensions and new controversies for research policy and evaluation. Research Evaluation, 20 (5), 365–375. https://doi.org/10.3152/095820211X13164389670392

Roshani, S., Bagherylooieh, M. R., Mosleh, M., & Coccia, M. (2021). What is the relationship between research funding and citation-based performance? A comparative analysis between critical disciplines. Scientometrics, 126 (9), 7859–7874. https://doi.org/10.1007/s11192-021-04077-9

Ryan, P. J., Spencer, T., Asheesh, B., & Bryson, P. L. (2020). Ten years of sports health: Authorship characteristics and levels of evidence. Sports Health, 12 (6), 573–578. https://doi.org/10.1177/1941738120922163

Santos, J. M., Horta, H., & Luna, H. (2022). The relationship between academics’ strategic research agendas and their preferences for basic research, applied research, or experimental development. Scientometrics, 127 (7), 4191–4225.

Shang, Y., Gunnar, S., Zhe, C., & Zhang, L. (2022). Gender differences among first authors in research focused on the Sustainable Development Goal of Gender Equality. Scientometrics, 127 , 4769–4796. https://doi.org/10.1007/s11192-022-04430-6

Shen, H., Xie, J., Ao, W., & Cheng, Y. (2022). The continuity and citation impact of scientific collaboration with different gender composition. Journal of Informetrics, 16 (1), 101248. https://doi.org/10.1016/j.joi.2021.101248

Snell, C., Sorensen, J., Rodriguez, J. J., & Kuanliang, A. (2009). Gender differences in research productivity among criminal justice and criminology scholars. Journal of Criminal Justice, 37 (3), 288–295. https://doi.org/10.1016/j.jcrimjus.2009.04.009

Squazzoni, F., Bravo, G., Farjam, M., Marusic, A., Mehmani, B., Willis, M., Birukou, A., Dondio, P., & Grimaldo, F. (2021). Peer review and gender bias: A study on 145 scholarly journals. Science Advances, 7 (2), 0299. https://doi.org/10.1126/sciadv.abd0299

Tahamtan, I., Safipour Afshar, A., & Ahamdzadeh, K. (2016). Factors affecting number of citations: A comprehensive review of the literature. Scientometrics, 107 , 1195–1225. https://doi.org/10.1007/s11192-016-1889-2

Thelwall, M. (2020a). Gender differences in citation impact for 27 fields and six English-speaking countries 1996–2014. Quantitative Science Studies, 1 (2), 599–617. https://doi.org/10.1162/qss_a_00038

Thelwall, M. (2020b). Female citation impact superiority 1996–2018 in six out of seven English-speaking nations. Journal of the Association for Information Science and Technology, 71 (8), 979–990. https://doi.org/10.1002/asi.24316

Thelwall, M. (2020c). Author gender differences in psychology citation impact 1996–2018. International Journal of Psychology, 55 (4), 684–694. https://doi.org/10.1002/ijop.12633

Thelwall, M., Bailey, C., Tobin, C., & Bradshaw, N. A. (2019). Gender differences in research areas, methods and topics: Can people and thing orientations explain the results? Journal of Informetrics, 13 (1), 149–169. https://doi.org/10.1016/j.joi.2018.12.002

Tricco, A. C., Thomas, S. M., Antony, J., Rios, P., Robson, R., Pattani, R., et al. (2017). Strategies to prevent or reduce gender bias in peer review of research grants: A rapid scoping review. PLoS ONE, 12 (1), e0169718. https://doi.org/10.1371/journal.pone.0169718

Yan, E., Wu, C., & Song, M. (2018). The funding factor: A cross-disciplinary examination of the association between research funding and citation impact. Scientometrics, 115 , 369–384. https://doi.org/10.1007/s11192-017-2583-8

Zhang, L., Sivertsen, G., Du, H., Huang, Y., & Glänzel, W. (2021). Gender differences in the aims and impacts of research. Scientometrics, 126 (11), 8861–8886. https://doi.org/10.1007/s11192-021-04171-y

Zuckerman, H., & Cole, J. R. (1975). Women in American science. Minerva, 13 , 82–102.

Download references

Open access funding provided by Library and Information Centre of the Hungarian Academy of Sciences. The work received support from Kingston University Impact Fund 2022–2023.

Author information

Authors and affiliations.

Library and Information Centre, Department of Scientometrics and Science Policy, Hungarian Academy of Sciences, Budapest, Hungary

Anna Kiss & Sándor Soós

Faculty of Education and Psychology, Eötvös Loránd University (ELTE), Budapest, Hungary

Anna Kiss, Sándor Soós & Andrea Petróczi

Pro-Sharp Research and Innovation Centre, Budapest, Hungary

School of Life Sciences, Pharmacy and Chemistry, Faculty of Health, Science, Social Care and Education, Kingston University, London, UK

Andrea Petróczi

Department of Movement Sciences, KU Leuven, Louvain, Belgium

Willibald Gebhardt Institute, University of Münster, Münster, Germany

You can also search for this author in PubMed   Google Scholar

Contributions

Conceptualization: Andrea Petróczi; Methodology: Anna Kiss, Sándor Soós; Formal analysis and investigation: Anna Kiss, Sándor Soós, Andrea Petróczi; Writing—original draft preparation: Anna Kiss, Sándor Soós; Writing—review and editing: Andrea Petróczi; Funding acquisition: Andrea Petróczi.

Corresponding author

Correspondence to Sándor Soós .

Ethics declarations

Conflict of interest.

The authors have no competing interests and the manuscript complies with the ethical of standards of the journal.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Validation of the LinkScore measure

In order to justify the introduction and use of the LinkScore measure (LS), various attempts have been made to provide an initial validation of this tool, via testing its convergent validity with respect to existing indices on knowledge integration. Most importantly, we investigated the agreement between the level of interdisciplinarity as measured by the LS versus a family of knowledge integration measures based on the Rao-Stirling index (RS), widely used in the literature (Leydesdorff et al., 2013). The reason behind this choice was twofold: (1) the RS is typically applied on a more granular level than our LS, namely, on the level of WoS Subject Categories, and (2) this measurement type accounts for the cognitive proximities of the research fields (WoS SCs) being covered by an actor’s research portfolio. Our goal in the validation attempt was to demonstrate whether the LS, as a more robust and more easily applicable and interpretable tool (not being dependent on extensive computations and access to the full content of indexing databases as is the RS index) sorts authors into ranked categories that mirror their levels of interdisciplinarity along the RS index.

To that end, a modified version of the RS index was calculated for each author in our sample (RSm). Since the main conceptual difference between the RS and the LS index was that the latter did not directly reflect the micro-level cognitive organization of the publication record, lacking cognitive proximities/distances as a factor in its calculation, we only retained this latter feature of the RS index to amplify potential divergence based on this theoretical (and technical) difference. In particular, we modified it to convey the „average cognitive distance” within an author’s publication set:

where n is the number of SCs associated with the publication set, s ij is the proximity of the i- th and j -th Subject Category ( i,j  ≤  n ) measured by the cosine similarity of their reference patterns (their vectors of aggregated references to other SCs) within a global science map. We used the global science map based on the Web of Science (that is, the similarity matrix of WoS SCs) published by Leydesdorff et al., 2013, downloadable from https://www.leydesdorff.net/ ).

As a means of statistical comparison between the RS and LS scores, as LS scores could most naturally be characterized as author categories w.r.t. the degree of knowledge integration, an ANOVA test was used (with post-hoc comparisons). The test showed that the differences in the average cognitive distance (RSm) were strongly significant between LS categories (F(4) = 461.8, p < 0.001***). The mean values of RSm are reported in Table  10 . Post-hoc comparisons and HSD Tukey tests were conducted to assess the pairwise differences of LS categories (or scores) in terms of RS scores (Fig.  9 ). The LS category mean values confirm that higher levels of knowledge integration on the LS scale implies higher mean cognitive distance, on average, that is, a greater extent of knowledge integration on the RS scale. Furthermore, the differences between the categories are, in most cases, significantly differ from zero, as conveyed by the Tukey plot. More precisely, there is a significant difference among lower levels (category 1, category 2), as between lower and higher levels of interdisciplinarity (category 3, 6, 9). Notably, higher levels do not differ significantly among themselves.

The Tukey plot of pairwise comparisons of the differences between LinkScore classes (along with RS scores)

These results can be interpreted as a fairly good initial confirmation of the use of the LS score, despite the low discrimination between its highest levels (3–9) by the RSm, since the latter categories are all meant to represent high levels of knowledge integration, while LS = 2 can be considered as moderate, and LS = 1 as the „zero” level. These watersheds are well reflected in the corresponding RSm ranges (Table  10 ).

Another concern that can emerge regarding the definition of the LinkScore measure is the selected aggregation method: namely, taking the maximum of the publication-level LS values to characterize the knowledge integration capacity for the author. An implication of this method is that authors with e.g. one highly interdisciplinary paper (LS = 9) are deemed more interdisciplinary than authors with several, perhaps many, but somewhat less interdisciplinary publications each (3 ≤ LS < 9), which is a potential bias. In addressing this bias, it should first be noted that such an implication is in line with our purposes to define knowledge integration capacity as the range of knowledge integration for authors, instead of the frequency or share of interdisciplinary papers in their record. In this sense, this behavior of the LS does not violate the conceptual validity of the measure. However, in order to further investigate the empirical relation between these two perspectives (range vs. frequency), we made a systematic comparison between two versions of the LinkScore metric, differing only in the aggregation method. In particular, we compared (1) the original LS with the maximum value and (2) an LS with the average value of the underlying publications’ LinkScore (the latter was meant to represent the „frequency-based” perspective on knowledge integration capacity). An ANOVA test was applied with post-hoc comparisons of the mean differences of individual LS categories in terms of their average LinkScore values. We found a significant overall difference between LS categories (F(4) = 3308, p < 0.001***), and a strict monotonic increase in the category means of the average LS values for successive LS ranks relying upon the maximum-based aggregation method (Table  10 ). Furthermore, there is a difference significantly differing from zero between each pair of LS categories, as reported in the post-hoc tests (Tukey plot, Fig.  10 ). All these results clearly evidence that the maximum-based LS-definition empirically aligns with the average-based LS-definition, that is to say, higher ranges of knowledge integration are associated with higher frequencies of interdisciplinary publications at the level of authors, which conclusion renders the potential bias in question less of a violation to construct validity even for a broader concept of interdisciplinarity.

The Tukey plot of pairwise comparisons of the differences between LinkScore classes (along with average LinkScore values)

Validation of the impact type categorization

The categorization of selected papers into the „impact orientation” classes defined in our research was based on a qualitative analysis of the papers’ abstracts. In order to improve and test the quality of the analysis we, besides canonical tests of reliability (see below), made an attempt to support this categorization with a quantitative approach, and investigate the convergent validity of the two approaches. To that end, we selected a quantitative indicator that can be argued to covary with the type of orientation (mainly scientific / mainly social) for individual publications. It has become quite common in scientometrics to use so-called „usage indicators” as proxies for measuring social impact. We considered these indicators as less applicable in the present context. It should be noted that there is vast criticism as to their content validity in the literature (Pulido et al., 2018 ), whether it is the social impact of research that these indicators convey. Nonetheless, our main reason was that the construct for which we coded the sample was not social impact per se, but rather the orientation of the paper towards „basic” or „pure” scientific research vs. direct applications (that is, the achievement of „social value”). Usage indicators in themselves are arguably not ideal for differentiating between these types.

Instead of usage indicators, we capitalized on a feature of publications that can directly be linked to the research goal in these terms, viz . funding data. It is reasonable to assume that the type of funding/funder imposes certain constraints on the content of the reported research. In particular, we considered funders with clear science-related missions (such as universities, research councils, and bodies funding basic research) as supporting basic research or scientific orientation, and funders with clear „social missions” or bodies involved in research valorization (such as anti-doping agencies, Olympic committees, governmental bodies concerned with sport and health policy) as supporting applied or – in this sense – socially oriented research. Typical examples from our sample are listed in the following Table  11 .

Given these considerations, the validation process was undertaken in the following steps. (1) We have coded the unique funding bodies listed in the „funding information” field (FU) of the WoS sample into a funder type variable with values {0, 1}. Funder type = 0 represented funders with science-oriented missions or basic research, and funder type = 1 stood for funders with social^societal or research valorization missions. (2) For each publication P we calculated a metric that detected the weight of funders with social missions among all the funders reported for P ( Share of funder type  =  1 ). Given the coding, for any publication P with n reported funders, this could easily be described by the formula

(3) In the final step., we compared the three Impact type categories by the funder type statistics of papers belonging to them.

In the sample, only a subset of papers (n = 90) contained sufficient funding information (at least one reported funder) for calculating the statistics described above. For this subsample we applied explorative statistics, the average of the Funder type measure in each category as a means of comparison. The results are shown in Table  12 . It is striking from the table that (1) the categories are well separated by the average Funder type, and (2) their differences are well aligned with their type of research orientation they represent. In particular, the smallest average value (~ 0.6), that is, the smallest average share of social mission oriented funders pertains to Impact type category = 1, coding for „pure” scientific research goals. The highest (in fact, 100%) value is tied to Impact category = 2, that is, primarily social missions in research. The mixed category (Impact category = 3) presents a value in between. In sum, the present quantitative attempt to support the qualitative analysis confirms the results of the coding procedure applied on the sample.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Kiss, A., Soós, S. & Petróczi, A. Impact as equalizer: the demise of gender-related differences in anti-doping research. Scientometrics (2024). https://doi.org/10.1007/s11192-024-05094-0

Download citation

Received : 02 August 2023

Accepted : 19 June 2024

Published : 29 June 2024

DOI : https://doi.org/10.1007/s11192-024-05094-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Research impact
• Anti-doping
• Interdisciplinary research
• Bibliometric analysis
• Find a journal
• Publish with us
• Track your research

Development co-operation

The OECD designs international standards and guidelines for development co-operation, based on best practices, and monitors their implementation by its members. It works closely with member and partner countries, and other stakeholders (such as the United Nations and other multilateral entities) to help them implement their development commitments. It also invites developing country governments to take an active part in policy dialogue.

• Development Co-operation Report
• Official development assistance (ODA)

Select a language

Key messages, charting development co-operation trends and challenges.

The OECD keeps track of key trends and challenges for development co-operation providers and offers practical guidance. It draws from the knowledge and experience of Development Assistance Committee (DAC) members and partners, as well as from independent expertise, with the ultimate goal of advancing reforms in the sector, and achieving impact. Using data, evidence, and peer learning, this work is captured in publications and online tools that are made publicly available.

Making development co-operation more effective and impactful

The OECD works with governments, civil society organisations, multilateral organisations, and others to improve the quality of development co-operation. Through peer reviews and evaluations, it periodically assesses aid programmes and co-operation policies, and offers recommendations to improve their efficiency. The OECD also brings together multiple stakeholders to share good and innovative practices and discuss progress.

Strengthening development co-operation evaluation practices and systems

The OECD helps development co-operation providers evaluate their actions both to better learn from experience and to improve transparency and accountability. Innovative approaches, such as using smart and big data, digital technology and remote sensing, help gather evidence and inform policy decisions. With in-depth analysis and guidance, the Organisation helps providers manage for results by building multi-stakeholder partnerships and adapting to changing contexts and crisis situations. 

Civil society engagement in development co-operation

National and international civil society organisations (CSOs) are key partners in monitoring development co-operation policies and programmes. Development co-operation can also be channelled to or through CSOs: 

Aid is characterized as going to CSOs when it is in the form of core contributions and contributions to programmes, with the funds programmed by the CSOs. 

Aid is characterized as going through CSOs when funds are channeled through these organisations to implement donor-initiated projects. This is also known as earmarked funding.

Development co-operation TIPs - Tools, Insights, Practices

TIPs is a searchable peer learning platform that offers insights into making policies, systems and partnerships more effective. 

Related data

Related publications.

Related policy issues

• Development co-operation evaluation and effectiveness
• Development co-operation in practice
• Development co-operation peer reviews and learning
• Innovation in development co-operation

• DOI: 10.37278/bhaktikaryadaninovatif.v3i2.735
• Corpus ID: 265728822

Meningkatkan Brand Awareness UMKM Di Wilayah Kecamatan Bojongloa Kidul Bandung Melalui Social Media Ads Menggunakan Video Promosi

• Nichi Hana Karlina , Budiman Budiman , Aggi Panigoro
• Published in Jurnal Bhakti Karya dan… 19 October 2023
• Jurnal Bhakti Karya dan Inovatif

Related Papers

Showing 1 through 3 of 0 Related Papers