Music affects viewers’ responses to advertisements. In this study we present the findings of an experiment that investigates the emotional and cognitive reactions of subjects’ brains during exposure to television advertisements with music congruent, and incongruent, with the advertisement content. We analyze the electroencephalography signals and eye-tracking behaviors of a group of 90 women watching six TV advertisements. The study's findings suggested that incongruent music generates higher levels of attention and advertisement recall. On the other hand, frontal asymmetry measured through electroencephalography was shown to be higher with congruent music. Similarly, cognitive workload was higher when the music was incongruent with the advertisement content. No significant differences were found in terms of advertisement likeability based on incongruent versus congruent music. The results demonstrated the validity of neurophysiological techniques for assessing the effects of levels of music congruence in advertisements.
Listening to music is one of the most pleasurable experiences for individuals (Salimpoor, Benovoy, Longo, Cooperstock & Zatorre, 2009). Its influence on behavioral intentions has been shown in a meta-analysis at the in-store level (Roschk, Loureiro & Breitsohl, 2017). In advertising, music is present in 86% of television advertisements internationally (Allan, 2008), in up to 94% of German commercials (Breves, Herget & Schramm, 2020), and is considered a "catalyst of advertising" (Hecker, 1984). It is often used as a background to advertisements to provoke consumer preference (Gorn, 1982), to increase levels of product recall and purchase intent (Lavack, Thakor & Bottausci, 2008; Oakes, 2007), and to generate emotions (Alpert & Alpert, 1990; Bruner, 1990; Stout & Leckenby, 1988; Ausin, Guixeres, Bigné & Alcañiz, 2017). Indeed, advertising studies have demonstrated the decisive emotional role of music in decision-making (Vermeulen & Beukeboom, 2016). The role of music as an emotion generator has also been demonstrated in brain activation studies (Schaefer, Desain & Farquhar, 2013) and in the brain cortex (Koelsch, Skouras & Lohmann, 2018). However, some research has argued that the mechanisms through which music elicits emotions are still unclear (Valla, Alappatt, Mathur & Singh, 2017), and has advocated analysis of emotions other than pleasure (Reybrouck, Eerola & Podlipniak, 2018).
Despite the abundant literature, controversy remains about the influence that music exerts on advertising response variables (Craton, Lantos & Leventhal, 2017). Some studies have demonstrated that music increases advertisement brand and message recall (Allan, 2008; Hecker, 1984; Hoyer, Srivastava & Jacoby, 1984), but others have disagreed on the effectiveness of music in increasing advertisement recall (Olsen, 1995; Stout & Leckenby, 1988). This discrepancy may be due to various factors, particularly to the study methodologies employed (Fraser, 2014), and to the difficulty of capturing feelings and emotions through self-reports (Salimpoor et al., 2009). In the present study we analyze the internal neurophysiological mechanisms that trigger emotional reactions to music in TV advertisements.
Congruence between music and visual content has also been used to explain this discrepancy in results (Jeong et al., 2011). Indeed, congruence facilitates the “processing fluency” effect (Breves et al., 2020) by merging the cognitive structures that ultimately positively affect consumers’ evaluations of TV advertisements and brands. However, when music and visual elements are incongruent, the viewer may have to reconfigure their meanings. Nevertheless, the relevant literature is inconclusive. For instance, while Moorman (2002) found that advertisement recall improved for advertisements that were congruent with the thematic content in magazines, Dhalen et al., (2008) demonstrated the benefits of incongruence for brand associations in thematically incongruent media.
The authors of the present study analyze, using neurophysiological measures, the role of (in)congruence in music and content in responses to TV advertising. The objective is twofold: First, to analyze the possible neurophysiological variations in outcome variables due to advertising featuring (in)congruent music; second, building on schema theory and the elaboration likelihood model (ELM) (Petty & Cacioppo, 1986), we assess how (in)congruent music affects cognitive processes (e.g., memory and level of likeability).
Two conceptual frameworks illuminate the analysis of music in advertising. First, schema theory (Bartlett, 1932) explains how individuals categorize pieces of information (Rifon, Choi, Trimble & Li, 2004). Second, the ELM (Petty, Cacioppo & Schumann, 1983; 1986), where music works as a low-involvement peripheral cue, or as a cue supporting arguments in a message (Eisend & Tarrahi, 2016). These conceptual frameworks and their hypotheses have been tested by means of self-reported studies. However, internal psychological responses to emotional stimuli, for example, unconscious processes and music, cannot be captured through self-reports (Pozharliev, Verbeke & Bagozzi, 2017). In recent times, consumer neuroscience-based studies (Karmarkar & Plassmann, 2019) have contributed to research into advertising (Chang, 2017; Guixeres et al., 2017). Advertising research using neuroscience has hitherto focused on four key constructs as indicators of the effectiveness of advertisements: attention, memory, affect, and preference (Pozharliev et al., 2017; Venkatraman et al., 2015). We retain these four key constructs, but add cognitive workload.
Our approach differs from previous studies in two ways. First, we analyze consumer responses by means of neurophysiological continuous data. This approach is consistent with the view of human cognition as a dynamic process (Thelen & Smith, 1994). Second, the limited capacity model of motivated mediated message processing (LC4MP) proposes that processing messages involves three simultaneous major activities, encoding, storage (i.e. linking of recently encoded cues to previously stored cues), and retrieval (i.e. memory activation) (Lang, 2000). Our approach attempts to cover both types of memory. Indeed, neurophysiological measures capture encoding and storing subprocesses, while self-report measures capture the retrieval process. Prior research has, on the other hand, focused mainly on the retrieval process by asking consumers about their recall, liking, feelings, and emotions.
This research contributes to the existing literature in three ways. First, by testing, through neurophysiological measures, how viewers respond to TV advertising accompanied by (in)congruent music. Second, by evaluating the impact of (in)congruent music in TV commercials on attention, memory, liking, cognitive workload, and preference. The remainder of the present study is organized as follows. First, we discuss the main features of the neurophysiological tools used. Thereafter, to develop the hypotheses, we review the literature on congruence and advertisement content. Next, we describe the method used to test the hypotheses; through an experimental design we evaluate cognitive variables of responses to advertising, attention, memory, advertisement likeability, cognitive workload as a proxy for processing effort, and frontal asymmetry as a proxy for advertisement preference. Then we discuss the results and implications, both conceptual and managerial and, last, we address the conclusions and limitations of the study.
2Literature review2.1Measurement of neurophysiological and emotional reactions to advertisingThe growing acceptance of neuroscientific research tools in consumer research is based on the accurate measurement of consumer processing, including unconscious processes (Ohme, Reykowska, Wiener & Choromanska, 2010), and the neurophysiological reactions of consumers to marketing stimuli (Karmarkar & Plassmann, 2019) in continuous real-time recording. Neuroscientific tools can accurately analyze responses to (in)congruent music by tracking unconscious and emotional effects. Indeed, growing attention is being paid to the psychophysiological analysis of music-induced emotions (Gingras, Marin, Puig-Waldmüller & Fitch, 2015; Ohme, Matukin & Pacula-Lesniak, 2011). However, no studies using neurophysiological tools have considered music and advertisement content congruence. This study adopts a multitool method by using the following measurements:
2.1.1Eye-Tracking (ET)ET registers where the experimental subject is looking at each moment, and the sequencing of his/her gaze (Bulling & Wedel, 2019). The growing literature on ET, now widely used as a measure of attention in advertisement studies, has shown that visual attention has systematic effects on brand memory, and can predict responses to advertising (Pieters & Wedel, 2004). More specifically, Wedel and Pieters (2000) found a positive, direct correlation between average fixation duration and recall.
2.1.2Electroencephalography (EEG)EEG measures electrical brain activity based on temporal resolution by means of electrodes placed on the scalp. Previous studies have investigated memory, emotions, and attention (Gordon, Ciorciari & van Laer, 2018; Guixeres et al., 2018; Lin, Cross, Jones & Childers, 2018; Shestyuk, Kasinathan, Karapoondinott, Knight & Gurumoorthy, 2019; Venkatraman et al., 2015). Frontal asymmetry is recognized as a valid indicator of approach-withdrawal tendencies that, ultimately, are associated with emotions (Ohme et al., 2010). Lin et al. (2010) found a link between emotional states, recorded through EEG, and music to an accuracy of 82.2% with joy, anger, sadness, and pleasure.
2.2Congruence of music with advertisement contentMusic congruence has been defined as the degree to which consumers perceive that background music is important or appropriate for the central message being communicated (Kellaris, Cox & Cox, 1993; MacInnis & Park, 1991). In communication research, congruency, fit, or the match-up effect, have been applied in different contexts, such as sponsorship (Rifon et al., 2004) and spokesperson characteristics (Lynch & Schuler, 1994). In the present study we focus on the effects of the congruence of music and TV advertisement content.
Congruence in advertising has attracted research attention, with contradictory results (Dahlén, Rosengren, Törn & Öhman, 2008; Oakes, 2007), also highlighted in more recent literature (Craton et al., 2017; Guido et al., 2016). From one perspective it has been argued that the fit between advertising messages and executional cues facilitates information processing (MacInnis & Park, 1991) and improves attitude toward brands and advertisements (Galan, 2009; Lavack et al., 2008). On the basis of schema theory it has been argued that advertisements containing incongruent elements are more difficult to process and recall, as more cognitive effort is needed to resolve their inconsistencies (Meyers-Levy, 1989). Kellaris et al. (1993) found that high-congruence music-message advertisements positively influenced information processing and improved recall and recognition. When congruence is low, however, music seems to be a distracter and hinders advertisement processing.
From another perspective, incongruent stimuli have been found to enhance advertisement processing (Dahlén et al., 2008), elicit consumer attention, and are more likely to be processed and stored than congruent stimuli (Peracchio & Tybout, 1996). Overall, these contradictory views call for further research with neurophysiological tools able to capture emotions and unconscious processing.
These results may be contradictory because music perception involves complex brain functions (Kumagai et al., 2017), and/or due to the methods used to measure its influence, which have mostly been based on self-reports (for a review, see Oakes, 2007). In this regard, unconscious reactions can be more reliable in capturing emotions, especially when the stimuli, that is, the music and the visual content, are presented simultaneously, as opposed to discretely (i.e., as in print advertisements); neurophysiological tools are more appropriate than self-reports for measuring unconscious reactions.
Congruence has been explained from a conceptual point of view by schema theory (Bartlett, 1932). This theory argues that all knowledge is organized into units. These knowledge units, driven by both cognitive and emotional interactions, are stored, related to previous knowledge, and retrieved when needed. Thus, a congruent unit of information elicits better comprehension. In addition, and consistent with the ELM (Petty & Cacioppo, 1983; 1986), incongruent music-content advertisements require greater elaboration and cognitive effort. As congruence between the spokesperson and the brand has positive effects on the knowledge of, and affection felt toward, brands (Misra & Beatty, 1990), it is anticipated that music-content congruence will have positive effects, from the advertiser's perspective, on the viewer. The literature also suggests that musical stimuli influence the emotional response of the viewer to visual stimuli, can modify consumer behavior in terms of likeability (Gorn, 1982), and enhance brand image (Kellaris et al., 1993).
The visual attention given to advertisements depends on a variety of factors, including content, message, music, brand, and advertisement characteristics (Calder & Malthouse, 2008). The literature on the effects of music in advertisements has found that listening to particular types of music activates different brain areas (Lin et al., 2010) and influences the cognitive processing of visual events (Boltz, 2001). The eye-mind hypothesis argues that cognitive processes are reflected in the idiosyncrasies of gaze movement (Just & Carpenter, 1980). Thus, longer fixation duration and fewer fixations represent more detailed processing (Horstmann, Ahlgrimm & Glöckner, 2009; Seneviratne, 2015). Accordingly, this study uses average fixation duration (AFD), fixations per second (FpS), and ambient vs. focal (AvF) attention, which has been shown to have high explanatory value in processes related to attention type (Holmqvist et al., 2011). These metrics are discussed in the methodology section.
As noted previously, incongruent cues require more information processing effort. Schema theory also suggests that, when incongruencies arise, humans need to increase their cognitive effort to resolve them. Complex tasks demand more processing resources and attention (Kallenbach, Narhi & Oittinen, 2007). Incongruent content can be considered as a more complex task than congruent content. Thus, the visual attention paid to TV advertisements with incongruent music will be higher than that paid to advertisements with congruent music. Therefore, the following hypothesis is proposed:
H1 The visual attention paid to advertisements with incongruent music is greater than that paid to advertisements with congruent music.
At the cognitive level, the literature has shown that frontal brain asymmetry is an indicator of user preference (Balconi, Stumpo & Leanza, 2014) and engagement with advertisements (Çakar et al., 2018). The concept of frontal asymmetry has received attention in the advertising research literature (Ohme et al., 2010; Schmidt & Trainor, 2001; Vecchiato, Cherubino, Trettel & Babiloni, 2013; Li, Walters, Packer & Scott, 2018). The EEG literature suggests that asymmetry is enhanced by pleasant music (Arjmand, Hohagen, Paton & Rickard, 2017). Accordingly, consumers will engage with advertising if the music facilitates cognitive processing and matches the cognitive hypothesis (Calder & Malthouse, 2008). As to the evaluation of music through EEG, specifically through measures of frontal asymmetry, correlations between oscillations in the alpha band and the frontal regions suggest preferential engagement. Thus, advertisement preference is associated with greater affective response, an effect on neural activity being observed. Studies in which this technique has been used confirm its ability to assess how to increase advertisement effectiveness (Tandle, Jog, Dharmadhikari & Jaiswal, 2016; Trainor & Schmidt, 2003).
Therefore, based on the proposal that frontal asymmetry is a valid indicator of the approach-withdrawal reaction, we propose that:
H2 Approach response is greater in advertisements with congruent music than in advertisements with incongruent music.
Measurements of the level of mental workload involved in the processing of audiovisua content can be useful in the analysis of how the brain processes advertisements. As discussed earlier, elaboration is higher for advertisements with incongruent music than for advertisements with congruent music. Evidence from other study types suggests that the presence of roadside advertisements increases vehicle drivers’ subjective ratings of mental workload (Young et al., 2009), and the time needed to respond to road signs and errors in driving (Edquist et al., 2011). Therefore, it is expected that distractions, such as incongruent stimuli, will cause more cognitive workload. Fraser (2014) found that images evoked by music focused the attention of experimental participants on the advertisement message and on the brand, and thus improved brand recall. As cognitive workload is based on cognitive processes, such as memory and task demand levels, distracting stimuli, for example, incongruent music, create higher workload. Therefore, it is expected that, in a non-congruent condition, consumers will face extra cognitive workload. Therefore, we propose that:
H3 The cognitive workload used is higher in advertisements with incongruent music than in advertisements with congruent music.
Sánchez-Porras (2013) noted that Coca-Cola attaches paramount importance to music, and that 94% of its advertisements are designed to match their background music with the feelings evoked by the images and, in the great majority (95.3%), the music is an integral part of the advertisement's visual action. Musical stimuli are especially effective in enhancing the effects of visual imagery, but some studies have shown that imagery enhances emotional responses to music (Juslin & Västfjäll, 2008). Furthermore, executional advertising cues (e.g., music) have a very significant effect, with low-involved consumers, on attitude toward brands, but also exert influence on central-route processing among both high- and low-involvement consumers (Macinnis & Park 1991).
Advertising likeability can be also used diagnostically for advertisements (Leather, McKechnie & Amirkhanian, 1994). Empirical findings have shown that the consumer's liking for an advertisement is directly related to his/her attitude toward the advertisement and brand, and toward his/her purchase intention (Du Plessis, 2005). Based on schema theory and ELM, in terms of the amount of cognitive effort needed, it is expected that congruent music will be more liked. Therefore, music-content congruence in advertisements might increase their likeability. Furthermore, advertisements perceived as more likeable produce more positive judgments and are more effective (Smit et al., 2006). Therefore, we propose the following:
H4 Advertisements with congruent music are liked more than advertisements with incongruent music.
Three competing research streams are associated with the influence of music on advertisement recall. The first stream states categorically that music helps consumers remember information about products and messages (Allan, 2008); in addition, it argues that music has a positive impact on memorization and the perception of advertisement duration (Galan, 2009). The second stream argues that music has a negative effect on the consumer's memory because it distracts his/her attention (Fraser & Bradford, 2013). The third stream combines the first two; the effect of music depends on two fundamental factors, the congruence between the music and the product, and the characteristics of the music. As suggested above, congruence involves less elaboration, aids memory and, therefore, congruent music is associated with higher recall than is incongruent music. Furthermore, the prior literature on congruence shows it has a myriad of beneficial effects, for example, the creation of positive attitudes, an increase in the credibility of sponsors (Rifon et al., 2004) and, in the match-up hypothesis, it has been shown that endorsements are more effective when there is a fit between the endorser and the product (Wright, 2016). In a self-report analysis, Guido et al. (2016) found significant differences based on the type of ending to the music that accompanied advertisements; they concluded that normal endings increase memorization of products and messages, and that truncated endings hinder memorization as they divert the attention of consumers toward the music, and away from the products and the messages. As an abrupt ending can be considered as a form of incongruence, thus, consistent with schema theory, we propose the following:
H5 Advertisement recall is higher in advertisements with congruent music than in advertisements with incongruent music.
The first stage consisted on selecting advertisements. The chosen advertisements had to fulfill the following requirements: (i) They had to last between 30 and 90 s to minimize the bias effect on recall caused by very short, or overly long, advertisements; (ii) they had to use only background instrumental music to avoid the influence of lyrics; (iii) they had to be absent of dialog/voice-overs to avoid the effect of language and ensure effective manipulation; (iv) they had to use music congruent with the images; (v) they had to show non-native brands so that the participants, who were of one particular nationality, would not previously have seen them. As a result, 45 advertisements were selected. A pretest, similar to tests carried out in past research (Alpert & Alpert, 1990; Brooker & Wheatley, 1994; Gorn, 1982; Kellaris & Cox, 1989; Macinnis & Park, 1991), was conducted to select between the 45 chosen advertisements to be used in the experiment. The participants were asked: "Do you think the music is congruent with the video?", using a scale of 1 (not congruent), to 10 (totally congruent). In the second stage, the 15 most congruent advertisements of the 45 advertisements examined in the first stage were manipulated by the removal of the original music and the addition of incongruent music. We chose two groups of advertisements, one with happy music, and one with sad music. Thus, sad classical music replaced happy classical music (Alpert, Alpert & Maltz, 2005; Alpert et al., 1990), and vice versa.
In the last stage, the 6 most congruent advertisements out of the 15 most congruent advertisements examined in the second stage were manipulated by the removal of the original music, and the addition of incongruent music. These 6 advertisements, now with incongruent music, were tested on a new group of 60 participants. The participants were asked: "Do you think the music is congruent with the video?", using a scale of 1 (not congruent), to 10 (totally congruent) (Fig. 1).
As to the music stimuli, two orchestral pieces, Samuel Barber's “Adagio for Strings”, and “Ode to Billy Jo”, were chosen to evoke sadness, and Beethoven's Symphony no. 6, and Well-Tempered Clavier, were chosen to evoke happiness (Alpert et al., 2015). The other two advertisements used sad/happy pop music. The music was taken from the database of the YouTube audio library (Dougan, 2016). The videos with the incongruous music were produced by professionals who made the harmony, timbre and rhythm sound the same as in the original versions (Table 1).
List of stimuli and features.
The participants were exposed to six advertisements. Half (45) watched advertisements with their original congruent music, and the other half watched the same six advertisements, but with newly inserted incongruent music, resulting in two scenarios. The advertisements were randomly distributed to avoid data-gathering bias.
The experimental design was based on viewing, through a 1920 × 1080-pixel monitor, a 30-minute naturalist documentary on marine life, with three ad-breaks. Two advertisements were shown during each two-minute ad-break, which is consistent with real-world television advertising. The advertisements were presented randomly to avoid primacy and recency effects, which might have produced higher levels of recall in the subjects based on order of participation (Murdock, 1962).
As dependent variables we evaluated the experimental participants’ cognitive levels (visual attention, frontal asymmetry, cognitive workload) and recall of six television advertisements. Two hours after they viewed the advertisements the participants were interviewed to measure advertisement recall and the level of likeability of the advertisements using a 7-point Likert-type scale. In this way we verified the relationships between the cognitive states, and ad recall, and the likeability responses of each subject.
3.3SampleThe neurophysiological literature has shown that women are better than men at detecting non-verbal cues (Hall, Carter & Horgan, 2000), and that “the gender effect for visual-plus-auditory studies was significantly larger than for visual-only and auditory-only studies” (Hall, 1978, p. 845). Previous research has indicated that males and females perceive music differently (Zander, 2006). To control for potential gender differences, only females were recruited. The initial sample consisted of 100 women with normal vision and hearing (mean age 36.2; SD = 5 years), recruited by a company based in the same city as the laboratory where the experiment was carried out. Ten participants were removed due to corrupted data in signals obtained in the experimental sessions, giving us a final sample of 90. The participants were assigned randomly to the two scenarios (advertisements with congruent music vs. advertisements with incongruent music). They watched the marine life documentary shown in the preliminary study. No mention was made of the significance of the advertisements. The institutional review board of the Politechnic University of Valencia approved the study based on the written informed consent of all subjects, in accordance with the Helsinki Declaration.
3.4Recording of the physiological signalsTable 2 contains a summary of the constructs, tools, and metrics used, related to each hypothesis. The ET and EEG signals were synchronized with the stimuli through iMotions software (https://imotions.com).
The EEG device was placed on the subjects while they listened to the explanation about the documentary that was about to be shown. The ET device was embedded in the desk monitor. The measurement began with a baseline task consisting of tests of attention, relaxation, and cognitive load for each of the participants (Berka et al., 2007).
The Tobii Pro-TX300 device was used to analyze the ET. This records at 300 Hz and has a built-in 23-inch monitor. The subjects’ gaze behavior was recorded, and a fixation detection algorithm based on speed, with a threshold of 30°/sec, measured the fixations and saccades. Total scan (TS) is the sum of the length of the saccades in degrees. It is greater if the viewers explore more. AFD, FpS, and AvF attention data were obtained for each advertisement. To calculate AvF the saccades were classified into ambient (long) and focal (short), using a threshold of eye movement of 4.8° (Zangemeister, Sherman & Stark, 1995), and dividing the number of ambient saccades by the number of focal saccades. A high AvF indicates that the visual exploration has been ambient, with long saccades, and a low value indicates a focal exploration, with small saccades.
Electrical brain activity was recorded using the B-Alert X10 device (Advanced Brain Monitoring, Inc.). It records at 256 Hz and includes 9 channels, using the international location system 10–20, positioned in the frontal, central, and parietal-occipital areas. First, corrupted channels were identified by applying the fourth standardized moment (kurtosis). In addition, if a channel contained more than 10% of flat signal the electrode was classified as corrupted. The EEG baseline was removed by applying a bass pass filter between 0.5 and 40 Hz, and the signal was segmented into epochs of one second. The intra-channel kurtosis level of each epoch was used to reject the epochs with high noise levels. To detect artifacts caused by muscular activation, blinking, and eye movements, an independent component analysis (ICA) (Gao, Lin, Yang, Wang & Zheng, 2010) was applied; a trained expert manually analyzed all the components, rejecting those related to artifacts. Finally, frontal asymmetry was calculated using the alpha band of the decontaminated signal. As noted earlier, right and left asymmetry are related to approach-withdrawal behavior (Davidson, 1977, 2004). We refer to approach-related tendencies (or left-hemispheric dominance) as "positive emotional reactions", and withdrawal-related tendencies (or right-hemisphere dominance) as "negative emotional reactions” (for a technical review, see Fischer, Peres & Fiorani, 2018; Harmon-Jones & Gable, 2009).
Specifically, we performed an EEG spectral analysis in each epoch using Welch's method, with 50% overlapping, and calculated the spectral power of the alpha band (8–12 Hz). Frontal asymmetry was computed using the formula:
To obtain the EEG-workload metric, a four-class quadratic discriminant function analysis (DFA) was conducted for each participant. Similarly, a metric benchmark was developed for all participants, and the model was individualized using a DFA coefficient. A linear two-class DFA model was used to assess the level of cognitive workload. A detailed explanation of the model is presented in Berka et al. (2007).
A questionnaire was used to measure the relationships between the subjects’ neurophysiological responses and their levels of self-reported liking for, and recall of, the advertisements. The questionnaire asked the subjects how much they liked the advertisements. The level of liking was measured on a 7-point, 3-item Likert-type scale, as used by Berryman (1984): "I think the commercial that I just saw is 1 (bad) to 7 (good).” As for the reliability of the test, Cronbach's alpha was calculated to be 0.72, which is considered acceptable (Cronbach, 1951). In the questionnaire the participants were shown one frame from each of the advertisements they had previously viewed, and two from advertisements that had not been screened (Pepsi and Reebok), and were asked which advertisement contained the frame.
4ResultsFig. 1 shows the results of the pretest; this analyzed the level of audiovisua congruence between the two advertisement alternatives, (in)congruent music, on a 7-point Likert-type scale. All the congruent advertisements presented averages between 6 and 7, and the incongruent averages between 1 and 2. Due to the non-Gaussian distribution of the data (p < 0.05 from the Shapiro-Wilk test), Wilcoxon signed-rank tests were applied with a Bonferroni correction. Significant differences were found in all the advertisements (p-value<0.001), which suggests that the audio manipulation changed the level of congruence of the advertisements.
Concerning H1, related to visual attention, Figs. 2–5 show the gaze patterns in terms of fixation per second (FpS), average fixation duration (AFD), ambient vs focal exploration (AvF), and total scan (TS), respectively. Due to the Gaussianity of the data (p > 0.05 from the Shapiro-Wilk test with null hypothesis of having a Gaussian sample), t-tests were applied to assess if there were significant differences between the congruent and incongruent advertisements. The incongruent advertisements presented more fixations per second (FpScon=2.086±0.523 vs. FpSinc=2.247±0.4016, p-value=0.0018), and longer fixations (AFDcon=330.3 ± 81.9 vs. AFDinc=353.7 ± 80.1, p-value=0.0043), indicating a higher level of attention. In addition, the incongruent advertisements presented more explorative gaze patterns as they showed a higher ambient vs focal ratio (AvFcon=1.263±0.571 vs. AvFinc=1.568±0.726, p-value<0.001), that is, the saccades were longer with the incongruent stimuli, which also provoked a higher total scan time (TScon=580.7 ± 327.8 vs. TSnc=688.6 ± 382.1, p-value<0.001).
Figs. 6 and 7 show the EEG responses in terms of frontal asymmetry (FA), confirming H2, and cognitive workload (WL), confirming H3. Due to the Gaussian distribution of the data (p > 0.05 from the Shapiro-Wilk), in both cases t-tests were applied to analyze if there were significant differences between the congruent and incongruent advertisements. The congruent advertisements showed a higher percentage of time with positive asymmetry (FAcon=0.510±0.248 vs. FAinc=0.452±0.213, p-value=0.005), suggesting that congruent stimuli are associated with higher approach behavior. In contrast, the incongruent advertisements showed a higher level of workload (WLcon=0.567±0.111 vs. WLinc=0.603±0.090, p-value<0.001), indicating that greater cognitive effort is needed to process incongruent stimuli.
Regarding H4, Fig. 8 shows the liking ratings of the congruent and incongruent advertisements, using a 7-point Likert-type scale. Due to the non-Gaussian distribution of the data (p < 0.05 from the Shapiro-Wilk), in both cases t-tests were applied. No significant differences were found.
Regarding H5, Fig. 9 shows the level of advertisement recall (RC) based on the congruence of the stimuli. Given the categorical nature of the data (recall vs no recall), a chi-square hypothesis test was applied to the congruent and incongruent advertisements. A higher percentage of subjects recalled the incongruent advertisements (RCinc=58.30% vs. RCcon=71.52%, p-value<0.008), rejecting H5.
5Discussion and implications
The results support the advisability of using congruent music in advertisements. The use of neurophysiological tools gives our findings more robustness than previous studies and sheds light on the specific effects of congruent music, as follows: First, the findings provided further evidence of the influence of music and its effectiveness in television commercials; second, our analysis goes beyond advertisement recall by examining unconscious responses such as visual attention, cognitive workload, and frontal asymmetry.
The study found significant differences in metrics related to visual attention, average fixation duration (AFD), fixations per second (FpS), and ambient vs. focal (AvF) attention, between advertisements with congruent music and advertisements with incongruent music, as anticipated by H1. The EEG recordings confirmed there were significant differences in frontal asymmetry. Thus, asymmetry is shown to be a very effective metric for measuring music and advertisement content congruence; higher levels of approach versus withdrawal reactions were observed in advertisements with congruent music, confirming H2. Significant differences in cognitive workload levels were found in advertisements with congruent music, when compared to the advertisements with incongruent music, which confirms H3, showing a higher cognitive effort in incongruent advertisements. Last, the responses to the questionnaire did not show that advertisements with congruent music are more liked than advertisements with incongruent music, as H4 proposed. This finding might suggest that reported liking is not an accurate measure for discriminating between incongruent versus congruent music in advertisements. Furthermore, we found that incongruent music helped the subjects remember the advertisements, contrary as predicted in H5.
These findings can be interpreted in line with schema theory, and with the value that the ELM attributes to music as an executional cue. Interpreting our findings holistically, the implications for managers are twofold. First, music congruent with advertisement messages is associated with lower visual attention levels and lower levels of recall. The assumption that incongruous music elicits higher memory performance can lead to complex interpretations based on existing theories. First, schema theory suggests that when consumers process a cue that does not fit well with previous schemes, probably learned over years, they allocate extra efforts to process it, which results in higher visual attention being devoted to it that lead to withdrawal approaches. This is consistent with the limited capacity model of motivated mediated message processing (Lang, 2000), and the underlying concept of attention and effort propounded by Kahneman (1973). The LC4MP proposes that the human processing system has limited capacity. There is no doubt that cognitive load reduces, in small or large proportions, the cognitive resources available for processing other cues. Second, on the basis of the ELM it can be argued that music acts as a peripheral cue that may contribute to attitude change. When consumers mentally process advertisements with incongruous music, this may increase their attention levels, but not necessarily focus them on the main message the advertisers wish to convey. Other studies have suggested that thematic incongruence can enhance viewers’ evaluations of advertisements (Dahlén et al., 2008). However, our findings suggest that conflict between music and message results in withdrawal behavior. It seems that conflict between auditory and visual cues resulted in higher visual attention being paid, and a withdrawal approach. Since the withdrawal approach is an emotional reaction, interpreted as a proxy for preference, it can be concluded that advertisers should be cautious about including incongruent music. The present study is based on a forced video watching activity and, therefore, visual attention has to be interpreted with caution. In addition, higher attention levels were associated with higher levels of advertisement recall, which is in line with previous research (McDaniel, Waddill, Finstad & Bourg, 2000).
Second, our results showed that content-music congruent advertisements generate lower cognitive workload. The finding that higher attention provokes higher mental workload is also in line with previous research (Kantowitz, 2000). Since performance and mental comprehension decline when mental workload is too high, or too low, the role of attention and its involvement with cognitive load in advertising is critical. The study methodology validates the use of implicit measures to assess the balance between attention and workload, which can be used by advertisers in the development of audiovisua advertisements. Moreover, in the set of advertisements used in the present study, the congruent stimuli provoked higher positive reactions than the incongruent stimuli. This is reflected in the fact that frontal asymmetry, which measures the behavioral approach system, and is associated with positive emotional states, was greater. The fact that no differences were found in likeability based on the self-assessments, but positive reactions were recorded by the electroencephalogram, shows that implicit measures are a powerful analytical tool for managers in their assessments of the emotions that advertisements can generate. This methodology can be applied to the decision-making processes involved in evaluating different alternatives of advertisements in terms of their audiovisua congruency.
6Conclusions, limitations and further researchThis study uses neurophysiological and self-reported measures to explain the effects of (in)congruent music in TV advertisements on unconscious, and conscious, responses.
We hope this study helps better explain responses to (in)congruent music in TV advertisements, and how they can be measured. The results for H1 showed a strong correlation between the quantity of fixations and attention level. In this case the incongruent music advertisements generated more attention and more mental processing. This is consistent, in that advertisements with incongruous music, at first, generate more surprise, which provokes a higher level of attention due to the viewer's inability to understand the connectivity, which creates a greater demand for resources, and an increase in neural processing resources. The approach constructs raised in H2 and H3 showed a strong relationship between the EEG measures frontal asymmetry and cognitive workload. This frontal asymmetry measure, that is, the increased activity in the alpha band in the left hemisphere (F3), suggests that the viewer undergoes an emotionally positive response; and negative responses were noted in the right hemisphere, associated with incongruent music (Moore, 2012). Cognitive workload measures the level of mental resources necessary to process information. The distractions provided by the incongruent stimuli caused more cognitive workload, as H3 predicted (Seneviratne et al., 2015). The test of H4 did not find any statistical differences in terms of likeability between congruent and incongruent advertisements, based on the self-assessment questionnaire. Finally, the test for H5, where we measured the effects of music on advertisement recall, showed that incongruent music is remembered more than congruent music. This shows the importance of measuring psychological constructs through both neurophysiological and traditional means; this allowed us to more comprehensively evaluate the responses generated by (in)congruent music in the advertisement videos.
Our findings suggest that incongruent music in advertisements demands higher mental processing resources than does congruent music. Thus, incongruent music generates more visual attention, higher cognitive workload and withdrawal reactions, than advertisements with congruent music. Overall, these findings suggest that incongruent music leads to higher visual resource allocation. However, the viewers showed a withdrawal reaction. Given that this study is based on a forced exposure to stimuli, it can be argued that, in real situations, if consumers have a high level of involvement with the advertisements and/or the brand, their visual attention will remain high. On the other hand, if their involvement is not high, the withdrawal reaction might mitigate visual attention.
This study has some limitations that restrict the generalizability of the results. First, the quantity and type of advertisements analyzed were limited. Although the advertisements were for products unknown in the country where the study was conducted, and the sample was homogeneous in terms of gender, it is possible that previous purchase experience with the product category could have affected the results. Schema theory would suggest that product category might act as a complementary unit of information. Our study did not control for attitude toward the advertising, which might affect the results. Similarly, the music genre might be controlled. Furthermore, the advertisements about social issues (i.e., Embrace Life and Domestic Violence) showed greater strength in the measured effects. Thus, for example, they had higher levels of asymmetry. Accordingly, future research might explore the moderating effect of involvement with the product category on the emotional impact of music congruence. Second, it seems that the type of music used might influence the subjects’ responses. Third, we intentionally excluded lyrics to avoid biasing the subjects’ responses. However, congruence between lyrics and music might also be influential. These three limitations should guide future research. Since our findings are restricted to women, future research might also explore the influence of gender by analyzing the effects of the music congruency of advertisements on men.
Two more future research approaches might address effects over time and the influence of other potential (in)congruent cues. Thus, a future study might consider the effect of (in)congruence over several exposures to the advertisements, and address if the effects remained stable or decreased/increased after multiple exposures to the same advertisement. Other cues might be considered, such as the role of message credibility and executional style (Eisend & Tarrahi, 2016), also using neurophysiological tools.
