Tracking unconscious response to visual stimuli to better understand a pattern of human behavior on a Facebook page

Abstract

As one of the fastest growing social media networks, Facebook is used daily by 7.91 billion users. Despite its worldwide influence, the cognitive perception of this social network has not been fully understood or researched. The current study examines human behavior when viewing Facebook pages to provide marketers information on how to enhance content and increase attention on their Facebook pages. A neuroscience approach using an eye-tracking device was used to gain insight into the reasons users chose to follow the Croatian Academic Union of the Faculty of Economics (HAZEF). The results showed that followers spent, on average, less than 1.5 s viewing a Facebook post, while the emotional analysis of facial expressions revealed a higher prevalence of negative moods and emotions such as sadness when viewing the HAZEF page. Negative moods decreased when posts contained images of human faces. Possible reasons for these findings are discussed, and recommendations are made to optimize the Facebook page for optimal attention and awareness of postings.

Online social networks (OSNs) are a popular way to gain personal exposure and communicate with others. Facebook, one of the world's leading OSN platforms, allows users to create a personal account with a photograph, basic demographic information (About Me section), and activities (Likes and Interests section). The public uses the network daily, mainly by engaging with content in their newsfeed. Even with the widespread use of social networks, experts still do not know enough about users’ cognitive perceptions of the posted content on personal profiles and pages (Seidman & Miller, 2013). Visual data and speech inflections are the only signals that viewers can access with online interactions, especially in OSNs. Therefore, online advertisers and social media users interpret social statistics based solely on such indicators. Since various forms of social facts are presented on Facebook in a unified visual aspect, it provides an excellent platform for researchers to study social data attention. Several studies have shown that certain features of a social media profile, such as written content and photographs, can influence the perception of the profile owner (Tong et al., 2008).

Facebook has steadily gained popularity, not only as a social media network, but also as a mass-marketing platform. Due to the variety of advertisements, visually appealing profiles, and content, this network has attracted a great deal of scholarly interest. Early research aimed to analyze Internet users and their behavior to discover the unique characteristics of web ads and advertisements promoted on Facebook. The ability of such advertising to elicit immediate comments from the audience is one of its distinctive advantages. The effectiveness of web advertisements has usually been evaluated using direct statistics, such as press and hold, or other marketing strategies (Hoffman & Novak, 1996). Statistics show that only a small percentage of people who see a web advertisement remember and engage with the text, such as the company (brand) name. In fact, web advertisements may be ignored by more than half of web visitors (Shankar & Hollinger, 2007). An important aspect in this context is that users inevitably examine an online advertisement before rejecting or ignoring it. A person cannot cognitively perceive, grasp, retrieve, or click on an Internet advertisement unless they see it. This demonstrates that Internet advertising has unanticipated tendencies, and it is crucial to comprehend the consequences of web advertisements that are rejected or avoided. Traditional conceptual methods of advertisement perception cannot investigate why users ignore or reject ads, and Facebook statistics do not measure this aspect (Drèze & Hussherr, 2003).

As one of the fastest-growing social media networks, Facebook recently surpassed Google on the total amount of time Americans spend on the respective sites (Wang & Hung, 2019). Owing to its marketing power, the network provides a platform that meets advertisers’ needs for consumer engagement and creates an environment in which information can continue to flow between customers and retailers (Dunay & Krueger, 2009). In addition, Facebook has significant appeal to businesses, generating approximately $2.5 billion in revenue from business ads (Vahl, 2011). Lacking the face-to-face and interactive nature of traditional customer marketplaces, this social network has made the Internet a highly successful business and advertising medium. Nevertheless, there is indeed skewed and incomplete knowledge of efficiently and adequately using web advertising on the platform (Wang et al., 2009). To shed light on this matter, this article addresses the following question: How can we track the unconscious response to visual stimuli to better understand human behavior on a Facebook page?

The focus of the current study is to investigate human behavior on websites and social networks, especially on Facebook, and to improve the content to attract consumer attention. To this end, neuroscience methods were used to determine why users follow the Facebook page of the Croatian Academic Union of the Faculty of Economics (in Croatian: HAZEF). By evaluating the current content, this study provides insight into the possibilities of gaining new followers on the said page. The answers from the questionnaire were compared with the neurometric data obtained to verify whether expectations and reality match. Functional magnetic resonance imaging (fMRI) is the most widely used neuroscience approach in many customer neuromarketing studies. However, due to rapid technological advancements and increasing affordability, electroencephalography, eye tracking, and subliminal measurements are becoming more common for market research purposes (Harris et al., 2018).

Unconscious processing of web pages

Almost all modern psychology assumes that basic cognitive processes can absorb knowledge unconsciously. Most people define unconsciousness as a "lack of awareness of signals or consequences during an activity." Unconscious functioning is described as “quick, spontaneous, reflexive, not limited by capability, and always initiated by a specific cognitive stimulus” (Yoo, 2008, p.3). Inspired by cognitive processing and associative learning research, Eighmey (1997) examined unconscious processing and showed that visual, auditory, or even both inputs could be processed unconsciously. Most web advertising contains both visual and auditory stimuli, and it is believed that this type of advertising can also be processed unconsciously. Unconscious processes and attention are inextricably linked. It is assumed that customers discriminate between advertisements depending on their interests and that special attention is required because of the limitations of the cognitive process. Compared to traditional media, the web is considered even more demanding because it is used to satisfy customer demand and desire for content, entertainment, and social contact (Eighmey, 1997). According to Shankar and Hollinger (2007), most people browse with the objective of meeting primary goals including; searching for information, watching videos, talking to friends online and focusing on obtaining sensory cues from the website. As a result, most, if not all, online ads go unnoticed (Shankar & Hollinger, 2007). However, Afef and Jamel-Eddine (2012) found that customers initially engage in unconscious processing of web advertisements when browsing websites where ads are placed. Web advertisements that attract attention are evaluated at a conscious level, but those that are rejected or averted are evaluated only at an unconscious level (Afef & Jamel-Eddine, 2012).

Unconscious response to visual stimuli

Outside of conscious awareness, visual data can influence conscious perception, motor response, and sometimes even conscious execution (Harris et al., 2011). Dehaene and Changeux (2011) stated that it is important to consider the extent to which supraliminal sensory data can be interpreted to understand the strength and limitations of subliminal image perception. If viewers can accurately “predict” the identification of visual stimuli without actually seeing them, it stands to reason that subliminal visual data could influence behavior similar to supraliminal visual input (Dehaene & Changeux, 2011). According to Song and Yao (2016), people with vision deficit who cannot consciously perceive sensory images in their flawed visual system due to injuries in the occipital cortex, but can correctly identify the signals, have shown that they process subliminal visual data unconsciously. Visual information considered unseen can be identified with the above reliability in healthy adult viewers, demonstrating an unconscious interpretation of subliminal visual input (Song & Yao, 2016). Dehaene et al. (1998) considered well-established transitions between perceptual awareness and the appropriate categorization of visual stimuli, but it is still unknown how stimulus properties influence the unconscious interpretation of unseen visual stimuli. According to their findings, the intensity and variety of signals containing subliminal visual data could alter the extent to which the subliminal sensory system can influence behavior. To test this theory, researchers examined how the brightness and complexity of the stimulus affected the amount of subliminal processing. They used the continuous flash suppression procedure to render a screening stimulus undetectable by presenting it to one eye, while a masked signal was presented to the other eye (Dehaene et al., 1998). According to a study by Song and Yao (2016), fluctuations in cortical activation triggered by the evaluative stimuli resulted in the identical evaluative stimulus being completely unseen in some trials but fully or partially seen in others. This study showed that the intensity and variety of signals with subliminal visual data could alter the degree of unconscious analysis without affecting subjective awareness (Song & Yao, 2016).

Tracking the unconscious

Over the last century, human-computer interaction studies have incorporated psychological and physiological web applications that enhance the computers’ perception of clients’ conscious cognitive and affective levels and increase their performance and resilience. However, the human state is not limited to cognition and emotion, but includes universal impulses and fundamental properties that constitute the combined unconscious. Psychosocial computing analyzes psychophysiological data such as brain function and galvanic skin response to assess user behavior during cognitive and emotional phases. Unlike typical interaction approaches, this approach does not require participants to provide any information (Ivonin et al., 2015). Fairclough (2009) explored the benefits of psychophysiological web applications for consumers, including increased adaptability, ease of use, and improved communication frequency band. This sparked the interest of researchers in the field of human-computer interaction (HCI) and led to the exploration of computer systems that can detect and improve the emotional and cognitive state of users (Fairclough, 2009). According to Ivonin et al. (2013), research on HCI has evolved significantly from simple platforms that depend on users’ activities at a physiological level to more complicated social patterns driven by physiological processing that also are able to incorporate a users’ emotional and cognitive mood. Despite these improvements, customer experience beyond the cognitive process, in the realm of primal reflexes and innate responses, is poorly understood and, except for a few outliers, remains a predominantly new area of HCI research (Ivonin et al., 2013).

Psychological and, more recently, neuroscientific research by Ivonin et al. (2013) has shown that the variables defining subjective existence are not limited to anatomical, mental, or emotional domains. Rather, they have focused on the conscious cognitive and emotional aspects that have been at the forefront of emotional and physiological programming. These phases were characterized by consumers’ understanding of the events that immediately surfaced in their consciousness. A significant part of human existence is associated with a deep level of psychology called the unconscious because it is not accessible to conscious thought (Ivonin et al., 2015). Since the scientific world has not yet fully understood the concept of the unconscious, there is no accepted definition. To minimize uncertainty and misinterpretation, Bargh and Morsella (2008) operationalized unconscious cognitive states as a low level of awareness of the impacts or effects of prompting stimuli rather than the provoking stimuli itself (p.78).

Eye-tracking analysis

The growing popularity of the Internet in everyday life requires a website to be easily accessible to consumers who wish to experience it. Consumers who are dissatisfied with the accessibility of a website will simply abandon it. As a result, the need for website accessibility ratings is on the rise, and eye-tracking is becoming increasingly popular among the strategies used for this purpose. Eye-tracking is a method that records a customer's gaze while looking at a stimulus. The gaze never stays fixed in one place for a long time, and the eyes move many times per second, with micro-movements often involving only a few pixels. Fixation is a generally motionless instant in which the eye moves to another area, while a saccade is a rapid movement between fixations. Eye-tracking technology can detect fixations and saccades using data collected by an eye tracker. A gaze plot can be used to show a specific sequence of consumer fixations and saccades on a panel or website, while heat maps can indicate how long each area of the screen was viewed. Accessibility professionals interpret these (and other) visual representations of eye-tracking statistics to indicate user confusion, pattern studying or scanning, or areas that users are not gazing at (Ehmke & Wilson, 2007).

When it comes to studying relationships between eye-tracking measurements and usability issues, Goldberg and Kotval (1999) represent one of the most well-known paradigms. They developed and tested a set of geographic eye-tracking measurements in the context of visual inquiry. The measurements were tested in a study involving the configuration of an application for drawing tool selection, which involved multiple layouts (Goldberg & Kotval, 1999). After testing a model for a web interface, Goldberg et al. (2002) examined various eye-tracking data. In addition to basic measures such as the number of fixations and mean fixation duration, they used two other eye-tracking measures: saccade amplitude and scanpath width. As focus is drawn from a distance, larger saccades may indicate more significant cues, while a longer scan path may indicate a less effective search (Goldberg et al., 2002). In the field of HCI research, Jacob and Karn (2003) provided a helpful summary of eye-tracking and eye-tracking statistics that included the number of respondents and activities and the eye-tracking measures used. They examined 24 studies published between 1950 and 2002 to identify the six most commonly used statistics: i) percent fixation total (in randomized trials), ii) gaze on each area of interest (seven studies), iii) mean cumulative fixation duration (six studies), iv) number of fixations on each area of interest (six studies), v) mean gaze duration on each area of interest (five studies), and vi) percent fixation total (five studies). It has been suggested that these metrics are not always the best options to analyze and that alternative metrics such as scan path or transition probability between AOIs (Area of Interest) should be used instead to investigate the effectiveness of the layout of the customer interface components (Jacob & Karn, 2003).

Wang et al. (2018) noted that the existing research either used the click-through-rate (CTR) to determine the number of people who responded to the appearance of the advertisement or used a survey to evaluate the effectiveness of the advertisements. However, due to lack of a general understanding of online advertisements, these results cannot adequately inform customers. Their study of a researcher's page on Facebook employed an eye-tracking device to capture the respondents’ eye movement habits and examine variations in visual attention and approval allocation. Eye-tracking metrics such as total fixation duration (TFD) and number of fixations (NOF) were analyzed for the specified interest areas of a selected Facebook account to reveal high areas of visual attention (Wang et al., 2018).

Materials and methods

Participants

Prior to the start of the Entrepreneurial Academy in Medias Res conference, a sample of 59 attendees, both males and females, between the ages of 20–55 were selected to participate in the neuromarketing study evaluating conference announcement posts and its causal effect on the arrival of all participants who registered for the conference. Some of them were full or part time students, while others were marketing professionals or entrepreneurs of SME's and startups. The required sample size was calculated a priori using G*Power (Faul et al., 2007) to ensure that the desired level of power and significant results were able to be achieved (see Appendix). Based on the G*Power output, a sample size of n = 43 was required to detect the effect with a power of 95% and a two-sided significance level of 5%.

All participants were notified of the study and provided a digital informed consent prior to participation. Data was handled following standard practices and in compliance with the GDPR and the European Code of Ethics for Research. Participants were also made aware that the research link could not be shared with third parties under the Copyright Act.

Materials

The online platform Sticky by Tobii Pro-for advanced quantitative research was used to track eye movements and measure emotional expressions when visiting the official Facebook page of the Croatian Academic Union of the Faculty of Economics (HAZEF). Simultaneously, a closed-response questionnaire was used to compare the cognitive and subliminal outcomes.

According to recent research, the average viewing error of Sticky in a natural environment (not in the lab) is 4% of the screen width and 6% of the screen height on a mobile device, which is sufficient to obtain scientifically valid results.

General procedure

Facebook's security system limits access from third parties when used for various software measurements; therefore, a simulation of the official HAZEF's Facebook page was made. This was done by pre-recording a 70-second video of the official page, which consisted of 17 recorded posts during the period of May 12th, 2021, through June 13th, 2021. A simulation was recorded in such a way that participants could scroll through each post (from top to bottom and vice versa) with the only restriction being they were unable to click “See more” on the posts. However, the research software “Sticky by Tobii Pro” tracked mouse clicks on video, which can be used as a metric of interest to read more on the FB post. The video could only be accessed from mobile devices to obtain real-time data, as it was statistically determined that more than 85% of users accessed Facebook from mobile devices. After receiving participation details with the link, participants were also given a series of images with calibration instructions to ensure they met the technical requirements of the study before the eye calibration test. Participants were then asked to browse the Facebook page as usual and then answer a closed-response questionnaire asking them to rate their reasons for visiting the HAZEF Facebook page.

Eye-tracking analysis

Quantitative and qualitative analyses were performed. First, the recording was divided into three areas of interest (AOI) including: the cover page with the HAZEF logo, conference announcement, and other posts. These were used to understand which parts of the Facebook page attracted the most visual attention and what subconscious preferences users have. An analysis was conducted using the dwell time (time spent looking at a specific AOI) and number of fixations (the average number of fixations within the AOI). One-way repeated measures, ANOVA was used to verify if there was a statistically significant difference in eye movement metrics between AOIs. A heat map of the static gaze visualizations served as a graphical representation of the fixation distribution, while facial coding was used to measure subjects’ emotions based on their facial expressions to understand the effect of HAZEF's Facebook page.

Results

An overlay of the visualizations with a color gradient showed that the gaze was directed to each AOI (Figs. 1–3).

Fig. 1.

Heat map of the Cover Page.

Fig. 2.

Heat map of the conference announcement.

Fig. 3.

Heat map of one of the other posts.

For the eye movement measurements, a one-way repeated measures ANOVA was performed (Table 1). The results indicate a significant difference in dwell time between the three areas of interest (F (2807) = 49.23, p = 0.00) (Fig. 4), and a post hoc Tukey test revealed that respondents spent significantly more time looking at the cover page (M = 1.47, SD = 0.50) and other posts (M = 1.55, SD = 0.81) than the conference announcement post (M = 0.37, SD = 0.24). However, no significant difference in dwell time was found between the cover page and other posts.

Table 1.

Descriptive statistics for eye-tracking metrics.

	(I) AOI	(J) AOI	Mean Difference (I-J)	Std. Error	Sig.	95% Confidence Interval
						Lower Bound	Upper Bound
Time Viewed	C. Page	Con.Announ.	1.10	0.16	0.00	0.71	1.48
		Other Posts	−0.08	0.12	0.77	−0.36	0.20
	Con. Announ.	Other Posts	−1.18	0.12	0.00	−1.46	−0.90
# of Fixations	C. Page	Con. Announ.	15.73	2.50	0.00	9.86	21.60
		Other Posts	−2.17	1.82	0.46	−6.45	2.11
	Con. Announ.	Other Posts	−17.90	1.82	0.00	−22.18	−13.62

Fig. 4.

Average amount of time spent looking at a specific AOI.

A statistically significant difference between the three areas of interest was also found in the number of fixations ((F (2807) = 48.44, p = 0.00) (Fig. 5). A post hoc Tukey test showed that the number of fixations within the cover page (M = 21.96, SD = 7.17) and within other posts (M = 24.12, SD = 12.40) were higher than the number of fixations within the conference announcement (M = 6.22, SD = 3.95).

Fig. 5.

Average number of gaze points directed at a specific AOI.

While visiting the HAZEF Facebook page, users subconsciously showed a higher prevalence of negative moods as opposed to a positive mood (Fig. 6). However, in posts containing images of people there was a slight decrease in negative moods.

Fig. 6.

Intensity of positive and negative emotions triggered during the Facebook page visit.

In addition to the neuromarketing measures, the participants were asked to complete a closed-response questionnaire. The results showed that the highest percentage of participants (61%) followed HAZEF's Facebook page to read interesting news, while a smaller number of participants (20.3%) tried to find a job on the Facebook page, 15.3% wanted to expand their contact base, and 30.5% followed the page because of business interests (Fig. 7).

Fig. 7.

Survey data on the reasons for following the Facebook page.

Discussion

Neuroscience methods were used to analyze human behavior on Facebook to understand how content should be modified to attract followers’ attention. Our choice of a webcam-based eye-tracking device is not accidental. Given the pandemic-related lockdowns and related lab closures, the neuromarketing research field transitioned to an online format to acquire insightful data with web-based technologies.

Contrary to state-of-the-art eye-tracking devices that are based on infrared light, webcam eye-tracking devices operate with visible spectrum light (Ferhat & Vilariño, 2016). While webcam eye-tracking devices are more sensitive to the movement and do not work optimally in low-light conditions, it is still possible to obtain accurate and reliable results by creating and conducting shorter experiments (Baunbæk Jensen, 2019). Hence, in the current study, a 70-second video of HAZEF's Facebook page was prerecorded and calibration check points were inserted automatically. This was done to account for the number of participants that were screened out due to ending their session based on a screening question or not meeting technical requirements as well as a control for excessive movement during the gaze data collection. As a result, a more in-depth understanding of human behavior and elicited emotions was obtained because Facebook is limited to user and interaction statistics that only include a quantitative view of the number of visitors, likes, shares, and comments.

Previous research has proven that users view websites according to the “F” reading pattern, in which the first words on the left of each line of text are fixated on more than subsequent words in the same line (Shrestha et al., 2007). Since our eye-tracking findings suggest the presence of the same reading pattern, the first step is to position the HAZEF's logo further to the left, as it does not stand out on the cover page with its current position.

Another major factor to consider when creating a post is color. One possible reason for the Entrepreneurial Academy in Medias Res’ conference announcement receiving significantly less attention than other posts on the page is the color combination used for the visuals. Instead of dull colors, which often evoke feelings of sadness, a better solution would be to use warmer colors as they have an attention-grabbing effect (Suk & Irtel, 2010). Moreover, the research results also revealed that 38% of registered attendees did not participate in the conference. Taken together, these results suggest that the conference announcement post was not well presented from an emotional (facial coding), perceptual (eye-tracking) or cognitive (survey) point of view.

Additionally, an emotional analysis of participants’ facial expressions suggests that visiting the HAZEF Facebook page evokes emotions of neutrality and sadness, which is consistent with the muted colors that are prevalent on the page. The possible speculations of elicited negative emotions being a reflection on the conference are disputed in this instance, as the research was conducted before the start of the conference to control for possible biases. Since the intensity of negative emotions and moods decreased with the presence of images with human faces, the use of such materials is a good choice to attract positive attention on the Facebook page. Such a shift in mood is not surprising since people's eyes are innately attracted to human faces, and they often spend some time looking at them (Simion & Di Giorgio, 2015; Tullis et al., 2009).

Another important finding is that participants spent an average of less than 1.5 s reading the posts on the page. Due to the short attention span of today's Internet users, the material presented must be brief and concise to capture attention and to promote retention of the material (Jeganathan & Szymkowiak, 2020). In the future, it is advisable to use more concise posts that will attract attention with their combination of text, selected colors, and visual format and hold the attention for a longer period.

While the highest percentage of followers would like to read interesting news or updates on HAZEF's Facebook page, eye-tracking metrics have shown that such information is not optimally presented as evidenced by fewer fixations and a shorter time spent on posts. However, it is important to point out that due to the technical limitations of Sticky all participants had a pre-set time of 10 s to answer the survey question. Hence, our research lacks insight into the individual response times for the closed-ended question. A scientific discussion opens here as to whether more insightful survey results evaluating the response latencies of participants could have been obtained. Implicit results based on the reaction time assess the degree of conviction, or in other words, highlight the extent to which participants truly believe what they say, which is particularly useful for evaluating consumers’ brand memory. However, when exploring the reasons for visiting the HAZEF's official FB page, the main objective was to obtain explicit answers that would help us understand what type of information followers are searching for.

This research has shown how neuromarketing methods, such as eye tracking and facial coding, are very successful online, especially to address the deeper questions on human behavior. Despite the use of online research with a Tobii Sticky user webcam in which a screen-based eye tracker operates at 250 Hz, the results from this study show how we can get accurate and useful information to improve the effectiveness of Facebook pages by implementing the neuromarketing tools in promotion as one of the elements of the marketing mix.

Acknowledgments

Research activities, neuromarketing equipment, design and performed the experiment, derived the model and analyzed the data was supported by the Institute for Neuromarketing, Croatia. The authors would like to acknowledge the support of Prince Sultan University for paying the Article Processing Charges (APC) for this publication. The research was conducted exclusively for the sake of development and its contribution to science. The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.