From institutional websites to social media and mobile applications: A usability perspective

Abstract

Mobile applications (MA) and social media (SM) platforms are changing Internet user behavior. This study aims to unveil within current academic literature, the fields where usability research has been focusing their efforts in the dimensions institutional websites (IW), SM and MA usability, and to suggest possible paths for future studies. Search was performed in peer-review journals, providing 302 published articles between 1994 and 2018. To examine the manuscripts, text mining (TM) was adopted to discover pertinent terms, and to reveal trends, gaps and opportunities for future research. Results show a gap on marketing and nutrition research fields and an increasing interest in the usability principles of SM technology in general. Moreover, many articles are associated with the health and medical area, suggesting a more mature development of these fields.

1

Introduction

New behavioral patterns are emerging based on the intensity of information channels which surround us. SM platforms, such as Facebook and Twitter, have supplanted the traditional website (Nah & Saxton, 2013), and consumers prefer to share their thoughts and experiences through SM instead of using websites (Yan et al., 2016). There are more than one MA for every occurrence of our life (Xu, Frey, Fleisch, & Ilic, 2016) and MA have exceeded the role of websites on sales (He & Liu, 2017). Beyond facts, statistical evidence reinforces a paradigm change. About forty percent of the worlds’ population are SM active users (Statista, 2018), SM platforms outperform being at the top online destinations (Hodis, Sriramachandramurthy, & Sashittal, 2015), five of the twelve most visited websites on the Internet are SM platforms (Alexa, 2018), and ninety percent of the time spent on a smartphone is using a MA (Alliance, 2017), revealing how indispensable SM and MA are on the users’ daily life. These evidences suggest that users, confronted with the possibility of choosing between an IW or a SM/MA, tend to choose the latter option. The degree of usability of each platform influences and determines their use. Usability is the ease of use and learnability of a human-made object such as a tool or device (Chow, Bridges, Commander, & Figley, 2014). Usability has emerged as a key domain in information systems (IS) and has developed efforts to understand the influence of several characteristics that influence usability of technology and design (Appendix 1). Technology refers to stability and performance of the IS, while design creates an environment that allows the user to positively interact with the IS (Koehler & Mishra, 2005). User frustration increases when usability principles are absent. The most important factors that influence consumer decisions to reject an IS are lack of usability of those technologies (Khasawneh & Kornreich, 2015).

The principles that enhance usability have influenced users toward a different path when it turns to Internet access (Kortum & Bangor, 2013). IW, SM and MA are IS that users tend to use to find relevant information about products and services, and are used as a business card and point of contact in the virtual world (Huang & Benyoucef, 2015). From this stand point, it is possible to infer that usability is a key concept that influences the user, and the achievement of a higher efficiency, efficacy and satisfaction of the SM and MA led to the identified behavioral change, revealing the propensity of users to choose these platforms instead of IW to find relevant information, turning IW less relevant in the online context.

Considering the impact that usability has on the acceptance of IS, the aim of this paper is to summarize the fields where usability research has been focusing their efforts in the dimensions of IW, SM and MA, and to suggest promising directions for future studies. There are various factors highlighting the significance and convenience of such review of the literature. First, the field of usability applied in the dimensions of IW, SM and MA has seen a growing number of publications, yet a careful review of existing work is absent. Literature reviews are fundamental to create a sustainable evolution in a field study. Second, a literature analysis allows researchers to align their studies toward the identified trends and gaps, and highlight areas where a plethora of research in a determined field reveals itself saturated (Webster & Watson, 2002). Finally, research so far did not consider the identified behavioral change to clarify an alternative research point of view (Rowe, 2014), and lacks a roadmap in terms of usability. Evaluating existing literature can prompt to a better understanding of the state of the art of the three dimensions, but it additionally distinguishes patterns in the development of the areas themselves. This literature review presents the analysis of 302 full-text scientific papers through TM to offer the current research trends in IW, SM, MA usability contexts, find patterns of information from the collected data, and translate them into valuable knowledge to uncover opportunities for further research and future applications.

2

State of the art of IW, SM and MA

2.1

Institutional website usability

An IW can be defined as the sum of related web pages under a single Internet domain under the control of the company manager (Aksakallı, 2012), with the purpose of serving as a point of contact of the company or organization in the world wide web, providing relevant information. This type of website has the function of providing the basic information about an institution (https://europa.eu/european-union/about-eu_en), contacts (https://www.europarl.europa.eu/portal/en/contact), service (https://online-learning.harvard.edu/) or product (http://www.pepsico.co.uk/what-we-believe/products), among other relevant information.

The global performance of an IW relies on a set of features and elements. Its quality is influenced by the dimensions system quality, information quality and service quality (Zhou, 2011). IW interactivity, informativeness, security, responsiveness, trust and perceived usefulness have the most statistically significant effects on user satisfaction and intention to use (Herrero & San Martín, 2012). Researchers suggest several approaches to evaluate IW based on quality, content, customer acceptance and satisfaction (Salavati & Hashim, 2015).

In a broader context, usability research has revealed the important role in forecasting positive attitudes, performance, satisfaction, better interaction techniques, among other findings. A sample of studies where usability was a subject of analysis in the IW dimension is presented in Appendix 2.

2.2

Social media usability

SM can be defined as a technological platform founded under the Web 2.0 principles, where users have the power to change its dynamic by introducing user-generated content (Chung, Andreev, Benyoucef, Duane, & O’Reilly, 2017). The participation on SM platforms involve social communication, building reputation, building career opportunities and generating monetary revenue, thus providing several benefits (Zhang, Trusov, Stephen, & Jamal, 2017). SM has been used for accessing shops, search for a place (e.g. TripAdvisor, Foursquare), to be updated (e.g. BBC news, Facebook page), look for a rating of a product (e.g. Amazon), search for an answer, question or advice (e.g. Quora), educational content (e.g. Wikipedia), listen to music (e.g. Spotify), watch videos (e.g. YouTube), share photos (e.g. Instagram), follow interests (e.g. Pinterest, Tumblr), find friends (e.g. Facebook), chat with friends (e.g. Twitter, WhatsApp), professional purposes (e.g. LinkedIn), among other interests.

Recent SM usability research has identified considerable contexts in which it has been a subject of analysis, such as the impact of usability on SM to increase purchase intention (Shang, Wu, & Sie, 2017), word of mouth and loyalty (Risius & Beck, 2015), among others. A sample of studies where usability was the focus of analysis in the context of SM dimension is presented in Appendix 3.

2.3

Mobile application usability

MA is a software that can be downloaded by and run on mobile devices, such as tablets or smartphones, with a native language of a specific platform (Franko & Tirrell, 2012). With the progress of mobile technology, namely smartphones and tablets, the adoption of MA changed the way we work and interact (Hsiao, Chang, & Tang, 2016) and its increasing use is supported by the number of MA downloaded (Statista, 2017).

The application of usability guidelines from other technologies on MA are fruitless given their characteristics differ from others, but a less probability of failure can be achieved by developing a MA that is user-friendly, easy to learn and more effective on the task performance (Kortum & Sorber, 2015).

The diversity of research on MA usability dimension has provided insights concerning its use in distinct contexts (Gao, Zhou, Liu, Wang, & Bowers, 2017). A sample of studies where usability was analyzed in the MA dimension is presented in Appendix 4.

Usability characteristics have been playing an important role toward the use of IS, influencing users to choose one platform over the other. This research aims to resume the stand point in the IW, SM and MA usability fields where researchers have been setting their efforts to find trends and gaps and suggest directions for future research.

3

Methodology

3.1

Journal selection

To select the relevant publications, the emphasis was set on finding the most influential peer-reviewed journals (Q1 Web of Science – WoS) on the researched dimensions. Scopus was chosen to find relevant literature about the relationship of IW, SM and MA, since it is one of the most widely accepted bibliographic databases where relevant publications are indexed. The keywords used to create the three queries were collected from the literature (Appendix 5) to eliminate the inherent subjectivity associated to a query. The queries consisted of a Boolean expression using AND/OR between the dimension and usability keywords, implying that any article should contain at least one dimension keyword and one usability keyword. Search was performed in January of 2018. The search steps and results can be observed in Fig. 1. On the first step, keywords associated with “literature review” were added to find literature reviews from each dimension. On the second step, a search refinement was applied, providing 142 IW usability literature reviews, 75 usability SM literature reviews, and 52 MA literature reviews. After the analysis of each literature review, were selected only literature reviews that provided the list of articles that supported the review (stage three), revealing two literature reviews for each dimension. From the list of articles that supported the six literature reviews were selected the Q1 Web of Science journals (fourth stage), revealing the most influential peer-reviewed journals. The final list comprised 14 journals for IW usability, 81 journals for SM usability, and 54 for MA usability, which were established as the sources for this study. To find the corpus for this study, the search query was again applied, but without the literature review keywords, and restricted to the most influential peer-reviewed list of journals. The outcomes provided an aggregate of 309 articles (156 for IW, 83 for SM, and 70 for MA). A manual analysis detected duplicates that were eliminated and led to a final dataset of 302 articles (153 for IW, 80 for SM, and 69 for MA) from 18 journals. Appendix 6 provides the number of articles per journal.

Fig. 1.

Steps to find the most-peer review journals.

Journal of Medical Internet Research and Computers in Human Behavior where the major data sources for this research, uncovering their overwhelming position for each dimension.

3.2

Text mining for literature review

To have some organization of the produced information and to maintain the scope within a manageable list of terms, three dictionaries were defined, one for each dimension (IW, SM, and MA), integrating a list of terms of one or more words (n-grams). By contemplating n-grams (Moro & Rita, 2018), the system can introduce some setting through the blend of a set of words (e.g., “social media”). The criteria to assemble and validate the dictionaries, reducing the inherent subjectivity, were as follows:

• •

  The 302 scientific articles’ keywords provided the list of relevant terms.

• •

  The keywords were grouped in clusters to reduce similar concepts into a common term.

• •

  With the possibility of adding or deleting terms, a panel of 11 independent multidisciplinary experts analyzed the produced dictionaries, validating them.

TM can be alluded as the search for shrouded information, patterns or trends, on large amounts of data (He, Zha, & Li, 2013). Full-text, excluding the reference section to guarantee that no term was captured from any publication title cited in the article, was analyzed permitting a full text evaluation of term frequencies. TM includes several steps of work over the collected raw text, such as converting all words in lower case, ensure that stemming is applied, i.e. reducing similar words into a common term (e.g., “web site” and “websites” are reduced to “website”) according to the dictionary (Moro, Cortez, & Rita, 2015). “R” statistical software was used to conduct the TM technique. The R tool provides an open source platform for conducting data analysis through a myriad of packages developed by an enthusiastic community (Cortez, 2014). Specifically, for the tasks underlined in this study, two packages were adopted, the “tm” for extracting terms into the document term matrix, and the “topicmodels” for producing the topics summarizing the findings. Appendix 7 presents the validated dictionaries for IW, SM, and MA dimensions (some of the similar terms were omitted to save space).

3.3

Classification of topics

To accomplish a format that allows a deeper analysis, topic modeling was applied. Topic modeling provides a structure that gathers articles in order to allow profound scrutiny and discover terms that often appear together in a document or in a large set of documents (Santos, Rita, & Guerreiro, 2018) For modeling purposes, the latent Dirichlet allocation (LDA) algorithm was chosen, as it is the most popular topic modeling technique. By supplying as input the document term matrix, it provides an integrated overview of the body of knowledge by grouping articles on the most relevant topics (Amado, Cortez, Rita, & Moro, 2018; Moro & Rita, 2018). Such model empowers to break down the relative pertinence of each term utilizing the beta (β) distribution value, which depicts the relationship between the theme and the given term (Calheiros, Moro, & Rita, 2017). This structure can help recognize which topics are catching more consideration from researchers and discover gaps for future research. The LDA product is a tridimensional table incorporating topics, terms, and articles distributed through the defined time-frame. Thus, for each topic it is conceivable to acquire a measure of its connection to one of the word reference terms through the β distribution. Likewise, for each article it is conceivable to check which topic it suits better (Canito, Ramos, Moro, & Rita, 2018). The LDA model is considered one of the most important probabilistic models in widespread use today (Moro et al., 2015). The three most critical terms for portraying every topic as stated in the β distribution were considered.

4

Results and discussion

The analysis involved the study of the topics obtained by running LDA on the gathered data. For each topic, there is always a dominant term, with a β value that matches it to closeness to that topic. A β closer to zero indicates a stronger connection between the term and the topic. The terms “Technology” and “Design” from Appendix 7 were included to contextualize the focus of investigation in each dimension. Appendix 8 discusses the LDA results.

Through the analysis of the Venn diagram presented in Fig. 2, it is observable that education, health, mobile device, and physical activity have received attention from the three studied dimensions, providing valuable knowledge in these domains.

Fig. 2.

Website, Social Media, and Mobile Application usability dominant fields.

Tables 1–3 in the appendix 8 provide detailed results and subsequent analysis concerning technology and design settings in each context, which are summarized in Table 1 in terms of the main trends and gaps in existing literature.

Table 1.

IW, SM, and MA literature trends and gaps.

	Institutional website		Social media		Mobile application
	Technology	Design	Technology	Design	Technology	Design
Business			TREND	GAP	GAP	GAP
Communication		TREND	TREND	GAP	GAP	GAP
Demography	GAP	GAP	TREND	TREND	GAP
Education	TREND	TREND	TREND	GAP	TREND	GAP
Health		GAP	TREND	GAP	GAP
Innovation	GAP	GAP	TREND	GAP	GAP	GAP
Marketing	GAP	TREND	GAP	GAP	GAP	GAP
Mobile device	TREND	TREND	TREND	TREND	TREND	GAP
Nutrition	TREND	GAP	GAP	GAP	GAP	GAP
Physical activity	TREND	GAP		GAP

Within the IW usability investigation, a careful analysis revealed that recently the focus of the academia has been in the design usability characteristics setting, unveiling a trend in the areas of communication, education, marketing and mobile device, with research gaps in the areas of demography, health, innovation, nutrition and physical activity. Research of IW technology usability setting has been considered in several contexts, namely, education, mobile device, nutrition and physical activity. The areas of education and mobile device are those that reveal a trend in both technology and design settings. In general, in the research in IW, usability started between 1997 and 2001 with 16 articles, but throughout the years the interest has been increasing, revealing a trend, with sixty articles published between the years 2014 and 2018.

Regarding SM usability research, results show that there are major developments in the technology setting. There are seven trends out of nine technology research areas, and seven gaps out of ten design research areas. The demography and mobile device areas are considered trends in both technology and design settings. However, for marketing and nutrition, both are underdeveloped in the considered research settings (technology and design). Marketing through SM platforms can promote consistent interactions between business and their target, building stronger relationships (Alarcón, Sepúlveda, Valenzuela-Fernández, & Gil-Lafuente, 2018), so that the understanding of usability principles in the technology and design settings can improve those relations by transforming the message via arranging the information in a clearer and easier way to read (Ku, Chen, & Zhang, 2013). In general, SM usability has revealed to be a trend. From 2009 to 2013 nineteen articles were published, however, from 2014 to 2018, there were sixty papers published in peer review journals.

Within MA usability studies, there are two trends in the technology setting (education and mobile device) and zero in the design setting. The application of a user-centered design is strongly recommended (Holzinger & Errath, 2007), and the research of the usability principles of design can result in the creation of a suitable user interface that can meet the consumers’ needs. It is understandable that the areas of business, communication, innovation and marketing expose gaps in the literature since the collected keywords from the 69 articles used to build the MA dictionary (Appendix 7) have not provided any keywords in the designated areas. Nowadays, having the possibility of making commerce through MA, namely in-app transactions (Hsu & Lin, 2016), it is crucial to understand what characteristics can augment the efficiency, efficacy and satisfaction for commerce purposes (Kumar & Reinartz, 2016). The absence of research on the aspects that improve usability in marketing through MA, suggests a gap in the literature. Therefore, it is essential to uncover usability of advertising in MA to understand their impact on companies for a sustainable and profitable future, especially on business models that depend solely on ads (Jiménez & San-Martín, 2017). In the technology setting, the analysis points to gaps in the literature of research of demography and health. It is relevant to find the usability technological characteristics that enhance the use of MA to mitigate the difference of adaption that different generations find when they are confronted with the use of mobile technologies (Gao et al., 2017). Moreover, in the design setting, the areas of education and mobile device also reveal gaps in the literature. To help the issue of learning by using MA, the understanding of design usability principles can become fundamental to make easier and satisfying for students the recording of lessons learned (Rueda, Benitez, & Braojos, 2017). A gap in the technology and design setting is found in the field of nutrition. Consumer health and nutrition technologies are important drivers to manage health of patients and reduce costs (LeRouge, Van Slyke, Seale, & Wright, 2014). A general analysis highlights a gap in the technology and design setting in several fields of investigation in the MA dimension. This gap should have to do with the fact that the interest of researchers started from 2013 forward, suggesting a recent focus of investigation and the beginning of a trend.

From a field analysis perspective, it is important to highlight that marketing and nutrition have not been under the radar of researchers in terms of usability on SM and MA dimensions. The use of SM and MA for nutrition purposes needs to be further investigated to provide knowledge to academia and nutrition professionals to use these platforms to reach their audience and provide efficiency, efficacy and satisfaction, and in the end, better results (Tobey & Manore, 2014). Usability research on marketing will reveal knowledge that can create a positive environment to amplify brand communities (Kaur, Dhir, & Rajala, 2016).

5

Conclusions

This paper reviewed an extensive amount of existing IW, SM, MA usability studies and outlined roadmaps for future research in ten specific research fields. Fig. 2 and Table 1 details each research area outcomes revealed by performing this literature review.

Based on these findings, there are three aspects that should be emphasized. First, two critical themes to investigate are the usability areas of Marketing and Nutrition that revealed themselves underdeveloped in terms of research in the three dimensions. It seems that researchers are not giving the deserved attention toward these two areas considering their relevance in the mentioned dimensions (Tsai & Cheng, 2012). Second, the results of this research highlighted the amount of papers associated to the health and medical fields and a growing interest on the usability technology of SM, while usability of technology and design of MA is less focused. By mentioning these deficiencies in the literature, researchers can redirect their work by opening new paths of research. Third, it is recommended that researchers take in consideration the new behavioral change that is occurring on the Internet to assure that the relevance of their work meets the identified behavior path and the user experience (Lemon & Verhoef, 2016).

From a future research point of view, it would be useful for the research community to map the efforts of the IW, SM, MA usability to provide a thorough analysis on each researched field. Specifically, future research could investigate the IW usability on user behavior related to trust and intention to use an IW, and the influence of usability characteristics, like navigation or layout, on trust building features, such as users’ reviews and user-institution interaction in mobile devices. From the SM usability stand point, future studies could explore the significance of usability to prevent lurking in online communities, and the impact of SM design features on security and privacy protection, such as privacy protection and data encryption, applied to different generations. Future research may also benefit from the understanding of loading time, esthetics and quality of information on perceived usability of a MA, and the impact of MA usability on first impressions, quality of perceptions and emotions in the context of advertising.

This review has limitations that need to be stated: the search scope was restrained to Scopus database and Quartile 1 ISI Web of Knowledge; regardless of the use of 11 multidisciplinary experts for dictionary validation purposes, there will always be an inherent subjectivity at the dictionary definition. Additionally, the professional profiles used for the panel of experts are restricted to a small set of professional activities. A wider range of profiles could lead to a richer dictionary, enhancing its quality; finally, a significant part of the body of knowledge gathered in this research is focused on the health/medical domains. Specifically, 54.97% of the articles were published in those areas, implying an over-representation of the health area, thus influencing the obtained results. Nevertheless, this fact also shows that health has been extensively explored, particularly within websites, exhibiting a high level of innovation.

This intensive review of identified articles makes an important conceptual addition to academia, by revealing the research focus, trends and literature gaps in IW, SM and MA usability research and suggesting the future path for academics. Researchers can use this valuable knowledge as an immediate reference from where to develop their work.

Declarations of interest

None.