Validity and Reliability in the Assessment of the Vulnerability of Social Networks

Abstract

Nowadays, measuring the impact and effectiveness of Social Networks is important for people who use them in an individual manner, due to social or academic interests, as well as for companies that use them to evaluate or promote their products and businesses. Particularly, it is necessary to monitor and evaluate qualitatively and quantitatively, if possible, this tool that is used for the dissemination of information of any kind. We have tried to create an instrument that aims to validate the reliability of social networks, taking Facebook our case study because it is one of the most widely used currently. This instrument is a tool that measures the “information vulnerability”; information that people decide to upload to the Internet as part of a social network. Regarding validity, a suitable solution focused on four factors was found: legality (self-regulation), social factors, psychological factors, and finally technology. It was concluded that the proposed mechanism can be a useful instrument that detects the information vulnerability storage in each social network.

Resumen

En la actualidad, la medición del impacto y la eficacia de las redes sociales es importante para las todas las personas que los emplean de manera individual, ya sea por intereses sociales o académicos, de igual manera es fundamental para las empresas que los utilizan para evaluar o promocionar sus productos y negocios. En particular, es necesario monitorear y evaluar cualitativa y cuantitativamente, de ser posible, la herramienta que se utiliza para la difusión de información. Se ha tratado de crear un instrumento que tiene como objetivo validar la fiabilidad de las redes sociales, tomando Facebook como caso de estudio, por considerarse una de las más utilizadas hoy en día. Este instrumento es una herramienta a manera de modelo, que mide la “vulnerabilidad de la información” que las personas deciden subir a Internet como parte de una red social. En cuanto a la validez, se encontró una solución adecuada centrada en cuatro factores: legalidad (autorregulación), aspectos sociales, aspectos psicológicos, y aspectos tecnológicos. Se concluyó que el mecanismo propuesto puede ser un instrumento útil que detecta la vulnerabilidad de la información almacenada en una red social.

Introduction

This study intends to validate the reliability of an instrument to measure the vulnerability of the information in social networks. This is due to the fact that during adolescence and youth, individuals require more approval from other people of the same age and communicate with them, and this is when they are most interested in belonging to a social network. Thus, in many cases, they show personal information that makes them easy targets for other people, who use these means of socializing for their own benefit, almost of any kind.

Owning to the importance that Information Technology has reached in recent years, and in particular social networks, people with other interests have also joined them, causing a rush of information and emotional situations that create social influences.

By 2011, in Mexico, the use of Information Technology at homes (INEGI, 2011) has shown considerable growth, facilitating access to Internet contents such as social networks. In addition, companies have been influenced and supported by Information Technology and have realized the importance of belonging to the most consulted and used social networks in Mexico.

However, behind this boom of dissemination and use of information in social networks, there is a problem with self-regulation that handles the media, and causes information to be vulnerable and to be misused by people or organizations without scruples.

Reliability can be defined as the ability of a product to perform its function as intended previously. In other words, the reliability can also be defined as the probability that a product will perform its intended function, without incidents during a specified period of time under indicated conditions. Running a reliability analysis on a product or on a system should include many types of tests, in order to determine the reliability of the intended product or system, to be analyzed.

For our case study, once the analyses are made, it is possible to predict the effects, changes and corrections in storage and handle the information that users provide within the social network. Several studies concerning social networks are related, linked and examined jointly, in order to determine their reliability under all possible perspectives, identifying potential problems, suggesting corrections, changes or improvements.

Nowadays, the importance of social networks in order to obtain emotional stability and social skills in the development of young people, is high. However, due to the misuse of information in these networks, there are socio-emotional problems, and later anxiety and depression. These could determine unpleasant outcomes such as suicide.

Moreover, companies know that social networks are very important tools for their business. However, they also know that there is insecurity and misuse of information that can affect them negatively. Therefore, in this context, the opportune detection of vulnerable information found in social networks becomes a priority, and the problem can be faced successfully, with proper management strategies and skills.

In Mexico there is no reliable and validated tool, capable of detecting the levels of information vulnerability in social networks. Our suggestion is:

• 1)

  Determine the variance proportion of the original data that the instrument explains from a proposed model.

• 2)

  Evaluate the content validity of this instrument.

• 3)

  Evaluate the internal consistency of this instrument.

Problem

There are a number of features that make social networks more or less vulnerable to information misuse, thereby affecting their users and in some cases hurting youngsters intellectually, emotionally and socially. In Mexico, there are no reliable and validated instruments to assess the information vulnerability in social networks. This research proposes a reliable and valid tool to measure the vulnerability of information contained in social networks. Assuming that this vulnerability is associated with the improper use of certain regulations, according to the information contained in these web sites with little or low security, this allows a third-party with proper technology to reach this information and use it improperly or incorrectly.

Main objective

This research aims to validate the reliability of an instrument to measure information vulnerability stored in social networks.

Conceptualization and Social Network analysis

The term social network refers to a structure and joint interconnected relationships (Mitchell, 1969). However, the term “social network” also refers to Internet platforms and with Social Network Analysis, and thanks to existing means it is possible to obtain several statements about this. In social networks, the social nature of the network is modelled as a graph of interconnected nodes.

Consequently, the applicable rules to the measurement of directed graphs can be applied to this network of nodes, and the degree of a node can be calculated by adding all input and output edges or counting each edge just once. In addition, as a part of Social Network Analysis, the density of a network can be measured. This network is limited by the maximum number of edges that may exist.

The greater the density on a network, the stronger the awareness among participants (Wellmann, 1981); additionally, it is also possible to calculate the local density in a subnet. First, define an actor and this provides a measure that describes the ratio between the real connections and all possible connections with other actors. Other interesting indicators are centrality measures defined by Freemann (1979) showing the popularity on the network.

Furthermore, based on graph theory, it is possible to map a social network as a graph. In the same way, the information, trends and preferences of the members of the network can be recovered.

Methodology

Type of study

A descriptive study was conducted with a quantitative methodology, through a questionnaire with the intention of a measuring ‘the vulnerability of information in social networks’, validating it and considering it reliable.

Population

The population consisted of a group of male and female students from the first senior level semester from the School of Mechanical and Electrical Engineering (ESIME, Escuela Superior de Ingeniería Mecánica y Eléctrica) of the National Polytechnic Institute (IPN, Instituto Politécnico Nacional) in Mexico.

Sample

A probability sample was chosen considering a confidence level of 95% with an estimation error of 5%. The sample included 160 students of both levels. Finally, 90 subjects (56.25%) of the total sample answered the questionnaire.

Of these, 37 (23.13% of the total sample) were females and 63 (39.38% of the total sample) were males and their ages ranged between 18 and 26 years.

Instrument

The questionnaire “vulnerability of information” was developed taking into account the technological, social, psychological and legal dimensions. This was based on the proposal of coping with information loss, arguing that there are individuals with personal and social characteristics, who are dedicated to investigate the vulnerability of social networks to extract information, in order to use it for other purposes. This questionnaire originally consisted of 39 items divided into: information risk (20 items), security strength of social networks (13 items) and social reaction from the misuse of information (6 items).

Procedure for data collection

Using the program SPSS v.10 (SPSS, 2011), a probabilistic study was conducted in which we chose a sample size of 100, just randomly applied to a group of male and female students from the first semester of senior level within the School of Mechanical and Electrical Engineering (ESIME, Escuela Superior de Ingeniería Mecánica y Eléctrica) of the National Polytechnic Institute (IPN, Instituto Politécnico Nacional) in Mexico. Their ages ranged between 18 and 26 years, considering that it is the young who are now in the majority of social networks users.

Later, students inside the school were asked for their voluntary research cooperation, ensuring data confidentiality.

A factorial analysis was performed on data obtained from the original questionnaire to identify main components grouped in items. After identifying the components, we proceeded to a content analysis for the factors conceptualization and the theoretical items analysis. This allowed us to evaluate the content validity. Lastly, the reliability of the questionnaire was determined by the response consistency of the items. It was answered with a 5-point Likert scale (Méndez y Peña, 2007), ranging from strongly disagree to strongly agree (as shown in Table 1). A strongly agree response corresponded to a value of 1 and 5 to a strongly disagree response.

Table 1.

5-point Likert scale

Consideration	Value
Strongly agree	1
In agreement	2
Indifferent	3
Disagree	4
Strongly disagree	5

All items were written positively, so that, a higher score means greater vulnerability.

Communalities Results

“Commonality” is the name for the variance proportion explained by the common factors in a variable. The commonality (h²) is the sum of the squared factor weights in each row.

Factor analysis (Repositorio Institucional de la Universidad de Alicante, 2013) begins its calculations based on what it is known as reduced matrix composed by the correlation coefficients between the variables and commonalities in the diagonal.

As communality cannot be found until the factors are known, it is one of the problems of factor analysis. Since the Principal Component Analysis does not assume the existence of any common factor, the communality takes 1 as an initial value. Other methods are used in different ways to estimate the initial commonality:

• •

  Estimating the highest correlation commonality in the i-th row of the correlation matrix.

• •

  Considering the commonality as the square of the multiple correlation coefficients between x and other variables. (SPSS gives it by default).

• •

  The average of correlation coefficients of a variable with all others.

• •

  Calculating from the two highest correlation coefficients of that variable, the following operation:

  h2=rxyrxzryz

The final commonality of each variable is given by:

h2=P1j2+P2j2+…+Pkj2

Communalities of the variables (items) were estimated by the extraction method named Principal Components. This analysis considered acceptable those items that had communalities above 0.3 an overall there were 23 of the 39 that met this condition. Items 5, 7 and 9 were better explained by the obtained factor model as the variance percentages were 60.8%, 60.8% and 60.6%, respectively. Items 10, 3 and 14 were the least explained as the single factor model can only reproduce 29.5%, 32.5%, and 32.5% respectively, of its original variability (see Table 2).

Table 2.

Communalities for items asked ESIME’s students to find the vulnerability of Social Networks

Items	Initial	Extracted
Do you think it is important to have an account on a social network?	1.0	0.322
Do you agree on the usefulness of social networks?	1.0	0.481
Do you think it is important to access frequently to a social network?	1.0	0.325
How important is the time you spend on the social network?	1.0	0.395
Are the networks you belong to easy to use?	1.0	0.608
When you belong to a social network, do you consider it is important to read the legal agreement before signing it up?	1.0	0.516
Do you think it is important to have knowledge about the security provided by the social network you belong to?	1.0	0.608
When you upload your personal information and photos on the social networks you use, do you think that information in your profile is in danger?	1.0	0.448
Are you accepted by all the people you send a friendship request in the social network you use?	1.0	0.606
Do you consider your personal information is at risk in the social network you use?	1.0	0.301
Have you ever had any problem in the social network you use?	1.0	0.458
Have you ever got a ‘blocked access’ from the social network you use?	1.0	0.351
Have you ever been a victim of bullying in social networks?	1.0	0.332
Have you ever experienced spoofing problems and loss of access to your account on the social network that you use?	1.0	0.325
Did you respond appropriately and satisfactorily to the social network you belong to, when you discovered that you were a victim of the misuse of your information?	1.0	0.413
Would you recommend the social network you use?	1.0	0.370
Do you usually send all kinds of personal information in your account (personal photos, family, friends, address, etc)?	1.0	0.496
If you make available your information to only some people do you think there is enough security?	1.0	0.475
Do you think the information in your profile and in your social network account could be in some kind of danger?	1.0	0.445
Do you think the social network, you belong to, has satisfied your expectations of quality of the service they offer?	1.0	0.362

Factor 1 (Legality (self-regulation)), Factor 2 (Social Aspects), Factor 3 (Psychological Aspects) and Factor 4 (Technology) these factors explain 44.5% of the total variance from the original data. It is observed that the first component explains 21.8% of the variance; whereas the rest of the components explain 22.7% of the variance (see Table 3).

Factors	Total	Percent variance	Cumulative Percent variance
Legality	5.024	21.845	21.845
Social	2.135	9.283	31.128
Psychological	1.621	7.046	38.175
Technology	1.444	6.280	44.455

Subsequently, the matrix of factorial structure was created, where the items are grouped in each of four factors. Variables were grouped into components according to the following criterion: factor loads greater than 0.3. In the case of the variables that were filled in two or more factors, these were grouped in the most factorial load factor and those corresponded to the conceptual component (see Table 4).

Table 4.

Factor structure matrix obtained from the survey ESIME’s students

Items	Legality	Social	Psychological	Technology
Do you think it is important to have an account on a social network?		0.524	0.684
Do you agree with the usefulness of social networks?				0.742
Do you think it is important to access frequently to a social network?		0.758	0.701
How important is the time you spend on the social network?		0.643	0.650
Are the networks you belong to easy to use?				0.726
When you belong to a social network, do you consider it is important to read the legal agreement before signing up?	0.514
Do you think it is important to have knowledge about the security provided by the social network, you belong to?	0.634
When you send your personal information and photos on the social networks that you use, do you think that information in your profile is in danger?	0.517
Are you accepted by all the people you send a friendship request in the social network you use?		0.551	0.496
Do you consider your personal information is at risk in the social network you use?	0.729
Have you ever had any problem in the social network you use?				0.404
Have you ever got a ‘blocked access’ from the social network you use?		0.714	0.650
Have you been a victim of bullying in social networks?		0.758	0.701
Have you ever experienced spoofing problems and loss of access to your account on the social network you use?	0.554
Did you respond appropriately and satisfactorily to the social network you belong to when you discovered that you were a victim of the misuse of your information?	0.353
Would you recommend the social network you use?				0.418
Do you usually send all kinds of personal information in your account (personal photos, family, friends, address, etc.).?		0.742	0.726
If you make available your information to only some people. Do you think there is enough security?	0.359
Do you think the information in your profile and in your social network account could be in some kind of danger?		0.359	0.326
Do you think the social network, you belong to, has satisfied your expectations of quality of the service they offer?	0.451

a. (Factor 1) Legality (self-regulation), (Factor 2) social aspects, (Factor 3) psychological aspects and (Factor4) technology.

The factors were conceptualized as follows: Legality (self-regulation) (Factor 1), social aspects (Factor 2), psychological aspects (Factor 3) and technology (Factor 4). After analyzing the validity of the items, in the proposed questionnaire a clear definition of the factors and indicators classified within each was obtained (see Table 5).

Table 5.

Specifications from the questionnaire to measure the “Vulnerability of the Information on Social Networks

Factor	Definition	Indicators
Legality (self-regulation)	There are laws that regulate the use of information in Information Technology, each Web site has the ability to self-regulate (creating its own policies) based on the existing laws of security to be offered to its users.	- When you want to belong to a social network, do you consider it is important to read the legal agreement before signing up? - Do you think it is important to have knowledge about the security provided by the social network, you belong to? - When you send your personal information and photos on the social networks that you use, do you think that information in your profile is in danger? - Do you consider your personal information is at risk on the social network you use? - Have you ever experienced spoofing problems and loss of access to your account on the social network you use? - Did you respond appropriately and satisfactorily to the social network you belong to when you discovered that you were a victim of the misuse of your information? - If you make available your information to only some people, do you think there is enough security? - Do you think the social network, you belong to, has satisfied your expectations of quality of the service they offer?
Social	Human qualities	- Do you think it is important to have an account on a social network? - Do you think it is important to access frequently to a social network? - How important is the time you spend on the social network? - Are you accepted by all the people you send a friendship request to on the social network you use? - Have you ever got a ‘blocked access’ from the social network you use? - Do you consider being a victim of bullying through the social network? - Do you usually send all kinds of personal information in your account (personal photos, family, friends, address, etc)? - Do you think the information in your profile and in your social network account could be in some kind of danger?
Psychological	Study mental processes in humans beings, animals and everything that is related to human behavior.	- Do you agree with the usefulness of social networks? - Are the networks you belong to easy to use? - Have you ever had any problem in the social network you use? - Would you recommend the social network you use?
Technology	Set of theories and techniques for thepractical use of scientific knowledge.

Reliability analysis

Due to its high sensitivity, Cronbach’s alpha (Cronbach, 1951) was used for detecting measurement error sources that are attributable to the sampling of situational factors.

Table 6 presents the reliability indexes corresponding to each of the four major factors. These four reliability indexes can be considered acceptable according to Hernandez, Fernandez and Baptista (Hernández et al., 2006), since the highest theoretical value of alpha is 1, 0.80 is generally considered an acceptable value, therefore the high the value of alpha, increases reliability (Bogliaccini et al., 2005).

Table 6.

Reliability indexes for measuring factor “Vulnerability of information in social networks”

Factors	Media	Standard deviation	Cronbach’s alpha
Legality	1.98	0.83	0.76
Social	1.57	0.80	0.69
Psychological	2.05	0.88	0.59
Technology	1.96	0.86	0.63

Results

Regarding the content validity of the evaluated questionnaire, a suitable factor solution with four factors that explain up 44.5% of the variance of the measured construct was found; in this case, the vulnerability of information on social networks.

The percentage of the obtained variance, although not satisfactory, represents an important step in validating the questionnaire, considering the novelty of the context in which it was applied.

According to the obtained responses, the possible merger of two factors, social and psychological aspects, was observed, because psychological aspects increase the social aspects.

On the other hand, it is possible to adapt and validate measurement questionnaires on information vulnerability as needed, in the case of social networks.

The reliability indexes of each information vulnerability questionnaire scales on social networks can be considered as satisfactory.

Conclusions

Nowadays, the importance of social networks is undeniable and they will continue to be valid in an immediate future. The obtained results for this sample show that the questionnaire could be very useful for early detection of any possible risk in any social network. For example: identity theft or information theft.

Two important factors, which encourage people to join social networks, are the social and psychological aspects and these tend to be confused. So, it can be considered that most people who join take them as only one aspect.