Year two of the Claraboya research project: changing faculty attitudes regarding academic library resources for improving academic performance in selected subjects and student satisfaction

ABSTRACT

This paper reports the research results of the Claraboya project, which through the implementation of attitudinal changes in teachers regarding reading suggestions, is focused on improving student academic performance and student satisfaction with the resources available in the campus library. The research methodology entails a comparison of pre-test result of seven indicators of satisfaction of 373 students against two random samples of students who responded to the 2013 Service Quality Survey. The research found that in six of seven variables students participating in the Claraboya Project Research Survey (cprs) exhibited higher average rates than those of Sample 1 and Sample 2 gathered in 2013. Moreover, the Claraboya sample exhibited statistically greater levels of satisfaction across six and five variables, respectively. To verify the second hypothesis, courses taught by the same teacher (Claraboya) were isolated and their averages compared. This hypothesis was rejected as only four of twenty-one exhibited significant differences. The scope and limitations of the study are included in conclusions.

RESUMEN

La presente investigación informa los resultados del proyecto Claraboya, el cual estuvo centrado en mejorar dos variables: 1) satisfacción del estudiante con los recursos disponibles en la biblioteca del campus, a través de la implementación de cambios actitudinales en los académicos acerca de las sugerencias de texto, y 2) el rendimiento académico de los estudiantes de los cursos participantes. Mediante un diseño preexperimental, se compararon los grados de satisfacción en siete indicadores entre una muestra por accesibilidad de 373 alumnos y dos muestras aleatorias de estudiantes participantes en la encuesta de calidad de servicio 2013. Los promedios obtenidos por los alumnos en la Encuesta de Investigación Proyecto Claraboya (cprs) son superiores a los promedios 2013 para las muestras 1 y 2 en seis de las siete variables analizadas. Además, hay un nivel de satisfacción estadísticamente superior en seis y cinco variables respectivamente, en comparación con las muestras aleatorias. Para verificar la segunda hipótesis, se extrajeron los cursos impartidos por el mismo profesor en 2012 y 2013 (Claraboya) y se compararon en sus promedios. Esta hipótesis fue rechazada, en la medida en que sólo cuatro de los 21 cursos mostraron diferencias estadísticamente significativas. Finalmente, se discuten los alcances y las limitaciones de este estudio.

INTRODUCTION

San Sebastian University (uss) has implemented several initiatives to enhance student learning. The investment in bibliography and data bases in the year 2013 came to US$ 638.000, bringing the collection up to more than 144.000 books, of which 56% are basic bibliographies for course subjects.

These improvements, however, have not resulted in commensurate progress in student competency in the use of lbrary resources. In this context and in recognition that efficient and effective use of the resources available in the library is as an essential element of teaching and learning processes, the Quality Assurance Vice Chancellorship (vrac), in conjunction with the Academic Vice Chancellorship (vra), has implemented a two-year pilot project to increase students’ use of library resources and their satisfaction with said resources.

There is not enough evidence about the effects of efficient management of library resources on teaching of a particular subject or students’ thoughts regarding their satisfaction with the bibliographic resources available to support their studies in such a course. This connection stands at the intersection of quality management in higher education, the use of information technologies, and the sociology of education, administration, and library science.

The processes of teaching and learning are strengthened to the extent that library resources are effectively used in classrooms, where course content is matched with the materials actually available. For teachers to fully utilize their materials and match course content to material available, however, they must go to the campus library regularly with an awareness of expected learning experiences. This will facilitate the teaching and learning processes, making them more efficient.

On the basis of enrollment numbers each year, Chilean universities acquire basic textbooks and the university libraries acquire complementary bibliographies. At the uss, there is a permanent and increasing effort to acquire texts used to prepare students for professional careers. Additionally, the educational project of the uss and the learning environment “Vive la Experiencia uss,” have been established to enrich the classroom teaching and learning processes with cultural content, while providing students greater autonomy in the search for information, the possibility of learning and applying bibliographic standards, and evaluating the appropriateness of fonts used in academic tasks (De la Vega and Fukushi, 2011). This initiative is termed: “information culture in the classroom.”

While one might suppose that the growing existence of texts in the library and aforementioned initiative described would result in greater student satisfaction with the texts available; however, this has not happened. In two consecutive studies of service-quality, rating on a scale from 1 to 7, the availability of texts in the library for courses in Universidad San Sebastián has been rated by students as deficient. In 2010, the score came in at 4.8, and in 2011 it dipped slightly to 4.7 (Universidad San Sebastián, 2010, 2011).

REVIEW OF LITERATURE

Though most research has focused on the use of library resources by students (Simisaye, 2012), there are several studies showing faculty assistance to the institutional library is often inconstant and insufficient (Popoola and Haliso, 2009; Münster, 2003).

Other studies have shown a decrease of 35% in printed books over a 10 year period (Camacho and Spackman, 2011), and ebrary (2007) reports that approximately 50% of respondents prefer to use online resources for research, class preparation and instruction, versus 18% who prefer print materials.

Although Popoola (2008) says that just a small percentage of social scientists regularly use formal databases as information resources, Münster (2003) has reported that 79% of the researchers visit the library weekly and 47% use the catalog at the same rate, though there were some differences across the diverse disciplines. For a sample of 987 graduate students, Kayongo and Helm (2010) indicate that 53.3% visit the library while 65.5% use library resources from home. Of those who visit on a weekly or daily basis, 34.8% come to sign out or return a book.

The numbers of uses seeking assistance could be driven by several causes, including insufficient technical support and access to collections, the presence of modern technology in contemporary libraries, or the huge volume of information, etc. (Jiao, Onwuegbuzie and Daley, 1997). All of this is very important to consider to the extent that there is evidence that effective use of library resources has an effect on the quality of teaching and of learning (Popoola and Haliso, 2009).

For example, Gaona and Villuendas (2011) report statistically significant correlations between academic performance and attitudes toward reading (ρ=.413, p <.001), and to frequency of strategies used to find information in libraries (ρ=.239, p <0.001).

Soria, Fransen and Nakerud (2013) analyzed use of libraries at the University of Minnesota during the Fall 2011 by first-year undergraduate students and found statistically significant differences in cumulative gpa (cumulative grade point average) between those who used at least one library service compared to students who did not use any.

These data are in line with research by Han, Wong and Webb, (2011) that found a correlation between the use of libraries and graduation gpa of students graduating between 2007 and 2009 at the Hong Kong Baptist University (hkbu). This positive correlation was also found by Stone and Ramsden (2013), who successfully demonstrated a statistically significant correlation between student achievement and e-resources use and book borrowing statistics. Similar findings are reported by Goodall and Pattern (2011). This pattern has also been found in high school students, where libraries visits combined with proper study habits were significantly correlated to achievement in science subjects (Aanu and Olatoye, 2011). The key mechanisms behind these correlations, apparently, are not only cognitive or associated with the development of critical judgment, but they are also motivational (Ilogho, 2011).

The fact that teachers do not adequately stimulate the use of institutional resources can occur for many reasons. For example, many of the faculties are part-time professors, with varying approaches to the use of resources and effective, valid management of bibliographies. Moreover, these approaches are distinct from those exhibited by tenure-track teachers in these areas (Washington-Hoagland and Clougherty, 2002). On the other hand, within universities, there are dissimilar expectations about the proper use of and interaction with libraries, and the fullest use of available resources (Stebelman et al., 1999).

In this way, faculty could assign certain readings for assessment that are not found in library, simply by providing copies to students. By doing so, they transmit the implicit message to students that the library's collection is not useful for educational purposes in that subject; and students might complete the course and later report dissatisfaction with resources available in the library. This dissatisfaction is not a product of an inconsistency between the course program and the available library resources, but between the professors’ readings and those available in library. In light of the research of Lau (2001) and Pierce (2009), who suggests that a basic goal should be to encourage students to include library materials in their learning strategies, this disconnection is potentially very serious.

In sum, an attitudinal change must take place in faculties (Dierking and Fox, 2013; Khan and Pred, 2002; Kaplan, Cook and Steiger, 2006); and not only where the connections between course content and the library's resources are widely known. Faculty must also model efficient library use through an ongoing effort to assign readings that are available at the university.

THE CLARABOYA DESIGN: THE FIRST EXPERIENCE OF THE 2012 PILOT

A preliminary version the program aimed at increasing student satisfaction with the available library resources through the implementation of an attitudinal change in the professors regarding assignment of readings was designed and assessed. The central activity of this stage was a seminar attended by 18 faculty members (González et al., 2014).

In that first year, the overall structure of the seminar and the mechanisms to assess student satisfaction with the resources of the library were designed. Finally, satisfaction levels for seven indicators of and accessibility sample of 175 students of 18 pilot courses were compared to random sample of students participating in the Service-Quality Survey of 2010 and 2011. A greater level of satisfaction in all indicators was found for 2012. Five of these differences were statistically significant against 2010 survey data and as well as versus to 2011 data (González et al., 2014).

Due to the success of this initial experience, a program with greater coverage was designed for the year 2013. This program was offered to all accredited majors and those in the process of securing accreditation at the USS Campus Concepción.

Aims

To purpose of this study is to assess the the second year of the Claraboya project, which aims to: 1) increase student satisfaction with resources available in the campus library through the implementation of attitudinal changes in teachers regarding assigned reading, and 2) improve academic performance of students in participating courses.

Method

Hypotheses

• 1)

  Students and teachers participating in the Claraboya 2013 program have a statistically higher level of satisfaction with the resources available in USS library than students participating in the 2013 Service-Quality Survey.

• 2)

  The mean scores obtained from Claraboya 2013 courses are statistically higher than the mean scores obtained in 2012 in the same courses.

Design

The research design entails intact groups, and pre-experimental, experimental intervention, and post-test stages. The 36 subject professors participating in the project attended the Claraboya seminar in order to train to make optimal use of library resources in connection with their respective course contents. After this intervention, students of those subjects were evaluated in two variables: 1) student satisfaction with resources available in the campus library for the course and 2) academic performance in the participant courses, which was done by comparing academic achievement of students enrolled with the same professor the year before.

Intervention

The intervention seminar titled the “Claraboya Workshop: optimizing use of bibliographies and electronic databases available in library by implementing digital indexing in subjects,” was targeted at part-time teachers for the central purpose of helping teachers through attitudinal change to develop strategies for optimizing the use of academic library resources for pedagogical purposes (Steintert et al., 2006).

The seminar attended by 36 teachers lasted 15 hours, five of which were face-to-face class time (July 25, 2013). Participant courses were required to meet the following acceptance components:

• 1)

  A group project presenting a disciplinary theme in Microsoft Powerpoint which considered: a) connection between course content and virtual and physical bibliographies available in USS libraries system, b) connections with subject planning, subject programs and suggested texts, c) modeling behaviors aimed at effective use of library resources, and d) modeling of the incorporation of library resources into the classroom

• 2)

  Personalized follow-up throughout the semester, in which teachers receive assistance with questions and difficulties they encounter when implementing courses contents. This monitoring program was implemented using both telephone and email communication.

Sample

For the first hypothesis, an accessibility sample was used, consisting of 373 students from 17 of the 19 participant major fields: Law (5.1%), Preschool Education (0.8%), Nursing (7.5%), Phonoaudiology (8.6%), Business Administration (4.6%), Kinesiology (8.9%), Medicine (5.9%), Veterinary Medicine (3.8%), Nutrition and Dietetics (3.8%), Dentistry (14%), Pedagogy of Secondary Education in English (2.7%), Pedagogy of Secondary Education in Language and Communication (1.3%), Pedagogy in Primary Education (7.3%), Psychology (12.1%), Chemistry and Pharmacy (3%), Medical Technology (9.4%) and Social Work (1.3%). These students (35.6% males, 64.4% females) were enrolled in courses in Claraboya in 2013.

Of the 36 original subjects (courses) participating in the seminar, students of 34 of them answered the survey. To obtain the two 2013 Service-Quality Survey samples, 746 persons were randomly selected from the corresponding databases to achieve 343 persons in each sample. This procedure provides two databases needed to test the first research hypothesis, consisting of 746 students who responded to the Service-Quality Survey of 2013 (in two separate random samples of 343 each) and 373 students for the experimental measurement of the 2013 Claraboya group.

To test the second hypothesis, courses taught by the same teacher in 2012 and 2013 with 20 or more students enrolled in both years were selected. The averages of the resulting 21 units of analysis were compared using the non-parametric Mann-Whitney U test. All the samples showed a non-normal distribution.

Instruments

cprsscale. To assess the effectiveness of the intervention, an adaptation of the Service-Quality Survey (which USS has been performed since 2009) was made. The 2013 version consists of 99 thematic questions, that cover the following dimensions: Library; Food and Canteen; Administrative Staff and Faculty Attention; Infrastructure, Furniture and Equipment; Technologies and Computer Science; Security; Photocopies; Communications; Green and Recreation Areas; Sanitary facilities and toilets; Student Assistance Services/Scholarships and State benefits; uss University Life; Clinical Fields/Practice Centers and finally General Satisfaction. The entire survey achieved internal consistency of 0.991 (Cronbach Alpha) in 2013.

The library assessment section of the instrument contains between 7 and 9 items - depending of the year - with which students assess each service on a scale from 1 to 7. For the purposes of this study, an instrument was created called the Claraboya Project Research Survey (cprs) that was applied in an online version using Surveymonkey. This instrument consists of seven statements that were reformulations of the items created for the Service-Quality Survey, but in this case focused on the specific participant courses in which students were enrolled (González et al., 2014) (Table 1).

Table 1.

Service-Quality Survey Items and associated cprs items.

Service-Quality Survey Items	cprs
Text availability in library	Text availability in library for the selected course
Library hours of operation	Library hours of operation when you need material required for the course
Laptop loan service	Laptop loan service in library for this course
Digital library databases availability	Digital library databases available in library that you have used in this course
Search catalog	Material search catalog for the selected course
study room availability in library	Quiet study classroom availability in library (for the purposes of this course)
Library study room comfort	Library study room comfort (when required for the selected course)

Upon final measurement, the cprs survey attained internal consistency of 0.851 (Cronbach's Alpha). Both the pilot study and this main research used the Surveymonkey online survey system, while the cprs was applied by email between January 10 and January 23, 2014.

Variables

The variables considered in this research are library resources satisfaction for specific courses (as measured by the cprs), the library service dimension (extracted from the 2013 Service-Quality Survey) and the global average in the 21 courses selected for 2012 and 2013.

Data analysis

To assess the differences between the scores of students participating in the study and the general population, two random samples of the 2013 Service-Quality Survey were extracted. Each samlple contains the same number of students as the experimental group. Once this was done, the means of these three groups—experimental group, Sample 1 2013 Service-Quality (2013 S1) and Sample 2 Service-Quality 2013 (2013 S2)—were compared using Mann-Whitney U test to assess the effectiveness of the program.

Moreover, the Mann-Whitney U test was used to compare the averages of the 2012 and 2013 courses.

FINDINGS

Table 2 shows the descriptive scores for the variables under study, namely satisfaction with library service (Service-Quality 2013 samples 1 and 2) and satisfaction with library resources for the selected course (by cprs).

Table 2.

Satisfaction with library resources.

	Sample	Mean	sd	Lowest Score Value	Highest Score Value
Text availability in library	2013 S 1	4.81	1.735	1	7
	2013 S 2	5.06	1.627	1	7
	cprs	5.47	1.614	1	7
Library hours of operation	2013 S 1	5.73	1.483	1	7
	2013 S 2	5.82	1.486	1	7
	cprs	6.20	1.238	1	7
Laptop loan service	2013 S 1	5.05	1.673	1	7
	2013 S 2	5.20	1.666	1	7
	cprs	5.64	1.433	1	7
Availability of digital databases of library resources	2013 S 1	5.32	1.513	1	7
	2013 S 2	5.46	1.502	1	7
	cprs	5.75	1.445	1	7
Search catalog	2013 S 1	5.47	1.410	1	7
	2013 S 2	5.57	1.457	1	7
	cprs	5.74	1.466	1	7
Availability of library study rooms	2013 S 1	4.47	1.799	1	7
	2013 S 2	4.70	1.829	1	7
	cprs	4.39	1.885	1	7
Comfort of library study rooms	2013 S 1	5.12	1.789	1	7
	2013 S 2	5.13	1.769	1	7
	cprs	5.48	1.640	1	7

2013 S1: Random Sample 1 of Year 2013

2013 S2: Random Sample 2 of Year 2013

The averages obtained in the cprs are superior to the averages for 2013 S1 and 2013 S2 for six of the seven variables analyzed: Text availability in library (2013 S1= 4.81; 2013 S2= 5.06; cprs= 5.47), Library hours of operation (2013 S1= 5.73; 2013 S2= 5.82; cprs= 6.2), Laptop loan service (2013 S1= 5.05; 2013 S2= 5.2; cprs= 5.64), Availability of digital databases of library resources (2013 S1= 5.32; 2013 S2= 5.46; cprs= 5.75), Search catalog (2013 S1= 5.47; 2013 S2= 5.57; cprs= 5.74), availability of library study rooms (2013 S1= 4.47; 2013 S2= 4.7; cprs= 4.39), Comfort of library study rooms (2013 S1= 5.12; 2013 S2= 5.13 cprs= 5.48).

To ascertain statistical significance of these differences, the Mann-Whitney U test for independent variables was conducted. The Table 3 shows these results:

Table 3.

Results of Mann-Whitney U test for the independent samples.

	Samples	N	Ranks	Mann-Whitney U	Asymp. Sig. (2-tailed)
Text Availability in library	2013 S-1	365	313.25	47540	^*0.000
	cprs	340	395.68
	2013 S-2	357	321.17	50756	^*0.000
	cprs	340	378.22
Library hours of operation	2013 S-1	361	315.77	48652	^*0.000
	cprs	342	390.24
	2013 S-2	357	324.61	51981	^*0.000
	cprs	342	376.51
Laptop loan service	2013 S-1	326	300.94	43099	^*0.000
	cprs	317	343.65
	2013 S-2	324	301.45	45018.5	^*0.001
	cprs	325	348.48
Availability of digital databases of library resources	2013 S-1	332	296.53	43170	^*0.000
	cprs	319	356.67
	2013 S-2	336	307.98	46865	^*0.004
	cprs	319	349.09
Search catalog	2013 S-1	346	314.28	48711	^*0.001
	cprs	327	361.04
	2013 S-2	343	321.97	51438	0.054
	cprs	327	349.7
Availability of library study rooms	2013 S-1	362	353.11	59507.5	0.618
	cprs	336	345.61
	2013 S-2	357	326.51	54439	^*0.033
	cprs	336	330.52
Library study room comfort	2013 S-1	350	311.12	52994	^*0.007
	cprs	340	380.89
	2013 S-2	358	326.96	52789	^*0.008
	cprs	333	366.47

2013 S1: Random Sample 1 of Year 2013

2013 S2: Random Sample 2 of Year 2013

*

p<0.05

In the 2013 S1/cprs comparison, statistically significant differences were found in six of the seven variables studied. These difference all favored cprs as follows: Text availability in library (p=0.00), Library hours of operation (p=0.00), Laptop loan service (p=0.00), Availability of digital databases of library resources (p=0.00), Search catalog (p=0.001) and Library study room comfort (p=0.007). Availability of library study rooms showed no significant differences (p=0.618).

In the same way (See table above), the 2013 S2/ cprs comparison produced statistically significant differences favoring the following five cprs variables: Text availability in library (p=0.000), Library hours of operation (p=0.000), Laptop loan service (p=0.001), Digital library databases availability (p=0.004) and Library study rooms comfort (p=0.008). Search catalog (p=0.054) produced no significant differences, while Availability of library study rooms showed significant differences (p=0.033), but in the opposite direction.

To check the second hypothesis, courses taught by the same teacher in 2012 and 2013 with 20 students or more enrolled in both terms were selected. The averages of the resulting 21 subjects (courses) were compared using the non-parametric Mann-Whitney U test. The results are shown in Table 4. To safeguard faculty privacy, acronyms for their names are used.

Table 4.

Mann-Whitney U results for the 21 selected subjects.

Subject	Year	Means	Ranks	Mann-Whitney U	Asymp. Sig. (2-tailed)
ek	2012	4.78	79.97	2110.5	0.29
	2013	4.62	68.48
an	2012	4.53	50.23	342	0.000*
	2013	3.79	28.77
fvv	2012	3.83	21.33	212	0.086
	2013	4.09	24.9
fcq	2012	4.09	29.53	351	0.170
	2013	4	27.13
ep	2012	3.68	117.06	6601	0.123
	2013	3.76	126.09
nk	2012	3.94	82.23	1890.5	0.000*
	2013	3.66	62.17
fpp	2012	4.21	64.99	1665.5	0.387
	2013	4.18	63.33
ff	2012	3.49	81.24	3127.5	0.000*
	2013	4.03	122.64
ft	2012	3.63	58.17	849.5	0.014*
	2013	3.95	77.07
hpb	2012	3.62	30.89	440	0.164
	2013	3.74	34.7
pm	2012	5.14	60.55	528	0.000*
	2013	4.48	33.21
pif	2012	3.96	34.13	204	0.000*
	2013	3.29	21
qoe	2012	3.18	45.82	989.5	0.000*
	2013	3.69	62.33
pci	2012	3.41	51.64	1090.5	0.236
	2013	3.16	47.83
bt	2012	3.62	41.84	853.5	0.2
	2013	3.78	45.84
mpb	2012	4.04	30.84	425.5	0.14
	2013	4.22	35.54
pap	2012	3.39	66.61	1923.5	0.069
	2013	3.58	76.03
pgv	2012	4	29.5	360	0.065
	2013	3.92	27.35
foo	2012	4.21	96.13	3798	0.063
	2013	4.09	87.89
igt	2012	3.88	48.38	1127.5	0.000*
	2013	4.34	66.54
pifii	2012	4.05	44.41	683	0.080
	2013	3.87	38.53

P<0.005

As shown in Table 4, statistically significant differences in favor of the 2013 intervention were found in 4 of the 21 subjects evaluated, corresponding to 19% overall.

DISCUSSION AND CONCLUSION

This paper reports on the research is the middle stage of a pioneer project in Chile for optimizing of the use of library resources for academic purposes. A higher level of satisfaction was found in six of the indicators measured by the cprs scale. The study found that six of these indicators showed significant differences against 2013 S1 and five (5) indicators against 2013 S2. These results serve to provide substantial support for the validity of the first research hypothesis.

An analysis of this variation reveals that the most important indicator for the purposes of this research, i.e., “text availability,” shows statistically significant differences in both comparisons. Since the S1 and S2 samples were selected randomly, these results corroborate the data found in the preliminary approach by the researchers (González et al., 2014) and they underscore the importance of implementing this program at the national level.

As mentioned before, these results come in addition to the other five and four indicators with significant differences found respectively in the two comparisons. Of these, the increased levels of satisfaction with library hours of operation may indicate a positive feedback: as students visit the library more frequently, they become increasingly familiar with and better adapted to the library schedule.

In the view of the authors of this study, the fact that satisfaction with the search catalog did not improve after intervention is due to the fact that students in the experimental courses were required to use the library by faculty. Moreover, participating faculty often provided catalogue codes needed to find required reading.

The second research hypothesis must be rejected because no effect on academic performance of students enrolled in the selected courses was found against the either the 2012 or 2013 grade average, a finding that is not consistent with the findings reported by Gaona and Villuendas (2011), Soria, Fransen and Nakerud (2013), Han, Wong and Webb (2011) and Stone and Ramsden (2013).

It is important to highlight, however, that the aforementioned research studied the correlation between the students’ academic performance and the use of library resources, but not the impact of an intervention on the course's overall performance in two different years.

This is important, to the extent that the development of skills for the use of existing library resources benefits students over the course of their entire academic life. On the other hand, there are many factors that affect the overall performance of a course, including the number of students enrolled and the individual characteristics of the students.

In fact, of the courses participating in the comparison, five (5) are offered in the first year of the respective majors, and there is evidence in the USS that the overall performance of students admitted in 2013 was lower than those admitted on 2012 (Programa crear, 2013).

Finally, it should be pointed out that, in terms of faculty satisfaction and teaching quality, the effectiveness of programs aimed at behavioral changes through attitudinal changes has been sufficiently documented (Dierking and Fox, 2013; Khan and Pred, 2002; Kaplan, Cook and Steiger, 2006), but student satisfaction and organizational development has not been documented to nearly the same extent (Steintert et al., 2006).

An analysis by major or by student within a full-fledged experimental design or an analysis of the effectiveness of the intervention, comparing participating students vs. unexposed students in a given year, are potential areas for further research. In the same vein, the limitations of this research were related to the generalizability of the results and the nature of the research design. The pre-experimental design does not utilize random selection of participants in the Claraboya Seminary or in a random a control group, and therefore cannot asses the second hypothesis or allow the researchers to draw conclusions or assert causal connections between the variables. These issues should be considered limitations of this research and areas for future study.