Introduction
Primary health care (PHC) is an area of special importance within the National Health System, as is the case in Spain, since the need to improve the viability, and therefore, the efficiency of the system involves having a PHC setting available that fulfils its function as the entrance portal, with a high problem solving and core capacity, that modulates the use of the rest of the system.1
Starfield defined a theoretical model to which PHC had to comply with, that included some basic features and other associated ones2 (Table 1). One of the main characteristics of this model is that the focus of attention of PHC is directed at the individual, and not at health problems.
Spain began PHC reform (PHCR) in 1984, that would enable profound changes to be made in the primary care field following the principles defined in the Alma Ata conference in 1977. Among the reforms that were established by the PHCR included the introduction of medical history which was non-existent in PHC then, the creation of primary care teams (PCT) instead of basically individual work, as well as the lengthening the clinic times for the population. PHCR assigned a very important role to PHC within the health system, which it has taken since then, with better evaluation of its professionals and giving them a greater role in the health system as a whole.3,4 This "new PHC" was adapted to the model defined by Starfield.
After these significant and drastic changes in the health system, the need arose to evaluate the service provision process and the impact of its intervention. A positive impact of PHCR has been observed in the system overall in different studies, and improvements in several areas have been highlighted, such as quality of the drug prescription, user satisfaction, the resolution capacity and even, an improvement in the health of the population.3,4 However, in recent evaluations of PHC processes, the lack of information to determine aspects such as accessibility, longitudinality and continuity of care has been highlighted.5,6 There is a need to increase and improve information so that centres can be compared as regards their care capacity, their structure, practice or processes, as well as the product or results of the care.2,7,8
Starfield and Shi, of the Johns Hopkins University, in the United States, have designed a set of tools to evaluate PHC, PCAT (Primary Care Assessment Tools),9 which consists of questionnaires that allow evaluating to what degree PHC fulfils these characteristics. The difference between them is the population to which they have to apply: a) the PHC user population, b) the managers of the PHC resources, and c) PHC providers. The tool directed at PHC users may be suitable for use in a Spanish context, where there is a National Health System and where the potential user of the public PHC system is population set, and because the theoretical model for which it has been designed is similar to the Spanish one in its basic characteristics.
This questionnaire collects information on the experience of users as regards each of the characteristics of PHC (Table 1) from the identity of a provider. It also includes other sections, such as the type of health care cover, the perceived state of health and sociodemographic characteristics.10 There are 4 versions available, 2 for adults (PCAT-AE; adult edition) and 2 for children (PCAT-CE; child edition), a long version (128 and 121 items, respectively) and a short one (59 and 68 items) in each case. The anticipated length of the original short versions is 20-30 min. The response categories are mainly 4 category Likert type scales. The measurement of the degree of compliance of each characteristic is obtained by calculating the average scores of the responses. The dimensions of the original versions of PCAT have an adequate internal consistency.9-11 Given that PCAT has been designed for use in the United States, a trans-cultural adaptation is required,12,13 because the language and the health and social environment of that country has significant differences compared to Spain.
The carrying out of the Third Catalan-Fifth Barcelona Health Survey in 2006 offered the opportunity to include questions to evaluate PHC, but with a limitation as regards the number of items, which had to be reduced. The objective of the study was to reduce and transculturally adapt the short version of the PCAT questionnaire for users (PCAT consumer client version) with the aim of obtaining an abbreviated version of the original tool, in Catalan and Castilian, conceptually similar to the original in English, culturally appropriate and viable for use in the Spanish population, and which would be useful for inclusion in health population surveys.
Methods
Selection of Questions
The research team, together with the people responsible for the 2006 Catalan Health Survey and the 2006 Barcelona Health Survey and the subsequent communication with the author of the original tool, decided to select a reduced number of questions from the PCAT tool for users (adult and 18 year olds versions), a condition required to enable inclusion in population health surveys.
Thus, it was decided to use PCAT-CE as the source of the questions for the questionnaire for the population up to 14 years old and the PCAT-AE for the population ≥15 years. These 2 age groups are similar to the 2 age groups that a specific health survey questionnaire is directed, but at the same time they are the 2 age groups around which the PHC is organised, as less than 14 year olds are seen in paediatrics and the rest in the general medicine clinics.
The following points were taken into account in the selection of the questions: a) objectives and contrast with the group of questions of the health survey, given that it had questions in other sections that could refer to aspects measured by PCAT for users; b) prioritisation of the PHC attributes measured, due to the time limit of the interview that would be dedicated to this section; and c) opinion of the author of the theoretical model and the original questionnaire in English (BS) on the process followed.
As with the original version, both questionnaires were administered by an interviewer, the questionnaires for children would be answered by an indirect informant, and the adult ones by the person him/herself.
Transcultural Adaptation of the Selected Questions
The adaptation of the users PCAT versions in Castilian and Catalan were carried out following the method of direct and back-translation recommended in the specialised scientific literature.12,13 Firstly, 2 translators, both with Castilian and Catalan as their mother tongue, and with sufficient command of English, both carried out independent translations into each language. The translators scored the degree of difficulty for each item that they found and the degree of conceptual equivalence with the original, on a scale of 0-10 (0 = no difficulty; maximum equivalence; 10 = maximum difficulty; minimum equivalence). The items and the response options proposed by the translators were systematically reviewed in a meeting with the research team and the translators, where they commented on the difficulties, and the most suitable alternative was selected as the first consensus versions of the tool. In the following stage, a back-translation was carried out by a translator whose native language was English and with a sufficient command of Castilian and Catalan. The differences between the back-translated version and the original version were evaluated in another meeting between the researchers and the translators, to propose alternative solutions. As a result of translation process a pre-test version of the questionnaires was obtained and a report was reviewed by the author (BS) of the original tool.
The clarity, acceptability and familiarity of content of the two pre-test versions of the questionnaires were evaluated using cognitive interviews with people of different ages, sex, educational levels, and language (Castilian and Catalan). Firstly, the questionnaire was administered and the time required for this part of the interview was monitored. The general opinion on the items of the persons interviewed were recorded, by means of 4 questions on difficulty, understanding and interest of the questions and suitability of the questionnaire language, which were answered using a Likert type scale of 5 options (1 = none; 2 = a little; 3 = moderately; 4 = a lot; 5 = very).
Lastly, paraphrasing and probing methods14-15 were used to assess the understanding of the questions by the person interviewed, by means of statements made by the same person to reformulate the question. The interviewers recorded their opinions, incidences and difficulties, as well as suggestions for changes, and these were analysed jointly with the research team, all of which led to making changes to the questionnaire. At the end of this phase the final pre-test versions were obtained, for children (≤14 years) and adults (≥15 years), in Castilian and Catalan.
Results
Selection of Questions
Firstly, it was decided only to evaluate the essential attributes. Fifteen questions were chosen for the adult version and 24 for the children less than 15 years old version, which enabled a PHC provider to be identified and to collect information on the dimensions of first contact, continuity, extent of services, and coordination. The contents of these dimensions and the number of questions for each one compared to their original versions are shown in Table 2. There are a total of 12 questions in the PCAT-AE and 20 in PCAT-CE, which correspond to the original questionnaire, while 3 and 4 questions, respectively, are additional and respond to the needs of the health survey.
In the extent or totality of services, where the availability and provision of certain services, basically preventive were measured, it was considered, on the one hand, that the Spanish health system is public, with a very homogenous service provision, and on the other, that the health questionnaire tool for adults already included preventive activities which are appropriate to PHC (anti-smoking advice, regular blood pressure measurements, influenza vaccination). Only the mental health services item of the PCAT-AE was selected in this dimension. On the other hand, the number of items selected from the child questionnaire (PCAT-CE) was greater, given that they were not included in other parts of the survey.
Adaptation of the Questionnaire
No conceptual differences were encountered during the translations. Few difficulties in the style of writing were found. One of them was the English acronym "PCP" (primary care provider), which is used in the formulation of the majority of questions to refer to the person or place identified as the PHC provider. In this case, a short equivalent phrase had to be agreed upon (Table 3).
During the translation and the consensus meetings other doubts arose which were recorded for assessment during the pre-test. These included understanding the term "appointment slip" or "authorisation," or "mental health services" and "sexual health," particularly for younger users. There were few specific observations that were different in Castilian and Catalan, and they were of minor significance and easy to resolve.
Later, cognitive interviews were carried out on 19 people of different ages, sex, education level, and language (Table 4). In the child version of the questionnaire, the 5 people interviewed were mothers, and in the adult version, the distribution by sex was the same. The stratum for low education level was only covered in the version for adults over 40 years. The young adult population (including mothers who answered the child version) had mainly secondary and university education. Finally, the Castilian and Catalan versions were used in a similar proportion.
Observations on the general aspects of the questionnaire and questions specific to the questions asked were recorded during the interview. All the observations were pertinent for the 2 languages, Castilian and Catalan.
As a result of the adaptation process, 6 minor modifications and the modification of 1 item (A8) have been proposed among the 24 items translated for the children´s version (CE), and 4 minor modifications among the 15 items translated for the adult population (AE). All the minor modifications consisted of grammatical points to make it easier to formulate and understand the questions.
The proposed modification to item AB was not due to a translation problem, but to the difficulty of the people interviewed to respond, without a time limit, to the question "How many times in total have you been there (their doctor or PHC centre)?," as the question was set in the original questionnaire. The modification was the introduction of a reference time period of one year. This implied that it became an indicator of the use of the services and not of loyalty to a service.
As regards the viability of the use of the questionnaire, it required, on average 5.2 min to complete the adult version (AE) and 5.6 min for the children´s version (CE) (Table 5). The degree of difficulty of the questionnaire, according to the opinion of the people interviewed, was very low (mean, 1.3), and was less so for the mothers of children (mean, 1.2) and higher for those who responded to the adult version (mean, 1.4). The level of understanding of the questions was practically maximum for the child version (mean, 4.4) answered by the mothers, and was also high for the adult version (mean, 3.9). The level of interest of the questions was, in general, moderate (mean, 3.3), with significant variability in the version for adults (range, 1-5). The suitability of the language was good for the child version and moderate for those who responded to the adult version.
TABLE 5 Feasibility of the PCAT Questionnaire: Administration Time and Opinions of the People Interviewed on the Difficulty, Understanding, Interest, and Suitability of the Language of the Questionnaire. Averages*
Discussion
A group of variables have been obtained to evaluate PHC from the perspective of the population to whom the service is offered, from the ESCA-ESB and the PCAT questionnaire for the adult (PCAT-AE) and child (PCAT-CE, for indirect informants) population, based on a theoretical model that establishes the essential characteristics that PHC has to fulfil. This work presents the process of selecting the items and the cultural adaptation of the questionnaires.
It is hoped that these tools will be useful for inclusion in health surveys in the Spain. For this reason, the least number of items possible was selected that would be capable of collecting the dimensions of the theoretical model that were considered priorities, taking into account the context of the Spanish National Health System and the background information that is already collected in these health surveys. The items thus selected would allow the degree to which PHC fulfils these essential attributes to be evaluated, such as, being first contact with the health services for the user, maintaining continuity of care, coordinating health care and carrying out and providing services considered necessary in this care setting. Additionally, its cultural competence will also be able to be evaluated. It must be pointed out that these versions do not collect all the dimensions included in the original tool, and have fewer items. The dimensions not included are, family and community orientation, in both questionnaires, and the services provided by the version for adults. This last dimension is normally included in health surveys, but by using other types of questions that record the statement that the person may have received certain preventive practices. The reduction in items affected the adult version more than the child one, given that the dimensions are made up of 1-4 items. In later evaluations of the tool, by means of a pilot study, the validity and reliability of the dimensions will be analysed, using the means calculated from the items that make up each dimensions, as was used in the original questionnaire, to compare its operation with the version with a higher number of items. Despite the limitation of reliability which this reduced number of items may assume, it is hoped to have a tool with acceptable measuring properties and a number of questions that will ensure its viability in long term studies.
As a result of the pre-test, the questionnaire, in general, was not difficult, was understandable, sufficiently interesting and with a good adaptation of the language. The pre-test was useful for introducing some modifications that were expected to improve the consistency of the questions in the questionnaire.
A systematic process of translation and adaptation was carried out, which is line with international standards and enables problems of linguistic and semantic differences to resolved, as well as familiarity with terms and concepts in a cultural context; in this case the organisation of the health system.
The consensus meeting between the researchers and the translators, the cognitive tests with people from the target population and the review of the process by the author of the theoretical model and the tool, ensure that versions obtained will include words and questions understood by the end-users of the questionnaire, and helped in validating its contents by measuring the characteristics that PHC is supposed to fulfil.
PHC is currently being evaluated using the information available, a fact that has led to deficiencies being detected for evaluating areas such as continuity and coordination of care. The PCAT user versions with the questions in Castilian and Catalan will provide another tool for evaluating PHC and will include new aspects in the evaluation, in the context of the Spanish and Catalonian health system. Also, by not just including the provisions available in the evaluation, but also the provisions perceived by potential users, it must be able to detect areas for improvement, such as, accessibility to services. The perspective of the population is essential in the evaluation of all fields of the health system, but in the case of PHC it acquires even more importance due to its coverage, since 70% of the population go to primary care clinics in one year.16
The main limitation of this process is that a translation and adaptation of the complete original tool could not be carried out, which implies that the evaluation that was made on PHC did not cover the theoretical model in its entirety. Also, by not being able to include complete dimensions, the measurement obtained will not have the same validity as the original tool. Even so, it is considered an important step to have made this reduced version available, since it could be included in the principal health surveys that are carried out in Spain in several environments (municipalities, autonomous communities, national). The questionnaires presented here will be included in ESCA 2006 and in the 2006 Barcelona Health Survey. These are the first of a group that it is hoped will be available in the medium term, given the recognised need to have tools available to evaluate the fulfilment of the dimensions of PHC from the perspective of those for whom the services are provided, the managers and providers of PHC. A new project has been started recently, financed by the Health Research Fund, which will enable the Castilian and Catalan PCAT versions to be adapted for PHC providers.
Acknowledgement
We would like to thank the management of the Third Catalonian Health Survey-Fifth Barcelona Health Survey of 2006, for being the driving force for carrying out this work.
Spanish version available www.doyma.es/173.909
A commentary follow this article (page 402)
What Is Known About the Subject
* A long process of reform of primary health care (PHC) has ended in Spain.
* This has brought about important changes in the provision of primary care services.
* There are areas of evaluation for which there are no evaluation tools for application in our country.
What This Study Contributes
* The methodology followed to obtain a group of items suitable for evaluating some essential attributes of PHC from the perspective of the population.
* These items can be easily included into wider health surveys that are carried out systematically.
The Epidemiology and Public Health Centres Network (RCESP) (C03/09) has financed part of this work.
Correspondence: Dr. M.I. Pasarín.
Agencia de Salut Pública de Barcelona.
Pza. Lesseps, 1. 08023 Barcelona. España.
E-mail: mpasarin@aspb.es
Manuscript received July 10, 2006.
Manuscript accepted for publication December 18, 2006.
