Development of a tool for screening adverse food reactions and food allergy in Portuguese children

Abstract

Introduction and objectives

A standardised questionnaire may be an excellent tool for epidemiological studies aiming at screening children with suspected food allergies. Thus, the aim of the present study was to develop a screening questionnaire for assessing children with suspected food allergy and to analyse its reproducibility.

Materials and methods

A questionnaire of adverse food reactions was developed by literary review of similar questionnaires validated in other countries as well as less well defined, non-validated Portuguese questionnaires. Peer review of the questionnaire by a panel of specialists and subsequent exploratory analysis was carried out by applying the questionnaire in children with confirmed food allergy. Test–retest analysis was performed by giving a face-to-face questionnaire to 159 children with suspected adverse food reactions, aged between three and 11 years. Temporal stability using Spearman Rho correlation test and reproducibility was studied using Cohen's Kappa index.

Results

115 children confirmed adverse food reactions that occurred with one or more foods. Retest was given about three weeks after the test, to 50 of these children who were randomly selected. The questionnaire showed good temporal stability (Spearman correlation coefficient of 0.834), and good reproducibility (only two of the 27 items had a Kappa index <0.60).

Conclusions

This questionnaire showed good temporal stability and reproducibility. Its validation for screening children with suspected food allergy will allow a standardised approach to diagnosis and comparison of results obtained in different centres.

Introduction

Food allergy involves reactions to foods in which an immunological mechanism can be demonstrated and which includes IgE-mediated reactions.^1,2 However, other mechanisms may be implicated in adverse reactions to foods, namely non-toxic mechanisms such as intolerance.³ Clinical manifestations of food allergy are diverse, but most frequently include mucocutaneous reactions, although anaphylaxis may also occur,^4–6 with children and adolescents having a higher risk.⁷ A final diagnosis of food allergy requires confirmation by in vivo and in vitro tests (“probable” food allergy), as well as oral provocation procedures (gold standard; “confirmed” food allergy), in specific situations.⁸ However, clinical suspicion may be based on a clear history of reproducible specific food-associated symptoms with resolution upon eviction⁴ in association with predictive thresholds for food-specific IgE levels.⁸ This is helpful for the characterisation of the suspect food, clinical features and their severity, thereby allowing the appropriate clinical management of the situation, avoiding too restrictive or unnecessary diets.⁹ It may also allow clarification regarding which foods must be avoided to prevent serious reactions on contact with suspect foods.⁶ In accordance with international guidelines,^4,10 skin prick tests (SPT) and determination of food-specific IgE levels should be focused on specific foods, guided by clinical history and food-specific IgE levels.⁸

The prevalence of food allergy may be increasing, at least for certain foods,¹¹ and is highest in children and then declines with age.^6,12–14 If assessment of the prevalence of food allergy is based on self-report (“perceived/possible” food allergy), the values show a broad variation (3–47%), depending upon factors such as age, geographical area, operational definition of “food allergy”, “food hypersensitivity” or “adverse food reaction” used, and the questionnaire methodology applied (by telephone, self-administered, interviewer-driven, etc.).^10,15–18 In this context, a standardised, reliable, easily completed and available questionnaire may be an excellent tool for epidemiological studies focusing on detecting children with suspected food allergies. Although it is possible to translate questionnaires validated in other languages, it is not always feasible to adapt such questionnaires for use with populations that are culturally different, which may compromise the validity of the data obtained.¹⁹ Furthermore, most epidemiological studies provide scant information on the questionnaires that were used,^14–18,20 even those that used the EuroPrevall questionnaire.^21,22 In this context, a previously developed questionnaire in Brazil underwent preliminary studies by some members of our team, in terms of reproducibility and was shown to have a high number of questions with a good Kappa index (≥0.6).²³

In Portugal, a couple of studies carried out in children at Allergy clinics used non-validated questionnaires.^24,25 Thus, the objective of the present report was to analyse internal consistency and reproducibility of this questionnaire for the study of adverse food reactions and food allergy in Portuguese children.

Methods

Development of the questionnaire

The first step consisted in a bibliographical search for validated questionnaires for application in children with suspected food allergies. We found no validated questionnaires for the Portuguese population. Questionnaires used in prevalence studies in other countries did not mention validation data^26–28 and EuroPrevall studies did not include readily available questionnaires.

We then built a preliminary version of our questionnaire, based upon the most frequent clinical manifestations of food allergy reported in other studies^{10,27,29–31} and upon the questionnaire previously tested for reproducibility and temporal stability in Brazilian children.²³ Guidelines from Portuguese⁹ and European³² scientific societies were taken into account. Our questionnaire included more questions (deemed clinically relevant) than the one used in Brazil, and was a confirmatory, second phase questionnaire, as seen in other studies.^20,21,31

Clinical data that are crucial for the diagnosis^32,33 were reflected in our questionnaire: nature of the suspect food, time lag between ingestion and development of symptoms, whether ingestion of the suspect food induced similar symptoms on other occasions; other triggers such as physical exercise, and when the previous reaction took place³² were also analysed. Questions on reproducibility of symptoms were also included.

Pre-test, logical and content validity

A pilot study was then performed with the questionnaire in twenty-four children from the paediatric outpatient Allergy Clinic of Cova da Beira Hospital Centre, with clinical, laboratory and double-blind, placebo-controlled challenge-confirmed (DBPCFC) food allergy (14 males and 10 females; mean age of 7.4 years, SD ± 3.4). This stage aimed at assessing the applicability of the questionnaire, logic, comprehension and adequacy of the questions from the point of view of the target population. It also allowed us to perform a first evaluation of the consistency of the questions. The time it took parents to fill in the questionnaire was timed and doubts reported by children and their parents were recorded. The questionnaire was regarded as thorough by the children and parents but with simple and easily understandable questions and adequately timed (duration of 7–12 minutes).

With the feedback obtained with this pilot study, we made some changes, namely in the sequence of the questions. We also simplified some of the questions and defined the parameters better, in order to turn open questions into closed ones.

Once completed, the questionnaire was sent to a panel of three Allergists with experience in food allergy, and comments obtained were used for analysis of logical or apparent validity, and content validity of the questionnaire. Thus, content validity was confirmed with a panel of children and their parents, Allergy experts and literature review.

The final questionnaire was also read by a specialist in Portuguese linguistics, to correct language errors.

Description of the questionnaire

In addition to questions regarding demographics, the questionnaire included 17 questions (Table 1), in seven domains. Domain 1 focused on confirmation of the presence of a previous adverse reaction to food. Domain 2 aimed at identifying the food which triggered the adverse reaction. The questionnaire only proceeded on from this point in the event that at least one trigger food had been selected. Domain 3 focused on the characterisation of the first reaction to suspect food(s), and included questions 3–9. These questions were answered separately for each identified trigger food, and included evaluation of reported symptoms and their severity, definition of the reaction as immediate or delayed, identification of eventual triggers, how the food induced the reaction, and identification of new foods that might have been neglected. Domain 4 included questions 10 and 11 and focused on procedures followed in response to the reaction. Domain 5 involved questions about the stability of reactions upon a new contact. Questions included how long ago the last reaction had taken place (item 12), subsequent ingestion of the suspect food and eventual reactions (item 13), changes in severity or tolerance to the food on subsequent contact (item 14), and the number of new episodes (item 15). Finally, Domain 6 included questions 16 and 17, on personal and family history of allergy, as risk factors.

Table 1.

Assessment of reproducibility of a screening questionnaire for the detection of adverse food reactions and food allergy.

Questions		Relative concordance			Kappa (CI 95%)
		n		%
Domain 1 – Confirmation of allergic reaction
1	Does your child have any health problem or reaction with any food or drink?	50	50	100	1.0 (1.00–1.00)
Domain 2 – Identification of suspect food
2	Which food or drink triggers a reaction?	47	50	–	–
	Milk	49	50	98	0.91 (0.74–1.08)
	Egg	50	50	100	1.0 (1.00–1.00)
	Fish	49	50	98	0.94 (0.82–1.06)
	Soy bean	48	50	96	0.89 (0.74–1.04)
	Peanut	50	50	100	0.75 (0.57–0.92)
	Meat	50	50	100	0.83 (0.55–1.10)
	Fresh fruits	50	50	100	1.0 (1.00–1.00)
	Fresh legumes	48	50	96	0.6 (0.24–0.96)
	Other	49	50	98	0.92 (0.82–1.03)
Domain 3 – Characterisation of the first reaction and associated features
3	When your child had the reaction, was that the first time that he/she ate/drank that food?	43	50	86	0.77 (0.62–0.93)
4	How long after having eaten the food did the reaction occur?	49	50	98	0.88 (0.74–1.01)
5	What type of reaction did your child have after having eaten/drunk that food/drink?	–	–	–	–
	Respiratory symptoms	47	50	94	0.82 (0.62–1.02)
	Gastrointestinal symptoms	45	50	90	0.77 (0.57–0.96)
	Mucocutaneous symptoms	49	50	98	0.94 (0.83–1.05)
	Cardiovascular symptoms	48	50	96	0.48 (−0.14 to 1.10)
6	How was the reaction triggered by the food/drink?	50	50	100	1.0 (1.00–1.00)
7	If your child smells that food or it touches his/her skin, does he/she have any reaction?	48	50	96	0.85 (0.66–1.04)
8	Were factors such as physical exercise, ingestion of medication or any other associated with the reaction to the foods?	50	50	100	–
9	Did your child ever have itchy, swollen or tingling lips, mouth or throat after having eaten any other food?	50	50	100	1.0 (1.00–1.00)
Domain 4 – Procedure followed after reaction
10	Was your child taken to hospital when he/she had the reaction to food/drink?	47	50	94	0.83 (0.68–0.99)
11	Did your child have to be given any medication when he/she had the reaction	46	50	92	0.87 (0.74–0.99)
Domain 5 – Stability of reaction upon new contact
12	How long ago did the last reaction take place?	40	50	80	0.56 (0.36–0.76)
13	After the first reaction, did your child eat the same suspect food again? Please describe the reaction, in the event that there was one.	49	50	98	0.88 (0.65–1.11)
14	If your child ate the food more than once, have reactions to it changed in severity over time, to the same food?	47	50	94	0.85 (0.69–1.01)
15	In total, how many episodes of adverse reactions to the same food did your child have?	47	50	94	0.89 (0.77–1.01)
Domain 6 – Risk factors
16	Does your child have any other allergies?	40	50	80	0.64 (0.46–0.82)
17	Does anyone in the child's family have any allergies?	43	50	86	0.79 (0.65–0.93)

CI: confidence interval.

Initial study

A shortened form of the final questionnaire consisting only of questions on demographics and two questions (questions 1 and 2) regarding the previous occurrence of food reactions and identification of the suspect foods was sent to all children from public Pre-Schools and Primary Schools of the region of Beira Interior (Belmonte, Covilhã and Fundão), Portugal. Of the 2474 children whose parents replied to this simplified questionnaire (Questionnaire 1), 176 children were detected as having possible, self-reported food allergy. The parents of 159 of these children then replied to the complete questionnaire (Questionnaire 2) (Supplementary materials 1 and 2) and 115 confirmed a suspicion of adverse reaction to at least one food (Fig. 1).

Figure 1.

Flow chart of the study design for analysis of reproducibility and temporal stability in Portuguese schoolchildren.

Test–retest

Reliability of the screening questionnaire was studied for intra-observer reliability (test–retest). From the group of 115 children with suspected food allergy, we randomly selected 50 children for test–retest analysis, who again had to fill in the same questionnaire. The retest was applied by the same researcher, 1–12 weeks (mean five, median three weeks) after the first application of the questionnaire. For the calculation of the retest results, the food associated with the most severe reaction was selected, if there were reactions to more than one food.

Statistical analysis

Spearman's Rho correlation coefficient (with a level of significance of p < 0.01) was used to analyse temporal stability, regarding values >0.70 in the absolute value as a strong correlation. Analysis of concordance and reproducibility of the questionnaire was performed using Cohen's Kappa test for each question. Cohen's Kappa results and their 95% confidence intervals were interpreted for levels of concordance: 0.00 – poor; 0.01–0.20 – slight; 0.21–0.40 – fair; 0.41–0.60 – moderate; 0.61–0.80 – substantial; >0.80 – almost perfect.³⁴ Data were analysed using the Software Package for Social Sciences (SPSS) version 20.0^®. A p value of less than 0.05 was regarded as significant with all tests.

Ethical considerations

This study was approved by the Ethics Committees of the Faculty of Health Sciences of the University of Beira Interior and Cova da Beira Hospital Centre. All parents/guardians signed a written informed consent. The questionnaire was approved by the General Board for Curricular Innovation and Development (Direcção-Geral de Inovação e de Desenvolvimento Curricular – DGIDC).

Results

Demographic data

The 50 children who were randomly selected in order to perform the test–retest had a mean age of 8.7 ± 2.4 years and 58% were male. Most of the children (69%) lived in an urban environment, 37 (74%) were atopic and 31 (62%) had family history of atopic diseases. The most frequently reported reactions to foods occurred with fresh fruits (n = 16); fish (n = 9); and eggs (n = 8).

Temporal stability

Analysis of temporal stability using the sum of the most relevant test/re-test replies to questions 4, 5, 10, 11 (which allowed characterisation of clinical manifestations and their severity) and 16, showed a Spearman's Rho correlation coefficient of 0.834.

Analysis of concordance and reproducibility of the questionnaire

Kappa Cohen's test was used for analysis of reproducibility (intra-observer reliability) of the test–retest for each question (Table 1). For the 17 questions, 27 analyses of concordance were performed. One of the questions (9) showed perfect concordance for one category, nine items showed an almost perfect concordance (Kappa between 0.81–1), fifteen items had a good or very good Kappa value (>0.6) and two (items 5 and 12) showed a moderate Kappa value (0.48 and 0.56, respectively).

The item related to identification of suspect food (item 2) showed a good or very good Kappa value (≥0.75) for all foods, except for fresh legumes (0.60).

Although most parents did not know whether the first reaction coincided or not with the first ingestion (item 3), we found a good concordance in replies to this question as well as in terms of the number of episodes (item 15) (0.77 and 0.89, respectively).

In terms of questions aiming at characterising the reactions, question 4, whose objective was to discriminate between early and delayed reactions, showed a relative concordance of 98% and a Kappa value of 0.88. Questions related to clinical features of the reactions were grouped into mucocutaneous, respiratory, digestive or cardiovascular manifestations (item 5) and showed good temporal consistency (≥0.77) except for cardiovascular symptoms (Kappa of 0.48). Item 6 aimed at identifying the contact route with the food (inhalation, cutaneous contact or ingestion), and showed a concordance of 100% and a Kappa value of 1. Item 13, which aimed at assessing the reproducibility of the reaction, showed a 98% concordance and a Kappa value of 0.88.

Questions regarding food episode-associated hospital visits (item 10) and medication prescribed (item 11) also showed an almost perfect concordance value (0.83 and 0.87, respectively), and aimed at assessing severity of the reaction.

In terms of the question regarding triggers/co-triggers such as physical exercise or drugs (item 8) all answers were “I do not know”.

Questions 16 and 17 aimed at assessing possible risk factors for the reactions and showed a Kappa value of 0.64 for other allergic symptoms in the children, and of 0.79 for allergic disease in children's families.

Discussion

In the present study, we developed and studied feasibility, reliability (test–retest reproducibility), face and content validity of a questionnaire for screening adverse food reactions in children. The questionnaire was shown to be simple to apply, to have good temporal stability, as well as good or very good reproducibility of most questions,^19,35 thereby suggesting its adequacy for screening children with adverse food reactions.

The questionnaire was called “Questionnaire 2 (Q2)” and was intended to be applied to children with suspected food allergies who had reported food-related symptoms in a preliminary, two question-long questionnaire (“Q1”). This means that we intended to follow a two-step approach for epidemiological studies, also used in EuroPrevall²¹ and other studies.^20,31 Nevertheless, our Q2 screening questionnaire was designed to be applied without the need for the initial Q1 questionnaire. Our full questionnaire involves a semi-structured interview, which makes it more powerful than a simple checklist, and its construction followed a theoretical model, based upon a robust theoretical review of the literature. In addition, it also continued the work of a previously published questionnaire in Brazil,²³ with the addition of questions deemed to be clinically relevant.

With its two initial questions, this questionnaire allows the identification of children with adverse food reactions who may be at risk of developing further reactions. Question 1 aimed at detecting the presence of self-reported adverse reactions to foods and question 2 aimed at identifying the trigger foods. Although these questions are not specific enough to exclude non-allergy related situations, they are potentially highly sensitive, thereby allowing the inclusion of all cases of adverse food reactions which will be more specifically studied by the subsequent items in the questionnaire. In fact, most previously used questionnaires,^18,20,22 namely those used in EuroPrevall²¹ and the Brazilian studies²³ included two similar initial questions.

As far as reliability is concerned, our questionnaire was assessed for intra-observer stability. In this context, temporal stability demonstrated high and significant Spearman Rho correlation values (r = 0.834; p < 0.01), thereby suggesting that it has good enough temporal stability (value close to 1) for it to be applied in our target population.

Kappa index has been studied in various studies in children, to assess concordance among various observers³⁴ but also for analysis of intra-observer reliability.^36,37 Our study used Kappa index for assessment of the reproducibility of the different questions and eventually proposing their modification or exclusion. All domains showed high levels of concordance, with only two questions in Domain 3 (characterisation of the first reaction) showing a fair concordance Kappa value (question 4, for cardiovascular symptoms; and question 12, regarding how long ago had the previous reaction taken place). Thus, all the remaining 15 questions demonstrated a substantial concordance (Kappa value > 0.6), and nine questions showed an almost perfect concordance (Kappa between 0.81 and 1). It is difficult to compare these parameters since we could not find any published validation of similar screening questionnaires for food allergies in the literature, apart from the Brazilian study.²³ The Brazilian study showed almost perfect concordance in two questions (“Has your child had any reaction when this food only touched his/her skin?” and “Did your child feel itching, swelling, or numbness in his/her mouth after eating fruit or raw vegetable?”) and this was in agreement with our study.

Our study showed that there was a good concordance in the identification of suspect foods except for fresh legumes, which may be due to the fact that in our population there is a low prevalence of adverse reactions to fresh legumes,^24,25,38 as has also been reported in other countries.^20,22 In fact, allergy to legumes is not regarded as a common food allergy.¹¹ Furthermore, whenever reactions to fresh legumes were reported in our study, these were mild or mostly based upon a single episode of symptoms, which may make it more difficult for the reaction to be remembered, as was demonstrated in a previous study on cow's milk allergy/intolerance.³⁹

In terms of characterisation of the first reaction (Domain 3), although most parents did not know whether it coincided with the first ingestion of the suspect food (question 3), we still found a good concordance in replies to this question (0.77), with values that were higher than those reported in the Brazilian questionnaire (0.55).²³

Still in Domain 3, answers concerning clinical characterisation of the reaction showed substantial concordance (≥0.77) except for cardiovascular symptoms (0.48), which may have been due to the fact that these symptoms were very seldomly reported in our sample. In terms of triggers such as physical exercise or medication (question 8), all answers were “I do not know” which, in spite of full concordance, did not allow us to further study this aspect. Thus, although these triggers are regarded as relevant in other studies,²⁹ we could not demonstrate that in our patients.

In Domain 4 (procedure followed after reaction), questions regarding going to hospital as well as medication given, are useful for assessing the severity of the reaction, and showed an almost perfect level of concordance (0.83 and 0.87, respectively), thereby suggesting that there may be a sharper memory of issues of greater frequency or severity.³⁹ These high levels of concordance were higher than those found in Brazilian children, in similar questions,²³ possibly because the reactions involved were of lower magnitude.

In Domain 5 (stability of reactions upon new contact), concordance was also almost perfect (Kappa value of 0.88) for answers about the number of adverse food reaction episodes. This is possibly because we narrowed parent choice by allowing them to choose from a range of numbers of episode rather than by asking them to fill in a single absolute number of episodes.

Still in Domain 5, in terms of time elapsed since the previous reaction (question 12), we only found a moderate concordance value (Kappa of 0.56), which was, nevertheless, higher than that observed in the Brazilian children (Kappa of 0.28).²³ Questions that deal with elapsed time and age of occurrence may show a low degree of concordance, due to memory bias. In fact, memory of the previous episode may depend upon the severity of the event and time elapsed ever since.^26,39 Another reason may be that these reactions may include both food-induced allergies as well as non-allergic adverse food reactions with different degrees of severity. In addition, after the initial interview, parents may have attempted to remember the episodes in order to give a more precise reply on retest. Finally, children may have developed tolerance to foods between test and retest as was demonstrated in similar studies in children.^39,40

In Domain 6 (assessment of risk factors), questions 16 and 17, which aimed at assessing risk factors, only showed good Kappa values (0.64 for other allergic symptoms, and 0.79 for history of allergies in the family). This may be partially explained by the fact the parents of some children may have become more aware that their children had an allergic disease or had a confirmed diagnosis of allergic disease between test and retest, as happened in some cases. In addition, low temporal stability may also have been due to a memory bias, as has been previously reported.^38,40

Our study has some limitations. Firstly, content validity was only assessed in twenty-four children with food allergies, which is a relatively low number of individuals. Nevertheless, a similar analysis was also performed in 50 children with suspected food allergies. Thus, we believe that the sample size used for this study may have been adequate according to item analysis recommendations⁴¹ Secondly, although most questions had very good consistency and reproducibility, some of the questions had low consistency and will need to be reformulated or removed by multiplex reduction analysis. Thirdly, reliability analysis of the questionnaire using alfa Cronbach's test was not carried out because the intrinsic nature of the items of the instrument did not allow it.³⁷ Fourthly, the time lag between test–retest was moderately outside the ideal time range (two weeks), although the median in our study was only three weeks, and the mean was five weeks, which is similar to other reports. Nevertheless, this time lag may have been associated with memory bias. In addition, our questionnaire will subsequently have to be validated regarding its sensitivity and specificity by comparing it to the gold standard technique of DBPCFC. Although our 24 children with DBPCFC-confirmed food allergy filled in the questionnaire, we did not perform such an analysis at this stage. Finally, although our questionnaire is simple and was used for screening children for adverse food reactions and food allergies, it may still be further improved by introducing Likert-type questions to increase its robustness for detecting specific food allergy profiles. Further studies are warranted, namely in collaboration with the Brazilian questionnaire.²³

Nevertheless, our questionnaire has the advantage of being simple to use, easily available, and follows criteria for diagnosis of food allergy.^4,10,21 This is particularly relevant in terms of its application for a rapid and simple diagnosis of possible food allergy in children, a population in which such a problem is highly relevant.^7,11

Thus, we believe our questionnaire may subsequently be applied in all Portuguese-speaking countries (250 million people) and has features that allow its application worldwide, given its availability, ease of application, global consistency and temporal stability.

Conclusions

We have developed the first questionnaire in Portugal for screening food allergies in children, which has very good internal consistency and reproducibility and can be easily applied. It holds good potential as a useful screening test for food allergies.

Funding

This study did not have any external funding. Costs regarding questionnaires and laboratory tests were paid for by internal investigator funds (Faculty of Health sciences and Cova da Beira Hospital).

Conflicts of interest

The authors declare that they have no conflicts of interest.

Acknowledgments

We would like to thank the Departments of Paediatrics of Cova da Beira Hospital and all nursing staff that helped with skin prick tests.