Satisfaction with a performance appraisal system in the Portuguese public sector: The importance of perceptions of justice and accuracy

Abstract

The present work aims at: (1) assessing whether the Portuguese public sector employees are (or are not) satisfied with their appraisal system; (2) dissecting the relationship between perceptions of justice and accuracy of performance appraisal and satisfaction with that appraisal. In order to attain the aforementioned objectives, we decided to conduct an exploratory empirical study, cross-cutting in nature, focusing on a population of 2247 individuals, from which we obtained 334 responses (n = 334). After statistical analysis of the data collected, with a descriptive and an inferential component, results point towards the fact that employees who have a better perception of justice and accuracy of performance appraisal tend to reveal greater satisfaction with it. However, in general terms, the elements of the sample considered the appraisal process as being unjust, inaccurate and were not satisfied with it.

1

Introduction

The current Integrated System of Performance Management and Appraisal in the Public Sector, known as SIADAP, was established by Law no. 66-B/2007, of December 28. Very briefly, we may consider that this system has introduced individual appraisal for all levels (services, managers, workers), in a logic of management by objectives, and forms the basis of career progression and awards. To that end, it integrates three subsystems, which operate in an integrated and consistent manner with the objectives set out within the scope of the planning system; objectives of the service management cycle; objectives set out in the mission statement of senior managers; and objectives set out to other managers and employees, namely:

• •

  Subsystem of Performance Appraisal of Public Sector Services;

• •

  Subsystem of Performance Appraisal of Public Sector Leaders;

• •

  Subsystem of Performance Appraisal of Public Sector Employees.

Moura (2012) posits that performance appraisal of the public sector, of its leaders and its employees is a critical tool for the attainment of a new culture for the Portuguese public sector, considering that only a demanding integrated appraisal model will detect imbalances and shortfalls of Portuguese public organisations, gear them towards achieving their goals and, simultaneously, engage all those who carry out their professional activity there.

Furthermore, this appraisal system is one of the most notable reforms in the Portuguese public sector of managerial inspiration, as it reveals the introduction of management by objectives in this sector, which can be considered as a switch matrix, by linking the goals to the results obtained, seeking to introduce a culture of merit and results appraisal, ceasing, thus, automatic and regular progressions based on seniority (cf. Madureira & Ferraz, 2010).

However, it should be noted that literature has revealed that reactions to performance appraisal and to the appraisal process have a significant impact on the overall effectiveness of the appraisal systems, as well as on their acceptance and validity (cf. Cardy & Dobbins, 1994; Cawley, Keeping, & Levy, 1998; Keeping & Levy, 2000; Kuvaas, 2011; Levy & Williams, 2004; Murphy & Cleveland, 1995). For example, Murphy and Cleveland (1995) establish that reactions are almost always relevant and an unfavourable reaction can condemn even the most carefully designed performance appraisal system.

This is the context of the present research study on the satisfaction of appraisees with a system of performance appraisal in the public sector, more exactly SIADAP. This study emerges after a gap in Portuguese literature has been detected: to the best of our knowledge, there is no scientific work focusing on this issue in the way that we will present hereafter.

But what is, specifically, the importance of a study that addresses satisfaction with a performance appraisal system, both in the public and in the private sector? It is very simple: several studies have concluded that the extent of the effect of performance appraisal on employee's attitudes and behaviours, such as professional performance, productivity, motivation or organisational commitment, derives from their satisfaction with that appraisal (cf. Cawley et al., 1998; Ilgen, Fisher, & Taylor, 1979; Kuvaas, 2006).

Therefore, the purpose of this study is to assess whether Portuguese public sector employees, whose performance appraisal derives from the application of SIADAP, are (or are not) satisfied with this appraisal system. To this end, we chose to carry out an exploratory empirical study, cross-cutting in nature, focusing on a population of 2247 individuals, from which we obtained 334 responses (n = 334), and a mix between ‘descriptive research’ and ‘correlational research’, considering the perspective followed by Carmo and Ferreira (2008).

Furthermore, and on the basis of Cook and Crossman's (2004) view, in literature the causes of satisfaction (or dissatisfaction) with performance appraisal have been attributed to a number of different reasons, hence it is essential to know them, given the influence of this satisfaction on other organisational variables, as referred above. Only by knowing them may organisations increase this kind of satisfaction.

In this work we will not analyse, obviously, all possible causes. The goal is to dissect the relationship between perceptions of justice and accuracy of performance appraisal and the satisfaction felt with that appraisal in the context of SIADAP, insofar as a ‘good’ performance appraisal system is generally defined as the one that provides accurate, complete and just appraisals of each individual's performance (cf. Murphy & Cleveland, 1995).

After defining of the goals of our research study, we may assert that the present work is organised around four main moments: theoretical construction and development; methodological framework of the empirical study; analysis and discussion of results; main conclusions attained.

2

Theoretical underpinnings

Literature has revealed that the effects of performance appraisal on employees’ attitudes and behaviours may derive from their satisfaction with that appraisal. For example, Kuvaas (2006) reported that satisfaction with performance appraisal revealed by the appraisees determines their performance at work. Moreover, other studies have shown that employee's satisfaction with performance appraisal affects other variables, such as productivity, motivation and organisational commitment (cf. Cawley et al., 1998; Ilgen et al., 1979).

The importance of satisfaction with performance appraisal on the part of employees and its preponderance regarding other organisational variables is established from the outset, as was seen in the previous paragraph. It is, thus, necessary to dissect the causes of this satisfaction (or dissatisfaction).

One of the directions followed by research studies in this area was the verification of the fact that the expressed dissatisfaction with the performance appraisal system may be related to organisational justice. And the result attained by these studies was positive: the relationship between organisational justice and satisfaction with performance appraisal has already been understood.

In this regard, Cook and Crossman (2004) state that literature suggests that individuals will only be pleased with the performance appraisal process if it meets the criteria of ‘justice’, whereas Mcfarlin and Sweeney (1992) have found that perceptions of justice influence different attitudes towards performance appraisal, including satisfaction with that appraisal. Similar findings have been attained by Tang and Sarsfield-Baldwin (1996), who found, in their study, evidence that the elements of organisational justice are important in the prediction of satisfaction with performance appraisal.

As an aside, a short reference to the perception of justice is made. In synthetic terms, judgments about justice are based on workers’ individual expectations and on the comparison of their results with those obtained by others within the organisation (Cook & Crossman, 2004). It should also be referred that organisational justice encompasses several dimensions, insofar as the different research studies on this topic have clearly demonstrated that human beings, in an organisational context, have clear concerns about the justice of outcomes (distributive justice), procedures (procedural justice) and interpersonal treatment (interactional justice, being possible to identify within it two facets or sub-dimensions: interpersonal justice and informational justice).

In short, from the preceding paragraphs, we may consider the hypothesis that the levels of satisfaction with the performance appraisal system are clearly related to the perceived justice and equity of the system by the appraisees (cf. Cook & Crossman, 2004; Kluger and DeNisi, 1996; Mcfarlin and Sweeney, 1992; Mount, 1983, 1984; Pooyan & Eberhardt, 1989; Selvarajan & Cloninger, 2012; Tang & Sarsfield-Baldwin, 1996). This leads to the first hypothesis of this research:Hypothesis 1

The perception of justice of performance appraisal is positively related to the satisfaction felt towards that appraisal.

On the other hand, we will also assess whether there is any relationship between the perception that appraisees have of the accuracy and precision of SIADAP and the satisfaction with performance appraisal resulting from this system.

Thus, we may start by considering that the appraisal's accuracy is something relative and belongs to the sphere of individual perception (Landy & Farr, 1983; Tsai & Wang, 2013). In other words, the level of accuracy or inaccuracy of the appraisal system will depend on the human perception and awareness regarding the process and the results of performance appraisal (cf. Boswell & Boudreau, 2000). However, it will, inevitably, be affected by something: the inaccuracies of the appraisal process, such as the halo effect, the horn effect, the leniency error, the amplitude constraint error, the central tendency error, the contrast error or the first impression error. Their existence raises, from the outset, pertinent questions about the accuracy (or inaccuracy) of performance appraisal.

That said, we should point out that the acceptance of the appraisal process by the appraisees increases when performance appraisal systems are perceived as accurate in their results (cf. Carroll & Schneier, 1982; Farh, Werbel, & Bedeian, 1988; Roberts, 1992). From this perspective, when workers perceive an inefficient and somewhat loose performance appraisal, they quickly deny the accuracy of that system (cf. Almeida, 1996; Blau, 1999; Roberts, 1998; Wilson, 1991), withdrawing any meaning or purpose from it, inasmuch as they stop accepting or trusting in it (cf. Cardy & Dobbins, 1994; Murphy & Cleveland, 1995; Tuytens & Devos, 2012). On the other hand, if workers feel that their appraisals are accurate, they are more likely to show positive reactions and to accept the appraisal system and participate in it, as well as comply with the actions, at the organisational level, that are based on those appraisals (Miller & Thornton, 2006).

Although Selvarajan and Cloninger (2012) state that literature has not directly addressed the aforementioned relationship yet (perception of the accuracy of performance appraisal and satisfaction with it), the truth is that these authors have advocated it, considering the following logic: if workers perceive appraisals of their performance as being accurate, they will be prone to truly trust them; therefore, the possibility of having favourable attitudes towards such appraisals is real. Hence, performance appraisals that are perceived as accurate are perceived positively by workers, and this may lead to higher satisfaction with the performance appraisal system.

This hypothesis was verified in the empirical study carried out by the aforementioned authors, that is, they concluded that performance appraisals perceived as being more accurate were related to higher levels of satisfaction with those appraisals. Hence our second working hypothesis:Hypothesis 2

The perception of accuracy of performance appraisal is positively related to the satisfaction felt towards that appraisal.

3

Research methodology

3.1

Characterisation of population

The target universe of this research is composed of the set of Portuguese civil servants who are subject to the appraisal of their performance through the precepts of SIADAP. It should be reminded that this is not to say that the target universe is composed of the set of all Portuguese civil servants. These two universes are distinct, given that the members of several special careers are not appraised by the aforementioned appraisal system.

Given the large dimension of this universe, we have chosen to include only the civil servants whose performance is appraised according to the SIADAP standards, working at the three existing public higher education institutions in a given city, district capital, from Centre mainland Portugal. In essence, we have one university, one polytechnic institute and one school of polytechnic nature, although not integrated.

We sent the questionnaire to all elements of the population, a total of 2247 individuals, through the various organisational units that make up the three selected HEIs (Higher Education Institutions). There was obtained, in return, 334 questionnaires completed and considered valid.

In terms of sociodemographic characteristics of the respondents to the questionnaire, the data presented in Table 1 allow us verifying that female gender predominates, as masculine subjects account for only 29% of respondents. Furthermore, there is a uniform distribution in terms of group age, except for the extremes: only 13 individuals had up to 29 years of age (i.e., 3.9%), whereas the number of individuals with over 60 years of age was even lower: 5 workers, or, if we prefer, 1.5%. With regard to educational qualifications, over 60% hold a higher education degree, although the number of individuals with the 12th grade of school should not be neglected (26.3%).

Table 1.

Sociodemographic characteristics of the subjects from the sample.

Variable	n	%
Gender
Female	237	71.0
Male	97	29.0
Age group
Up to 29 years of age	13	3.9
Between 30 and 39 years of age	108	32.3
Between 40 and 49 years of age	103	30.8
Between 50 and 59 years of age	105	31.4
Over 60 years of age	5	1.5
Educational qualifications
Less than the 9th grade of school	6	1.8
9th grade of school (3rd Cycle of Basic Education)	17	5.1
12^th grade of school (Secondary Education)	88	26.3
Technological Specialisation Course	15	4.5
Bachelor	9	2.7
Graduation	86	25.7
Post-Graduation	49	14.7
Master	61	18.3
Ph.D	3	0.9

Source: own elaboration.

Drifting our analysis into the professional characteristics of the subjects from the sample, depicted in Table 2, we find that almost half of respondents (49.4%) were senior technicians, whereas the second most represented career was the one of ‘technical assistant’, with 36.5%. The values of the other categories are quite distant from these: operational assistant with 7.5%; holders of leadership positions with 3.3%; and workers integrated in other categories and/or careers, accounting for 3.3%.

Table 2.

Professional characteristics of the subjects from the sample.

Variable	n	%
Professional category
Operational Assistant	25	7.5
Technical Assistant	122	36.5
Senior Technician	165	49.4
Leader	11	3.3
Another Category	11	3.3
Institution where the professional activity is carried out
Alfa Higher Education Institution	96	28.7
Beta Higher Education Institution	229	68.6
Gama Higher Education Institution	9	2.7

Source: own elaboration.

Finally, regarding the institution where respondents carried out their work, the majority (68.6%) reported the Beta Higher Education Institution (university), whereas 28.7% reported having a contractual relationship with the Alfa Higher Education Institution (polytechnic institute). Only 9 (i.e., 2.7%) are connected to the Gama Higher Education Institution (school of polytechnic nature, although not integrated).¹

3.2

Operationalisation of variables

The operationalisation of the concepts used in this research was carried out mainly through the use of scales developed in previous empirical studies, which presupposes purified scales as to reliability and validity. Thus, we present, hereafter, the selected scales that were part of the questionnaire submitted to the pre-test. When necessary, the items of the chosen scales were adapted to the context of this research (Table 3).

Table 3.

Adapted scales in the delineation of the questionnaire used in the pre-test.

Variable	Scale author
Perception of distributive justice of performance appraisal	Colquitt (2001); Korsgaard and Roberson (1995)
Perception of interactional justice of performance appraisal	Moorman (1991); Colquitt (2001)
Perception of procedural justice of performance appraisal	Colquitt (2001); Own elaboration
Perception of the accuracy of performance appraisal	Vest, Scott, and Tarnoff (1995); Brudney and Condrey (1993)
Satisfaction with performance appraisal	Greller (1978); Colquitt (2001)

Source: own elaboration.

It should be added that all items of the scales may be characterised as “closed questions” and mandatory in nature, in a Likert scale of five points/levels, from ‘strongly disagree’ to ‘strongly agree’. The questions from the last group, related to the biographical and demographic variables, follow the “close question” philosophy, albeit the Likert scale is not used. The only exception is the request for a final comment, optional, which is, naturally, an open question, given its nature.

This first version of the questionnaire was subjected to a preliminary study applied to individuals from the research target universe but not to the elements to be surveyed at the implementation of the final study. Specifically, the questionnaire was sent via email to various public institutions, both from the local, regional and central administration.

After assessing the reliability of the scales used, performed on the basis of the calculation of the item–total correlation and internal consistency of each of these scales using the Cronbach's alpha coefficient, without forgetting the reading of the numerous suggestions and comments, some problems were found that were needed to be corrected. Accordingly, some corrections were made: at the level of items’ form and order; and the reshaping, removal and addition of items in the scales used. For example, no changes arising from statistical analysis were performed, given that results equal or higher than 0.70 were attained for the Cronbach alpha coefficient when assessing the internal consistency of each of the scales used; moreover, values equal or higher than 0.20 were attained for item–total correlation. However, comments and suggestions made by respondents within the scope of this preliminary study led to the reformulation of some items, in order to make them more concise, direct and noticeable, or to the removal of some items of scales, when they were redundant, so as to shorten the questionnaire.

A new questionnaire with the same five variables resulted from the aforementioned changes: (1) “Perception of distributive justice of performance appraisal” composed of five items (example: “The final rates I attained are fair”); (2) “Perception of interactional justice of performance appraisal” composed of six items (example: “The appraiser was able to avoid personal biases”); (3) “Perception of procedural justice of performance appraisal” composed of eight items (example: “The procedures used in the appraisal of my performance are fair, appropriate and unbiased”); (4) “Perception of the accuracy of performance appraisal” composed of five items (example: “The performance appraisals were accurate and precise”); (5) “Satisfaction with performance appraisal” composed of six items (example: “I am pleased with the final rates I received”).

To note that the latter group continues to include questions about age, gender, educational qualifications, professional category, etc., in order to characterise the respondents. In addition, this group maintains an open question, allowing respondents to add their opinion on the issues included in the questionnaire.

3.3

Data collection instrument and procedures

As this research is cross-cutting, in temporal terms, the questionnaires were administered between May and June 2013. More precisely, we used the self-administered online questionnaire survey, based on the LimeSurvey platform and available from the website survey.iscac.pt, whose access link was: http://survey.iscac.pt/index.php?sid=75229⟨=pt.

We cannot fail to mention that the contact was not made at the individual level, but rather at the institutional level, with the email described previously having been sent to each department of the three chosen HEIs, with a request for disclosure amongst their workforce by so as to be self-administered by them. There was a dual purpose regarding this option: to obtain permission from the various organisations; and to reach the entire population in a timely and error-free manner. Later, when the answers stopped occurring for three consecutive days, a reminder of this commitment was carried out, in a way identical to that reported above.

After the period of data collection, the data attained were exported from the LimeSurvey platform to Excel and, subsequently, to the Statistical Package for the Social Sciences software, version 20.0 from 2012, aiming at their statistical processing.

4

Analysis and discussion of results

So as to verify the reliability of the items that made up the final versions of the scales used, which was confirmed, the item–total correlation and the internal consistency, using Cronbach's alpha, were calculated.

Before carrying out the test of the hypotheses of this research, it was necessary to perform a previous task with a very clear goal: to have all scales scored in a range from 0.00 to 100.00 points. Thus, we were able to calculate the results presented in Table 4, on the basis of the responses obtained.

Table 4.

Results observed for the scales used.

Scale	x¯	Md	s	xmin	xmax	p
Interactional justice	57.65	58.33	24.74	0.00	100.00	0.000
Distributive justice	39.12	25.00	31.57	0.00	100.00	0.000
Procedural justice	41.10	40.63	21.45	0.00	100.00	0.072
Accuracy	35.75	35.00	20.25	0.00	100.00	0.000
Satisfaction	37.69	37.50	21.29	0.00	100.00	0.001

Source: own elaboration.

The values of the measures of central tendency (mean and median) reveal that the respondents tended to show that they have a low perception of accuracy of that appraisal (35.75; 35.00); a low satisfaction with the appraisal (37.69; 37.50); and a reduced perception of distributive justice (39.12; 25.00) and procedural justice (41.10; 40.63). Results also show that the subjects from the sample have a reasonable perception of interactional justice (57.65; 58.33).

Clues about some of the reasons behind these such low values may be drawn from the comments to the aforementioned open question. Thus, the considerations we have found for the scenario described above include:

• •

  Perceived existence of favouritism: “In the cases I am aware of, SIADAP does not reflect the actual employees’ performance. It is used to promote favouritism and has no connection whatsoever with merit”; “Appraisers continue to rate employees very much by the personal favouritism criterion, and not by the actual capabilities and professional skills”.

• •

  Imposition of quotas for the best ratings, given that, in no. 1 of art. 75 of Law no. 66-B/2007, of December 28, the maximum percentage of 25% is set for the final qualitative appraisals of ‘relevant performance’ and, from these, 5% for the recognition of ‘outstanding performance’: “From the moment that there are quotas in an appraisal system, it cannot be fair”; “The quota system imposed by SIADAP is an instrument of perversion of the appraisal system”;

• •

  Lack of practical effects from SIADAP, given the interdiction on remuneration revaluations, even if employees meet the legal requirements in terms of performance appraisal (cf. art. 47 of Law no 12-A/2008 of February 27): “Given the budget cuts and the impediment to career progression, SIADAP does not bring any personal benefits”; “The performance appraisal system becomes virtually useless in face of freezes on career progression”.

The latter topic leaves the door open to another factor that may have influenced the results obtained by this research: the current global economic and financial crisis, mainly because of the profound implications for public finances. This crisis had, as a clear result, troubled times for all workers from the Portuguese public sector, as a consequence of the many austerity measures made real or not (examples: wage cuts, freeze on career progressions and promotions; decrease in the number of civil servants through dismissal; increase in the weekly working hours of civil servants in 5 h, from 35 to 40 h; increase in the contributions to the health subsystems that are specific of the public sector; loss of holiday and Christmas bonuses, etc.). Thus, when the survey reached the subjects of the surveyed population, uncertainty and insecurity about the professional future, among other factors, of Portuguese civil servants was enormous, which may have influenced respondents’ perceptions about SIADAP.

Table 4 also presents the results of the significance of the normality test (Kolmogorov–Smirnov). As it may be observed, except for the scale used to measure the perception of procedural justice (p = 0.072), all others revealed frequency distributions that deviate significantly from the normal or Gaussian distribution (p < 0.050). This led us to opt for non-parametric techniques to test the hypotheses.

We carried out some further studies to the collected data, using the Mann–Whitney U test and the Kruskal–Wallis test. Thus, we compared the results of the diverse scales used in term of gender, age group, academic qualifications, professional category and institution of work. The main conclusion to be drawn is that results are similar, regardless of age group, gender, etc., with the following exceptions:

• •

  function of academic qualifications: there are significant differences at the level of procedural justice (p = 0.019). The comparative analysis of measures of central tendency (ordinal mean, mean and median) revealed that employees with higher academic qualifications tend to show lower perceptions of procedural justice.

• •

  function of the professional category: the existence of statistically significant differences at the level of the perception of procedural justice (p = 0.016), in which leaders showed a greater perception of procedural justice, opposed to senior technicians, who showed the lowest perception of this type of justice.

So as to test hypothesis 1 «the perception of justice of performance appraisal is positively related to the satisfaction felt towards that appraisal», we studied the correlation using the Spearman correlation coefficient and the respective significance test. The hypothesis was tested in all scales of justice and, as it may be verified in the results displayed in Table 5, all correlations are positive and highly significant (p < 0.001). We conclude that the hypothesis is confirmed, that is, employees who have a better perception of justice in performance appraisal tend to show greater satisfaction with it.

Table 5.

Correlation between satisfaction and perception of justice of performance appraisal.

Variables	Satisfaction
	rs	p^a
Interactional justice	+0.68	0.000
Distributive justice	+0.79	0.000
Procedural justice	+0.76	0.000

a

Unilateral test.

Source: own elaboration.

So as to test the second hypothesis «the perception of accuracy of performance appraisal is positively related to the satisfaction felt towards that appraisal», we studied the correlation between the values of two variables applying, again, the Spearman correlation coefficient and its test significance. The result reveals a positive and highly significant correlation (p < 0.001). We conclude that workers who have a better perception of the accuracy of performance appraisal tend to show greater satisfaction with that appraisal (Table 6).

Table 6.

Correlation between the perception of accuracy and the satisfaction with performance appraisal.

Variables	Satisfaction
	rs	p^a
Accuracy	+0.80	0.000

a

Unilateral test.

5

Conclusions

One of the goals of this research study was to analyse a possible relationship between perceptions of justice and accuracy of performance appraisal with the satisfaction felt with this appraisal, within the context of SIADAP, which led to our two working hypotheses. Results allow concluding that these two hypotheses were corroborated, which points towards the fact that the results of this research are in line with the literature that underpinned the construction of the theoretical framework of this study. In sum, it may be concluded that employees who have a better perception of justice and accuracy of performance appraisal tend to show greater satisfaction with it.

The problem is that, observing the values of the measures of central tendency, we may infer that respondents have a low perception of the accuracy of performance appraisal (35.75; 35.00) and are less than satisfied with that appraisal (37.69; 37.50). In terms of perception of justice regarding performance appraisal, the scenario is a little more diverse, although only the perception of interactional justice exhibits positive and reasonable values (57.65; 58.33). Furthermore, when comparing the results of different scales used according to gender, age group, academic qualifications, professional category and institution of work, the main conclusion to be drawn is that, for the most part, the results obtained in the diverse scales are identical by gender, age group, academic qualifications, etc. The reasons that may explain these values could entail the perception of the existence of favouritism, through the enforcement of quotas for the best ratings and through the lack of practical effects from SIADAP.

The scenario described above is, to say the least, a cause for concern, considering (1) the influence that satisfaction with performance appraisal has on employees’ attitudes and behaviours; (2) that dissatisfaction and feelings of injustice towards the performance appraisal process may result in its failure; and (3) that the adverse reactions may result in failure of the performance appraisal system. In sum, given the above and the results obtained by this research, we may consider that the current relevance of SIADAP, as performance appraisal system, is extremely limited.

In order to reverse this scenario and ensure that SIADAP fulfils its purposes and may be considered an indispensable tool for achieving a new culture of merit for the Portuguese public sector, the reasons underlying this dissatisfaction must be met – or mitigated. To this end, future research studies should focus on the analysis of the factors that lead employees to show this dissatisfaction and these negative perceptions about this appraisal system and about their performance appraisals. Only then may administrators – and the very legislator – envisage what is it that needs to be changed so as to increase this kind of satisfaction.

We are aware that, as in any other research project, this study has some limitations, namely: the existence of items from some scales with low item–total correlation, although their presence does not substantially impair the scales’ internal consistency; problems with the external validity of this research, since the methodological choices may jeopardise the generalisation of the results obtained, in the context of the Portuguese public sector.