Translation and adaptation into the Romanian language of the Personality-Related Position Requirements Form (PPRF)

Abstract

The present research aimed at the translation and adaptation from English into the Romanian language of the Personality-Related Position Requirements Form (PPRF), developed by Raymark, Schmit, and Guion. Within it, the job of operational non-commissioned officer (NCO) employed in the mobile structures of the Romanian Gendarmerie was analyzed. Four subsamples of participants were used: two subsamples of subject-matter experts in relation to the analyzed job, 87 job incumbents, and 32 supervisors, one subsample with 133 analyzed job incumbents and one subsample composed of supervisors of the participants from the previously mentioned subsample. The vast majority of reliability coefficients calculated for evaluating the inter-rater agreement had very good values both for the inventory's dimensions and subdimensions. Regarding the concurrent criterion-related validity, acceptable results were found by using the partial- and full-weighting approaches for criterion-related validation of job analysis tools.

Resumen

Esta investigación aborda la traducción y adaptación del inglés al rumano del Formulario de Requisitos del Puesto Relativos a la Personalidad desarrollado por Raymark, Schmit y Guion. En este context se analiza el puesto de suboficial operativo en las estructuras de la gendarmería rumana. Se utilizaron cuatro submuestras de participantes: dos de expertos en el puesto analizados, 87 ocupantes del puesto y 32 supervisores, una submuestra con 133 ocupantes de los puestos analizados y otra con supervisors de los participantes de la submuestra previamente mencionada. La mayoría de los coeficientes de fiabilidad calculados para evaluar el acuerdo entre jueces tenia muy buenos valores tanto para las dimensiones como para las subdimensiones del inventario. En lo que respecta a la validez concurrente relacionada con el criterio se obtuvieron resultados aceptables utilizando los métodos de ponderación parcial y complete para la validación relacionada con el criterio de los intrumentos de análisis.

Job performance is influenced by situational factors (e.g., characteristics of the job, of the organization, and of the colleagues) and by dispositional factors (e.g., declarative knowledge, procedural knowledge and skills, motivation, intelligence, abilities, personality characteristics). Although personality characteristics can explain only a part of the job performance's variation, the identification of the relationship between these and job performance was an issue for the work and organizational psychology in the context of the attempt to ensure the person-job fit through personnel psychological selection.

By the tenth decade of the last century, abilities and intelligence were used as psychological predictors of job performance, and personality characteristics were not considered such credible predictors. This situation was driven by the publishment of research results, such as the qualitative review of Guion and Gottier (1965) and the quantitative meta-analysis conducted by Schmitt, Gooding, Noe, and Kirsch (1984), after which the general conclusion drawn was that personality tests have no predictive validity suitable for their use in personnel selection (Hurtz & Donovan, 2000). The explanation proposed by some researchers for the above conclusion was that the true predictive validity of personality was hidden by a lack of a common approach to the organization of personality traits used as predictors (idem).

An exception to the situation described above is related to emotional stability, a personality characteristic considered by specialists particularly needed for working in high-stress jobs. “Personality measures have long been used as a screening test for detection of potential emotional instability or maladjustment in sensitive, high-stress or high-security jobs”, such as those of the police officers or military personnel (Kanfer, Ackerman, Murtha, & Goff, 1995).

Since the tenth decade of the last century, with the increasing confidence in the robustness of the Five-Factor Model of personality (FFM), researchers have adopted this model in research regarding psychological selection of the personnel (Hurtz & Donovan, 2000). The result of this research trend was the demonstration of the relationship between personality traits and job performance in several singular research and meta-analyses (e.g., Barrick & Mount, 1991; Barrick, Mount, & Judge, 2001; Mount, Barrick, & Stewart, 1998; Hurtz & Donovan, 2000; Rothmann & Coetzer, 2003; Salgado, 1997; Salgado, 2003; Tett, Jackson, & Rothstein, 1991).

Meta-analyses have shown that two factors within the FFM, conscientiousness and emotional stability, have validities that generalize in the prediction of overall job performance (Barrick & Mount, 2005). Most meta-analyses conducted indicated that the conscientiousness factor is more strongly related to overall job performance than emotional stability factor (Barrick et al., 2001). Conscientiousness has been identified as one of the best predictors of job performance in the United States and Europe in the research conducted by Barrick and Mount (1991) and in that of Salgado (1997) (Rothmann & Coetzer, 2003). The other three factors of the FFM (agreeableness, openness to experience, and extraversion) do not predict overall job performance but success in particular jobs or related with specific criteria (Barrick et al., 2001).

The demonstration of the relationship between personality and job performance stimulated the interest of many researchers for the development of job analysis tools designed to identify the personality dimensions that predict performance in specific jobs. Raymark, Schmit, and Guion (1994) have developed such a personality-based job analysis tool–Personality-Related Position Requirements Form (PPRF). They conducted a study which included the collection of 260 descriptors of different jobs to determine whether this tool could reliably differentiate these, and they found such evidence (Raymark et al., 1994).

Little research has been conducted for determining the psychometric properties of personality-based job analysis tools, especially regarding the determination of their validity, like those of Hogan and Holland (2002), Meyer, Foster, and Anderson (2006), and Meyer and Foster (2007). There is a very important difference between personality measures and personality-based job analysis tools from the point of view of their subject: personality measures assess the person and personality-based job analysis tools assess the job. The determination of criterion validity of a personality-based job analysis tool has its specificity because the relationship between the results of such a tool and the performance criteria in a specific job cannot be directly assessed. Meyer et al. (2006) argue that, for the validation of a personality-based job analysis tool, the results obtained at it must be interjected in the relation between a personality measure's results and job performance. According to the previously mentioned authors “one way of doing so may be by weighting the HPI (an FFM personality measure) results based on the results of the PIC (Performance Improvement Characteristics - a FFM personality-based job analysis inventory) (e.g., placing more weight on those HPI scales that were deemed most important in job analysis as PIC scales)”.

Research by Meyer et al. (2006) had as objective the predictive validation of PIC. The meta-analytic results obtained by using an algorithm created for this goal indicated that performance was significantly predicted by the partial-weighted predictors (ρ = .24) and that “performance in a given job was better predicted when the predictor (HPI) was weighted according to job analysis data (obtained through the use of PIC) from that job than when weighted by job analysis data from another job” (in the latter case, ρ = .07). The algorithm implied that within each study (job), the PIC scales were rank-ordered based on the importance ratings provided by the PIC: the highest ranked PIC scale was given a weight of “3”, the second highest a “2”, the third highest a “1”, and the remaining four PIC scales were weighted “0”, thereby eliminating them from the subsequent analyses.

Meyer and Foster (2007) examined the utility of three approaches to validate the PIC:

• -

  Partial-weighting - which use the same algorithm created by Meyer et al. (2006).

• -

  Full-weighting–“the percentage of total possible for each PIC scale is calculated. Then, each individual's HPI scores are multiplied by these PIC percentages to arrive at weighted scale scores. The seven weighted predictor (HPI) scales were then summed and correlated with performance”.

• -

  Profile similarity–“employs a metric of similarity between predictor (HPI) scores and PIC scores, sometimes referred to as a profile correlation index (PCI: Timmerman, 1996)”.

Meta-analyses were computed for each of the three weighting approaches in the afore-mentioned study and among the findings were the following: profile correlation indexes were predictive of performance (ρ = .21), performance was predicted by the full-weighted predictors (ρ = .12), and the 90% confidence interval did not include .00, but the magnitude of the estimated parameter is not substantial; performance was significantly predicted by the partial-weighted predictors (ρ = .21).

Full- and partial-weighting approaches have been utilized for the concurrent validation of the Romanian PPRF version.

The Present Study

The present research aimed to translate and adapt the personality-based job analysis tool, the Personality-Related Position Requirements Form (PPRF), developed by Raymark et al. (1994), in compliance with the international best and newest methodological guidelines in the adaptation of the psychological tests field (e.g., ITC, 2005; Muñiz & Bartram, 2007; Van de Vijver & Hambleton, 1996). A secondary objective was the development of the ideal personality profile of the operational non-commissioned officer (henceforth, NCO) serving in the mobile structures of the Romanian Gendarmerie, which include the personality dimensions and subdimensions necessary for the effective job performance, along with their related development levels.

Method

Ethical Statement

The informed consent was obtained from all the participants in the research. To this end, each participant completed the suitable informed consent form. Any information that may help to the identification of the participants has not been and will not be made public and/or disclosed to third parties, from inside or outside of the Ministry of Internal Affairs of Romania.

Participants

The research sample consisted of four subsamples of participants:

• -

  Two subsamples of subject-matter experts–one analyzed job incumbent subsample, with 87 participants, and one subsample composed by supervisors of the analyzed job incumbents, with 32 participants (most of them immediate supervisors–93.75%, 30 participants). The participants included in the two subsamples had at least 5 years of service in the previously mentioned jobs and are rated at least very well at the yearly job performance appraisal over the last three years. They completed the PPRF inventory that was translated and adapted in the Romanian language to analyze the target job of the present research.

• -

  One subsample with analyzed job incumbents, with 133 participants who completed the CP5F personality questionnaire. The characteristics of the participants included in this subsample were as follows: 131 men and 2 women; the average age in years –36.6 years–, standard deviation –6.29 years–, minimum age –24 years–, maximum age –51 years–, the average length of service in the analyzed job in years –11.12 years–, standard deviation –5.38 years–, minimum age –2 years–, and maximum age –25 years.

  To generalize the research results to the whole groups of the operational NCOs employed in the mobile structures of the Romanian Gendarmerie and of the subject-matter experts (analyzed job incumbents and supervisors of them), it was attempted to ensure the statistical representativeness of the subsamples used, this goal being achieved through random selection of the participants included. Thus, initially, clusters sampling was carried out by selecting through the simple random selection technique a number of units (county inspectorates at national level and battalions of the Bucharest General Directorate of Gendarmerie) in the case of each of the subsamples, and finally at the level of the selected units it was again used simple random sampling to select participants in the research.

• -

  One subsample of participants’ supervisors from the subsample of the analyzed job incumbents. They completed a job performance appraisal form for each of their analyzed job incumbents.

The Analyzed Job

In the present research it was analyzed a job picked so to have a large number of incumbents in the Romanian Gendarmerie and to be important in terms of carrying out the missions of this institution: operational NCO employed in mobile structures. People employed in this job have an executive function, which implies carrying out missions in the field of public order and safety, such as:

• -

  Ensuring public order at meetings and events (protests, marches, demonstrations, processions, picketing actions, promotional actions, commercial actions, cultural and artistic activities, sport activities, and religious and commemorative activities), taking place in public places and involving crowds of people.

• -

  Intervention in and restoration of public order when it was disturbed by any action or forces that contravene the laws in force.

• -

  Maintaining public order in touristic resorts.

• -

  Actions to combat crime in markets, fairs, areas, and places with high criminal potential.

• -

  Performing procedural acts at the request of magistrates and together with them (mobile structures, n.d.).

Procedure

The job of operational NCO employed in the mobile structures of the Romanian Gendarmerie was analyzed by two subsamples of subject-matter experts (analyzed job incumbents and supervisors of them) through the use of the Romanian version of the PPRF. In order to establish the concurrent validation of that measure, the participants from the subsample of analyzed job incumbents responded to the CP5F questionnaire and their supervisors appraised their job performance for last six months.

Measurement Instruments

Personality-Related Position Requirements Form (PPRF). This inventory was developed by Raymark et al. (1994) to identify the FFM personality dimensions and subdimensions required for the efficient performance in a specific job. The inventory contains 107 items intended to identify the level of 12 subdimensions (actually 11, because one of them, emotional stability, is a dimension) and 5 dimensions. Patrick H. Raymark, one of the inventory's authors, expressed explicitly the agreement to the author of the present study for the use of the PPRF for research purposes.

The inventory was translated from English into Romanian independently by two Romanian connoisseurs of the English language and then the first form of translation was created, which involved the agreement between the two translations. The reverse translation of this form from Romanian into English was performed independently by two specialists in English. The final form of the reverse translation was made, so there was an agreement between the two reverse translations. Finally, the original version of the inventory was compared with the final form of the reverse translation, on this occasion a good fit between them was noticed. The pilot administration of the translated inventory at the level of a group with 28 staff from the Ministry of Internal Affairs of Romania (NCOs and officers, serving in the Romanian Gendarmerie and in the Romanian Police) did not reveal the need to achieve substantial changes.

The scoring procedure for the PPRF items is the following: 0 = not required, 1 = helpful, and 2 = essential.

Five-Factor Personality Questionnaire [Chestionarul de Personalitate cu Cinci Factori - CP5F]. This questionnaire was developed by Albu (2008) after the model of the Five-Factor Personality Inventory (FFPI) developed by Hendriks (1997). The CP5F questionnaire contains 130 items, and has the following scales: extraversion (23 items), agreeableness (24 items), conscientiousness (25 items), emotional stability (21 items), autonomy (22 items), and social desirability (15 items). Response options to questionnaire items are 1 (suits me very little), 2 (suits me less), 3 (suits me about half), 4 (suits me much), and 5 (suits me very much).

Job performance appraisal form. Through this form it was requested the appraisal by the supervisor of the analized job incumbent of the performance of at least six tasks, duties, and responsibilities listed in the job description, the most important and frequently performed. The rating scale used had six steps, ranging from 1 (exceptional) to 6 (inadequate). An overall job performance rating was calculated for each participant from the subsample of analyzed job incumbents by averaging the ratings of job performance indicators mentioned by their supervisors. The appraisal had to be made for a period of the last six months, completed when it was done. To lower the possibility of results’ distortion by many existing appraisal biases, a briefing was written in which various such items that may occur in the appraisal process were presented.

Results

Results Obtained in the PPRF

The results obtained by Raymark et al. (1994) using an approach related to the comparative judgment law of Thurstone in applying the case V indicated that certain items must have a coefficient of 2 to take account of their importance to their dimensions (or subdimensions) (Touzé & Steiner, 2002). Therefore, the scoring method of the inventory's items, presented by Raymark et al. (1994), requires the multiplication by 2 of these items’ scores (those more important for their subdimensions). Also, in the calculation of the PPRF subdimensions’ mean scores, the divisor is equal to the number of items not multiplied by 2 plus twice the number of items which have to be multiplied by 2. All mean scores of the PPRF dimensions and subdimensions presented have been calculated in the above specified manner.

The mean scores of the PPRF dimensions and subdimensions and other descriptive statistics from the total subsample of subject-matter experts and from the subsamples of experts are presented in Table 1.

Table 1.

Descriptive Statistics of the PPRF Scores from the Total Subsample and from the Subsamples of Experts.

PPRF dimensions and subdimensions	Number of participants			Mean			Standard deviation
	1*	2*	3*	1	2	3	1	2	3
Surgency	119	87	32	0.98	1.04	0.78	0.40	0.38	0.39
General leadership	119	87	32	1.04	1.12	0.82	0.50	0.48	0.51
Interest in negotiation	119	87	32	0.85	0.94	0.65	0.54	0.55	0.48
Ambition	119	87	32	1.04	1.09	0.89	0.50	0.51	0.44
Agreeableness	119	87	32	1.19	1.24	1.04	0.38	0.36	0.40
Friendly disposition	119	87	32	0.91	0.95	0.80	0.50	0.51	0.46
Sensitivity to interest of others	119	87	32	1.13	1.19	0.97	0.54	0.53	0.52
Cooperative or collaborative work tendency	119	87	32	1.52	1.58	1.36	0.40	0.37	0.44
Conscientiousness	119	87	32	1.42	1.46	1.32	0.30	0.32	0.36
General trustworthiness	119	87	32	1.23	1.28	1.12	0.37	0.34	0.42
Adherence to a work ethic	119	87	32	1.53	1.57	1.43	0.36	0.36	0.34
Thoroughness and attentiveness to details	119	87	32	1.50	1.53	1.40	0.52	0.51	0.54
Emotional Stability	119	87	32	1.77	1.78	1.74	0.38	0.37	0.43
Intellectance	119	87	32	1.19	1.26	1.03	0.44	0.42	0.45
Desire to generate ideas	119	87	32	0.83	0.86	0.72	0.51	0.52	0.46
Tendency to think things through	119	87	32	1.56	1.65	1.34	0.47	0.42	0.54

Note. 1*: total subsample of experts, 2*: job incumbent expert, 3*: supervisor expert.

Student's t-tests for independent samples were calculated to compare the mean scores of the five PPRF dimensions from the level of the analyzed job incumbent experts and from the level of the supervisor of the analyzed job incumbent experts, thus the following statistically significantly differences being identified, all indicating higher ratings of the job incumbent experts in relation to those of the supervisor experts: in the case of surgency, between the scores of job incumbent experts (M = 1.04, SD = 0.38) and those of supervisor experts (M = 0.78, SD = 0.39), t(117) = 3.262, p = .001, d = 0.67; in the case of agreeableness, between the scores of the job incumbent experts (M = 1.24, SD = 0.36) and those of the supervisor experts (M = 1.04, SD = 0.40), t(117) = 2.590, p = .011, d = 0.52; in the case of conscientiousness, between the scores of the job incumbent experts (M = 1.46, SD = 0.32) and those of the supervisor experts (M = 1.32, SD = 0.36), t(117) = 2.076, p = .040, d = 0.41; and in the case of intellectance, between the scores of the job incumbent experts (M = 1.26, SD = 0.42) and those of the supervisor experts (M = 1.03, SD = 0.45), t(117) = 2.544, p = 012, d = 0.52. The results obtained for the emotional stability dimension indicates that there is not a statistically significant difference between the ratings’ level of this dimension necessary for the efficient performance in the analyzed job at the level of job incumbent experts (M = 1.78, SD = 0.37) and the same type of ratings conducted at the supervisor experts’ level (M = 1.74, SD = 0.43), t(117) = 0.523, p = .602. The agreement of ratings between the two categories of experts probably highlights the importance of the emotional stability dimension for the efficient performance in the analyzed job.

Also, Student's t-tests for independent samples were calculated to compare the mean scores of the PPRF subdimensions from the analyzed job incumbent experts’ level and from that of the supervisors of the analyzed job incumbent experts. Statistically significant differences were identified in the following cases, all indicating higher ratings of the job incumbent experts in relation to those of the supervisor experts: in the case of general leadership, between the scores of the job incumbent experts (M = 1.12, SD = 0.48) and those of the supervisor experts (M = 0.82, SD = 0.51), t(117) = 3.020, p = .003, d = 0.60; in the case of interest for negotiation, between the scores of the job incumbent experts (M = 0.94, SD = 0.55) and those of the supervisor experts (M = .65, SD = 0.48), t(117) = 2.613, p = .010, d = 0.56; in the case of sensitivity to interests of others, between the scores of the job incumbent experts (M = 1.19, SD = 0.53) and those of the supervisor experts (M = 0.97, SD = 0.52), t(117) = 2.068, p = .041, d = 0.41; in the case of cooperative or collaborative work tendency, between the scores of the job incumbent experts (M = 1.58, SD = 0.37) and those of the supervisor experts (M = 1.36, SD = 0.44), t(117) = 2.733, p = .007, d = 0.54; in the case of general credibility, between the scores of the job incumbent experts (M = 1.28, SD = .34) and those of the supervisor experts (M = 1.12, SD = 0.42), t(117) = 2.036, p = .044, d = 0.42; in the case of tendency to think things through, between the scores of the job incumbent experts (M = 1.65, SD = 0.42) and those of the supervisor experts (M = 1.34, SD = 0.54), t(117) = 3.235, p = .002, d = 0.64. The results obtained in the cases of the subdimensions ambition, friendly disposition, adherence to work ethics, thoroughness and attentiveness to details, and desire to generate ideas indicated that there is not a statistically significant difference between the ratings’ values of these subdimensions necessary for the efficient performance in the analyzed job at the job incumbent experts’ level and at the supervisor experts’.

Results Obtained with the CP5F Questionnaire

The descriptive statistics calculated using responses to CP5F items of participants from the analyzed job incumbents’ subsample is presented in Table 2.

Table 2.

Descriptive Statistics of Raw Scores Obtained with the CP5F Questionnaire by the Participants from the Job Analyzed Incumbents’ Subsample.

CP5F dimension	Number of participants	Mean	Median	Standard deviation	Minimum	Maximum
Extraversion	133	91.44	92.00	10.55	60	114
Conscientiousness	133	109.2	110.00	10.00	67	125
Agreeableness	133	98.85	98.00	9.088	69	120
Emotional stability	133	93.70	96.00	8.03	68	105
Autonomy	133	78.05	78.00	7.24	51	94

Results Obtained with the Job Performance Appraisal Form

The technique of job appraisal by the supervisors of the analyzed job incumbents was used to appraise the criterion used for the validation of the PPRF translated and adapted into Romanian language. The appraisal form designed to capture the job performance criterion has enabled each supervisor to specify a maximum of 10 professional performance indicators that capture the tasks, duties, and responsibilities listed in the job description, with the greatest importance for the successful performance on the job and with the great frequency of execution. Through the previously mentioned way it was tried to ensure the relevance of the criterion and to avoid its deficiency, anticipating that, the essential performance indicators for the performance in the analyzed job–that probably are best known by supervisors–would be assessed. The number of indicators used by supervisors ranged from six (used in 43.6% of the forms) to 10 (7 indicators were used in 3% of the forms, 8 in 18%, and 10 in 35.3%).

Since the use of an instrument with variable number of items may represent a factor that bias the measurement, the existence of such a bias was tried to be identified.

Student's t-test for independent sample results revealed no statistically significant difference between the means of the job appraisal ratings of the analyzed job incumbent participants for whom six professional performance indicators were mentioned (M = 2.20, SD = 0.45) and the means of the job appraisal ratings of the analyzed job incumbent participants for whom seven or more indicators were mentioned (M = 2.30, SD = 0.20), t(74, 73) = -1.527, p = .131. After calculation of the Mann-Whitney's U test, it was determined that the median of the ratings of the participants for whom six performance indicators were mentioned (2.16) is significantly lower than the median of ratings of the participants for whom seven or more indicators were mentioned (2.25), U = 1652, p = .017, r = .206. The distributions of the two variables, the scores of the participants for whom six indicators of professional performance were specified (W = .953, p = .025) and the scores of the participants for whom seven or more indicators were specified (W = .958, p = .014), are not normal.

Also, the study tried to identify the differences between the means of ratings based on the first six performance indicators and the means of ratings based on all professional performance indicators mentioned at the level of participants evaluated based on more than six indicators of job performance. To this end, the paired samples t-test was calculated, resulting in the identification of a statistically significant difference between the means of ratings where only the first six indicators mentioned by supervisors were taken into account (M = 2.35, SD = .28) and in the case of using all professional performance indicators mentioned (M = 2.30, SD = 0.20), t(74) = 2.56, p = .012, d = 0.20.

Based on the previously mentioned results, it was concluded that there were indices of a bias influence in the case of ratings conducted based on a variable number of indicators of job performance. Therefore, to ensure criterion measurement's accuracy the retaining of a constant number of indicators is justified (i.e., in the case of the present research just of the first six professional performance indicators specified by supervisors). This measure was supported by the fact that at the level of the 133 appraisal forms filled by the supervisors of analyzed job incumbents the performance indicators were listed in descending order of importance and of the execution frequency of the tasks, the duties and the responsibilities surprised by them.

To compare the distributions of the ratings’ means based on the first six performance indicators mentioned and of the ratings’ means based on all professional performance indicators mentioned on the level of the total sub-sample of analyzed job incumbents, a paired samples t-test was calculated. A statistically significantly difference was identified between the ratings’ means of the analyzed job incumbents for whom only the first six indicators mentioned by supervisors were used (M = 2.287, SD = 0.37) and the ratings’ means of the analyzed job incumbents for whom all professional performance indicators were used (M = 2.259, SD = 0.34), t(132) = 2.520, p = .013, d = 0.078. Note that although there is a statistically significant difference between the two kinds of means, the effect size value is very small, which means that the difference is unimportant from a practical point of view.

Determination of Psychometric Properties of the PPRF Translated and Adapted to Romanian

Reliability. The present research aimed to determine the reliability of the PPRF translated and adapted to Romanian through inter-rater agreement estimation, the established way to determine the rating tools’ reliability, one of these being the PPRF. This agreement was determined by two ways: the calculation of Cronbach's alpha coefficient for each PPRF dimension and subdimension and the calculation of intraclass correlation coefficient for each PPRF dimension and subdimension. In both cases, the raters were treated as items and various items contained in a scale were considered observations of these items.

The results obtained by calculating the two types of coefficients at the level of PPRF dimensions and subdimensions using the participants’ answers from the total subsample of experts are presented in Table 3.

Table 3.

Values of Inter-rater Agreement Indicators in the Cases of the PPRF Dimensions and Subdimensions at the Level of Total Subsample of Experts.

PPRF dimension or subdimension	Alpha coefficient value	Intraclass correlation coefficient value (result of F test with true value 0/p-value)
Surgency	.966	.955 (29.074/.00)
General leadership	.977	.967 (44.587/.000)
Interest in negotiation	.953	.933 (21.464/.000)
Ambition	.975	.964 (40.701/.000)
Agreeableness	.973	.967 (37.767/.00)
Friendly disposition	.975	.968 (41.459/.000)
Sensitivity to interest of others	.882	.851 (8.514/.000)
Cooperative or collaborative work tendency	.968	.963 (32.754/.000)
Conscientiousness	.985	.982 (68.023/.00)
General trustworthiness	.994	.993 (165.961/.000)
Adherence to work ethic	.969	.967 (33.518/.000)
Thoroughness and attentiveness to details	.920	.897 (13.191/.000)
Emotional Stability	.881	.839 (9.002/.00)
Intellectance	.986	.980 (75.607/.00)
Desire to generate ideas	.964	.947 (28.137/.00)
Tendency to think things through	.940	.915 (17.711/.00)
Dimensions mean	.958	.944
Subdimensions mean	.949	.933

The values of inter-rater agreement indicators show a very good inter-rater agreement of the Romanian version of the PPRF at dimension level, with the exception of the values obtained for the emotional stability dimension that indicates a good agreement. Also, most of the calculated values indicate a very good inter-rater agreement of PPRF subdimensions. The exception, indicating a good level of agreement, is the sensitivity towards the interests of others subdimension for alpha coefficients, and in the case of the sensitivity to the interests of others and the thoroughness and attentiveness to details subdimensions for intraclass correlation coefficients.

Criterion validity. In the present research a concurrent validation of the Romanian version of the PPRF was pursued. The two approaches for criterion validation of the mentioned inventory were two of the three validation approaches of the job analysis tools addressed by Meyer and Foster (2007): full-weighting and partial-weighting. We proceeded in the following manner: a) mean scores obtained at the PPRF dimensions by participants included in the total subsample of subject-matter experts were converted into coefficients obtained through dividing of them with 2; b) full-weighted total scores were calculated by adding together the results of the multiplication between the coefficient for each PPRF dimension obtained by the aforementioned way and the scores of the corresponding CP5F scale (the full-weighted total score = PPRF Surgency coefficient x CP5F Extraversion scores + PPRF Agreeableness coefficient x CP5F Agreeableness scores + PPRF Conscientiousness coefficient x CP5F Conscientiousness scores + PPRF Emotional Stability coefficient x CP5F Emotional Stability scores + PPRF Intellectance coefficient x CP5F Autonomy scores); c) partial-weighted total scores were calculated by adding together the results of the multiplication between the coefficients of the three PPRF dimensions with the highest means calculated (Emotional Stability–3, Conscientiousness–2, Intellectance-1) and the scores of the CP5F correspondent scales (the partial-weighted total score = 3 x CP5F Emotional Stability scores + 2 x CP5F Conscientiousness scores + 1 x CP5F Autonomy scores).

For the actual concurrent validation, a Pearson correlation analysis was performed, along with a simple linear regression analysis, where the dependent variable was the ratings’ mean of the first six indicators of job performance specified by supervisors of the analyzed job incumbents and the predictor variables were the full-weighted total score of the CP5F questionnaire, respectively the partial-weighted total score of the CP5F questionnaire. The following results were obtained:

• 1.

  When the full-weighted total score in the CP5F questionnaire was used as predictor variable, a negative correlation, statistically insignificant, was found between the full-weighted total score of the CP5F questionnaire and the ratings’ mean of first six professional performance indicators specified (r = -.140, p = .109), and the regression model predicted 1.9% of the variance. The model was not a good fit for the data (F = 2.601, p > .05).

• 2.

  When the partial-weighted total score of the CP5F questionnaire was used as predictor variable, a negative correlation, statistically significant, was found between the partial-weighted total score at the CP5F questionnaire and the ratings’ mean of the first six professional performance indicators mentioned (r = -.211, p = .015), and the regression model predicted 4.5% of the variance. The model was a good fit for the data (F = 6.122, p < .05). The coefficients of the regression model with the partial-weighted total score as the predictor variable and the mean of first six professional performance indicators specified ratings as dependent variable are presented in Table 4.

  Table 4.

  Coefficients of the regression model with partial-weighted total score as predictor variable and the mean of first six professional performance indicators specified ratings as dependent variable.

  	B	Standard error of B	β
  Constant	3.311	.415
  Partial-weighted total score	−0.002	.001	−.211

The second way to establish the criterion validity of the PPRF using the same approaches mentioned above, the full- and partial-weighting. Also, the same statistical analyses and the same variables were used, except that in the case of the two types of predictor variables (full and partial-weighted total scores) raw scores for each dimension of the CP5F questionnaire were divided with their number of items to eliminate their influence in the predictor total score. The inspiration for this division is originated in the research by Foster, Johnson, and Gaddis (2008), who in the calculation of algorithms to predict the scores on specific competencies, determined to be critical for job success in jobs from the professionals job family through the use of the HPI personality measure, made adjustments (dividing the raw score of HPI predictor scale by its number of items) to account for the different number of items contributing to each HPI scale, since raw scores were used for each HPI predictor. The results obtained were the following:

1. When the full-weighted total score of the CP5F questionnaire with the relativization of the CP5F dimensions’ scores was used as predictor variable, a statistically significant negative correlation was found between the full-weighted score of the CP5F questionnaire and the ratings’ mean of the first six professional performance indicators specified (r = -.202, p = .020), and the regression model predicted 4.1% of the variance. The model was a good fit for the data (F = 5.576, p < .05). The coefficients of the regression model with the full-weighted total score with the relativization of the CP5F dimensions’ scores as the predictor variable and the ratings’ mean of first six professional performance indicators mentioned as dependent variable are presented in Table 5.

Table 5.

Coefficients of the Regression Model with Partial Weighted Total Score as Predictors Variable and the Mean of the First Si Professional Performance Indicators Specified as Depende Variable.

	B	Standard error of B	β
Constant	3.291	.426
Full-weighted total score	−0.037	.016	−.202

2. When the partial-weighted total score of the CP5F questionnaire with the relativization of the CP5F dimensions’ scores was used as predictor variable, a statistically significant negative correlation was found between the CP5F partial-weighted total score and the ratings’ mean of first six professional performance indicators specified (r = -.214, p = .013), and the regression model predicted 4.6% of the variance. The model was a good fit for the data (F = 6.272, p < .05). The coefficients of the regression model with the partial-weighted total score with the relativization of the CP5F dimensions’ scores as predictor variable and the ratings’ mean of first six professional performance indicators specified as dependent variable are presented in Table 6.

Table 6.

Coefficients of the regression model with the partial-weighted total score with the relativization of the CP5F dimensions’ scores as predictor variable and the ratings’ mean of first six professional performance indicators specified as dependent variable.

	B	Standard error of B	β
Constant	3.325	.416
Partial-weighted total score	−0.040	.016	−.214

When the mean of all the appraisals of the job performance indicators mentioned by supervisors was used as dependent variable and the full-weighted total score with or without the relativization of the CP5F dimensions’ scores, respectively of the partial-weighted total score with or without the relativization of the CP5F dimensions’ scores, were used as predictor variables no statistically significantly correlation coefficients were obtained, and the models were not a good fit for the data.

Discussion

The composition of the subject-matter experts’ sample and its statistical representativeness are very important aspects for the job analysis’ results. Regarding the first aspect, the literature review reveals that many researchers use for job analysis samples with mixed composition, job incumbents (top performers) and their supervisors. While job incumbents have a better knowledge of their job, their perception of it may be distorted by biases such as overvaluing the necessity of features valued by the society for an efficient job performance. Therefore, a look from the outside, of the supervisor, is likely to be less distorted by such biases but has the weakness of a less profound knowledge of the analyzed job. There is not a wide agreement among specialists on the ideal proportion of analyzed job incumbent experts and supervisor experts; for example, there were samples with 22.85% supervisor experts of the analyzed job incumbents from a total of 140 experts (CSPB, 2003) and 12.5% supervisor experts from a total of 80 experts (Touzé & Steiner, 2002). Regarding the size and the statistical representativeness of the subject-matter experts’ sample, the literature review points out that some specialists are not concerned with these issues. For example, Foster, Gaddis, and Hogan (2009) used between 8 and 23 experts (supervisors and high performing job incumbents) to analyze three jobs. Other specialists consider that the experts’ sample must be statistically representative. McCormick and Jeanneret (1988, as cited by Whetzel & Wheaton, 2007) present a way of determining the number of job incumbent experts participating in job analysis depending on the total number of job incumbents. In the present research the formation of statistically representative subsamples of experts was attempted since such an approach is useful from a practical standpoint, the analysis results being able to be generalized with a high degree of confidence to the entire analyzed job incumbents group.

At the subject-matter experts’ total subsample level the following decreasing hierarchy of the ratings’ means of the personality dimensions’ importance for the efficient performance of the operational NCO employed in mobile structures tasks was obtained: emotional stability (1.771), conscientiousness (1.420), intellectance (1.194), agreeableness (1.188), and surgency (0.978). The comparison of the dimension ratings’ means from the levels of the two categories of experts (analyzed job incumbents and supervisors) revealed that in most cases the job incumbent experts did statistically significantly higher appraisals than the supervisor experts, a statistically insignificantly difference being identified only in the case of the emotional stability. About half of the subdimensions importance ratings’ means given by job incumbent experts (in the cases of general leadership, interest to negotiation, sensitivity to interests of others, cooperative or collaborative work tendency, general credibility, and tendency to think things through) was significantly higher than the means of the same ratings carried out by supervisor experts.

The fact that the first two important dimensions for the efficient performance in the analyzed job, according to the participant experts, were emotional stability and conscientiousness was expected in terms of duties, tasks, and responsibilities of the analyzed job, and also given the fact that these two dimensions are positively correlated with job performance in almost all the jobs (Barrick et al., 2001). It can be noted that the emotional stability dimension has a higher ratings’ mean than conscientiousness, contrary to the situation revealed by several meta-analyses that indicated a stronger connection with the conscientiousness of overall job performance than that of emotional stability (idem), but accordingly with the opinion of specialists that in high-stress jobs emotional stability is particularly important. A third important dimension identified, intellectance, is quite surprising at first sight in terms of job analyzed tasks but the analysis of its component subdimensions’ means help to explain the situation: the mean of the desire of generating ideas was 0.83 and the mean of the tendency to think things through was 1.56. A similar situation was presented by Raymark et al. (1994) in the case of the fireman job but their means were lower than those calculated in the current research: the mean of the desire of generating ideas mean was 0.77 and the mean of the tendency to think things through was 1.20. For an operational NCO employed in mobile structures, the desire to generate ideas is not important to efficient performance in his job, but the tendency to think things through is important from the point of view of the consequences of the possible work errors: some work errors can increase the risk of physical integrity impairment and the risk of disciplinary or criminal punishment. In the case of the subdimension in question, the score at the total subsample's level was increased by the ratings’ levels of the job incumbent experts, with a mean of 1.65, significantly higher than those of the supervisor experts, with a mean of 1.34.

A subdimension with a high level of development, which is not included within the first three PPRF dimensions rated as important for efficient performance in the analyzed job, is cooperative or collaborative work tendency, which had a ratings’ mean of 1.52, much higher than the mean of the dimension in which is included, agreeableness, 1.188. This result can be explained by the fact that the operational NCO employed in mobile structures job involves teamwork.

The values of the inter-rater agreement indicators show an excellent degree of agreement regarding the rating of the vast majority of dimensions (except for the agreement on the emotional stability dimension, which is good), for both types of indicators, Cronbach's alpha coefficient and the intraclass correlation coefficient. The mean value of the PPRF dimensions’ alpha coefficients was .958, with values ranging from .881 (emotional stability) to .986 (intellectance). The mean value of the intraclass correlation coefficient of the PPRF dimensions was .944, with values ranging from .839 (emotional stability) to .982 (conscientiousness). Touzé and Steiner (2002) obtained an alpha coefficient mean at the level of the PPRF dimensions in a microsample of nurses’ supervisors of .944, with values ranging from .90 (intellectance) to .98 (extraversion), and in a sample of nurses a mean of .98, with values ranging from .97 (intellectual openness) to .99 (agreeableness). The values obtained for the Romanian version of the PPRF are close to those obtained by the two authors mentioned above, except for those obtained on the emotional stability dimension. The analysis the reliability of the Performance Improvement Characteristics Inventory (PIC) dimensions revealed a mean of alpha coefficients of .81, with values ranging from .76 (adjustment - emotional stability) to .87 (agreeableness) (Hogan & Holland, 2002). In the case of the Romanian version of the PPRF, the alpha coefficient values obtained at the level of the dimensions were larger than those of the PIC dimensions, presented above. It is observed in both afore-mentioned inventories that the lowest alpha coefficient is found in the case of the emotional stability dimension.

Most values of inter-rater agreement indicators of PPRF subdimensions indicate an excellent degree of agreement, excluding the values of agreement that indicate a good level of sensitivity to interests of others subdimension, for both types of inter-rater agreement indicators, and of thoroughness and attentiveness to details subdimension in the case of the intraclass correlation coefficient.

The Pearson correlation coefficient between the full-weighted total score in the CP5F questionnaire with the relativization of the scores on the CP5F questionnaire dimensions and job appraisal ratings’ mean when using the first six indicators specified by the supervisor of the analyzed job incumbent of -.202 indicates the existence of a psychological measuring instrument located at the limit between depending on circumstances (instrument category with values between r = .11 and r = .20) and likely to be useful (instrument category with values between r = .21 and r = .35) in the psychological selection of personnel. The correlation coefficients between the partial-weighted total score in the CP5F questionnaire with and without relativizing scores on the CP5F dimensions and the ratings’ mean of the first six performance indicators specified indicate an instrument likely to be useful, according to the same criteria previously mentioned. Saad, Carter, Rothenberg, & Israelson (1999) state that validity coefficients indicating that the instrument is likely to be helpful are typical for individual tests because they are more likely to measure or predict fewer aspects of job performance than batteries of tests.

The fact that the values of correlation coefficients obtained by using the partial-weighted total score in the CP5F questionnaire (with or without relativization of raw scores of the CP5F dimentions) as predictor variables are higher than those obtained by the full-weighted total score in the CP5F questionnaire (with or without relativization raw scores of CP5F) as predictor variables’ is in agreement with the conviction expressed by Meyer and Foster (2007) that partial-weighting is an algorithm that “mimics practice more closely”.

The results obtained in this research by using full- and partial-weighting approaches for validation of a job analysis tool without relativization of CP5F dimensions raw scores are relatively similar with those obtained by Meyer and Foster (2007).

The fairly low values of correlation coefficients between predictors and criterion obtained may be explained by the fact that criterion reliability, involving job performance appraisal by supervisors, is low because the differences in training, experience, and frame of reference among raters can determine different scores of overall job performance rating for the same person rated. The effect of criterion unreliability is the underestimation of criterion validity in the population of interest (SIOP, 2003). It should also be noted that the direct relationship between scores of the PPRF and criterion was not evaluated. The predictor variable had a mixed character and was obtained based on two variables: the coefficients obtained from PPRF dimensions ratings’ means in the case of analyzed job and the scores obtained on the CP5F questionnaire dimensions by the respective job incumbent participants.

In the context of the previously mentioned issues regarding the criterion validity of the PPRF translated and adapted to Romanian, the maximum validity coefficient value obtained, -.214, and the maximum explained variance of overall job performance ratings, 4.6%, indicate an acceptable criterion validity of the Romanian version of the PPRF.

The acceptable values of validity coefficients obtained are an argument for the decision of the present research on the proportion of supervisor experts in the total subsample of subject-matter experts, 26.89%.

The information on the concurrent criterion validity of the Romanian PPRF version obtained in the present research is of interest not only for this inventory but also for the same psychometric property of the CP5F questionnaire.

Conclusion

The personality assessment practice in organizations has been developed with the publication of research findings at the beginning of the tenth decade of the last century that showed that personality traits, specifically those from the Five-Factor Model of personality, predict employees’ job performance in various organizations. In this context, the concern for the development of the personality-based job analysis tools has appeared, including the PPRF. The objective of the present research was the translation and the adaptation of this inventory into Romanian. The efforts made in this regard tried to meet the international standards relating to the translation and the adaptation of standardized psychological assessment instruments, specifically those concerning translation and reverse translation of instruments for the development of the version in the target language, as well as those related to determining the psychometric properties of the version in the target language (reliability and validity).

The vast majority of the reliability coefficients values calculated for evaluating the inter-rater agreement were very good both for the dimensions and subdimensions of the inventory, and in the case of the concurrent criterion validity acceptable results were obtained using the partial-weighting approach for the validation of job analysis tools without the relativization of the raw scores in the CP5F questionnaire and through the use of partial- and full-weighting approaches with the relativization of raw scores in the CP5F questionnaire.

Another result of the present research was the development of the ideal personality profile of the operational NCO employed in the mobile structures of the Romanian Gendarmerie. The most important dimensions and subdimensions found for the efficient performance of the job tasks found are: emotional stability (with the highest ratings’ mean), conscientiousness (the second ratings’ mean), intellectance (thanks to the contribution of tendency to think things through subdimension), and the cooperative or collaborative work tendency subdimension (from the agreeableness dimension).

Implications of the Study

The Romanian version of the PPRF obtained can be used with a good degree of confidence in applied research, involving job analysis in organizations. The ideal personality profile of a specific job incumbent obtained through the Romanian version of the PPRF can be used both to ground through the empirical evidence of the results of psychological assessment of the respective personnel category or the candidates for the access in that job, and to establish the priority objectives of the psychological formation of that job incumbents. Also, the results obtained through the PPRF can be used for the criterion validation of the FFM personality measures in the specific job incumbent categories.

Limitations of the Study

A possible limitation of the present research is that for concurrent criterion validation of the Romanian PPRF version the criterion used was not surprised by a standardized assessment tool. But this feature is simultaneously an advantage: granting a supervisor the possibility to choose job performance indicators probably increased their degree of relevance to the analyzed job.

Conflict of Interest

The author of this article declares no conflict of interest.