Article published by José Alberto León Alonso in El Diario de Avisos on 01/18/15.
Gabriel Táuriz, in his article “Selection of public employees and competitive examinations: what does the evidence tell us?”, on the Politikon blog, analyzes in detail our system for selecting public employees through competitive examinations. Personnel selection can have a great effect on society if we think about the selection of public employees. The performance of judges, prosecutors, teachers, doctors, police, health inspectors, tax inspectors, and labor inspectors has great importance for the performance of a country. And, as Hunter and Schmidt concluded in their study “Intelligence and job performance: Economic and social implications”, there are large differences between individuals in job performance. Are competitive examinations the best method to access public service? Although it may seem incredible, no one in Spain has studied its validity as a selection method.
To determine if a personnel selection system is good, the so-called face validity is used, which measures the degree of effectiveness with which an applicant's future job performance can be predicted or forecast based on test scores. It is used as a prediction indicator for future employment and is expressed as a percentage of success in the selection. When the different prediction methods are contrasted, the conclusion is that the best predictor of job performance is intelligence, measured by psychometric intelligence tests. The predictive validity of intelligence tests is around 65%. That is, 65% of the candidates selected through an intelligence test ended up doing a good job once hired. This validity is greater the more complex the position and decreases in less complex positions. In highly complex positions the validity amounts to around 75%; In those of low complexity it is around 55%, and in very low complexity positions the validity still remains around 40%.
There are other good instruments with a validity that is close to that of intelligence tests. For example, simulations of the work carried out in the position (work sample test) have a validity of 55%. And structured behavioral description interviews, where questions are made that evaluate the candidate's behavior in aspects important to the position, have a validity of around 50%. The difference is in the cost of each tool. There are multiple psychometric tests on the market capable of evaluating intelligence for little money, so the cost of implementation is low, and hundreds of candidates can be evaluated at the same time in a short period of time. Nothing to do with the complexity of truthfully simulating the tasks of the position, specifically developed and at high costs. So therefore, selection through intelligence tests is better not only for pure predictive power but also for the low costs of its application, both for the candidate (the “training” for the test does not last more than a few weeks since, after all, one has the intelligence that one has and it does not improve over time) and for the evaluating organization.
As for the competitive examinations, their effectiveness has not been studied in the only developed country in which they are used, Spain, since everyone seems to believe that they are the best possible system. The “job knowledge tests” are what could most closely resemble a competitive examination, and their validity is around 45%, twenty points lower than that of intelligence tests. Furthermore, this validity basically depends on the similarity between the content of the exam and the tasks and functions that will actually be performed in the position. If the knowledge examined does not resemble that necessary for the work to be carried out, the validity of these exams will be greatly reduced. And no one is aware that in most of our oppositions, content not related to the position itself, but of a general nature, is examined. So it is not unreasonable to think that competitive exams will actually have a predictive validity much lower than the 45% indicated for job knowledge tests. If we add to this that the opposition requires complete dedication from the candidate for a long time, which represents an enormous cost for what is, essentially, a selective process in which the majority of candidates will be rejected, it is not surprising that Táuriz concludes that the social and personal cost of the opposition system is enormous. That time could be used to work or to train and develop new skills. Furthermore, it has an undesirable effect on equality of opportunities, given that only those who can live on their own for a long period of time can afford to study a demanding opposition. The costs for society are similar, derived from the loss of productivity associated with having thousands of individuals locked up memorizing topics instead of working, developing skills, etc.
Even worse is the selection based on merit. These are found within what is known in scientific literature as “training and experience evaluations” (T&E) by “point method”. The predictive validity of these instruments is only 11%. Despite its popularity and supposed “justice”, this data invites us to rule out its use. They are a poor gauge of future performance at work.
Thus, I conclude, like Táuriz, that the logical thing would be to abandon the competitive examinations and opt for selection through intelligence tests, with greater empirical and scientific support. The oppositions cost candidates a lot of time, they are regressive for those without financial support behind them, and they offer a much lower capacity to predict future job performance than we would have if we applied an intelligence test. The selection procedure that has proven to be the best should be used, instead of being the only European country that continues to use a 19th century selection system such as competitive examinations. But, like so many things, it is not done. Country.
