Forensic science in a process of transition

Abstract

In recent years, the use of forensic investigations has increased not only due to the advent of new technologies, but also an increased awareness of what forensics can offer. Forensic science has been thrown into a process of transition due to the introduction of objectivity as a concept derived from quality assurance, statistics and probabilistic reasoning. In addition to the basic requirement of a piece of evidence's scientific validity, other values have emerged aimed at improving the integration of the forensic chain from the scene of the crime to the court. For this reason, forensic science is looking towards a future where the standardisation of its disciplines will guarantee the reliability of forensic evidence, thereby facilitating a common language and shared understanding of significant findings, to support the legal process and the implementation of justice.

Resumen

En los últimos años es notable el incremento del uso de investigaciones forenses debido al advenimiento de las nuevas tecnologías y a una mayor conciencia de lo que la ciencia forense tiene que ofrecer. La introducción de la objetividad, como concepto derivado del aseguramiento de la calidad, la estadística y el razonamiento probabilístico han situado a la ciencia forense en proceso de transición. Partiendo de la exigencia de la validez científica de la prueba, aparecen valores añadidos encaminados a la mejora de la integración de la cadena forense, desde el lugar de los hechos hasta el juzgado. Por estas razones, la ciencia forense mira hacia un futuro en el que la normalización de sus disciplinas sea garantía de la fiabilidad de la prueba, lo que permitirá un lenguaje común y la comprensión compartida de los resultados significativos, para asistir a los procesos judiciales y a la aplicación de la ley.

Introduction

I refer to the Letter to the Editor recently published in this journal calling attention to individualisation and the need for a commitment to collaborating with a more open mind, as this significantly contributes to the forensic process.¹ With this article, I offer a view of events and issues, such as the one referred to, that define forensic science as a sector in transition. Diverse circumstances have led to a change in the role of forensic laboratories. Now they are able to investigate more and a greater variety of evidence, extracting more information from less material. They have moved up from the supporting role to the leading actor in numerous investigations. They provide quick and reliable information on crime scenes, suspects and victims, and that is where the experts performing specialist tests in the different fields of forensic science are directing their efforts.²

The test methods are categorised as subjective and objective. Subjective tests are based on experience, heuristic techniques and intuition. Such methods are useful, but they have a different type of value and they have to be interpreted in a specific way. The most important advance in the history of forensic science was the introduction of objective methods and the concept of probability to measure the degree of uncertainty of a result.³

This objectivity has to do with validated methodologies, verified and calibrated equipment, detection and quantification thresholds, certified reference materials, estimation of uncertainty, and accuracy and precision in measurements. It is also related to guaranteeing the repeatability and reproducibility of the results, the selectivity, the specificity and the traceability of the procedures, taking part in inter-laboratory comparison exercises and being subject to internal and external audits carried out by accredited experts and by national assay accreditation organisations. Some scientists have summarised all these requirements into two principles: transparency and quality control.⁴

The end users—both existing and potential—the press and the public are now more aware of the extent of forensic capabilities and that in turn is generating an increasing demand. Forensic investigation has gradually taken on a more central and high profile role and is becoming a useful tool for the State Security Forces and Law Enforcement Bodies in the defence of the nation, as well as for others responsible for maintaining justice, social order and security.²

The current situation in forensic science

One of the clearest and most important trends in forensic science is its remarkable growth in the last 15 years. This is the result of three main factors: the introduction of new technological capabilities; the increase in awareness among users of forensic services of the value of forensic science efficiency; and the arrival, not yet in Spain, but in other European countries, of new clients outside the traditional forensic field.

The increased use of forensic investigations is not only due to the advent of new technologies but also the fact that the general population has more information about what forensic science can offer.

The rapid development of analytical methods has led to great importance being given to the inherent “quality” required to measure the data; this hand in hand with the two essential criteria of reliability and utility, meaning that the analytical results need to allow reliable decisions to be made.²

Until twenty or thirty years ago, forensic science had more in common with the arts and crafts than with a mature science, and some areas were even still in the pre-scientific era, as reflected in the National Academy of Sciences (NAS) report, “Strengthening Forensic Science in the United States: A Path Forward” (2009).⁵ Obviously, if the interpretation made by the forensic scientists is lacking in objectivity or a strong scientific basis, the value of the information and the interpretation provided by the forensic laboratories decreases considerably.

Forensic scientific development has not been as fast as it should have been precisely because of this culture of the artistic and the manual (already on the decline in the vast majority of forensic laboratories), combined with the demand for better performance from laboratories and the peculiarity that some scientists were not always in the habit of sharing or organising their knowledge acquired over time, preferring to keep it firmly to themselves.⁶

The above-mentioned report presents a critical view of the weaknesses in the scientific foundations of a series of forensic disciplines routinely used in the criminal justice system and, although not exempt from extreme factors, it was undoubtedly the beginning of a profound change to advance the forensic disciplines, especially in terms of improving the systems and the organisation of structures, better qualification of personnel, generalised adoption of best practices in terms of uniformity and applicability, and the obligatory nature of certification and accreditation programmes.

Some time later, another scientific study by Laurin J.E. on how forensic science is developed and applied in criminal cases focused on the set of known problems related to the significance of integrity and scientific reliability in the context of investigative decisions.⁷

Subsequent documents and international conferences, such as “The Future of Forensic and Crime Scene Science”, “Improving the effectiveness, efficiency, quality, and operations of forensic science laboratories” and “Forensic science on trial” have argued that a third era of change in forensic science is to come, identifying the key factors for forensic innovation: increased capacity (“more”) and quality (“better”), reduced time (“faster”) and costs (“cheaper”) and improved integration of the steps in the chain (“the easier the safer”).

The 2009 document, which should be considered as the most important and influential 21st century publication on the future of forensic science, emphasises the lack of a robust science due to factors such as the lack of training and continuous training of experts, and the lack of rigour in the requirements for certifications and accreditation programmes and compliance with standards and effective control.

This continues to affect the practice of forensic scientists themselves and those who interact with forensic disciplines, including prosecutors, judges and lawyers.⁸

Moreover, evidence suggests that the level of development and scientific evaluation varies substantially among the different forensic disciplines, and, in some, there is a notable lack of published studies with peer reviews establishing the scientific basis and the validity of certain methods used in forensic science.⁹

Scientific validity in forensic science

The concept of scientific validity plays an important role in the legal system. Within the framework of a judicial process, the expert evidence requires very high scientific standards and the testimony of the expert must be based on reliable and scientifically valid and accepted methods.

The September 2016 report of the United States President's Council of Advisors on Science and Technology (PCAST) differentiates between scientific or foundational validity and validity as applied. Scientific or foundational validity is the reliability of a method to be repeatable, reproducible and accurate in a scientific environment. Validity as applied is the reliability of the method in practice, which includes the risk of human error.³

The report is not aimed at all activities associated with forensic science, but addresses those based on the comparison of characteristics, and although not exempt from criticism by many experts, it is a call for forensic analysis methods to be validated under real-case conditions.

They reviewed the available empirical evidence related to seven tests: single-source and simple-mixture DNA analysis; DNA analysis of complex-mixture samples; bite analysis; fingerprint analysis; firearms analysis; footwear impression analysis; and hair analysis. On the basis of their review and evaluation of the available studies, they arrived at the problematic conclusion that only two of the seven (single-source and simple-mixture DNA analysis and fingerprint analysis) had foundational validity.

Summed up very briefly, they suggest that other analyses, such as bite marks, did not meet the standards to achieve scientific validity. The DNA analyses maintained foundational validity and validity as applied, but not accurately for complex mixtures of several components in which each contributor participated in different proportions. Brands of tyres and shoes required more empirical evidence to support their scientific validity, and for fingerprints, although foundationally valid, validity as applied needed to be established, especially due to confirmation bias, contextual bias and lack of expert competence.³

In particular, it should be a requirement that expert testimony on forensic science be the product of “reliable principles and methods” which have been “reliably applied [...] to the facts of the case”. Thus, for example, a certain toxicological analysis may be scientifically valid, but if the expert who analyses it does not do so with the appropriate standards, it is not valid as applied.

Five factors have to be considered when evaluating the validity of a method: (1) whether or not the theory or technique can be (and has been) proven; (2) whether or not the theory or technique has been subject to peer review and publication; (3) the existence and maintenance of rules that control the operation of the technique; (4) the degree of acceptance of a scientific technique within a relevant scientific community; and (5) the known or potential error rate of a particular scientific technique.¹⁰ On this last point, it is worth highlighting the numerous notifications and publications¹¹ coming by way of the media from the United States, especially in the last two years, about forensic cases being reviewed despite there already being a conviction.

The so-called “Innocent Project”, which was launched by lawyers Barry Scheck and Peter Neufeld at the Cardozo School of Law to demonstrate the innocence of a good number of convicts through DNA tests, has highlighted not only the fragility of traditional means of proof, such as testimonies, especially of witnesses (70%), and confessions, but also of the scientific evidence on which some of these convictions were based.¹²

From another point of view, error rates in some centres, such as the Netherlands Forensic Institute (NFI), are compiled and sometimes published; this treatment is useful for quality improvement and benchmarking, and contributes to an open research culture that promotes public confidence.¹³

Statements such as, “my conclusions are 100 percent certain”, have “zero”, “essentially zero”, “small”, “insignificant”, “minimal” or “microscopic”, or the possibility of error is so remote as to be “practically impossible” cannot be scientifically supported. All laboratory tests and feature-comparison analyses have error rates, even highly automated tests.

The PCAST publication which, as mentioned, analysed the scientific validity as applied and foundational validity for different types of tests, recommends calculating and including this data in forensic reports.³

In some disciplines, the error rates are simply not known, while in other cases, a particular technique could have well-known error rates and, conceptually, you can aim to work out how to do it better or talk about the uncertainty. Either way, as regards the concept of error, the most important thing is to know about it and then to report it.¹⁴

Statistical evidence and probabilistic reasoning today play an important role in the expansion of criminal investigations, prosecutions and trials, particularly in relation to the scientific evidence (including DNA) produced by the experts.¹⁵

The introduction of a mathematical approach developed and derived from the application of Bayes’ theorem represents a profound change in terms of evaluating the meaning of scientific evidence, as it is applied to the set of circumstances related to a supposed criminal act.¹⁶

Both the 2009 report and others have pointed out that, essentially, forensic science does not yet have a well-developed “research culture”.

It is important to emphasise that this research culture includes principles such as: (1) the methods should be presumed unreliable until their foundational validity has been established on the basis of empirical evidence; and (2) even then, scientific questioning and review of the methods should follow on a continuous basis. The idea that extensive “experience” in case work can be a substitute for the empirical studies of scientific validity is unacceptable.

Having a system that ensures quality is essential to obtain reliable results and reduce the probability of error, as it provides the infrastructure to promote qualified performance improvement. The ultimate objectives of a quality system are to minimise the existence of errors and develop and foster an environment for improving processes and services.

A well-developed quality programme provides quality products or services, and it can be stated that the opposite, i.e. not developing or not following quality control practices, can lead to unreliable results. It needs a strong commitment, with the use of validated and documented protocols, tested reagents, calibrated equipment, appropriate control samples, detailed and documented documentation requirements, and an independent review of operations, results and interpretations in order to obtain reliable results. If these principles and categories of a quality system are maintained, the desired objectives can be met.¹⁰ It should be mandatory for laboratories to have validation data to be used in the interpretation of report results and which are available to the judicial system.¹⁷

The science used in the forensic process, understood as a process that starts at the site where the incident took place (crime scene) and finishes in the court (or, where applicable, oral hearing), needs to be robust and objectively reliable; in terms of whether or not the validation of certain methods which have been used for a long time is still fit for purpose, and with regard to the new methods related to new technologies. However, it is also naive to think that “all forensic science” must be probabilistically quantifiable in a precise way, just as it is also limiting to think that forensic science is based only on measurement.

The fact that a test works scientifically does not necessarily mean that it has a forensic significance, as application criteria have to be taken into account. “Forensic validation” will therefore be far behind the availability of new technology and methods validated in other areas.¹⁰

All this is in line with another concept which also has to be considered in forensic processes, that of interoperability. A method that we use in one country must be transferable to another so that the data can be shared. The database structures need to be such that the information can be shared with common technical standards and quality criteria.

The need for standardisation of forensic science in the various disciplines is therefore patent, with the understanding that such standardisation embraces the entire aforementioned forensic process.

That need was declared openly by European bodies such as the European Network of Forensic Science Institutes (ENFSI) which, in its 2004 document, “Performance Based Standards for Forensic Science Practitioners”, had already compiled the necessary standards in the different phases of the process.¹⁸ It was also recognised by bodies from the US, Australia and other countries, eventually leading in 2012 to the proposal to form an ISO Technical Committee, ISO/TC 272, which has 22 participatory members, including Spain. This group developed ISO 18385 “Minimising the risk of human DNA contamination in products used to collect, store and analyse biological material for forensic purposes-requirements”, in force since February 2016, and is in charge of standardisation and guidance in the field of forensic science, which includes the development of standards that pertain to laboratory and field-based forensic science techniques and methodology in broad general areas such as the detection and collection of physical evidence, the subsequent analysis and interpretation of the evidence, and the reporting of results and findings which helps promote international standardisation and the exchange of information in forensic matters.¹⁹

The added value of forensic science

The concepts of effectiveness, efficiency and value vary depending on the organisation within which they have been defined. The effectiveness of the police will be different from the effectiveness of forensic doctors or toxicologists, which in turn differs from the effectiveness of the criminal justice system, and so on.¹⁷

To what extent do the results of the forensic investigation answer the question we are asking? To what extent is the question related to the crime investigated? How robust and reliable are the results of the forensic investigation? The first question refers to the strength of the evidence or the so-called probative value of the evidence generated by the investigation; the second indicates the relevance of the investigation; and the third relates to the quality of the forensic information.⁸

When experts in the various forensic disciplines present the results of examinations, tests or measurements in reports or testimonies, what types of quantitative or qualitative statements should they provide to indicate the accuracy of the measurements or observations and the significance of these findings?

These statements can describe the accuracy of the measurements (or, conversely, the uncertainty of the measurement), the weight of the evidence (the degree to which the measurements or observations support particular conclusions) or the likelihood or certainty of the conclusions, and should be based on: (1) the existence of a relevant database that describes characteristics, images, observed data and experimental results; (2) a statistical model that accurately assesses the strength of the inference in question or describes the process that gives rise to the data linked to the question posed; (3) information on the variability and errors in the measurements or in the statistics or inferences derived from the measurements; and (4) a statistical statement on the probative value of any comparison or calculations made; for example, how rare or common is it for a positive association to be observed when two indications arise from the same or different sources⁹?

Estimates of real, but unknown, values in a population are based on the study of samples which are subject to sampling uncertainty. It should therefore be standard practice to report the degree of accuracy of such estimates, as well as the range the choice of coverage requires, for example 95% or a credible 99% interval.²⁰ This is important, as forensic science in particular, with its intrinsically less controllable variables, is more about probabilities than certainties.

Obviously, some methods are very standardised, and if the samples are collected correctly and are not mixed, there is a very high probability that the analytical result will be correct. This is where forensic science is equivalent to the clinical model, where set samples are exposed to a testing regime and the results are accepted at face value. However, experts should resist the temptation to accept that the clinical model is right for forensic science and need to look for organisational models that adapt to the particular challenges of forensic science.

Reducing forensic science to the use of advanced technology devices that produce results which are delivered under the name of evidence will never provide an effective aid to justice, and even runs the risk of compromising it.

Existing accreditation systems, such as those that conform to ISO17025, confirm the quality system in the laboratory, but much more is needed for the general forensic process. More research needs to be carried out on the effectiveness of the processes from the crime scene to the court, not just the processes performed strictly in the laboratory.¹⁰

Full accreditation guarantees that the quality procedures are being met, that the error rates are known and monitored and that the methods comply with the state of the art. However, not being accredited does not mean that the results are not valid, as long as they are validated for the intended purpose,²⁰ when it can be stated that, “if through the control of samples, proficiency tests, blind tests and validation it is demonstrated that the forensic methodology is extremely robust and leads to almost zero random distribution without systematic errors, excellent quality has been achieved”.

Nowadays, it goes further, with new points of view being presented on the added value of forensic science, and great challenges are being formulated for future forensic innovation. These general challenges have been defined not from the perspective of solving current problems, but from a “what if” point of view; in other words, identifying new sciences and technologies that could significantly advance in the areas of forensic expertise. Advances such as these, which would allow forensic science to provide answers in criminal investigations where answers are not currently possible, will be made with the aim of maximising the informative value of forensic investigations, also an important aspect of the evaluation and interpretation of cases.⁸

A view to the future

In criminal proceedings, the role of the forensic scientist is to help the court to administer justice through the provision of unbiased expert opinions and explanations. This requires independent opinions to be given, derived from reliable procedures, which can be understood and evaluated by the decision maker.²¹

Forensic laboratories can increase the value of the information they provide in at least three ways: (1) increasing reliability through objectivity and scientific foundation; (2) providing more information at the level of “activity”, i.e. information that reveals how the different pieces fit together; and (3) developing tools and methods in the investigation of trace evidence which were not previously available. Currently, this is becoming “the quest” for forensic science, although they must go hand in hand with research and development programmes and a laboratory strategy that increases productivity and efficiency.²

The quality of forensic science is believed to be high, but there are still problems, such as the gaps between the different steps that make up the forensic process and the criteria necessary to be able to rely on the forensic evidence, reports and conclusions; which is even more complicated when they come from other countries.

The personal competence of the forensic expert is considered very important, as they will be in the group of experts or appear as the author of a report. The expert has to be able to explain the tests and methods in a way that is understandable to everyone in the forensic chain, and this should be part of their training. Additionally, members of the judicial system (judges, prosecutors) need to have knowledge in order to understand the forensic experts and their reports, and vice versa; the experts need to understand the judicial system.

One of the challenges that can arise is when experts have to appear in international cases, where not only are there language differences, but also differences between judicial systems. Particularly in these cases, operating procedures have to be impeccable and properly accredited in order to ensure that any discussion is not based on the process. We also need to be able to provide proof of expert training and, although possibly difficult to achieve, an international (EU) registry could be valuable in this instance.

The best solution would be to organise accreditation and certification in a similar way throughout Europe. However, with all the different forensic fields, this is not going to happen overnight. Nonetheless, the proposal of the ENFSI Chair in Amsterdam in 2016 was to begin in certain fields, a first step already having been taken in the harmonisation of forensic science with the Best Practice Manuals (BPM) published for the different specialist areas in 2015; although voluntary, they are a good starting point. Application of the BPM, not very widespread to date, requires both external pressure (from the EU or members of the judicial system) and internal pressure (experts and the laboratories themselves) on the forensic laboratories.

Equally important are proficiency tests. One step forward would be to perform proficiency tests throughout the forensic process, rather than just for a particular field. However, proficiency tests are not enough, as not all personnel in a laboratory would necessarily have to perform the inter-laboratory comparison exercise; therefore, it would also be important to individually certify the different experts.²²

Many years ago, discussions on how to assess the probative value were less structured and formalised than they are today and there was greater diversity of opinions. Nowadays, although there is still some disagreement, in general the number of scientists and legal experts who remain sceptical about the measure of evidence, the Bayes factor (often called the likelihood ratio), has reduced significantly. This development has been encouraged and supported by the fact that the rules and recommendations in different forensic disciplines substantially back the use of this measure of probative value. However, progress towards a more generalised consensus on the principles is still fragile, as it brings up new problems about which points of view differ.^23–26

Bayesian inference may be the only reliable tool for probabilistic inference and answers the most difficult questions related to reporting in forensic science. However, in the courts, to prove statements about the facts, a minimum of philosophical realism is needed, as some European legal experts are demanding. Therefore, concepts such as truth, certainty, doubt, evidence and belief need to be clarified under such a basic philosophical perspective, or at least defined according to the philosophical conception on which they are based, and this has yet to be done in forensic guidelines.²⁷

Although it is an indispensable condition to be based on a demonstrable scientific foundation and on the idea of standardisation, we know that the fundamental restriction in forensic science, as in science in general, is that the available information is limited and incomplete. This means that it is impossible to make categorical conclusions about events of legal interest. The estimation of uncertainty implies the need to determine the degree of belief that can be assigned to a particular event or proposition that is already uncertain, such as whether or not the suspect is the donor of the recovered trace, or the toxic substance the cause of death, and it is in this context that inferential sciences, including statistics, can offer a valuable and substantial approach.²⁸

The PCAST report referred to includes a series of recommendations. In general, the report underlines the importance of considering the criteria for the admissibility of expert opinion, and recommends the preparation of a best practice manual and development of adequate training programmes from the organisations. It also stresses the importance of investing in a strategic plan for forensic R&D.

Much has been said about improving the quality of the scientific work which supports the report issued by the forensic expert(s), but little has been done to meet the challenge of ensuring that the reports capture both the value and the limitations of the findings, expressed in a way that is understandable to a wide range of users, including the legal experts. Without a common language and a shared understanding of the significant results, forensic science as a recognised discipline will not progress and it will not be able to assist the judicial processes or the application of the law in the processing of crimes.²⁹

This is the framework that will define forensic science in the future.

Conflict of interests

The authors declare that they have no conflicts of interest.