Standards Development

Scope

Variability is present in all sensory tests. Sensory test variability is of particular concern because humans are the essential source of data. Concern about variability is expressed in discussions about accuracy, precision, repeatability, replication and reproducibility in sensory testing. These terms will be defined as they apply to sensory testing.

The sources of variation in sensory testing will be identified. With these terms precision, accuracy, repeatability, replication and reproducibility, defined along with identified sources of variability in sensory testing, this document will cover the elements of each can be achieved in sensory tests.

Further, how to determine repeatability, replication, and reproducibility in the three main types of sensory testing, descriptive analysis, discrimination, and consumer tests, will be covered, along with the use and benefits of determining each for sensory tests.

Purpose

Precision, accuracy, repeatability, replication and reproducibility are all challenging in sensory. These concepts are most important when a sensory finding is challenged, either by a business team, a legal team, or a regulatory body. Precision and accuracy depend on having an objectively defined reference value. With no such reference in most sensory testing, repeatability, replication, and reproducibility must be determined over repeat testing. At a minimum, repeatability must be established if a sensory finding is going to the basis for resource intensive decisions, or if the finding is going to be challenged. However, replication provides even stronger evidence that sensory data are reliable support for making business decisions or a claim. Reproducibility, while almost never established in sensory testing, provides the sensory scientist with the confidence that the finding is generalizable beyond the specific conditions of a single sensory test.

Repeatability is established regularly in most forms of quantitative descriptive analysis usually by repeated trials in the same session. Repeatability is not usually established in discrimination testing, unless the same assessor completes more than one trial in a session. Repeated trials in a discrimination test are usually done to increase the power of the test, rather than determine the consistency of each assessor’s response in repeated trials. Given the resources needed to complete a consumer test, consumer tests are rarely repeated. However, most sensory scientists compare the results of a single consumer product test with the results of historical tests similar in both the respondents and the products tested.

Replication is important in sensory functions that are organized globally. Global sensory functions operate more efficiently if the findings of one panel in one geography are similar to the results of a different panel in a different country. Cross-site testing to establish similar findings means testing does not need to be repeated in different geographies. Results replication allows the company to test only once and use those findings across different R&D and manufacturing sites. Ideally, within specific conditions, a descriptive analysis result found by one panel in one country should yield similar results found by another panel in a different country. How sensory scientists might determine the replicability of descriptive analysis panels in different geographies will be provided.

Replication in discrimination tests and consumer tests presents unique challenges. At a minimum, discrimination test replication provides evidence that the finding is generalizable beyond a specific single test consisting of a specific group of assessors. Results replication in discrimination tests could strengthen support for sensory claims and can help weather legal challenges.

Reproducibility of sensory results may be a long-term goal for sensory profession. If achieved, results that are reproducible may be used to demonstrate a sensory result is generalizable beyond the specific conditions of a single test.

Addressing repeatability, replication and reproducibility will augment the usability of sensory findings both within industries and outside of the sensory profession itself along with increase the credibility of sensory test results.