Open-Ended Digital Twins: How We Think About Validation
How we validate open-ended twins at Outset: hold-out design, deterministic yardsticks, and measured reference levels.
Abstract
A digital twin is a system that simulates an individual person in their ability to respond to questions and conversations. Nearly all public validation of these systems has happened on closed-ended instruments: surveys, scales, discrete choices. We assert that the more consequential frontier is open-ended responding, and measurement of twins' performance directly in that frame. From the closed-ended and open-ended literatures we derive five commitments for designing and validating our digital twin generation architecture: information-maximizing hold-out processes, determinism underlying validation instruments, thoughtful incorporation of measured reference levels, and grading twins on the distribution of responses their person draws from rather than the string the person happened to say.