Thanks to Raj Samuel for this: an Xbox application that’ll bring together real and virtual characters, even allowing crossovers of objects between the real and virtual spaces.
What’s most interesting here isn’t the flashy passing-a-sheet-of-A4 across the divide, or even the high-resolution little boy that looks more photographed than drawn. It’s the way subtle cues, like sweep of eye or droop of shoulders, have been incorporated into the communication between human and avatar. Milo’s head-hanging when she asks if he’s done his homework, Claire’s jump towards the screen to pick up the goggles – that’s 70% of what matters in communication, the nonvocal things. And yet that 70% is being driven by what must be a fairly small set of cues.
Two-thirds of the entire repertoire of human empathy has been reduced to a relatively small number of facial/gestural ‘primitives’, the same way incredibly complex graphics can grow from a few formulae involving triangles and mandelbrots. And with just those few coded nudges and winks, these virtual characters can give more empathy and companionship than many people – the old, poor, or lonely – get in real life.
The real test will come when shoot-em-up games get to this level. When a character’s coding makes it as intelligent and sentient as, say, a mouse (computer game characters today already have considerably more neurons than ants) do we have a right to just shoot millions of them indiscriminately? I mean, the virtual Milo is just the first primitive example, but even so I’d think twice if told to hold his head under the surface of that pond.