If you want to be really good at something, you pretty much do it as frequently as possible for as long as possible, making headway little by little until you achieve the desired result. Training an artificial intelligence model isn’t terribly different, and that’s basically how a team at Sony AI — the tech giant’s research division — taught its agent Sophy to be the fastest in the world at Gran Turismo. Also much like humans, the better Sophy got at GT, the more it was kind of a jerk about it.
Sophy is a collaboration between Sony — the PlayStation people — and Polyphony Digital, the developers of Gran Turismo. (“So-phy” — get it?) Sophy is really fast, but unlike, say, the “AI” in Mario Kart, it’s not fast because it cheats. It plays by the same rules as human players, with the same tools. The difference is it learned to be good by running thousands of times on a network of PS4s in Sony’s cloud, while being reinforced positively or negatively depending on the outcomes of its behaviors.
Sophy’s skill developed over time, as you can see in the fascinating making-of video embedded below. And I promise you, it really is interesting. If you’re anything like me, your eyes glaze over and you feel a head cold coming on whenever somebody mentions the words “AI” or “machine learning” within earshot, but Sophy’s case is really intriguing because of how it got where it got.
For example, one engineer at Polyphony Digital says that when the Sony AI team first brought Sophy to the studio to race against real people, it was occasionally quick but really messy. It couldn’t drive in a straight line.
Then it went back to the drawing board to develop core competencies, until it became faster than the world’s top FIA Gran Turismo Championship players, albeit only on hot laps. The researchers brought Sophy back to Polyphony and pitted a team of four Sophy-controlled cars against four expert human drivers. The Sophy team took first place in two of the three, but totaled fewer points overall than the warm-bodied contingent and dropped the set.
The trouble is that Sophy learned how to be fast, but it didn’t know how to be fast without driving like a complete jackass. “I think we all underestimated how hard it would be to get the sportsmanship side of it right,” Peter Wurman, director and project lead at Sony AI said, “and [teach Sophy] to do that without being overly aggressive or overly timid in the face of competitors.” Cut to a clip of Sophy punting a pair of rivals braking way too late into the first chicane at Dragon Trail Seaside.
Once Sophy stopped doing things that made it look bad, it was almost as rapid in traffic as when it had the asphalt all to itself. Out of this newfound superhuman intuition, a few unconventional techniques it was employing became apparent. Gran Turismo producer and Polyphony Digital chief Kazunori Yamauchi shed light on one example of Sophy’s unique driving with GTPlanet:
“I am a racer and learned techniques on how to drive fast, like slow-in-fast-out,” Gran Turismo creator Kazunori Yamauchi said. “Gran Turismo Sophy does not learn that way. I think that after Sophy launches into the world, the textbooks about driving will have to be changed. For example, when Sophy goes into a curve it actually turns and brakes. Usually, when you go into a curve, the load is only on the two front tires, but Sophy has the load on three tires: two in the front and one in the rear as well. It allows the car to break as it is turning and is not something human beings would be able to do, conventionally. What ultimately happens is that it is driving fast-in, fast-out.”
I’d hedge that some of Sophy’s tricks that work in Gran Turismo likely wouldn’t pan out as well in an actual car. Nevertheless, Sophy is playing the game with a deeper understanding and a different philosophy than the best players in the world, and recording laps anywhere from half a second to 1.5 seconds faster than them as a result of it. The way it threads the needle at Dragon Trail’s infamous “death chicane,” for example, requires a level of precision and bravery all but the best drivers in the world could muster — and even they likely wouldn’t pull it off as consistently.
Supposedly Polyphony Digital will bring Sophy to Gran Turismo 7 in some fashion in a post-launch update — not just as a competitor but also, potentially, a driving coach. Personally, I think GT7 could use an AI with fast hands and decent racecraft, because that’s something GT’s computer-controlled adversaries have never had. I’d be happy enough with a watered-down version that’s just ordinary quick though; leave the galaxy-brain fast version to the professionals.