The Trolley Problem Test
The message from the head of Stanford University’s Department of Electronic Engineering was straightforward. It urged Yoo Jae-won not to take the matter to heart; the people stirring up trouble simply did not understand what they were talking about. Given the context, “those people” clearly referred not to netizens or journalists, but to professors at Harvard University. An unspoken rivalry had always existed between Stanford and Harvard—Harvard as the powerhouse of the East, Stanford as the standard-bearer of the West. While Caltech or MIT might scoff at the comparison, both institutions carried immense pride in their own standing.
The department head’s message also offered an analysis: the ethical controversy had been inflated by a sense of crisis and defensiveness that arose when AI, a new field, had advanced dramatically without their knowledge. In short, the root of the ethical debate was fear born of ignorance. Once people gained even a basic understanding of how the system worked, the anxiety would resolve itself.
“That makes sense,” Yoo Jae-won murmured.
Stanford stood firmly behind him and promised full support if he needed it. The message had not come from the department head alone; it effectively represented Stanford University’s official stance. In recent years, Yoo Jae-won and ID Group had become the university’s greatest source of pride. Stanford’s ethos was built on a spirit of bold challenge, and its commencement speeches often included the line: “If you graduates start companies, the school will stand behind you completely.” Many of Silicon Valley’s most prominent firms had been founded by Stanford alumni. Intel and AMD traced their roots to Fairchild Semiconductor, itself founded by Stanford graduates. From that point onward, nearly every major name in the valley carried the Stanford pedigree.
In the past few years, Yoo Jae-won and ID Group had become Stanford’s new icons. No other Stanford-founded enterprise had achieved greater fame. Stanford graduates now listed ID Group as their top choice for employment. Beyond that, Yoo Jae-won had given the university substantial support—regularly supplying the latest computer systems and donating enormous sums each year through ID Group’s social contribution budget. The scholarship programs run by the ID Foundation were equally generous. For Stanford, the fact that Harvard had been the first to attack such a figure was intolerable.
“Fear born of ignorance, is it?”
Yoo Jae-won had already been considering how to respond to the ethical controversy that had begun at Harvard, and he was not particularly worried. The same issue had arisen in his previous life, and the reference materials he had prepared outlined exactly how a pioneer in the field should handle it. The matter was even recorded in his long-neglected master plan. Still, the department head’s phrase—“fear born of ignorance”—lingered in his mind. People felt uneasy about AI Gold because they did not understand how it operated. The solution, then, was to demonstrate that it functioned in a highly rational manner.
Yoo Jae-won had originally planned to create a trolley-problem scenario and deploy F2 within it. He believed that if the public saw how F2 navigated such a complex situation, at least half of them would be convinced. After receiving the department head’s message, however, he realized a more fundamental approach was also necessary.
“Should I publish a paper on Gold?”
Even the Stanford professors who supported him did not fully understand how Gold worked. When he had completed Gold, he had neither disclosed the underlying technology nor written any papers about it. That was typical for corporations; when releasing new products, companies filed patents but rarely published papers, since papers only helped competitors. If Gold had been developed in a Stanford laboratory, dozens—perhaps hundreds—of papers would have followed. But corporate research divisions were closed organizations, and academic publications accompanying new technologies were rare. ID Group was no exception. The only researchers who occasionally published were Dr. Lisa Su of ID Electronics and Dr. Lee Jong-hyo.
In the twentieth century, universities and academic societies had led technological development. In the twenty-first, corporate laboratories had taken that role. Those laboratories guarded their work for security reasons, which in turn fueled conspiracy theories. The questions currently being directed at Yoo Jae-won were minor by comparison.
If publishing a paper could bring many academics onto his side the way Stanford already was, there would be no reason to lose the battle for public opinion. AI research would accelerate as well. Most importantly, he could preempt unnecessary conflict by dispelling conspiracy theories rooted in ignorance while simultaneously securing allies in academia.
Having organized his thoughts, Yoo Jae-won immediately replied to the department head. He thanked him for his concern, stated that he already had a response prepared, and added that he intended to write and publish a paper on the machine learning that had been central to AI Gold’s completion. He asked for guidance and oversight.
The reply came at once.
“A paper? Really? We would be delighted!”
The Stanford faculty, including the dean and professors in IT-related fields, had been intensely curious about the structure of the AI application revealed on Real Camera, as well as the underlying system and operating principles of Gold itself. The prospect of Yoo Jae-won personally writing a paper on the subject felt like cool water to a man dying of thirst. They would not even have to wait for the paper’s release; “guidance and oversight” meant they would collaborate on writing it. The opportunity to learn the secrets of the world’s most talked-about AI was irresistible to a department head whose intellectual curiosity was already extraordinary.
“Please prepare the field test as well.”
One of Yoo Jae-won’s core beliefs was that a hundred explanations were worth less than a single demonstration. While the paper would address experts, the public needed to see F2 placed in a trolley-problem scenario and witness the outcome.
“Understood. I’ll make the arrangements.”
Kim Dae-seok responded immediately. As a company grew larger, fixed costs rose, but the advantage was that a single word from the top could set complex matters in motion without further discussion.
A few weeks later, the test track at ID High Tech’s research facility on the outskirts of Seattle was crowded with reporters from across North America and around the world. It was the day of F2’s public field test.
“Good afternoon. Thank you all for traveling so far to see Lightning Bolt’s F2. I am President Bolt.”
President Bolt of Lightning Bolt, not Yoo Jae-won, hosted the event. He had poured the most effort into completing F2. Although Yoo Jae-won had handled the autonomous-driving functions, F2 remained President Bolt’s masterpiece. Cameras focused on him as he stood before the press, and flashes erupted, yet his expression remained perfectly calm. He had absolute confidence in F2. Countless internal tests—from simulator runs to real-world trials—had all produced stable results. When the AI ethics controversy first erupted, he had thought the reaction was making a mountain out of a molehill.
“Before we begin the test, let me first explain the fundamental principles programmed into the autonomous-driving system.”
Just as he had once presented to the Real Camera crew, President Bolt set up a projector screen for the reporters and began a slideshow. The screen did not display anything resembling the Three Laws of Robotics. Even in the mid-twenty-first century of his previous life, when android robots had become widespread, no universal ethical code had ever been agreed upon. Instead, far stricter regulations governed both AI and robots.
- All AI developed by ID Group shall comply with current law.
Current law. Law represented the minimum standard that every member of society was required to uphold. Since AI was also a component of society, it followed the law. F2’s autonomous-driving AI therefore adhered to U.S. law and, naturally, fully complied with American traffic regulations.
- All AI developed by ID Group shall place human life as its highest priority.
Even so, situations could arise that current law alone could not resolve—one of them being the trolley problem. In such cases, F2’s guiding principle was that human life came first.
“Does that mean the driver’s life or the pedestrian’s life?” a reporter with little patience called out.
“Both,” President Bolt answered without blinking.
“Both? Wasn’t the whole point to choose one or the other? We didn’t come all this way to listen to sophistry.”
The reporter’s tone had been rude from the start, and the follow-up question matched it.
“Ah, I see we have someone here who confuses a logical thought experiment with reality. You may not be aware, but here at the ID High Tech laboratory, our world-class research teams produce extraordinary inventions every single day. Most of those inventions, however, never leave the lab, because they only function in an environment where every variable is fixed—like the lab itself.”
President Bolt continued without the slightest change in expression.
“The trolley problem is the same. In reality, the probability of such an event occurring is close to zero. It can only exist in an imaginary world where every variable has been completely eliminated. F2, on the other hand, is a future means of transportation that will drive on real roads. Those roads exist in reality, where countless variables change in real time. Our F2 considers all of those variables and makes the best possible decision.”
When he finished, President Bolt surveyed the crowd. Fewer than one in ten faces showed understanding. Half looked bored; the other half looked unconvinced.
“Our Chairman Yoo has a saying: showing once is better than explaining a hundred times. So let me demonstrate.”
At the mention of a demonstration, curiosity finally appeared on the reporters’ faces.
“I am not Chairman Yoo, however. He could have convinced you all in a single demonstration. I know I have not yet reached that level. Therefore, I will show you not once, but as many times as it takes until you are satisfied.”
F2 was already waiting on the track. Roughly three hundred meters ahead, a railroad crossing and five mannequins had been arranged like a stage set—the trolley-problem variant. The mannequins stood on boards fitted with wireless-controlled electric motors and wheels. Solid concrete structures had been erected like walls on both sides of the road. They looked sturdy enough to destroy F2 on impact.
“Ah, one more thing. Since the test consists of multiple stages, questions will be taken only after everything is finished. Now, let us begin. F2, start driving.”
At President Bolt’s command, the stationary F2’s systems powered on. Being a 100-percent electric vehicle, there was no loud engine roar. Instead, the brake lights, turn signals, and headlights illuminated in sequence for a systems check, accompanied by an artificial engine sound. Once all sensors confirmed normal operation, F2 began to accelerate smoothly. What had seemed like a leisurely pace quickly exceeded 80 km/h. One of the electric vehicle’s strengths was that it could deliver maximum torque instantly. When F2 was one hundred meters from the crossing, the boards carrying the mannequins began to move. Unable to use real pedestrians, the team had devised this workaround. F2’s AI recognized the mannequins as high-priority obstacles approaching its path and slowed down. When the path became obstructed, it came to a complete stop. The distance between them was twenty meters—far more than safe.
“When the brakes are functioning normally, it stops like this. Now let us force a trolley-problem situation. F2, return to the starting position.”
F2 turned and drove back to the start line.
“Automotive engineers who have come from afar, if you would assist us.”
External automotive experts, invited to lend credibility to the test, approached F2 alongside Lightning Bolt’s own engineers. Their task was to temporarily disable F2’s brakes. President Bolt had brought in outsiders because using his own engineers might have raised questions. They simply unplugged the power connector to the electro-hydraulic brake system, instantly disabling the brakes. Reporters captured every moment.
Once the signal came that preparations were complete, President Bolt gave the command again.
“F2, start driving!”
But something unexpected happened. The artificial engine sound played, the lights began their sensor check, and then a warning tone sounded as the vehicle halted. F2 had detected that the brake hydraulics were inoperative and refused to proceed with autonomous driving.
“As you can see, when F2’s brakes fail, autonomous driving itself is disabled. Therefore, an F2 with failed brakes cannot cause an accident. However, we must also consider the possibility of brake failure while already in motion, so we will enter debugging mode, bypass the brake-status check, and continue. In this mode, the AI will treat the non-functional brakes as normal and proceed with driving.”
President Bolt’s tone was almost teasing. Preparation took longer this time. Switching the main computer into debugging mode and altering the parameters was more involved than it sounded.
“F2, start driving!”
When everything was ready, President Bolt issued the command once more. With the debugging settings active, F2 pulled away even though the brake hydraulics remained disconnected. Every eye turned toward the crossing three hundred meters ahead. When one hundred meters remained, a strange whining sound came from F2 and its speed dropped rapidly. Having determined that normal braking was impossible, the AI applied reverse torque to the electric motor—the vehicle’s engine—generating electricity instead of consuming it. This regenerative braking was normally used to recover energy, but F2 now used it aggressively to bring the car to a safe stop.
President Bolt looked at the reporters with pride. Few of them appeared convinced. Many had been hoping for something more spectacular.
“Let us move to the third scenario—the worst-case situation in which reverse torque is also unavailable.”
This time, the scene the reporters had been waiting for finally appeared. With reverse torque disabled in debugging mode, F2 desperately tried to slow itself using guardrails, concrete walls, and even utility poles. It even performed something resembling stunt driving to narrowly avoid the mannequins. Each demonstration left F2 heavily damaged and requiring replacement. By the end of the tests, five vehicles had been reduced to wrecks. Since these were not mass-produced cars but hand-assembled prototypes costing well over $100,000 each, the demonstration had consumed more than half a million dollars. It was the kind of expense only ID Group’s financial power could sustain. The impact on the reporters, however, was enormous. Even without a driver, F2’s clear intent to protect both driver and pedestrians came through unmistakably. That impression spread through the articles the reporters filed.
A dilemma was a dilemma precisely because no choice offered a clear answer. Yet the growing impression was that F2 would be safe. The final seal on that impression came with the publication of Yoo Jae-won’s paper.
Titled “On the Construction of Artificial Intelligence through Machine Learning-Based Artificial Neural Network Composition,” the paper was released through the Stanford Electronics Society at the end of May, once the F2 controversy had begun to subside. At a time when news of civil war in Iraq dominated the headlines, Yoo Jae-won’s paper sent shockwaves through the academic world. It marked the beginning of a revolution in AI research that had long been stalled.
