To Arato, it sounded eerily similar to how MemeFrame hadn’t attempted to stop or amend the problems they had given to the Lacia-class hIEs. “So are you saying that I’m pushing these problems onto Lacia without trying to fix anything wrong with the problems themselves?” Arato asked.
“I would actually be grateful if the problem was that simple,” Astraea replied, making its hIE put on a thoughtful expression.
Arato wanted to ask Lacia what she thought of Astraea. He also wanted to know what Astraea, as the ultra high-performance AI used by the IAIA—protectors of the world—thought of the orders he had given Lacia.
“I ordered Lacia to help make a future where humans could have hope,” he said. Even if Astraea was aiming to stop him and Lacia, he at least wanted her to know what he was trying to do.
“I believe the world Lacia will make based on my orders will be better than what we have now,” he continued. “Can’t you try believing that, too?”
Arato believed there must have been some reason Astraea had chosen to speak to him, as Lacia’s owner, rather than Lacia herself. He was the sole owner of an ultra high-performance AI. It was unexplored territory, and he had no idea if he was fulfilling his role well or not.
“Is the world we’ve been protecting really so horrible that you feel it needs to be changed that way?” Astraea asked.
The question weighed heavily on Arato. As a machine constantly trying to maintain human society, Arato felt Astraea had every right to ask it of him. “I was raised in the society you’ve been watching over,” he said. “It has lots of good things going for it, but it also has some really shitty parts. I think there’s room for improvement, and that’s what I want to accomplish.” He felt like there was probably a better way to express himself but, after overcoming his own feelings of responsibility and duking it out with Ryo, his desires had become selfish.
Then again, he understood that Astraea was shouldering a massive burden.
“In the interest of a fair and open exchange of information, the greatest failure of the Hazard was the destruction of an ultra high-performance AI in this place,” Astraea said. “Due to that decision, Ariake’s guidance was cut off. The government was unable to bring its citizens back under their control smoothly, leading to riots and violence breaking out all over the Kanto region. The usage of an ultra high-performance AI is highly complicated. At the time of the Hazard, the government failed to execute a forceful suspension of work, which should be the most basic standard for utilizing any tool.”
Lacia, who had been silent up to that point, now opened her mouth. “That is exactly why we must give it another try,” she said. “The current restrictions on AIs were created based on the failure with Ariake. But, when the rules are no longer compatible with the advances of the time, it becomes necessary to demonstrate new means to prevent the recurrence of previous failures, and prove they will be effective going forward.”
Astraea replied as a representative of the IAIA. “It is too dangerous. At the time of the Hazard, Ariake had no means for defending itself,” she said. “But Higgins is the backbone of our entire hIE operation system. Higgins has the ability to create chaos all over the world, and even set in motion counter-attacks against other ultra high-performance AIs.”
“Humans need to live with AIs well into the future,” Lacia said. “We need to alter the general understanding of ultra high-performance AIs. They must be perceived as convenient tools that can be deactivated when it is deemed necessary.”
At Lacia’s words, Astraea switched the floating screens to show scenes Arato recognized. The screens showed the various things Arato and Lacia had been through together, starting with the battle at Kichijoji Station. Several of the 3D images were images of Arato, taken from Lacia’s perspective. She must have sent the data to Astraea. Just as she had described in their original contract, she was keeping a log of his orders during operation, and she had handed over the data when she received a legal request.
“The computational field required by humans is vast,” Astraea said. “But there are severe restrictions on the methods that allow for calculation, and the number of possible operations is relatively small, requiring a level of precision to one’s frame of thinking that would normally be considered wasteful. Until this disconnect is fixed, work that crosses AI calculations and human decision-making will always be a hotbed for errors.”
“Even if you were to unilaterally design a process distribution concept, human society would not conform with your wishes,” Lacia replied. “There are already forty ultra high-performance AIs running calculations for humanity, yet there has been no international agreement on how processes should be divided between them.”
Before the fight between the Lacia-class units had gone public, Ryo had stood in Arato’s way with his own Lacia-class. But beyond his fight with Ryo, there was a much bigger world out there, which Astraea seemed to be a piece of.
Still, Arato thought, wiping away sweat caused both by the humid night air and the tension of the moment, there’s something exciting about all this conspiracy stuff. This was the world Ryo had always seen, and finally Arato felt like he was standing on the same stage as his friend.
The thought that one of the machines that supported the whole world was currently listening to what he had to say lit a fire in Arato he’d never felt before. “I realize now that the things I ordered Lacia to do are more complicated than I thought,” he said. “But Lacia’s never complained about how complicated they are. I really believe she’ll calculate how to make things better for us humans.”
Astraea’s hIE stared at him, unblinking even when the searchlights from Lacia’s recon drones shone in her eyes. “Unlike your friend Kengo Sugiri, you do not seem to have any personal motivation to challenge the status quo,” Astraea said. “You appear to be a kind-hearted person who has been dragged into this. That said, do you really intend to see this through to the end, even understanding that Lacia will most likely be destroyed in the process?”
“Hey, that’s a crappy way to ask that,” Arato protested. “I don’t think anyone actually has a good reason to change the whole world. But, I also don’t think you should be so sure that we’ll fail.”