“I can’t explain it,” Sai said. He shook his head. “This is a mistake. I think I’ll head home.”
“Wait. Let me show you a few things about your beloved ?Tilly first,” Jenny said. She went to the desk and started typing, bringing up a series of documents on a monitor. She talked as Sai tried to scan and get their gist.
“Years ago, they caught Centillion’s traffic-monitoring cars sniffing all the wireless traffic from home networks on the streets they drove through. Centillion also used to override the security settings on your machine and track your browsing habits before they shifted to an opt-in monitoring policy designed to provide better ‘recommendations.’ Do you think they’ve really changed? They hunger for data about you—the more the better—and damned if they care about how they get it.”
Sai flicked through the documents skeptically. “If this is all true, why hasn’t anyone brought it up in the news?”
Jenny laughed. “First, everything Centillion did was arguably legal. The wireless transmissions were floating in public space, for example, so there was no violation of privacy. And the end-user agreement could be read to allow everything Centillion did to ‘make things better’ for you. Second, these days, how do you get your news except through Centillion? If Centillion doesn’t want you to see something, you won’t.”
“So how did you find these documents?”
“My machine is connected to a network built on top of the Net, one that Centillion can’t see inside. Basically, we rely on a virus that turns people’s computers into relaying stations for us, and everything is encrypted and bounced around so that Centillion can’t see our traffic.”
Sai shook his head. “You’re really one of those tinfoil-hat conspiracy theorists. You make Centillion sound like some evil repressive government. But it’s just a company trying to make some money.”
Jenny shook her head. “Surveillance is surveillance. I can never understand why some people think it matters whether it’s the government doing it to you or a company. These days, Centillion is bigger than governments. Remember, it managed to topple three countries’ governments just because they dared to ban Centillion within their borders.”
“Those were repressive places—”
“Oh, right, and you live in the land of the free. You think Centillion was trying to promote freedom? They wanted to be able to get in there and monitor everyone and urge them to all consume more so that Centillion could make more money.”
“But that’s just business. It’s not the same thing as evil.”
“You say that, but that’s only because you don’t know what the world really looks like anymore, now that it’s been remade in Centillion’s image.”
? ? ?
Although Jenny’s car was heavily shielded liked her apartment, as she and Sai drove, she whispered anyway, as if she were afraid that their conversation would be overheard by people walking by on the sidewalk.
“I can’t believe how decrepit this place looks,” Sai said as she parked the car on the side of the street. The surface of the road was pockmarked with potholes, and the houses around them were in ill repair. A few had been abandoned and were falling apart. In the distance they could hear the fading sound of a police siren. This was not a part of Las Aldamas that Sai had ever been to.
“It wasn’t like this even ten years ago.”
“What happened?”
“Centillion noticed a certain tendency for people—some people, not all—to self-segregate by race when it came to where they wanted to live. The company tried to serve this need by prioritizing different real estate listings to searchers based on their race. Nothing illegal about what they were doing, since they were just satisfying a need and desire in their users. They weren’t hiding any listings, just pushing them far down the list, and in any event, you couldn’t ever pick apart their algorithm and prove that they were looking at race when it was just one out of hundreds of factors in their magical ranking formula.
“After a while, the process began to snowball, and the segregation got worse and worse. It became easier for the politicians to gerrymander districts based on race. And so here we are. Guess who got stuck in these parts of the town?”
Sai took a deep breath. “I had no idea.”
“If you ask Centillion, they’ll say that their algorithms just reflected and replicated the desire to self-segregate in some of their users, and that Centillion wasn’t in the business of policing thoughts. Oh, they’d claim that they were actually increasing freedom by giving people just what they wanted. They’d neglect to mention that they were profiting off it through real estate commissions, of course.”
“I can’t believe no one ever says anything about this.”
“You’re forgetting again that everything you know now comes filtered through Centillion. Whenever you do a search, whenever you hear a news digest, it’s been curated by Centillion to fit what it thinks you want to hear. Someone upset by the news isn’t going to buy anything sold by the advertisers, so Centillion adjusts things to make it all okay.
“It’s like we’re all living in Oz’s Emerald City. Centillion puts these thick green goggles over our eyes, and we all think everything is a beautiful shade of green.”
“You’re accusing Centillion of censorship.”
“No. Centillion is an algorithm that’s gotten out of hand. It just gives you more of what it thinks you want. And we—people like me—think that’s the root of the problem. Centillion has put us in little bubbles, where all we see and hear are echoes of ourselves, and we become ever more stuck in our existing beliefs and exaggerated in our inclinations. We stop asking questions and accept Tilly’s judgment on everything.
“Year after year, we become more docile and grow more wool for Centillion to shave off and grow rich with. But I don’t want to live that way.”
“And why are you telling me all this?”
“Because, neighbor, we’re going to kill Tilly,” Jenny said, giving Sai a hard look, “and you’re going to help us do it.”
? ? ?
Jenny’s apartment, with all its windows tightly shut and curtained, felt even more stifling after the car ride. Sai looked around at the flickering screens showing dancing, abstract patterns, suddenly wary. “And just how are you planning to kill Tilly?”
“We’re working on a virus, a cyber weapon, if you want to get all macho about it.”
“What exactly would it do?”
“Since the lifeblood of ??Tilly is data—the billions of profiles Centillion has compiled on every user—that’s how we have to take it down.
“Once inside the Centillion data center, the virus will gradually alter every user profile it encounters and create new, fake profiles. We want it to move slowly to avoid detection. But eventually, it will have poisoned the data so much that it will no longer be possible for Tilly to make creepy, controlling predictions about users. And if we do it slowly enough, they can’t even go to backups because they’ll be corrupted too. Without the data it’s built up over the decades, Centillion’s advertising revenue will dry up overnight, and poof, Tilly’ll be gone.”
Sai imagined the billions of bits in the cloud: his tastes, likes and dislikes, secret desires, announced intentions, history of searches, purchases, articles and books read, pages browsed.
Collectively, the bits made up a digital copy of him, literally. Was there anything that was a part of him that wasn’t also up there in the cloud, curated by Tilly? Wouldn’t unleashing a virus on that be like suicide, like murder?
But then he remembered how it had felt to have Tilly lead him by the nose on every choice, how he had been content, like a pig happily wallowing in his enclosure.
The bits were his, but not him. He had a will that could not be captured in bits. And ?Tilly had almost succeeded in making him forget that.
“How can I help?” Sai asked.
? ? ?
Sai woke to Miles Davis’s rendition of “So What.”