Amid all the profound changes people have made in response to Covid-19, the infrastructure that undergirds the internet experienced a shift in usage patterns, too, as people traded office hours for home isolation. The companies that handle these systems have mostly been able to manage users' new needs. “You essentially took the peak and extended it over a far longer period of the day,” says Ben Treynor Sloss, Google vice president of engineering. “The usage went way up, but it was mostly that the use looked more like peak most of the day, rather than that the peaks went up dramatically.” Some services, though, saw usage spike well beyond normal.
Google prepares for emergencies on a regular basis through its disaster and incident response tests, or DIRT. In these exercises, around 10,000 employees at a time will simulate handling some sort of crisis, ranging from a localized natural disaster to a Godzilla attack. The Covid-19 pandemic, though, turned out to exceed even the company’s most dramatic scenarios.“We had typically simulated a regional-level event,” says Treynor Sloss. “We’d never done DIRT for a global-level event, in part, if I’m being honest, because it didn’t seem likely.” There was also a practical concern: Convincingly mocking up an incident with worldwide impact would risk downgrading the experiences of actual Google users, a cardinal sin in the world of DIRT.
All of which meant that Schaevitz, who led the incident response for Google Meet, and the teams involved had to figure things out on the fly. Especially as it became clear that they were taking on far more new users than their most ambitious early projections.“In the beginning, we started planning for a doubling of our footprint, which is already huge. That’s not the normal growth curve. We soon realized that wasn’t going to be enough,” says Schaevitz. “We kept trying to make progress on building more runway … so that we would have time to figure out a solution if things would arise on a longer time horizon rather than just every day waking up and being like, what’s newly on fire today?”
Sit Up Straight
How to Make Your Video Calls Look and Sound Better
Everyone's Video Chatting! But Which Service Is Best for You?What Happens Next?Videoconferencing Needs to Climb Out of the Uncanny Valley Complicating the challenge was that the Google engineers involved in the response were themselves working from home, spread across four offices in three countries. “All the people who worked on this—and this is a large number of teams—even the people working on it in the same place have actually never been in the room together since this started,” says Schaevitz, who is based in Zurich, Switzerland. On a technical level that proved manageable enough; as you might imagine, Google prioritizes web-based tools that can be accessed from anywhere. But coordinating the 24-hour-a-day operation remotely required setting up redundancies for more than just bandwidth. In a blog post detailing the response, Schaevitz described how everyone in an incident response role was assigned a “standby,” basically an understudy who could step in if the principal got sick or had to take time away. (An especially prudent measure during a global health crisis.)
Both are structured in the same way as Voice & Audio Activity , with options to erase records one by one, a day at a time, or all together via the Delete activity by link, if you click Manage activity .You do have options when it comes to temporarily stopping Google tracking you, as well—staying signed out when using YouTube for example, not signing into Google Chrome, or using incognito mode when searching and browsing in Chrome.