Microsoft's Meaningless Remote Work Study - And The Anti-Work Pro-Office Debate

I was on vacation last week, and nevertheless, everybody wanted to send me this Nature study of Microsoft and remote work, and I have had the displeasure of reading the entire thing. The long - and man, is it long! - and short (it isn’t) of the study is that people in remote work tend to speak to people outside of their circles of work less and collaborate with those they have weaker ties with less. People had more unscheduled calls, more meeting hours, more video and audio hours, and more IMs sent. The massive, endless study, a study I did not enjoy reading and believe may have given me gout, mostly dances around making any firm conclusions. It also doesn’t really explain how the study took place, nor how one measures “knowing yourself” or “getting ahead in a wider organization” or “fostering creativity.”

See the below chart:

Now, if you’ve got several pieces of sandwich meat in your brain, maybe this chart proves something, but my natural thought here is simple: what the hell does any of this mean? While these categories make sense, how were they evaluated? Actually, while I have you, how did you evaluate each person? Also, hold up a second:

Here, we use rich data on the emails, calendars, instant messages, video/audio calls and workweek hours of 61,182 US Microsoft employees over the first six months of 2020 to estimate the causal effects of firm-wide remote work on collaboration and communication.

In addition to the whole remote work debate: you used what? Excuse me? How did you use that data? What did you have access to? What parts of what things did you use, and in what way did you use them? How did you qualify each element of this study?

This study, as lengthy and boring as it is, doesn’t really get to a point of any sort - other than the fact that this is one company out of many, it also fails to describe exactly how things were measured. Were calls listened in on? How did they measure how one’s workday started and ended? And take this wonderful, citation-free paragraph:

…we found that remote work caused employees to communicate more through media that are more asynchronous—sending more emails and many more IMs. Media richness theory, media synchronicity theory and previous empirical studies all suggest that these communication media choices may make it more difficult for workers to convey and/or converge on the meaning of complex information.

In any case, this study took place from December 2019 to June 2020, meaning that it had maybe three and a half months of actual remote work that was also, I’d argue, very much work-in-progress and during the worst of the pandemic. It was a Microsoft-paid study, and one must ask a very simple question: if you were doing this study with the intention of actually finding something, why did you cut it in June 2020? If your real interest is in actually evaluating what remote work does, why would you use an incredibly flawed test group - one that had been rushed into remote work and was likely dealing with the crisis of a global pandemic. One might also have included something about laying off 1000 people in June, too, because that might mess with the data. Or maybe that’s why it was cut off in June.

Everything about this study feels manufactured to produce a result that suggests that remote work is insufficient, by using a non-specific evaluation of vaguely-described data, cut from a date range in which, to quote my friend Kasey, “no one could find their ass with a flashlight.” Microsoft has 182,000 employees worldwide, and it’s clear based on their interviews about going remote that they had to basically work out how to take the company remote as they went along. Every company - even those who were totally remote - basically spent March through June freaking out, and to use that data and that data alone to evaluate remote work’s efficacy (even in the mealy-mouthed non-specific way this study does) is corrupt at its core.

And that’s before you get to how the data was collected, handled and evaluated. You have all of these IMs and calls and calendar invites, but very little in the way of what any of that data actually showed beyond time spent doing stuff. See below:

For each employee, we observe (1) their remote work status before the COVID-19 pandemic, and what share of their colleagues were remote workers before the COVID-19 pandemic; (2) their managerial status, the business group they belong to, their role and the length of their tenure at Microsoft as of February 2020; (3) a weekly summary of the amount of time spent in scheduled meetings, time spent in unscheduled video/audio calls, emails sent and IMs sent, and the length of their workweek; and (4) a monthly summary of their collaboration network.

What exactly is a “monthly summary of their collaboration network”? For a term, they use several times, “collaboration network” isn’t actually defined - nor is what a “collaboration” is. Is a collaboration an IM? A call? A call over 30 minutes? A Microsoft Teams message? They spend a lot of time talking about strong and weak ties between people, ego networks, and so on, but no time saying what exactly defines a tie, or any of these terms they’re using.

In any case, the larger conclusions they come to are that remote work means we make fewer new “collaborators” and “shed fewer existing ones.” Microsoft says that this makes an organization “less dynamic,” which is bad apparently, and goes on and on and on about how remote work causes “siloing” in an organization. This term technically means information is kept within distinct groups but carries with it a stigma of giant walls between people that stop information flowing.

The study, crucially, does not actually seem to evaluate whether siloing of information is happening, nor does it seem to evaluate something that would’ve been useful within context - whether people are getting their work done on time and to a high quality.

Naturally, headlines have grown out of the study’s vague conclusions that suggest that “innovation and productivity are down,” despite the study failing to measure either productivity or innovation. If anything, Microsoft successfully measured whether people were doing something during the period of study, but not what that something might have been (beyond “I’m in a meeting”) or whether work was done.

Maybe I’m paranoid, but it’s almost as if Microsoft deliberately did this study as a means of finding a conclusion they wanted to, which is that remote work “silos communication,” and thus it’s not good to have everybody remote. The story may have changed since then - Microsoft has now postponed their return to the office “indefinitely” - but it feels as if Microsoft’s game here was to push hybrid work (likely due to Teams’ new hybrid features) and because, like every big company, they don’t like having a big campus with nobody in it.

Arguably the biggest problem I have with the study (other than the date range) is the fact that it doesn’t actually seem to evaluate work. It evaluates communication, which is part of work, and connections, which are in some cases tangentially part of work, and then makes conclusions about work without evaluating the work being done. By removing productivity and execution from the study, they’re not actually evaluating remote work.

The Repetitive Strain of The Remote Work Debate

It also makes the same stupid conclusions that every anti-remote thing does - assuming that the status quo is good and that new things are bad. The study repeatedly talks about how we’re not “making new connections” but fails to evaluate whether new connections actually improve business operations or lead to distractions in the workplace. Somehow, in a 60,000-person study that took place over the course of months, nobody thought to evaluate whether the things that they measured actually mattered at all. It’s a big academic paper in a huge science journal that comes to the vague, unsupported conclusions of every anti-remote op-ed in the New York Times - that collaboration leads to innovation, which is good, and we all know what it means, and the office is better for that, for some reason.

This is the problem with all of these things I write about - that we are still, from the trillion-dollar multi-national all the way down to 10-person organizations, blindly assuming that physical human contact is good, and that talking to more people always leads to good outcomes. Vacuous terms like “collaboration” and “innovation” mean nothing, especially when they are undefined, or defined by someone who is not an actual worker, which I’d define as someone who is not in an executive role at a large organization or someone who has the title “manager” (if you read this and take offense, please do email me at nobodycares@theranos.biz).

I think this problem is that a large number of organizations, big and small, are in total denial about remote work. They continue to say totally illogical things about how we  need the “collaboration” and “spontaneity” of the office, and that remote work “cuts down on innovation,” not because they actually believe that these things happen or have evidence that they exist, but because admitting that they don’t means reconciling with a vast amount of organizational and fiscal waste within their company.

This is why you see so many of these articles that are extremely emotional but thinly sourced. I’m yet to see a single anti-remote work piece that discusses how business is worse because of remote work. There are always vague “oh yeah, you’ll never close a big deal over Zoom!” anecdotes, but no big deal that they’ve lost because they had to do it over Zoom because most likely said deal was done over Zoom and it was fine.

These pieces - including this study - fail to actually address or discuss the efficacy of remote work but focus on the hard-to-impossible-to-measure tenets of “collaboration” or “innovation,” things that sound very important and scary to lose. I cannot find what collaboration is lost outside of Microsoft’s vague idea that we speak to fewer people outside of our groups. Innovation itself is a truly insane thing to bring up in the evaluation of remote work because there doesn’t seem to be any definition of what innovation is, what its outcomes are, how it’s measured, or indeed proof that more innovation comes out of the office than comes out of remote work.

I can’t get over the myopia of most of this coverage. The loudest voices are mostly old men who are pissed off about people not being in an office they don’t visit or research that mostly reveals how poorly researchers understand work. Most discussion of remote work seems to start with a default assumption that what we did before was right - that constant conversation with other people was good, that being in the office for all of these years had a reason beyond “we like seeing you,” and that there is no better way to do things other than doing exactly what we did in the office but on the computer.

The malfunctions that keep happening are because organizations need to - even if it means making nonsensical hybrid work strategies - feel as if working in the office actually had a point. The vagueness of the “loss of collaboration” conversation mostly suggests that whatever collaboration is, it either isn’t lost in remote work or doesn’t seem to have any real effect on our ability to do work.

And I think that’s what is really frustrating me - for all of the gnashing of teeth about remote work, there seems to be a dramatic lack of focus on the success of the business. If a Microsoft study about the efficacy of remote work is avoiding the consideration of both how work was done and the quality of said work, it’s hard to not think slightly crazy, paranoid thoughts about how big businesses are doing everything they can to demonize remote work and remote workers post-pandemic.

We need to escape the default assumptions that we have about work, starting with the embarrassing thought that we may have societally been doing inefficient, wasteful things for many years. While that may be easy for you and me to do, large enterprises are going to keep freaking out about this until they realize that there is no avoiding a future where a large number of jobs will be done remotely.