A Conversation about Supervising Artificial Intelligence at Work

[00:00:00] Imagine you're a lawyer, right? You are sitting in federal court in New York. The stakes are incredibly high and you've just submitted this brilliant 10-page legal brief. Oh, this is the classic nightmare scenario. Right. So it cites six ironclad precedents. The logic is flawless. The judge reads it, looks down at you from the bench, and slaps you with a $5,000 fine. Just brutal.

[00:00:24] And why? Because the computer program you used to help write that brief completely hallucinated every single one of those court cases. I mean, they didn't even exist. Yeah, completely fabricated. And you, in the rush of your busy week, just, you forgot to actually check the machine's work. So, welcome to the era of bot sitting. It really is the defining cautionary tale of our current technological moment.

[00:00:47] I mean, it perfectly encapsulates this massive collision between what we expected artificial intelligence to do for us and what is actually demanding from us in return. I am so glad we are finally talking about this because usually, you know, when a new workplace technology comes along, the narrative's entirely about liberation. Right. The whole work less, produce more pitch. Exactly. You plug in the washing machine, you turn it on, and boom, you have your Saturday afternoon back.

[00:01:13] It's clean. It's binary. We just love the idea of technology lifting the burden off our shoulders. But then you step into the world of generative AI in the modern office, and, well, that clean narrative completely shatters. Completely. We were promised margaritas on the beach while algorithms wrote our quarterly reports. Instead, you know, you might be sitting there right now staring at a screen, meticulously double-checking a chatbot's output line by line.

[00:01:42] And wondering why you feel even more exhausted than you did before the AI was even installed. Yes. Okay, let's unpack this. Our mission for this deep dive is to explore this exact phenomenon, which just recently got a name. They call it bot-sitting. Which is such a great term for it. It really is. Yeah. And to do this, we are looking at a massive, incredibly comprehensive new article published in the Human Capital Leadership Review. It's by Dr. Jonathan H. Westover, PhD. Yeah. And the title really says it all.

[00:02:11] Bot-sitting? The Hidden Labor of Supervising AI and How Leaders Can Make Oversight Visible, Valued, and Sustainable. Westover's research is so crucial right now because he isn't just looking at, like, the theoretical capabilities of these large language models. Yeah, he's not doing the Silicon Valley hype thing. Exactly. He's taking a hard empirical look at the lived reality of human beings at their desks. He pulls together survey data, labor economics, and some deeply complex cognitive psychology.

[00:02:39] Basically to explain why we are spending all day managing these systems instead of doing our actual jobs. So, spoiler alert for you listening, you are almost certainly doing a ton of invisible labor right now. Oh, undoubtedly. We're going to dissect exactly what that labor looks like, why human biology actually makes us terrible at doing it, and the frankly terrifying, sometimes hilarious, but very expensive organizational disasters that unfold when this human oversight fails.

[00:03:07] To really grasp the scale of those failures, I mean, we have to start by measuring the gap between the sales pitch we all heard a few years ago and the reality on the ground today. Right, because the pitch was, uh, AI will handle the tedious stuff and you'll be freed up to do deep critical thinking. Which sounds amazing. Which, and to be fair, there are plenty of studies showing that generative AI can make people faster at certain isolated things. But there has been this quieter, parallel story building up in the background.

[00:03:35] And the business publication Founder Reports finally put some hard numbers to it in October of 2026. They did. They surveyed over 2,000 employed adults in the U.S. who use AI at work on, like, at least a weekly basis. And what I love is that they didn't just ask, how much time did you save? No, they asked a much more revealing question. They asked, how much of your time actually goes into overseeing, checking, and fixing what the AI produces? The answers just blew my mind.

[00:04:03] Over 82% of weekly AI users said they are spending time every single week just overseeing and fixing the output. Every single week. But the really wild part is the upper extreme. Nearly 16% of these workers are spending six hours or more a week just managing the AI. Which is almost a full eight-hour work day. Gone. Dedicated entirely to babysitting a piece of software. And that is exactly why the term bot sitting resonates so strongly.

[00:04:31] I mean, you aren't collaborating with a peer. You are supervising a subordinate who requires endless vigilant attention. There was another data point in that survey that I found so funny but also a little tragic. They isolated their respondents who had actual experience training human employees. Like real managers who know what training takes. Yeah. And 60% of them said that getting good work out of an AI takes at least as much effort as training a brand new human hire. And didn't a chunk of them say it was actually harder? Yes.

[00:05:01] A third of them said it's actively harder than training a human. Which completely destroys the narrative of frictionless automation. I mean, think about how you train a human. You hire them. You spend maybe a month or two onboarding them. Building up their mental model of the company. Right. It's an upfront investment. Exactly. It takes time early on. But eventually they get it. They internalize the context. They understand the unwritten rules of your specific workplace. You can trust them to run on their own. But the AI doesn't learn like that, does it?

[00:05:30] No, because generative AI doesn't actually build a persistent mental model of your business between sessions. Unless, you know, you have a highly customized, incredibly expensive enterprise architecture. Which most of us definitely do not have. Right. For the average worker, every time you open a new chat window, the AI basically has amnesia. It's a perpetual state of onboarding. You have to explain the rules, the context, and the tone all over again. It feels exactly like managing a wildly overconfident intern. You know the type.

[00:05:59] Oh, we all know the type. They read a million words a minute. They have access to every Wikipedia article ever written. But they occasionally just make up a totally fabricated fact with a completely straight face. And worse, they absolutely refuse to admit when they don't know something. They'd rather lie to you beautifully than just say, hey, I'm not sure. That is a remarkably accurate analogy for how large language models function. I mean, they are probabilistic engines designed to predict the most likely next word in a sequence. They are designed to sound plausible, right?

[00:06:29] Not to be factually accountable. Exactly. And just like that overconfident intern, you can't just blindly trust a 50-page report they hand you. You have to verify every single claim. And that verification is where the massive time sink happens. Westover brings in some fantastic data from the GleanWork AI Index from 2026 to really quantify this. Yeah, let's talk about the math here because the Glean data surveyed, I think it was 6,000 digital workers across the U.S., the U.K., and Australia. That's right.

[00:06:57] And the respondents estimated that AI saves them about 11 hours a week. Which sounds incredible. Which sounds incredible. I would love 11 hours back. Right? Right. But then the exact same workers reported spending 6.4 hours a week just making the AI usable. Oh, man. And apparently 36% of AI sessions require substantial rework or just a total reset. This is the crucial economic distinction that almost every organization is currently getting wrong.

[00:07:25] AI is delivering value. We shouldn't deny that. Right. Saving 11 hours and spending 6 1⁄2 hours means you are still technically netting out about 4 1⁄2 hours of saved time. Exactly. But corporate leadership is only looking at the gross savings. They see the 11 hours and they start adjusting productivity targets upward. Oh, of course they do. They are completely blind to the 6.4 hours of brand new, highly intensive labor that has just been added to the workers' plate. But wait, let me play devil's advocate for a second. Isn't this just a transitional phase?

[00:07:55] Like, surely we had to bot-sit the first spell checkers or early search engines and now they work seamlessly. It's a very common assumption. So why wouldn't we just wait six months for the next software update? Won't the AI just get better and eliminate this bot-sitting phase naturally? It fundamentally misunderstands the architecture of generative AI. Look, a spell checker is rules-based. It checks a word against a fixed dictionary. If it's wrong, it's wrong. Okay, that makes sense. Generative AI is probabilistic.

[00:08:23] It doesn't have a rigid database of truth. It is constantly hallucinating, by design. That's literally what allows it to be creative and generate novel text. So the hallucination isn't a bug, it's the feature. Exactly. As the models get larger and more sophisticated, their outputs become more fluent, more persuasive, and more complex. Which actually makes the errors harder to spot. Wow. Because you aren't checking for a simple typo anymore. Right.

[00:08:53] You are checking for subtle, logical flaws hidden inside perfectly formatted, highly persuasive pros. So the bot-sitting doesn't go away. It just requires a higher level of expertise to do it well. You hit the nail on the head. And that is reflected in another study Westover Sights from the Society for Human Resource Management. The 2026 SHRM study, right? Yeah. They found that the amount of time spent correcting AI output actually rises with seniority. Directors and above are doing the most bot-sitting in the entire organization.

[00:09:22] Which is wild. You would think the junior analysts would be the ones cleaning up the AI's mess. Why are the directors doing it? Because of accountability. Let's say an individual contributor uses AI to draft a strategic brief. They spend 20 minutes on it, think it looks great, and send it up the chain. Right. They check it off their list. But when that brief lands on a director's desk, the director is the one who actually has to present it to the board. The director has the domain expertise to realize the AI misunderstood a nuanced market dynamic. Ah.

[00:09:50] So the director's the one who ends up spending three hours tearing the document apart and rewriting it. Exactly. The burden of verification flows relentlessly upward to the people whose reputations are actually on the line. Wow. And Microsoft's 2026 Work Trend Index completely backs this up. Right. They surveyed 20,000 knowledge workers. Yes. For years, critical thinking was always the undisputed number one emerging skill for knowledge workers. But in this latest index? Quality control of AI output leapfrogged it. It did.

[00:10:20] Checking the computer's homework is now the most vital skill in the modern economy. That is incredibly sobering. So to you, the listener, I want you to mentally calculate your own net AI time this week. Think about that gross time you theoretically saved using an AI tool. Now subtract the real costs. Right. Subtract the time you spent tweaking your prompt over and over because it wouldn't get the tone right. Subtract the time you spent Googling a weird statistic it generated to see if it was real.

[00:10:47] And subtract the time you spent manually editing the final document because it sounded like a soulless robot wrote it. Exactly. That difference? That is your personal bot-setting burden. And that burden is made up of very specific, cognitively demanding tasks. If we are spending over six hours a week doing this, we need to understand exactly what that labor consists of. Right. It's not physical labor. And it's not traditional knowledge work where you stare at a blank page and type. It's an entirely new category of work.

[00:11:16] I found Westover's breakdown of this so illuminating in section two. He maps out the anatomy of this invisible labor. And when you look at the sequence of what we're actually doing, it's exhausting. It really is. It starts way before the AI even generates a single word. Right. The first major component is context provision. The data shows workers spend about 2.3 hours a week just on this. Just giving it background. Because you can't just tell an enterprise AI, write a project proposal.

[00:11:46] If you do that, you get generic nonsense. You get work slop. You get work slop later, but yeah, it's bad. Exactly. So to get something good, you have to gather the client's historical data. You have to find three examples of past successful proposals. You have to outline the specific budget constraints. And you have to synthesize all of this into a massive, highly structured prompt. You are essentially doing all the architectural thinking before the AI types a word. I feel this so deeply.

[00:12:15] I will sometimes spend 45 minutes just organizing PDFs and formatting text to feed into the prompt box, just so the AI doesn't get confused. Yep. Feeding the beast. And then once you hit enter, and it spits out a draft in three seconds, the hard part isn't even over. You move right into the next phase, which is monitoring and review. And this phase requires a completely different type of cognitive energy. You spend another 2.2 hours a week just reading. But you aren't reading for pleasure. No. And you aren't even reading for information.

[00:12:43] You are reading defensively. You are inspecting the output meticulously to judge whether it's complete, if it followed your constraints, and if it's actually fit for purpose. You haven't changed a single comma yet. You are just trying to assess the damage. It's like being a health inspector in a restaurant kitchen. You are just looking for violations. And inevitably, you find them, which drags you into the third phase. Correction and rework. This is where you actually get your hands dirty. You are fixing the factual hallucinations.

[00:13:12] You are stripping out those weird, repetitive AI buzzwords it loves to use. The ones that make it sound so artificial. Right. And in many cases, as we saw with the Glean data, where over a third of AI sessions require substantial rework, you realize the AI completely lost the plot halfway through the document. And you have to rewrite whole sections from scratch. Honestly, sometimes I find that rewriting an AI's bad draft takes me longer than if I had just started with a blank page myself. Why do you think that is?

[00:13:41] Because when you start with the blank page, you only have your own thoughts to wrestle with. When you edit AI, you have to untangle its weird, circular logic before you can even figure out what it's trying to say. That cognitive untangling is incredibly taxing. And it leads to the final and perhaps most critical phase of bot sitting. Override and escalation. Which is the decision to just scrap it entirely, right? This is the executive function. This is the moment a human being has to look at the screen and say, this is garbage.

[00:14:10] I am throwing this out and doing it manually. Or, you know, reporting a systemic problem to IT. It takes actual courage to override the machine. Especially if your boss thinks the machine is infallible. It really does. When you lay it all out, curating context, defensive monitoring, heavy reworking, and the authority to override, that isn't a shortcut. That is a full-blown job description for a manager. But it's a job description that doesn't exist on paper.

[00:14:37] Which brings us to one of the most profound sociological concepts in Westover's paper. Oh, the 1999 concept. Yes. He connects this modern phenomenon to a foundational concept called invisible work. Theorized by researchers Susan Lee Starr and Anselm Strauss back in 1999. I loved this part. What exactly did Starr and Strauss mean by invisible work back before AI was even a thing? They were studying how complex organizations like hospitals or large corporations actually function day to day.

[00:15:07] And they realized that the formal organizational charts and the official job descriptions completely ignore a massive percentage of the actual labor keeping the place running. Like the glue holding it all together. Right. They called it articulation work or repair work. It's the informal coordination, the smoothing over bad processes, the quiet fixing of errors before they become crises. It's the work that by definition goes unnoticed if it's done perfectly. And bot sitting fits this perfectly.

[00:15:34] What's fascinating here is that if you spend three hours painstakingly correcting a highly flawed AI draft and then you email the flawless final version to your boss, what does my boss see? Your boss sees a flawless document produced in an era of AI efficiency. The dashboard that tracks software usage just logs that you used the AI tool and completed the task. So the company claims a productivity win. Exactly.

[00:15:58] The success of the human bot sitter is almost always recorded on the company dashboard as the AI's success. The disaster was averted behind the scenes and the system looks like it's functioning perfectly precisely because you sacrificed your own time to silently repair it. But let me push back on this a bit. If that's true, if I'm a senior director doing six hours of uncredited, invisible work, aren't I essentially taking a massive productivity pay cut? What do you mean?

[00:16:27] Like my company thinks I have this magic wand that makes me twice as fast, so they give me twice as many projects. But in reality, I'm spending all my time untangling the magic wand's mistakes. I'm basically doing two jobs for the price of one. You are absolutely taking a productivity pay cut. And we don't just have to rely on intuition to prove this. Westover highlights a fascinating study coming out of Denmark conducted by researchers Humlum and Vestergaard in 2025. This wasn't just a survey where they asked people how they felt, right? Exactly.

[00:16:56] This is what makes the Danish study so powerful. Denmark has incredible centralized administrative data. The researchers were able to link large scale surveys of workers' day-to-day tasks directly to their official labor market records. So their payroll, their job codes, their actual economic output. Right. So they could see the real economic reality. And they found that introducing AI doesn't just automate old tasks. It actively creates entirely new tasks for a significant portion of the workforce. Wow. How many people?

[00:17:26] About 8.4% of workers saw brand new tasks added to their roles. And here is the punchline. 59% of those new tasks were simply the labor of enabling and managing the AI. It was just workflow integration and quality review. Yes. As Westover notes, the new labor being generated is just the labor of making the automation function. So your feeling of taking a productivity pay cut is entirely valid. We have literally been assigned a second invisible job.

[00:17:56] Okay. So we know the work is happening and we know it's hidden from the balance sheets. But I want to pivot to why this specific type of work feels so uniquely draining. Section three. The human machine friction. Yes. Why is it so incredibly hard for a smart, capable human being to just read a document and spot the errors? To understand why we fail at bot sitting, we have to look away from Silicon Valley for a moment

[00:18:22] and look at decades of research in aviation, nuclear power, and medicine. Environments where human beings have been monitoring complex automated systems for a long time. Exactly. And the researchers in those fields have identified two massive psychological traps. The first is automation bias. And the second is automation complacency. Let's break those down. What is automation bias?

[00:18:45] Automation bias is our inherent tendency as humans to over rely on a recommendation simply because it comes from a computer. We just grant the machine an unearned authority. How does that actually play out in a daily workflow? It happens in two ways. You have errors of omission where you fail to notice a problem because the automated system didn't flag it. Like the spell checker. If there's no red squiggly line under the word, you just assume it's spelled correctly even if you're staring right at a glaring typo. Right. Your brain defaults to the machine's judgment.

[00:19:15] The computer didn't beep so everything is fine. What's the other way? Errors of commission. This is much scarier. This is when you follow the automated system's advice even when you have other perfectly valid information right in front of your face that contradicts it. You literally trust the machine more than your own eyes. It's the classic GPS disaster. The navigation app told me to turn right so I turned right and drove my car directly into a lake even though I could clearly see the water through my windshield. We surrender our judgment completely. So that's automation bias.

[00:19:44] What about automation complacency? I think of this one like highway hypnosis. Highway hypnosis is the perfect metaphor. If you've been driving on a straight, empty highway for three hours, your brain conserves energy by essentially tuning out. You stop actively scanning the road because nothing has happened for hours. You become complacent. Automation complacency is what happens when you are monitoring a system that is usually highly accurate.

[00:20:10] If the AI gets the summary right 99 times in a row, by the hundredth time, your vigilance has plummeted to near zero. Why expend the intense cognitive energy to check it when history tells you it's going to be fine? But then a deer jumps out onto the highway or the AI hallucinates a massive financial liability. And because you were hypnotized, you crash. Exactly. So what's the solution?

[00:20:33] The ultimate goal as defined by human factors researchers like Lee and C back in 2004 is something called calibrated trust. Calibrated trust. Meaning you don't want zero trust where you ignore the machine entirely. Right. And you don't want blind trust. You want your reliance on the system to exactly match the system's actual reliability for the specific task in front of you. Which sounds totally reasonable in theory. But with generative AI, achieving calibrated trust feels impossible.

[00:21:01] It is incredibly difficult. And Westover uses this incredible concept to explain why. The jagged technological frontier. The jagged frontier. This is from a massive 2023 experiment by researchers at the Boston Consulting Group or BCG alongside scholars like Delacqua. Yes. This study completely unlocks our understanding of why botsitting is so hard. Walk us through how they set this up. They took 758 of their own highly skilled management consultants.

[00:21:30] These are very smart, very highly paid people. They had them perform a series of complex knowledge tasks. Some used AI and a control group did not. Exactly. The researchers designed the experiment around this concept of a frontier. Imagine a landscape. Everything inside the frontier is territory the AI has mastered. Everything outside the frontier is territory where the AI fails. But the key is that the border isn't a straight line. It's wildly jagged.

[00:21:59] Because of the way large language models process information probabilistically, they can be absolute geniuses at incredibly complex creative tasks like synthesizing vast amounts of qualitative data, but utterly fail at a task that seems to a human to be adjacent or even simpler. Like strictly adhering to a rigid set of logical constraints. Exactly. So for the tasks that were inside that jagged frontier, the tasks the AI was genuinely good at, the results were staggering.

[00:22:28] The AI users crushed it, didn't they? They crushed the control group. Consultants using AI were 25.1% faster and their work quality was massively higher. It was an undeniable productivity miracle. But here comes the trap. Here comes the trap. The researchers then gave the consultants a task that fell just outside that jagged frontier. A task that to a human observer looked very similar in difficulty to the first one.

[00:22:55] But it relied on a specific type of logical deduction that the AI struggled with. And the result was an absolute disaster. On that task, outside the frontier, the consultants using the AI were 19 percentage points less likely to produce a correct solution than the consultants who didn't use AI at all. 19 percentage points worse. Because they couldn't see the frontier. They were totally blind to it. To the human consultant, the prompt looked similar. The AI's output sounded just as confident and beautifully formatted.

[00:23:25] So they just trusted it. They applied the trust they built on the first task to the second task. They fell straight into automation bias. Because the frontier is completely invisible to the user, you have to maintain maximum vigilance every single time you hit enter. Even if the tool just gave you a brilliant answer five minutes ago. And as we discussed with highway hypnosis, human beings are biologically incapable of maintaining maximum vigilance in a highly repetitive environment. No, we just can't do it. No, we just can't do it.

[00:23:53] And it's not just tired consultants who fall for this. Westover brings up a study on medical professionals that is, frankly, terrifying. Oh, the radiologist study. The Dratch study, published in 2023. This study isolates the sheer power of automation bias better than almost anything else. They took 27 radiologists, right? Yes, and had them read mammograms to detect breast cancer. Alongside the scans, the doctors were given AI-suggested assessment categories. But the researchers set a trap. They deliberately provided some incorrect AI suggestions.

[00:24:22] They wanted to see if the doctors would rely on their own medical training or if they would be swayed by the machine's bad advice. Okay, so when the AI was right, everyone did well. But when the AI was deliberately wrong, how did the humans perform? Human accuracy absolutely plummeted. For the inexperienced doctors, their accuracy fell terribly. But what's crazy is the veterans. Surely the veterans, the senior attending physicians who have looked at tens of thousands of these scans over decades,

[00:24:51] surely they looked at the bad AI advice, laughed, and made the correct diagnosis. You would hope. But even the very experienced specialists saw their accuracy drop to just 45.5%. Wait, really? Mm-hmm. Less than a coin flip. Less than a coin flip. More than half the time, a veteran doctor with years of specialized training was pulled toward a medically incorrect diagnosis simply because a computer suggested it on the screen next to them. Here's where it gets really interesting.

[00:25:18] This is where we have to talk about why generative AI is so uniquely dangerous here compared to human error. How do you mean? If I'm a senior doctor and a brand new intern walks up to me with a diagnosis, and they are sweating, stuttering, and avoiding eye contact, I immediately know I need to double check their work. Humans give off surface cues of incompetence. We signal when we are guessing. That is a brilliant point. Yeah. An AI never sweats. An AI never stutters. Right.

[00:25:47] When an LLM hallucinates, it does so with perfect grammar, pristine formatting, and absolute unwavering confidence. It strips away all the evolutionary cues we use to gauge trustworthiness. Yes. When we read beautifully formatted text, our brains automatically conserve energy by assuming the underlying facts are as pristine as the grammar. The cognitive dissonance of reading a perfectly written lie is just too much for our brains to process on the fly.

[00:27:14] hallucinates. It invented a fake policy. It told the customer they could go ahead and book a full price ticket immediately and then apply for a retroactive refund within 90 days. Which sounds totally plausible. It sounds like a real corporate policy. So the customer did it, and when they applied for the refund, Air Canada denied it, pointing to their actual policy, which stated bereavement fares must be arranged before travel. The customer took them to a civil

[00:27:40] resolution tribunal. And the defense Air Canada used in court is just breathtaking. Oh, it's wild. They essentially tried to argue that the chat bot was a separate legal entity. They blamed the bot. They said, hey, we aren't responsible for what the rogue AI says. And the tribunal essentially laughed them out of the room. They rejected it completely. The ruling was clear. If you choose to deploy an automated system on your website to interface with the public, you own every single word it generates.

[00:28:07] The failure wasn't that the AI hallucinated. The failure was that Air Canada deployed a system without the necessary human oversight constraints to prevent it from inventing policy. And then bringing it back to the hook of our deep dive, there is the infamous lawyer case, Mata v. Avianca in 2023. Right. The nightmare scenario for any professional who relies on research. Two lawyers submitted a brief in federal court containing citations and quotes from previous court

[00:28:34] cases. But they had used ChatGPT to help draft the brief. And ChatGPT did what it does when it doesn't know an answer. It pleased the user by inventing one. It completely fabricated the legal precedents. The cases didn't exist. And again, the judge didn't fine OpenAI. The judge sanctioned the human lawyers $5,000 because the failure wasn't the software. The failure was the humans skipping the verification step. They didn't bot sit. Exactly. And it gets even more expensive in the corporate consulting world. Westover highlights a

[00:29:04] massive scandal in Australia from 2025 involving the global consulting firm Deloitte. What happened there? They had secured a highly lucrative contract with the Australian government's Department of Employment. Deloitte's team utilized a suite of generative AI tools to rapidly draft a major report. But they didn't catch the hallucinations. No, they didn't. They shipped the report and an external academic reading the final published document noticed that several academic references were completely fabricated. And a supposed quote

[00:29:33] from a prominent court case was entirely invented. Wow. What was the fallout? Deloitte was forced to refund an installment of the contract worth $440,000 Australian dollars, nearly half a million dollars instantly wiped out. Not to mention the massive, incalculable damage to their reputation in the public sector. All because a human reviewer rubber-stamped a beautifully formatted hallucination. And it even happens in media, where fact-checking is supposed to be the core competency.

[00:30:02] You're talking about King Features in 2025. Yes. They syndicated a summer reading list supplement that was published in major papers. A freelance writer had used AI to generate the book recommendations. And the AI recommended books that literally did not exist. It invented titles and authors. And it sailed right through the editorial process and into print. When you look at all these cases together, the airlines, the lawyers, the consultants, the newspapers, the common denominator is that none of these are actually AI failures.

[00:30:30] If we connect this to the bigger picture, these are human oversight system failures. The organizations bought the technology to save money, but refused to invest in the human infrastructure required to manage it safely. Exactly. But while these multi-million dollar lawsuits make the headlines, Westover points out that there is a quieter daily toll on individual workers that is perhaps even more damaging to the economy overall. Yes. And he uses a brilliant visceral term to describe it. Work slop. Oh, work slop.

[00:31:00] It is just the perfect word for it. It sounds gross. It comes from a 2025 study by Niederhofer and colleagues. Work slop refers to AI-generated content that looks highly polished and professional on the surface, but when you actually read it, it is completely devoid of substance. It's generic. It's repetitive. It's fluent. Empty calories. And the study found that we are drowning in it.

[00:31:23] In their survey, a staggering 40% of employees reported that they had received AI work slop from their own colleagues. Wait, 40%? That's almost half the office getting spammed with beautifully formatted nonsense by the person in the cubicle next to them. But here is the real tragedy of work slop. It doesn't just annoy the recipient. It actively steals their time.

[00:31:46] The study found that dealing with that work slop costs the recipient an average of two hours to interpret, fix, or entirely redo. Oh my god. So imagine user A is tired. They prompt an AI to write a project update. It takes them five minutes. They feel great. They save time. Right. They send it to user B. But the update is vague, contradictory, and misses the key constraints. User B now has to spend two hours deciphering the slop, tracking down the real data, and rewriting it.

[00:32:14] User A didn't eliminate the work. They just weaponized the bot-sitting burden and dumped it onto their co-worker. The cost of poorly checked AI is simply displaced across the organization. This reminds me so much of Robert Karasek's job demand control model from the late 70s. It's this classic framework for understanding workplace stress and burnout. How does that connect here? Well, Karasek proved that the most psychologically damaging, high-stress jobs aren't necessarily the ones with the most work.

[00:32:44] The highest stress occurs when the demands of the job are incredibly high, but the workers' control over how the work is done is incredibly low. Which perfectly describes the modern bot-sitting nightmare. Exactly. Think about you, the listener, right now. Your boss forged you a 50-page market analysis that a junior analyst generated in three seconds using AI. The demand is massive. But you're accountable for verifying every fact in this massive document by 5 p.m. But your control is zero. You didn't write it.

[00:33:13] You don't know the logic behind it. You don't know what sources the AI used. You are trapped in a high-demand, zero-control situation. It is a recipe for severe burnout. Which brings us to the most vital part of the deep dive. We've mapped the invisible labor, we've diagnosed the psychological traps, and we've seen the disasters. How do we actually fix this? How do organizations and the individuals listening build a sustainable path forward?

[00:33:40] The source outlines five clear evidence-based responses. A playbook, really. Let's dive into that playbook. Section 5. I look at the first two responses as being all about budgeting and tiering. And response number one is the most fundamental shift an organization can make. Make oversight visible. Stop pretending bot-sitting is free. To prove why this is necessary, Westover contrasts a massive failure of measurement with a massive success story. Let's hear the failure first.

[00:34:09] The failure comes from a 2025 study by Becker on software developers using AI coding assistants. They took experienced developers and gave them AI tools to complete real-world coding tasks. What happened? Well, before the experiment, the developers predicted the AI would make them about 24% faster. After they finished, they subjectively felt like the AI had sped them up by about 20%. So the vibes were great. They felt incredibly productive. But what did the actual stopwatch say?

[00:34:38] When the researchers looked at the hard data of task completion time, the AI tools had actually increased the time it took to complete the tasks by 19%. They felt 20% faster. But they were actually 19% slower. Because human beings are terrible at estimating the micro costs of bot-sitting. We don't notice the 10 seconds here and the two minutes there spent prompting, waiting, reading, catching a bug, fixing the bug, and reprompting.

[00:35:07] It feels like fast action, but it aggregates into a massive time sink. This divergence between perceived productivity and actual measured productivity is why organizations cannot just deploy AI and rely on vibes. Okay, so that's the failure. What does the success story look like? Success looks like the Permanente Medical Group. They rolled out an ambient AI scribe system. Oh, let's define that quickly.

[00:35:34] Ambient AI scribes essentially listen to the doctor-patient conversation in the exam room and automatically draft the clinical notes, right? Exactly. It's a massive potential time saver for doctors. But Permanente didn't just throw the tech at the doctors and say, congratulations, your paperwork is gone. They designed the workflow around the reality of bot-sitting. They explicitly mandated that the AI output was strictly a draft. The workflow required the physician to sit down, review the draft, edit it for medical accuracy,

[00:36:04] and formally sign off on it before it entered the record. They built the invisible labor into the official process. And they budgeted time for it. And because they did that, they achieved genuine, sustainable net savings. Across over 7,000 physicians, they legitimately saved over 15,000 hours of documentation time correctly. It works, but only if you acknowledge and structure the oversight, which leads right into the second response in the playbook. Tier verification by stakes. Right.

[00:36:33] You can't scrutinize every single AI output with the intense rigor of a medical record. If I use AI to draft an email suggesting lunch spots for our team meeting, and it hallucinates a restaurant that closed down three years ago, the consequence is zero. We just walk down the street to a different place. You don't need to spend 20 minutes verifying that output. But if you use AI to draft a legal filing, the consequence is your career. And the legal system is actively forcing this tiering right now.

[00:37:00] After that catastrophic Mata v. Avianca case we discussed, a U.S. district judge in Texas, Brantley Starr, issued a groundbreaking certification order. It's a fascinating precedent. Judge Starr mandated that any attorney appearing in his court must formally certify that no portion of their filing was drafted by generative AI. Or, if it was drafted by AI, the attorney must certify that a human being explicitly checked the accuracy of every single AI-generated claim

[00:37:27] using traditional authoritative legal databases. He didn't ban the technology. He just legally mandated the bot sitting. I love that. And entire court systems are adopting this internally. The California Judicial Council passed rules requiring courts to implement policies ensuring their own staff rigorously verify any AI output. And the New York State Unified Court System? Established formal attestation protocols. For high-stakes outputs, a quick passive read-through is no longer acceptable.

[00:37:56] You must perform independent verification. And you must put your professional reputation on the line by signing your name to it. Okay, moving to the third response in Westover's playbook, upstream quality. This is the idea that the best way to handle bot sitting is to fix the errors before the human reviewer ever even sees them. Because downstream review, leaving it all up to the final user, is the most expensive and dangerous place to catch a mistake.

[00:38:21] To illustrate this, Westover contrasts the media company CNET with the financial giant Morgan Stanley. CNET is the cautionary tale here. In 2023, they decided to dramatically increase their output of financial explainer articles using an internal AI engine. But they kept humans in the loop, right? They had editors. They did. But they relied on their standard human editorial review process to catch the AI's errors. And that was their fatal mistake.

[00:38:49] A standard editorial process is designed to catch the mistakes that human writers make. Typos. Clunky transitions. It is not designed to catch AI errors. Because AI doesn't make typos. It makes up math. Exactly. An editor reading a fluently written article about interest rates might not stop to pull out a calculator and recalculate the compound interest formula the AI smoothly inserted into paragraph 4. Because of this structural mismatch in oversight,

[00:39:16] CNET ended up having to issue massive public corrections on 41 out of the 77 AI stories they published. Over half their AI output was factually flawed. So how do you fix that? What did Morgan Stanley do differently? Morgan Stanley's wealth management division took an upstream approach. Before they ever let their financial advisors use their internal AI tool, they built an immense infrastructure of quality control.

[00:39:42] They had senior advisors and prompt engineers rigorously test every single use case against real-world, highly complex financial scenarios. They mapped the jagged frontier for their employees. Exactly. And crucially, they implemented what software engineers call regression suites. Let's explain regression suites for a second because this is a great concept. It's basically like an automated daily fire drill for the AI. That's a great way to put it.

[00:40:08] You have a standing set of complex test questions that you know the correct answers to. Every single night, you run the AI against those questions. Because large language models can drift, right? Right. Their behavior changes as they are updated. The regression suite ensures that the model is still giving compliant, accurate, tested templates today, just like it did yesterday.

[00:40:29] By building all that quality control upstream, Morgan Stanley ensures that when the individual advisor opens the chatbot on a Tuesday morning, they aren't starting from scratch. Which makes the downstream bot sitting manageable. Okay. The fourth response in the playbook is about resisting bias by intentionally designing human roles to combat that devastating automation complacency. And this is so critical that governments are actually legislating it.

[00:40:55] The European Union's new AI Act, specifically Articles 4 and 14, legally requires that any high-risk AI system must be designed in a way that allows natural persons to effectively oversee it. But what does effective oversight actually mean in practice? The legislation specifies that the human overseer must have real authority to override systems. Authority. That is the magic word. Because think about a low-level claims adjuster at an insurance company.

[00:41:25] They are told to use an AI to process claims. And their performance is measured entirely by speed. Right. If they spot an AI error, but investigating it will take an hour and ruin their metrics, they don't have real authority to override the system. They're just a human shield for liability. They're just a compliance checkbox.

[00:41:44] If you want a human to effectively bot-sit, you have to protect their time, shield them from pure speed metrics, and give them the genuine psychological safety and organizational authority to say, the machine is wrong and I am stopping the line. Which brings us to the fifth and final response in the playbook. And this one is the most drastic. Setting boundaries. Sometimes the only way to manage bot-sitting is to realize the AI shouldn't be doing the task at all.

[00:42:12] Westover calls this hitting the automation boundary. We are finally seeing major corporations try to automate everything, hit the boundary, and actually walk it back. The pushback is real. And it's happening in customer-facing roles where human nuance is critical. Look at Klarna, the massive payments company. They made global headlines by bragging that their new AI assistant was doing the work of hundreds of human customer service agents. But then reality set in.

[00:42:39] By 2025, they realized that completely removing the human layer was fundamentally damaging the quality of their service. So they reversed course. They announced they were bringing back pools of human call center agents. It's almost like they realized customer service isn't just about answering an FAQ. It's about de-escalating human emotion, which AI cannot do. And there was a huge case with the Commonwealth Bank of Australia, too. Yes. The bank tried to eliminate 45 direct customer service roles, claiming their AI voice bot had reduced call volumes.

[00:43:08] But the frontline workers and the union fought back with actual data. They proved that call volumes were actually rising, and the calls were becoming more complex and angry because the AI couldn't solve the nuanced problems. It got so bad that branch managers were being forced to jump onto the phones to handle the overflow. So they didn't reduce the work. They just pushed the bot sitting onto the local bank managers. In August 2025, the bank had to issue a public apology and entirely reverse the 45 AI job cuts after union pressure.

[00:43:37] Even McDonald's hit the boundary. They were running a highly publicized pilot program with IBM to use AI voice ordering at their drive-thrus. But they abruptly ended the pilot due to accuracy concerns. What all these companies learned the hard way is that measuring automation success purely by cost per transaction is a massive dangerous trap. Is it safe to say that removing the human entirely doesn't eliminate the work? It just outsources the bot sitting to the angry customer. I completely agree.

[00:44:06] Because on paper, an AI chatbot handling a customer query costs pennies while a human agent costs dollars. It looks like a massive win for the AI. Right. But if you ignore the repeat context because the AI didn't actually solve the problem and the abandoned purchases because the user just gave up, you aren't actually eliminating costs. You are just hiding them. Exactly. This has been incredibly tactical for organizations.

[00:44:32] But as we move into Section 6 of Westover's paper, the focus shifts to building long-term capability and the future of expertise. This is an almost existential question for the listener. Addressing your daily workflow is one thing, but how do you protect your own career and your own expertise over the next decade? What happens to the very concept of a human expert? To answer this, Westover resurrects a classic, brilliant paper from 1983 by a researcher named Lizanne Bainbridge.

[00:45:01] The paper is titled Ironies of Automation. I found this concept chilling. What is the core irony Bainbridge identified? Bainbridge observed that when engineers automate a process, they naturally automate the easiest, most routine, most predictable parts of the job. Because if it were hard and unpredictable, the machine couldn't do it. That makes total sense. But here is the irony. By automating all the easy stuff, you leave the human operator with an arbitrary collection of residual tasks.

[00:45:28] The hardest, most complex, most unpredictable edge cases. The exceptions. And simultaneously, by taking away the routine work, you deny the human the daily, repetitive practice they desperately need to keep their skills sharp. Oh wow. If you never manually steer the ship through calm waters, you won't have the internalized physical intuition required to steer it when a hurricane hits and the autopilot fails. Exactly. You lose the foundational expertise required to handle the complex exceptions. Applied to knowledge work today, that is terrifying.

[00:45:58] Think about a law firm or a consulting agency. And the junior staff, the analysts, the first year associates, their whole job used to be doing the routine work. Writing the first draft. Pulling the case law. Doing the reps. Doing the reps. But if organizations use AI to do all that entry-level work, and the junior staff never have to stare at a blank page and reason through a problem from scratch, they will never develop the craft.

[00:46:25] And that craft, that deep, hard-won internal mental model of the domain, is precisely what is required to successfully bot sit. It is the only thing that allows you to spot when the AI is subtly wrong. Think back to the radiologist study. Why did the veterans survive the AI's errors better than the juniors? Because the veterans had decades of doing it the hard way. They built a massive internal database of subtle visual cues. The junior doctors hadn't built that compass yet.

[00:46:54] This raises an important question. How do we train the next generation of experts if AI takes away all the entry-level practice work? It is the defining human capital challenge of our era. Organizations are going to have to deliberately and artificially preserve opportunities for young professionals to do core work entirely without AI. This means the most valuable skill for the listener isn't just knowing how to prompt the AI, but knowing how to independently verify the truth. Precisely.

[00:47:23] The people who can act as the ultimate, reliable, human backstop to that invisible, jagged technological frontier, they will be the most scarce and valuable assets. Okay, this has been an incredibly deep and vital journey. Let's recap what we've discovered today.

[00:47:41] We started by confronting the shattered promise of immediate AI liberation, facing the reality that over 82% of weekly AI users are doing the hidden labor of bot sitting. And many are doing it for six plus hours a week. We explored the anatomy of invisible labor, context provision, defensive monitoring, correction, and escalation.

[00:48:05] We unpacked the human-machine friction, learning how automation bias and the jagged frontier hijack our cognitive defenses, making even veteran doctors fall for articulate lies. And we saw the catastrophic legal fallout and the daily misery of work slop. And finally, we walked through the playbook. Budgeting time, tiering verification, building upstream quality, demanding real authority to override, and fiercely protecting the human boundary.

[00:48:36] So what does this all mean? The core realization is that bot sitting isn't a sign that AI is failing. It's a sign that AI has entered the real world where outputs have actual consequences. And to build on that summary, I want to leave the listener with a final provocative thought concerning capability erosion.

[00:48:55] If genuine expertise is only built by doing the hard manual work that AI currently bypasses, will doing it without AI become a premium training commodity in the future? Oh, that is fascinating. Like a luxury good in the labor market. Exactly. Will the deliberate lack of AI use become a mandatory curriculum or even a high-status luxury symbol for the elite professionals of tomorrow just to prove they actually possess human judgment? The artisanal, handcrafted knowledge worker.

[00:49:24] I absolutely love that reframing. Thank you so much for guiding us through this incredibly complex landscape today. And to you, the listener, I want you to take a hard, honest look at your week ahead. Think about how much invisible labor you are doing. Make your bot sitting visible, valued, and sustainable. Until next time.