On this week’s episode of Inside Outside Innovation, we talk about the changes in lean analytics, how the human in the loop is a lie we tell ourselves, AI innovation, and the first thing to break when moving fast. Let’s get started.
Inside Outside Innovation is the podcast to help innovation leaders navigate what’s next. Each week, we’ll give you a front row seat into what it takes to grow and thrive in a world of hyper-uncertainty and accelerating change. Join me, Brian Ardinger, and Mile Zero’s Robyn Bolton as we discuss the latest tools, tactics, and trends for creating innovations with impact. Let’s get started.
Podcast Transcript with Brian Ardinger and Robyn Bolton
Angel Investing, AI Travel, and What’s Ahead
[00:00:30] Brian Ardinger: Welcome to another episode of Inside Outside Innovation. I’m your host, Brian Ardinger, and with me, I have Robyn Bolton. Hello, Robyn. How are you?
[00:00:46] Robyn Bolton: I am good. How are you, Brian?
[00:00:47] Brian Ardinger: I’m doing well. I’m actually heading to Minneapolis tomorrow for the Angel Fest conference. Meeting with our former colleague, Emily Kist, who’s with Groove Capital, Mickayla Russard, who spoke at the IO Summit in 2024.
They host an annual event called Angel Fest, and we’re excited to go up there and represent the Nebraska Angels and see what’s going on north of what’s going on in our world from angel investing. So that’s what’s going on in my world.
[00:01:12] Robyn Bolton: Awesome. Well, probably by the time these airs, I will either be on or be back from my AI-planned summer vacation. So, long-time listeners of the podcast may remember where I talked about my husband and I’s usual vacation to Key West was upended by the Key West regulation changes about short-term rentals, and so AI planned a trip to Turks and Caicos for us.
[00:01:36] Brian Ardinger: Oh.
[00:01:36] Robyn Bolton: So, the proof is about to be in the pudding. The next podcast, I will report back as to whether or not you should let AI be a travel agent.
[00:01:46] Brian Ardinger: That’s right. You’ll probably end up in St. Barts, but who knows?
[00:01:50] Robyn Bolton: You know, at some point it doesn’t really matter as long as it’s warm and sunny and there’s a beach.
[00:01:55] Brian Ardinger: That’s right. Sand is sand.
[00:01:57] Robyn Bolton: Sand is sand.
Lean Analytics in the Age of AI
[00:01:59] Brian Ardinger: Excellent. Well, let’s get started with some interesting articles that we’ve come across over the last week or so. First one is from our friend Ben Yoskovitz, and it’s an article that he wrote about lean analytics, and it’s Lean Analytics Reconsidered.
So, people familiar with the podcast know Ben and Alistair Croll. They wrote Lean Analytics back in the early days of the whole lean startup movement, and now it’s become one of the, I think, one of the core Bibles out there when it comes to, you know, how do you measure and understand what you’re building in the world of technology and that.
Ben goes back and looks at; the book’s probably a dozen years old now. How much does it hold up, the frameworks that they developed, and what’s changed in the world of AI? And what I liked about the book is it really kind of walks you through the stages that a company can go through, and the stage gates that you have to go through to kind of create traction and create product market fit, et cetera.
You know, everything from starting at empathy, stickiness, virality, revenue, and then finally scaling. And so the core portions of the book as far as the stages you go through and the, what you’re trying to solve at each particular stage rings true. But he talks about some of the things that are different when it comes to AI and how that may be changing the game from that perspective. What were your thoughts?
Rethinking Engagement Metrics for AI Products
[00:03:07] Robyn Bolton: Yeah, this was really interesting. I mean, it was thrilled, not surprised, that the principles hold. I mean, if you have good, grounded principles, they should be able to survive a lot, and granted, AI is a lot, but it was great to see that the principles hold. What really struck me was one of the shifts, shift three, on engagement is directional, and basically this is talking about how traditionally you look at engagement, and the more time spent on site is good.
You want people to spend a lot of time on a site. You wanted people to spend a lot of time in a session, and AI is essentially flipping that on its head, and it just made me think of like, yeah, the more time I have to spend on a site is actually more indicative of the more time I’m struggling to get done what I need to get done.
So he breaks it down into there’s time spent struggling, there’s time spent with AI doing the work on user’s behalf, time spent exploring or creating, and then zero user time tasks completed. And I just saw my AI usage, my site usage in all four of these categories, and realizing I go back to the tools, the sites, et cetera, that don’t make me use a lot of time or engages me in the right amount of time.
But if I spend too much time struggling, which given my attention span is like 10 seconds, it is not good. It is not a good metric to be like, “Oh, she spent three minutes on the site.” That is bad for you, my friend.
Why Quality Is Becoming a First-Class Metric
[00:04:38] Brian Ardinger: Yeah, so if more engagement, the more money, and as long as dynamics works, that’s great, but you have to be at least aware of it.
The other shift that I think is kind of interesting is he calls it quality is a first class metric now, and again, it goes back to this idea that just because you can build it and it, you can build it 80% doesn’t necessarily mean it’s going to be the right solution out there for the marketplace. An 80% good product versus a 95% good product feel completely different to a product or a user, and so, you know, how can you build for quality of those interactions and that, because everybody can get to an 80% now.
If we think about, again, what is making AI stand out and then where can you stand out and differentiate yourself, it comes back to a lot of those human things, taste, quality, access to the customers, relationships. And I think we have to continue to pound that home again, because we’re living in a world where commoditization of the actual product is becoming more easily built.
[00:05:34] Robyn Bolton: Plus, he paired that point with a Ron Swanson GIF.
[00:05:37] Brian Ardinger: GIF, yes.
[00:05:38] Robyn Bolton: From Parks and Rec, which just tells you how important that point is.
The Problem with Human-in-the-Loop AI Innovation
[00:05:43] Brian Ardinger: Excellent. All right, the second article is from Alvis Ng. He has an article in Medium called Human in the Loop is the Lie We Tell Ourselves. Again, we hear a lot about this as we’re talking about AI buildouts, and, you know, making sure that you have a human in the loop.
Make sure that the agents aren’t doing things that you don’t know what’s going on. And he goes into quite a bit of detail to say that it’s really a lie that we’re telling ourselves. The fact that we oftentimes put a human in the loop, but we make it so, so easy, or there’s no friction to click on that button.
This agent comes back and says, “Hey, check this.” You just automatically click it and go to the next thing and let it run its next test. And so, we’ve got to be very cognizant of the fact that what friction are we building, and making sure that that human in the loop is actually a friction point that allows us to test and try and say, “Is this on the right path? Is there something wrong?” Versus just making it an automation checkpoint or automation on the road that people have to go through, and it actually is defeating the purpose.
[00:06:41] Robyn Bolton: Yeah, I mean, one of the sentences he has here is, “You did not review a decision. You reviewed an artifact.” I find myself; I find so many other people I talk with falling into that, as you look at the output, the headline, and it makes sense to you, and you’re like, “Yep, great.”
And you don’t take the time, and so you’re reviewing the artifact, you don’t take the time to really dig in deeper, and pressure test the logic and double-check the resources. And so yeah, a human’s in a loop, but technically the brain is not in the loop.
Adding Real Friction to AI Decision-Making
[00:07:14] Brian Ardinger: And the fact that we are pushing more and more quantity out.
[00:07:18] Robyn Bolton: Mm-hmm.
[00:07:18] Brian Ardinger: It’s almost impossible to do actually all the tests that you need to do, you know, in a timely manner. When the machine can go double, triple, whatever, faster than you can, it makes it that much easier or tempting to say, “Okay, yeah good. Go, keep going, keep going, keep going.” But again, I think this was an important article to talk about the fact that-
[00:07:36] Robyn Bolton: Yeah.
[00:07:36] Brian Ardinger: You know, keep in mind what we’re actually doing and what are we actually creating, and make sure that what we say we’re creating is actually what we want to create in the process.
[00:07:45] Robyn Bolton: Yeah, it’s just so important, especially as more and more we get evaluated by the quantity of output. You know, the speed of decisions. You know, if you’re doing something wrong, you’re gonna have to do it over, so you’re not really saving time. You’re not saving energy, and we need to bring back human in the loop.
AI Automation vs. AI Augmentation
[00:08:06] Brian Ardinger: Excellent. All right, the third one is from Harvard Business Review, Why Companies That Choose AI Augmentation Over Automation May Win in the Long Run. And, this is a nice summary of, if you think about a lot of folks are jumping into the AI bandwagon, looking at it as a way to automate and execute on things that they already know, rather than using AI as a way to augment or change or transform, what people are building.
You know, leaders are making a choice about their strategy, looking at improving the bottom line and automation, headcount reduction, versus, again, using it as a way to augment and make things better, faster, stronger rather than just cost savings.
And the article goes on to look at and say that companies that are ahead of the game are looking at it less from the automation perspective and more from the augmentation perspective.
[00:08:53] Robyn Bolton: I think this is another example of and not or. Companies are going to have to do the automation just because it’s going to become a standard of doing business.
You know, if you haven’t driven out costs and made things more efficient, like, you’re going to fall behind financially, and then that’s going to impede your ability to grow, do things different, et cetera. And so yes, you have to do the automation, and you have to do the augmentation because that is where, you know, we can amplify what humans do.
We can amplify creativity. We can move appropriately faster, not faster breakneck speed. But this is where kind of innovation and growth can come from, is the augmentation.
Employee Perception and AI Adoption
Another point that I thought was really important to be made was that leaders also need to stop underestimating employee perception. And the behaviors that result from employee perception. So if employees think that they are training their who’s going to replace or what is going to replace them, they’re going to resist. If they see AI and tools as ways to help them be better, but they can never replace them, the resistance level falls. Not saying it goes away. They see the technology as more of a help than an existential threat.
[00:10:17] Brian Ardinger: Exactly. And I think that goes a long way to, again, trying to understand what are the motivations and the fears that people have. AI overlords have not done a good job of the marketing of the message where if they’re constantly saying, “We’re going to destroy all the jobs. Here, keep using this.
[00:10:32] Robyn Bolton: Yes …
[00:10:32] Brian Ardinger: particular device.” Tough marketing message to swallow for most people.
[00:10:36] Robyn Bolton: Yes. You have to use the device this many times on this many things so that it can learn faster, and you’re like, “No, this is a trap.”
When AI Speed Breaks Team Communication
[00:10:44] Brian Ardinger: All right. Well, the last article of the week is from Dave Rupert, and it’s When Moving Fast, Talking Is the First Thing to Break. So, Dave talks about, AI can help us build faster, but w- in that process, a lot of times what breaks is we stop talking to each other. When the AI expert is telling me that I’m great, and it’s moving me along, and I can get answers quicker and faster than asking my colleague, one of the first things to go is asking my colleague for feedback or insight or things along those lines.
And so it’s a cautionary tale again to think about what is breaking when we’re introducing and using these particular tools and be sure to understand where those breakages can happen and how to overcome them.
[00:11:29] Robyn Bolton: This was such a great reminder. I mean, it, in a lot of ways, it feels similar to one of the tenets around even just customer insights and understanding your customer.
That you have to go talk to them and spend time with them, understanding the why behind the what they do, and companies that just send out surveys being like, “Did you like this service?” aren’t going to get the insights that will enable them to leapfrog their competition or even move into the next phase of their growth.
Why Conversations Still Matter in Innovation
And so that in customer insights we’ve always known has been really, really important to have the conversation, and this was a wonderful article highlighting, yeah, those conversations also need to happen within the business and with your colleagues. And yes, it’s so much easier to have a conversation with Claude.
Yes, Claude and Gemini and everybody else loves to reassure you about how brilliant you are. You know, if you’re brilliant but wrong, it’s just going to create more work, more pain. I think he says, “It makes future conversations more difficult thanks to higher sunk costs and deeper entrenched opinions.”
[00:12:36] Brian Ardinger: I think the other thing that this kind of emphasizes the fact that in the past, building technology products, it was all about minimizing the friction, making it as fast, as efficient as possible.
And I think what we’re seeing now is that friction oftentimes is an advantage or avoiding friction because you don’t want to have that conversation that either slows down the decision or makes you feel like you’re in the wrong or whatever. Sometimes those friction points actually are value creation spots.
Building for Productive Friction
[00:13:03] Robyn Bolton: Yeah.
[00:13:04] Brian Ardinger: So how do you build for friction, which is kind of, again, a different way of thinking that we’ve been doing in the past, where try to remove all the friction. And so I think consciously thinking about where, where does friction add value in the, in whatever we’re creating for the f- customer or for the colleagues that we work with, I think is an important thing to think about.
[00:13:22] Robyn Bolton: Yeah, absolutely. I mean, we talk about sparks of ideas all the time. It’s sparking. Sparks are a big term in the innovation space, and you aren’t going to get sparks without friction. So there does need to be friction in the process. We can’t take it all out, because then otherwise we take out all the sparks of creativity and ideas.
[00:13:43] Brian Ardinger: That concludes another episode of Inside Outside Innovation. Thanks for coming out, and we’ll see you next time.
[00:13:48] Robyn Bolton: See you next time.
[00:13:54] Brian Ardinger: That’s it for another episode of Inside Outside Innovation. Today’s episode was produced and engineered by Susan Stibal. If you want to learn more about our teams, our content, our services, check out insideoutside.io or if you want to connect with Robyn Bolton, go to MileZero.io, and until next time, go out and innovate.
Articles Discussed
- Lean Analytics, Reconsidered – Ben Yoskovitz
- Human in the Loop Is the Lie We Tell Ourselves – Alvis Ng
- Why Companies That Choose AI Augmentation Over Automation May Win in the Long Run – HBR
- When Moving Fast, Talking is the First Thing to Break – Dave Rupert
