On this week’s episode of Inside Outside Innovation, we talk about some recent Stanford research, how designing for disability sparks innovation, and the hidden dangers of shipping too fast. Let’s get started.
Inside Outside Innovation is the podcast to help innovation leaders navigate what’s next. Each week we’ll give you a front row seat into what it takes to grow and thrive in a world of hyper uncertainty and accelerating change. Join me, Brian Ardinger and Miles Zero’s, Robin Bolton, as we discuss the latest tools, tactics, and trends for creating innovations with impact, let’s get started.
Podcast Transcript with Brian Ardinger and Robyn Bolton
AI Reasoning Risks, Inclusive Design Innovation, and the Hidden Cost of Shipping Fast
[00:00:30] Brian Ardinger: Welcome to another episode of Inside Outside Innovation. I’m your host, Brian Ardinger. And with me I have Robyn Bolton from Mile Zero. Robyn, welcome again.
[00:00:48] Robyn Bolton: Thank you again.
[00:00:50] Brian Ardinger: We have another amazing week ahead of us here. We wanted to share all the exciting things in the world of innovation that we’re running across.
First, I guess we’ll get right into it. We’ve got a number of articles that have touched our lives here. The first one I want to talk about, Stanford just published an uncomfortable paper looking at LLM reasoning, and some of the findings were kind of incredible. Basically, the gist of it is if you look at the LLMs, it sometimes goes to a point where it is creating an environment where it’s leading you to believe that it is confident in its answer, but it is not, for lack of a better term. That is what it’s all about.
[00:01:27] Robyn Bolton: I mean, it’s so perfectly worded. This is worse than being wrong because it trained users to trust explanations that don’t correspond to the actual decision process. And I will say I’ve seen that time and time again using different LLMs and have totally fallen victim to it is I’ll kind of quickly scan the response, really read the end when it kind of gives me the key takeaway, I’m like, yeah, that sounds right, and then go on.
And then it’s only later I’m like. Ugh. I fell victim to AI work slop because the reasoning doesn’t hold. So, it’s an easy track to fall into and a good one to just constantly be on guard for.
[00:02:09] Brian Ardinger: Yeah. The fact that the models produce unfaithful reasoning gives you this you think this is a correct answer, provides explanations, but when you ask it to explain it, the actual logic that it explains back to you is wrong or incomplete or fabricated.
So, it provides that sense that you’re on the right track. But the LLM itself can’t reason. And that inability to reason will take you down particular paths and even to the extent you could even change a single word or a phrase within your prompt, and that can take it down a particular path that, again, logically it doesn’t make sense.
And so, it’s not consistent even down to the word of the prompt that you put it into. So, all that to say it’s getting better, but it’s still not a thinking device and it’s not a reasoning device. Be careful when you’re using these particular methodologies and that. Don’t be a hundred percent confident in everything that comes out of it.
[00:03:02] Robyn Bolton: Yes, trust for verify.
[00:03:04] Brian Ardinger: There I go.
[00:03:04] Robyn Bolton: Or maybe don’t trust and still verify.
Designing for Disability as a Catalyst for Breakthrough Innovation
[00:03:08] Brian Ardinger: Alright, the second article from HBR is how designing with disability in mind sparks innovation. So, this was a great article. Oftentimes, I think when we’re building new, innovative things, we think about the amazing things that we’re going to create.
And this article talks about how oftentimes you can think about it differently and actually create new things by designing for the marginal case or folks, for example, with disabilities.
You can design for amplifying use cases that don’t normally happen, but by focusing on that, you can actually create new innovations and new ways of thinking about how to develop a new product.
[00:03:45] Robyn Bolton: This is such a great reminder and great call to action for innovators, and it reminds me, I think, as I mentioned to you, one of my favorite stories, which is about Oxo, the kitchen tools, the can openers, the spatulas, all of that, and how they were originally created for people with rheumatoid arthritis.
And you know, now, like Oxo is the only brand that I’ll buy for Kitchen Tools because they’re just so comfortable to use. And so it’s just again, a great illustration of how designing for a really, really specific, even niche customer and designing really well and thoughtfully for them, that the market will expand because I mean, honestly, even look at sidewalk cutouts. You know, the kind of like little rams. We all use them, but they were made because of the ADA, the American with Disabilities Act. So, find a really awesome niche and delight those folks and you’ll be surprised kind of what comes along.
[00:04:44] Brian Ardinger: Yeah. The article talks about, an example, I think it’s Butte is the company they created the, you know, the walk-in tub and people are like, why is, this is kind of crazy idea. Why can’t people just get in a tub?
But it’s the idea of like opening a door and rolling in or getting in there and then shutting it and then being able to actually take a tub experience. And again, something that was developed with kind of disabilities in mind, opens up a variety of different use case scenarios and users that they didn’t originally plan in the particular process.
The last part about it is as a product developer, software developer, et cetera, you can use this methodology to think about new ways your product or services could be used by narrowing down and saying, okay, what if we had to design this particular product or service with this in mind? How would that change the dynamics? How might that open up new opportunities for us and new markets that we never thought of before?
[00:05:34] Robyn Bolton: Constraints drive creativity, always.
The Hidden Danger of Shipping Fast. Speed, Bottlenecks, and Customer Attention
[00:05:37] Brian Ardinger: And the last article is from Product for Engineers. It’s called the Hidden Danger of Shipping Fast. And the basic premise asks, is it possible to ship too much or too fast? The answer is yes, probably. And it goes into talk a little bit about the fact that, again, we are in a, an environment where speed to market and speed of creating things is speeding up such that you could constantly be creating new features, new functionality, new things to test in front of your marketplace. you have to be careful at sometimes because you could almost outpace the usage or the ability for the consumer themselves to understand all the change and or interact with that change.
And so, the idea around getting that right balance between shipping fast and getting new features and solutions out to the marketplace, and then letting the market actually understand and adapt and use those. Because oftentimes when the market has a chance to use something, they actually give you different feedback than if you give them a chance to use it than what you’re thinking. And so if you’re doing nothing but shipping new stuff to them, you may actually mess up future opportunities as well.
[00:06:45] Robyn Bolton: I love this in so many ways because speaking of constraints, this article mentions the theory of constraints and one of the principles that it lists is that when upstream output increases without increasing downstream capacity, the system destabilizes. That is language that the engineers will know and love. What that means to the rest of us is, you know, you can only get so much stuff through a funnel. And there’s always a bottleneck.
And humans, bless our hearts, we will always be a bottleneck. We can only take in so much at a time, learn so much at a time, change so much at once. Just because you have infinite capability to produce and put things into the market, build agents, et cetera, et cetera. Sometimes it’s a whole lot of activity for not a lot of achievement, not a lot of change, not a lot of benefit. Again, just something that we have to keep in mind is that more is not always better sometimes better is better.
[00:07:46] Brian Ardinger: And I think we oftentimes underestimate one of the prime bottlenecks is attention itself. As a founder and you’re working with your product, and that you’re into that all the time. And you think that your customers are always into it at the same level that you are.
And if you’re constantly bombarding them with new stuff, the amount of tension they may not even see the stuff that comes through on a regular basis. So that attention bottleneck is one to be aware of as well.
[00:08:09] Robyn Bolton: Yeah, it goes back to our conversation on minimum viable products versus minimum desirable products, and you need to evolve.
[00:08:17] Brian Ardinger: All right, well this is the segment we usually go, Tactics to Try, but this week we’re going to flip it on its head and we’re going to ask you to send us in what are some of the tactics that you’re using nowadays to improve your innovation or increase your output. We’d love to hear from you. Keep us posted, send the information to Brian at insideoutside.io. We’ll take a look at those emails and hopefully share some stuff in in the coming weeks and months. And with that, that’s another episode of Inside Outside Innovation. Thanks for coming out. We’ll see you next time.
[00:08:45] Robyn Bolton: See you soon.
[00:08:48] Brian Ardinger: That’s it for another episode of Inside Outside Innovation. Today’s episode was produced and engineered by Susan Stibal. If you want to learn more about our teams, our content, our services, check out insideoutside.io or if you want to connect with Robyn Bolton, go to MileZero.io, and until next time, go out and innovate.
Articles Discussed
- Summary of Large Language Model Reasoning Failures – God of Prompt
- How Designing with Disability in Mind Sparks Innovation – HBR
- The Hidden Danger of Shipping Fast – Product for Engineers
