When to Stop and Think: Decision Making Using Pivot Triggers

Tom Kerwin: And then you realize that most of the stuff nobody's using and nobody wants it and it's just making your life harder. So how do you get that to stop? How does a company decide what are we gonna work on? And how do we stop doing things that aren't working much, much sooner.

Ryan Hatch: Welcome to exploring product, where we go behind the scenes on what it really takes to bring new products to market too often, people focus just on the success stories. Our aim is to flip the script. We try to unpack what product teams actually go through when trying to bring new products to market. I'm Ryan hatch. 

Rob Kaminski: And I'm Rob Kaminski. every day. We're trying to build products that our customers love and we know just how messy and difficult product work can be. We don't have it all figured out and we're okay with that. Join us on our journey. As we explore the world of creating new products.

Ryan Hatch: Today we are super pleased to welcome Tom Kerwin. Tom is an experienced product leader with a background in design research and strategy. And Tom, thanks so much for joining us. 

Tom Kerwin: Thanks for inviting me on. I've been, uh, yeah. to to let people know. I've been chatting with Ryan over the past few months and, uh, just we've had super good conversations.Uh, So I'm really, really excited to have this one today. 

Ryan Hatch: Wonderful. So recently, Tom, you wrote a blog post on pivot triggers and I read it Rob and I read it and were like, wow, this is, this is cool stuff. And we're really excited to unpack that with you together today. So really excited to dig into, to, pivot triggers.

Tom Kerwin: Yeah, can't wait, I've been, I've been talking about it a lot, so happy to talk about this [laughs] as well. 

Ryan Hatch: Yeah. Great. Well maybe we can even start off with. So why, why pivot triggers? You know, maybe you could talk about some of the problems you saw, you know, what led you, what's the background story that kinda led you in, in this direction.

Tom Kerwin: Uh, and that is a great question because they are many, many years in the, in the forming and it's kind of the, the the current state of the journey that I suspect all of us as product builders have gone on, which is how do you figure out what to build And how do we learn faster about what is and isn't working?

Um, so I think that the overarching problem is every company always has more to build than we have time and money to do, Where there are more initiatives running than anyone could even talk about, let alone actually figuring out what to do and and make it happen. And e- every single employee in a company usually has their idea that they really think the company should be doing.

If only they could get them to do it. But we also know. From our past experiences, usually the all initiatives are not equally valuable. Um, there's usually one or two that have gone incredibly well and been super valuable and then loads of others that, that haven't really worked. And I know so many product companies who stumbled across an initial idea when they were a tiny startup and they were really lean and mean and scrappy and they get that and then it it kind of works and it catches on and they they build that.

And then they hit, well, we'll, we'll do that again. We want the next idea. And I think people f- forget what it was actually like coming up with that first idea and they think they can do it in a nice clean way. And you end up with this sort of a creation mode of building a product where you just adding more and more stuff on.

And then you realize that most of the stuff nobody's using and nobody wants it and it's just making your life harder. So how do you get that? to stop? Um, how do you help, it help? How, how does a company decide? What are we gonna work on and how do we stop doing things that aren't working much, much sooner? Um, often we think we have to finish it and then look back and go, oh, well that worked or that didn't work.

I guess we just wasted that 12 months. Or actually you can not often not say that 'cause that's too painful to realize. So how do you figure that out much, much quicker. And so, so yeah. to, to go Back in time a little bit. And look at the, the past, I started out as a, as a designer researcher, primarily at different product companies driving several different places and I kept seeing these projects happen where uh, there's an example I can think of from like five years ago where This big initiative started, everyone got alignment. There was loads of energy behind it. The stakeholders bought in and it was gonna repurpose part of their product line to focus on this specific need that they'd seen out in the market. And They realized, look, there's these people with this problem, and we can go and solve the problem for them using this thing. And so off, they went big plans and they talked to the researcher on the project. In this case, wasn't me. It was someone else they talked to him said, oh, can you find us some, some customers who have this need, and then we'll we'll talk to them and we can figure out how to make this flow really good.

And he came back after a couple of weeks. He said, I actually can't find anyone with this need. No, nobody has. it. Are we sure we wanna keep doing this. They said, go, yes. You just go, go, keep looking, go and find some, or get some people who nearly have the needs, some adjacent ones. And he went and looked again. Couldn't find anyone. So captain, are you sure we wanna do this? I've talked to some people and they have this different need. We could do that instead. No, no we're doing this off. we go. Keep going. And so It went on like that. I'm sure you can tell how this story ended. The product launched after six months of hard graft and the team really like pulling hard and heavy lifting after which they managed to get one customer. And then that customer was declined from being able to use the product. And three months later, we'll just sunset that and put it away. But we all learned a lot and you know, you've got to learn new failures and everything like that sort of really, really, be good. Only We knew after two weeks, we knew after two weeks.

So why couldn't we get that zombie initiative to die sooner? And I really struggled with this and found that one thing is painful. to kill ideas. So once that project had all of that energy pumped into it, all of those people's time and emotional investment. If you kill that project, you suddenly turn all of that investment into a sunk cost.

And suddenly it's your fault. The person who found out that this is not, not working is the person who now cost the company, all of that invested energy and all of that. you don't wanna be that person. It's also. Really ineffective. 

Ryan Hatch: no, no one wants to be wrong. No one wants to take the fall. I mean, w- w- what is it?

Tom Kerwin: I think it's, I don't, ...I don't know if it's quite that and the- there's certainly elements of that. I'm sure you've seen elements of that as well in the world. Nobody wants to be wrong and it, ye- yeah, I think it's, it's painful to kill your darlings. Isn't it? It's painful to realize that that thing that you thought was gonna work is not gonna work and.

I think really it's the sunk cost fallacy comes in a lot, but we've spent so much time and energy on this. We have to make it work. And even if you realize that it's, a, it's gonna be, it's just uphill and that there is no, there, there you kind of, yeah. You know, I've seen, I've seen so many teams. I think I wish I knew exactly why it happened. Do you have any ideas, What, what do you think it is? 

Rob Kaminski: I think for me you know, when, I, when I being sometimes being that person of like, it's not gonna work. And for those that maybe hold that back, I think there's this part of being, not a team player. Like you're not supporting the team or the company's vision by saying. that. And so you're sort of like, okay, like, I'll keep it a little quiet.

I'm still supportive of the idea. And what you're really doing is enabling it to continue a little bit further and build up a little bit more momentum. 

Tom Kerwin: So [crosstalk 00:07:56]. 

Rob Kaminski: I think that's one thing to get drivers there, but 

Tom Kerwin: Yeah. actually another one I think, I think you're exactly right. And another thing that just occurred to me is when you all have this belief that something is gonna work and then the world says, actually, you're wrong." Well, you've got to do the very heavy, emotional work of dismantling and reassembling your model of how the world works. 

Rob Kaminski: Mm-hmm [affirmative]. 

Tom Kerwin: And nobody really wants to do that. [laughs]

Rob Kaminski: [laughs] 

Tom Kerwin: Or if you do, it's, it's expensive. It takes time and energy, and it takes a safe environment where it's okay to. do that. Um- 

Rob Kaminski: yeah, 

Tom Kerwin: and, but that's what we mean by learning. How do you enable learning? Well, when you learn, you, you develop a better model of the world and a better understanding of, of how things interact, 

Rob Kaminski: Yeah. 

Tom Kerwin: but it doesn't come easy. 

Rob Kaminski: It also it triggered me when you talked about, um, finding the first product that works. And I think about startups and entrepreneurs, who are, you talked about and being much more scrappy. I think that the stakes are different, right? You, you have to be really honest about the reality of the world you're in. because-

Tom Kerwin: Yeah.

Rob Kaminski: ... that's ...what ultimately is gonna drive that first revenue. Whereas when something's already working, you can work more in that hypothetical space or existing companies and trying new things of like, "Oh, we already know how it works and you can kinda be ignorant of it where startups don't, if if you don't figure out the real world and what the market says, you're gonna die in that space. Yeah. 

Tom Kerwin: Yeah.  It keeps you honest. Definitely. 

Rob Kaminski: [laughs] 

Tom Kerwin: Although it [inaudible 00:09:21] here about stuff that used to be the case that startups would just they'd write a business plan proposal with no evidence and then go out and get billions of dollars of funding and then spend three years building it and then launch it to critics and then quietly fold up. Um, yeah, that was that was glory days when they?

Rob Kaminski: [laughs] Sure. Why do you think that doesn't happen anymore? Is it, is it harder to raise money or is it. is, is The the mindset changed for for those founders? 

Tom Kerwin: I think from what I've seen, things changed in what it takes to raise money. and the, the people who are funding are looking for some traction rather than looking for an idea.

Rob Kaminski: Yeah.

Tom Kerwin: ...unless you're in I've, I've heard the crypto space 

Rob Kaminski: [laughs]

Tom Kerwin: ...and in the crypto space, you can write a white paper and then people will throw money at you to go and try it. 

Rob Kaminski: [laughs] 

Tom Kerwin: And I don't know if it's true, but that's what I've heard. 

Rob Kaminski: Yes. 

Tom Kerwin: So I think that it all goes in cycles. It's like, well, what's, what do you need to see to, in order to want to invest your money? What do you believe about the market and where it's going? to go? 

Rob Kaminski: Yeah, 

Tom Kerwin: Um-

Ryan Hatch: it's probably investors smartened up on, on the need for traction, but it's also the cost of starting cost, Cost of getting further of cost of launching actually also dropped, right. Technology is accessible to everybody. What's really interesting Tom, to me about, you know, that story you shared Is we, we think, um, well, we'll, you know, we're gonna be these great researchers. We're, we're gonna be great at customer interviews. We're gonna, we're gonna learn to understand the market and, you know, being product strategy or whatever you wanna, like, call yourselves. Um, let's go out and do this user research customer research and come back and present it.

And and then what? and then it's just like this battle between. You know, how do you, how do you do that? Where people are undervaluing customer research. We think we can save it because we're gonna know the market, but it's kinda turn deaf on the way and shoot when it's shared out. 

Tom Kerwin: Totally tone deaf. And that was the next big learning, I think so first it's, it's painful to hit, to kill ideas in the first place.

Second research makes for a really bad gatekeeper. And there's a, there's a thing there where if you're going out and you're finding, yes, I'm going out and I'm finding reality. And I I used to actually say that the user researchers are like the, the eyes and ears of the organism of the company and we're going and checking out what the real world is like.

And then bringing that information back and telling the organism. but It's it denies sort of core truth, which is the, I mean, the evidence shows us the evidence doesn't persuade anyone. And yet, as people who believe in the power of evidence-based design, we don't take that evidence into account because we believe too hard that evidence should work. It's a bit of a weird wrapping up thing. What I found was in fact, um, coming back with the answer and a bunch of evidence to try and persuade people. Almost never works unless it's persuading them of what they already wanted to believe in the first place. The only thing that I've seen that works is them going on the experience themselves and finding their own conclusions, which may be different from the conclusions we think they should have.

But at least it's it's that experience. ...Again, it's exposure to reality, bringing people along on the journey together and experiencing it together. 

Rob Kaminski: Has that changed the way that you conduct your research early on? what are some of the ways you bring those folks with you? Is it, is it truly with you or are you putting them in situations where there they are gonna see it for themselves?

Tom Kerwin: That's a really good question. Um, and absolutely yes, changed the way I did research. [laughs]. So when I I'm sure, you know, if, if you were around for user research in the, in the earlier days of the, late '90s and the early 2000s, we used to write reports, we'd go and do all the research and do the studies and got card sorting, used to take weeks.

And then you'd create all these graphs and evidence and write up the report and show the methodology and then handed over. And then people would find it on their shelf and never look at it. And it's completely unworkable. So I started to do much more. Um, open research and we'll just, we'll just sit you down and you'll watch the customers and get that awakening to like, oh, they're different than what I thought they were like.

And that, that can be really instructive where I took it finally and, where pivot triggers comes in. And this is the first stage of the process is kind of going on a theatrical time travel journey together to time-travel to after the project is finished And two Different worlds that we might find ourselves in one that is everything's worked amazingly.

And the other one where it's gone so badly we wish we'd never started in the first place. And one thing I've noticed is that it's, it's an uncomfortable thing to do. And it's, based on the, pre-mortem an idea by Gary Klein, um, which is incredibly powerful. One thing I've noticed is if you get the stakeholders and all the builders involved in that often when they're exploring that bad world, two things occur to them. one, which is I didn't actually think about this failing before. And the second thing is, oh, and all of these sound incredibly likely, and they start, there's suddenly a realization that maybe we do need to make sure that we're not going to end up in the bad world, 

Rob Kaminski: Yeah.

Tom Kerwin: ...Um, ...rather than just assuming the good world, so that's that's a really important step, I think.

Rob Kaminski: Yeah. 

...So I want you to go deeper on that with the first step of the process. One of the things that really stood out to me in what you propose is in, in your language might be different. So, correct me if I'm wrong, but letting the stakeholders, uh, start with their idea. That's something. just Sharing the headway. Like we often try and pull them back to like, what are, what are the, what's the problem, right?

We move them into the problem space, but you talk about them wanting to start with the idea. And so you, you let them do that. Can you tell us more about how that works and getting started? 

Tom Kerwin: Sure. Well, I mean, that is the third lesson that I have to learn really painfully, 

Rob Kaminski: Okay.

Tom Kerwin: ...which is just too hard to start with outcomes or to start with the problems and so I think, um, I don't know if this is totally mangling sort of evolutionary biology, and I shouldn't be thinking about this or using this argument, but for most of humanity history, it was kind of, you, you would come up with an idea and get on with it, and and that was the way you worked and and you didn't analyze the problem of a bear running at you.

And and think about all the options you might tac- use to tackle the bearing or you. Just, you just get the hell out of there. And That is a kind of a natural way that we leap to problem solving. And I noticed this over many, many years, I kept trying to pull people to Don't give the solution, tell the problem and let the designers and the researchers and the engineers and the product people as figuring out the solution.

Rob Kaminski: Yeah. 

Tom Kerwin: Um, and it, it just kept not working and it kept people love to talk about their idea. And so we're fine, let's, let's go with that. And it's kind of the jiu-jitsu move of it. Absolutely. We'll go with your idea now let's play that forwards and figure out what it, why it is. You think that idea is so important, what it is that you think it's solving, what, what opportunity you think is tackling and then let's And then as, as I already said, let's look at the opposite. What, and what if it doesn't work out and what if it actually makes things worse? Let's look at that world as well. 

Rob Kaminski: Mm-hmm [affirmative].

Tom Kerwin: ...And make sure we've considered the kind of possibilities. of the idea. not just the, uh, I think it's Luca Delena talks about the, the, uh, I forgot what it's called the control heuristic and effectively the idea. We're going way off topic here.

Rob Kaminski: [laughs]

Tom Kerwin: ... uh, but it's in that your, conscious mind that comes up with lots of ideas, like a stakeholder, uh, or like, like your team comes up with loads and loads of ideas, but what determines whether you actually act on the idea or not is actually the basal ganglia in your brain, and that doesn't have access to all the logic and the abstract thinking that only has a way of measuring the expected emotional outcome. How does it believe based on past experience, we're gonna feel immediately after we start doing that action and it's gonna pick the thing that, that it, feels good and not the thing that feels bad. And I think that roadmaps are a bit like that roadmaps like that with organizations, the things that go on the roadmap or the things that feel good, that feel like they're gonna work out, 

Rob Kaminski: Mm-hmm [affirmative]. 

Tom Kerwin: not the things that we've analyzed and rationally decided and based on evidence. Um, so that, and that's really. I think ...You have to work with how things work. [laughs] Not against them. 

Ryan Hatch: Yeah.  Yeah. I've worked with a natural flow. if people are naturally tending towards solutions, let's not try to fight it. Let's try to actually walking to that. Right. 

Tom Kerwin: Yeah. 

Ryan Hatch: Um- 

Tom Kerwin: It took me so many years to get that. 

Ryan Hatch: Yeah. Yeah.

And well, maybe this is like a natural transition for us. I know you have a visual presented, uh, to, to kinda share with us that explains pivot triggers. I can kind of Pull that up. Maybe you can start to kinda walk through, Hey, what is pivot triggers? Kinda like your steps to the process. And then, you know, we can kinda unpack it and ask questions as you go.

Tom Kerwin: Absolutely cool. Oh, that's just kind of weird. thing. That does not help. Right. There we go. Here's the visual. So yeah, what, what you're seeing here is uh, it's part of a cheat sheet. Um, so I put this together after our chat with Ryan, actually, you said this, this would be really helpful. You're absolutely right. Um, so this is the end result of two pivot triggers workshops.

Um, so one of the things it developed over a long time and and this has started to now solidify into a, into a process that's quite repeatable uh, and doesn't take too long and it breaks down into two workshops, really two collaborative sessions with loads and loads of post-it notes, which I know everybody loves.

Uh, The first workshop is these steps one to four. And the second Workshop is steps five to eight. The first workshop is the one we've been talking about already. This idea of let's outline the initiative and then let's time travel to these two worlds. So we start by talking about the initiative that step one, we're gonna write down the title and a couple of details.

What do we think this is gonna involve? The more crisp we can get about what is gonna be. the, the better it is for the next stage. I think so in the next stage, we don't start with this [inaudible 00:19:50]. 

Ryan Hatch: and,  and what is one, what is one, is that their vision? Like what, what is, what is  that? 

Tom Kerwin: it's the, what are we gonna do? So for some examples, it might be, um, we want to make a reporting product that helps, um, helps our customers to make better decisions. That might be one example, or it might be, we think that. Um, customers are struggling to book interviews with their clients and we want to try, uh, we want to integrate currently in order to help them to better book interviews.

There's a a couple of examples. So an initiative, something we're gonna try. Um, so yeah, those, those are couple examples or it might be, 

Ryan Hatch: [crosstalk 00:20:35].

Tom Kerwin: ...we're gonna build...we're gonna build a feature. We're gonna build a product, something. like that.

 Um, so that's an example, and then he might lay out some of the, some of the plans you have around, that. um, and, and get a little bit crispy it.

Um, so then once it, once you've done that and everyone's on the same page, it's important that this workshop includes, Uh, multiple levels of, of people who are involved. So the stakeholder or the main clients, and also the the team who are gonna be involved in this. Um, so the next thing we do is all wander off into separate parts of the board so that we've got our own space and we're not, we're not shouting out ideas We're writing our own ideas down.

So we get the maximum variety. of, of inputs, the biggest diversity of opinions. And we just describe, We, I, I, we, run the scenario. We're gonna imagine we jump in a time machine and we're traveling forwards to six months after this initiative has finished. And we step out of the time machine to realize we're in a world where, the, the, the project has worked, worked incredibly well better than anyone expected. And we're gonna uh, just write down what spots what's happening in that world. Describe the world from different perspectives. And usually I throw in a few prompts, like, well, let's see, what's it like for the team? What's it like for the the customers? What's it like for the market as a whole? What's the press saying about us?

What's the, as the CEO pop champagne and, and starts, and and says how happy he is with the, with the team. what's he saying? And so we describe all of those things and then come back together with all of our different elements and start to cluster them. And we do this silently as well. Wh- What I tend to do is move all of the, Um, we get everyone to move all the post-its to, to the left of, of a marker and then gradually grab ones that are similar and move them across and start to cluster them into these, these green groups here.

Or there actually is the yellow groups that we we say then for each of those groups, we're gonna name that group and that's the green post-it. So we're getting to, to themes of what that success might involve. So that's the first part and that's all nice and fun of me. We come out really happy that the project's been amazing.

And then I say, okay, now we're going to step back in the time machine. And now we're going to travel to a parallel universe. We're still six months in the future, but in this world, the project has gone horribly. The CEO's phoned up and he's screaming down the phone at us ...he doesn't know what's going on.

It's it's a nightmare. It's so bad. We wish we'd never even started the. project. And that's, that's I think that's the important framing. It's not the project was a good idea, but things went wrong. It's we wish we never even started this thing. What does that world look like? And so then, uh, send them off into the corners again, and and everyone writes down their descriptions of how terrible things would be.

And then we come back together and again, the same process we collect all of the post-its together and then gradually cluster them into groups and then name the groups with these orange post-its. So those are, those are the first few steps. What I've found almost always happens is there, there are some groups which align.

And so you- your sort of good world and bad world is sort of mirror opposites of each other, mirror images of each other. And then other groups which don't, and there's there's things which don't really have a, a, a, fixed opposite. At this point. We've now drawn out a picture of two possible future worlds. And the next thing we want to do, we, don't want to take all of that through. 'Cause as you can see on this board, there's a load in here and people come up with all sorts of crazy ideas. So often there's things like, oh, alien, alien invasion happens [

Ryan Hatch: [laughs]

Tom Kerwin: ...or there's another global pandemic. These are things which aren't really in our control and we can't uh, really influence.

So probably not worth us worrying about too much. Uh, so what, what I then do is, is get people to do a little. vote And just put uh, three dots. And if you're in something like mural or Miro, it has tools for doing this. If you're in person, I'd do this, um, just do the dots with a, with a marker pen, but you wanna do a secret vote first, rarely. So don't let people see what other people are voting for Everybody, Everybody picks the three negative world things that they're the most concerned about happening. And then we see what those votes are. And you can just about making out on this. visual, There are little dots in the corners of some of these tickets.

Rob Kaminski: Mm-hmm [affirmative]. 

Tom Kerwin: And with that, we end workshop one. And then normally, so this is the thing that happens with, with pre-mortems. So Normally, what happens with the pre-mortem is we go through the exercise and everybody has a nice time and then it gets put in a drawer and we don't look at it again. So the important thing we're going to do is use that premortem to determine a plan of action, 'cause this is the other thing I realized companies all need and want.

They don't want a generic general sense of we're going to go out and research and learn. They want to plan for what's happening when, and they want to know what's what to expect at different stages. along the way. So the next workshop is with the team. And it's important that you include the P all the people who are going to be involved in building the thing, running the thing, making the thing happen.

So usually that's your product managers, uh, your designers, your data, scientists, your engineers, customer success, people. um, whoever else is is working on that thing, we get together and we take the set uh, of behavior that the set of descriptions, of the, good world and the bad world that we had uh, described, and then you can see on the next step, step five, we're describing behaviors.

So we separate out the good world and the bad world. And in between, we're gonna start listing behaviors that would be able to see things that somebody is doing, or that you would see a system doing that would be visible detectable to us so that we can start to tease out well, which world. are we in? 'Cause we can to see these things different in those different worlds.

Rob Kaminski: And you do all this without the stakeholder, right? That was that's the one person who may be drops off into this second workshop. 

Tom Kerwin: Yeah. I mean, the stakeholder is very welcome to join. I've noticed that often stakeholders are quite busy and they don't really necessarily want to get into the weeds at this stage.

Sure. But it's, cool. If they wanna join, there is a flip side in that it then changes the dynamic on the team, as I'm sure you've noticed. as well. Um, so this, I think it's, it's up for grabs. None of this is set in stone. None of this is the final version and the ultimate way to do anything. It's a, it's a set of concepts, which I think could be applied in many different settings.

Uh, but I'm one of the things, I'm one of the reasons I'm having this conversation. One of the reasons I'm really keen to share this is I wanna see people take these ideas and and apply them in their own place. Adapt them, change them and come back with something better that I can then steal off them. Um, so yeah, as I say, we're at step five, so we're describing these behaviors.

That's all great. And we're starting to then get at the idea of signals that we could start to detect, but what's, what's really important. I found in a lot of places is there are leading and lagging indicators. And it's hard without a lot of years of experience and, and great product sense. It's hard to tease out exactly what those are really, really easily.

Um, but that's where the next step comes in. So step six, we take those behaviors. 

Ryan Hatch: Okay. Talking about before we get to step six let's let's so to put it, to tie a bow on workshop one, one through four, how long is that? Is that an hour and a half? Is that is that ...How long do you usually try to get for that with the stakeholder?

Tom Kerwin: So usually these that are used to do this in about 90 minutes, but that was with more talking. So if you've got a particularly chatty lot and you'd like to discuss all of the things 90 minutes is about right. If you wanna be tight for time and zip through this, you can do it in 45 minutes. If you keep your eye on the clock and you don't have much discussion.

Ryan Hatch: Got it. 

Okay. So, so behaviors but just before we move on to the step six, like, what, are your prompters with, with the team? What are these Are these customer behaviors, these team behaviors? Like what, what, how do you, how do you go from this failure scenario where you're scared and you know, all these things that you said actually see much more likely than, than I thought, or hadn't thought about this before. How do you go from there to two behaviors? What is the prompting 

Tom Kerwin: really good question. So one of the things I should mention actually is one of the benefits of having a diverse range of roles involved in in defining those worlds is you, you end up with a real range. It, It often comes down to, uh, a, decent understanding of Marty Kagan's product risks, Um, so Silicon valley product groups, ideas. So there are some things which are about the feasibility and the technical hiccups that we're not necessarily thinking about when we're designing a business idea. Um, there are some which are about the demand side issue, the desirability of this. Does anyone even want this?

Uh, there are some about the viability risk. Can we make this work in a way that actually is delivering value to us as well as other customers? Or is this just gonna be too hard? And The other one usability risk or is it just too hard for the customer to use? so all of these things come into play and we we get a rich picture, all of those things.

So when it comes to behaviors, it kinda depends which of those angles you're looking at and what I've generally seen as you, you tend to end up with a bunch which are, um, a bunch which are around the customer behaviors and a bunch of which are about the technical system. And then sometimes some which are about our team behaviors and Ideas some that have come up before is we spent too long, gold-plating before we shipped. Um, and that was uh, that was the thing that caused the project to fail. There's another one, which was, we actually couldn't get the data that we needed from enough of our customers to be able to make this work.

And it would take too much, too much cleaning of data to, to make it viable. And then others, which our customers start to use this, but it actually hurts. them. And, And they, it doesn't work or they start, they think it's a nice idea. They start to use it and then it doesn't work and, and it ruins their trust with us.

And so all of these different stories start to come out about the ways that things can go wrong. And based on those, I tend to try to prompt people on the behaviors that it's, it's something you, you it's, somebody doing something or it's a system. Doing something like a piece of technology doing something or not doing something in a way that we can see and detect. And that's, that's the behaviors. 

Ryan Hatch: Yeah. 

Tom Kerwin: Does that clarify it for you? 

Ryan Hatch: Yeah, yeah. 

Tom Kerwin: It is one of the hardest steps. I think, um, people aren't necessarily used to doing it, but I've found that it's a really helpful bridge from the world is like this to what signals would we see? What metrics could be used to measure it? 

Rob Kaminski: Hmm, There you go.

What signals or metrics could be used? And do you have like a template like custom for, for desirability stuff like market facing? Do you have a template? Do you use something like we often use something like customer will or will customer blank for a customer behavior? Something like that? 

Tom Kerwin: That sounds good.

I actually haven't used a template so far, um, tended to keep it more freeform, but I have noticed that means that some rather peculiar descriptions can slip in sometimes. Um, 'cause again, we're all, we're all working on this together and it keeps it really snappy and and it keeps everyone engaged and and involved and owning it together.

If we're all sort of adding behaviors and describing this stuff.

Rob Kaminski: Yeah. Is there any filters you put on it to decide which of those come in? You know, you talk about, you brought in all the different stakeholders, which are gonna bring in their lens on things, you know, usability might be one area.

Desirability might be another. Is there any prioritization that happens like, or is there a scenario where one of those would, we'd say, yeah, we're aware of that, but we're we're not gonna bring it into the way that we create these triggers in our decision-making. 

Tom Kerwin: That is a perfect segue into the next section, which is 

Rob Kaminski: Of course.

Tom Kerwin: ...how do we sequence these behaviors?

So what what we then do as a team is we, we grab all of the behaviors from the, the section. We just made it at part five and we dragged them down into this sequence behaviors, um, table it's a table, that's all it is. Uh, and you can see here, it's something I stole from John Cutler, this idea of pace layers for timescales that we might.

Be thinking we could see these behaviors in. So the first question is when would we see these behaviors? And we drag all of the behaviors down into uh, one of the columns. And usually they end up mostly in one to three months, one to three quarters. So if it's something that's about, um, customers start to, uh, churn. they start to, to, you know, our retention goes down well, that's, that's really lagging.

That's gonna be after one, three quarters, but if it's customers hit hit the buy button, then that's gonna be seen in one, three months after we've launched the initiative. So then the question becomes, right, this is the really meaty part of of section six. How, which of these could we provoke earlier if we were clever about it? And that's when we start to d- design our, our triggers and our probes, um, how are we gonna provoke the world product and get some signals back that tells us which of these behaviors are are likely to happen? Um, so classically, if it's, if it's one of these things, like the customer hits the buy button, once it's shipped, well, what could we do in one to three hours or one to three days that would figure out well, will, the customer hit the buy button?

Ryan Hatch: I love that pushing the team to be scrappy, right. Hey, you know, we, we almost think of it like a customer journey line. Like these customer will steps, right? They will visit the website. They will click sign up. They will purchase, they will create a profile. They will blah, blah, blah, blah. And often that gives, like that is kinda is the, the flow.

And you can just go to the first step and say, let's just solve for this. But I do like how. What you're doing is, is almost taking that to the next level and saying, well, yeah, these might happen one to three months down the line, you know, assuming some kind of like, you know, velocity team velocity is a lot of assumptions, even, even in the timescale, but I love how you're pushing the team to like, how could we learn that faster?

How might we accelerate our learning speed? 

Tom Kerwin: That's exactly it. And that's that's, ...I think. I'm sure you've seen the same thing. I've noticed that most teams, most of the time wait way too long to experiment and probe, and they think things have to be much more ready than they are before you can put it out there.

And we see this with build test learn. There's an awful lot of building happens before any testing and learning in most cases. And, uh, one of the things that, I've seen this prod teams to do is to start without code and to start trying things. Um, doing it by hand or one of the examples, um, that that really was, was where pivot triggers was born from was actually a, a team.

And this was a reporting product that we thought would change customer behavior. And we put the team together and eight days later we sent out a really scrappy. Google sheet, which was ugly as seen. the designer was crying, but it it went out and it provided value to customers immediately. We can send it to all our customers, the data wasn't there.

Uh, we couldn't do it. Uh, we can do it beautifully. It wasn't the perfect data science model that we thought we'd want. And it took us four days as a team collective to like bundle together and put this thing together uh, and send it out the first time. But once we got that out and we got the feedback from the world, we knew that there was something there and it was worth continuing to work on it.

And that's 'cause we'd set the pivot triggers beforehand. We knew that this was a probe that would give us a signal back that would tell us, is it worth investing in this more? And that really is the heart of the whole thing. If we then pull this through from the sequence behaviors into the signals and the triggers and the probes, what you're seeing down here with the, the pink and the black, uh, post-it notes is the pink post-it notes are the sort of here's the probe we're gonna run. Here's the experiment we're gonna, we're gonna try, here's the, the email, we're gonna send a few notes about how we're gonna, oh, here's the technical spike we're gonna do to check that our data is in the right shape for this. And then the, the black thing is the, the, the pivot trigger. And that is saying, if we see that fewer than X customers want to talk to us, or, uh, less than 70% of our data is in decent form, then we will stop and think again, 

Rob Kaminski: [inaudible 00:37:25] failure, failure, criteria. 

Tom Kerwin: failure, criteria, Yeah. And it's really looking at that, that bottom line of, if it's less than this, we're gonna question whether it's worth continuing with this project as it is.

Is it worth continuing to to invest in. this As it is. So one important thing to note is 'cause sometimes people get really scared about this. We're not saying this will kill the project dead at that moment. So if, if fewer than three customers reply to our email, the project is immediately dead. There are no other questions.

Everybody move on. No, It's not, it's not like that at all. It's saying we're just setting a date and saying by the end of two weeks, we wanna see that three customers have replied and are interested and are ready to, talk to us. Otherwise, we've gotta think about whether we're talking about this the right way, whether it's connecting with them correctly, whether we've understood enough about what we're really trying to do.

Uh, we'll have a think about that and and we'll be able to figure out because we've done something we've taken action and we've learned something more about the world. And then the final place that I've mentioned is the blue ones, the signals, which are then getting at those metrics. So these are the metrics we're gonna use. 

Ryan Hatch: Yeah, I was just gonna ask that what signals and behaviors, w- w- what's the difference here? So behaviors and then signal is, is what, like what percent like uh, what is a signal? 

Tom Kerwin: So the behavior is, is what somebody is doing. And the signal is how you would measure that. So most of our metrics are kind of proxies for real-world stuff and we can't generally directly see our customers because we're working in digital.

So we're having to infer their behavior. Using the metrics that we have available. Um, one of the things that often sets teams wrong though, I think is they they look at the metrics they have available in the dashboard that they currently have. And then they infer what, what customers are doing based on that. And often that is chasing Boodles or making up stories and it doesn't really reflect reality. Whereas we start the other way around and say, well, these are the behaviors we care about. And then we triangulate those with some sort of Metrics, we tend to get to a better result. I found. 

Rob Kaminski: Yeah, I really liked this, Tom. It, It makes me think of like, especially when you get down to your triggers and probes with the, the pink and black post-its here, your experiment building to a degree, you're figuring out what actions do we take to try and elicit those behaviors and view those signals so we can make a decision. And I, I think what's powerful here is Multiple experiments at once. I think this is where even sometimes we get stuck as well. We would look at some, you know, maybe not a similar board, but something similar of what's a key behavior for this thing to work. Sometimes we get very focused on that first behavior. What you've done here is you're allowing multiple behaviors to be probed for Simultaneously so that you're not halting the project. You're not creating blockers across different stakeholders. And you can take it a little bit further with, with which I think is is pretty exciting to see if if I'm understanding this correctly, in, in how [inaudible 00:40:26]-

Tom Kerwin: You, you've hit that nail is squarely on the head. And one of the key things I think that this helps is you get to do discovery and delivery at the same time.

So within the first and and what the stakeholder wants to know, generally, the person who's funding, the project wants to know when can I have something? When can I see some results? When can I have the thing that I care about? And with this, we ended up with having a, a timeline laid out. So at the end of two weeks, we will have done this and we will have this signal back that tells us something about the, whether this is worth investing in.

By the end of four weeks, we will have done these Probes and we'll have this information. We'll have a working prototype that does something like this. And then the the stakeholder can see exactly what they're gonna get. 

Rob Kaminski: So tell tell me more about that You're, you're what you're describing is you're taking you finished workshop to, right? 

Tom Kerwin: No. 

Rob Kaminski: the ideas, You have your, your set of experiments, your triggers, and your probes. And Now you're gonna measure those signals Do you, then do some work. It sounds like to communicate all of this back to the stakeholders that maybe didn't partake in this second step. And, And what does that look like in that communication of at all?

Tom Kerwin: That, That is exactly right. And uh, that's, that's one of the, the most fun parts. So it is, um, I think then it, it falls to usually the product manager at this stage or the product designer to someone like that to pull together the the plan and lay this out in a, in a clear way, usually in whatever format the company tends to talk in.

So whatever road-mapping tool or, um, project management tool they use, we can put these things in and we can set these up as milestones or something like that. That's what that's a pretty common. thing. And we can start to then have a conversation with the stakeholder about what we think this pivot trigger means why we think it's, it's not gonna be worth continuing.

If, if we don't see the signal back and we can have a conversation with a stakeholder about, well, is that right? Do they believe that? Are they happy with the the way we've laid this out? Is the signal high enough for them or is it too high that, would they be happy if, if we didn't meet that bar or would they want to see a lot more in order to know that it was worth doing?" And I've seen that happen several times where suddenly a project that was quite nebulous becomes super clear and the state is able to say, actually, no, if it's only that much, well, that's not worth our time doing. We're gonna need to see more than that. And it resets everybody's expectations.

And I've also seen, uh, one of my favorite moments was, um, the, the team with the reporting product after the first probe came back. And it was so sort of exciting and, and we thought, wow, we could do something really cool with this. We set about, um, defining our own new, exciting initiative where would use this as a beachhead product for the company.

It'd be like a sales tool and we could use it, put it out to the world and it would be incredible. And we shared this plan and, and our pivot triggers and that the timeline we'd laid out and the leadership team said. Yeah, that would be cool, but we actually don't want you to do that right now. That's not fitting in with our bigger strategy.

We need you to go back and do uh, something more like this, and we'll, we'll save this in the bank for later."And the team were able to say, yeah, okay. We believe that you've heard us. And you've understood what we were thinking. 

Rob Kaminski: Mm-hmm [affirmative]

Tom Kerwin: ...And we understand that it's not fitting the plan right now. So yeah, we went back and bang.

We bashed out another set of pivot triggers for the, the other version and cracked on with that. And it was a you know, switch in a day, rather than sometimes those sorts of things. I've seen them drag out for weeks. 

Rob Kaminski: right. 

Tom Kerwin: Um, so that was really exciting. 

Rob Kaminski: That's super impressive. 

Ryan Hatch: I'm really interested in the output of this, right?

Like this sounds great. Like getting to like, okay, we can run these experiments in the next day, three days, three weeks. Sounds amazing. My question 

Tom Kerwin: Sounds too good to be true [laughs] 

Ryan Hatch: Yeah. My question is like, how do you get buy-in? You know, it it goes back to like wanting to build a thing and my, my ideas are, you know, how do you get them to, you know, the client or stakeholder in this case, to actually get the whole team to wanna go learn these things first, as opposed to.

Well, we're just gonna start building and you can go do your little experimentation thing on the side. One the two people kinda like go do your little learning, you know, just, just almost to placate you. 

Tom Kerwin: Yeah. Well, that is uh, a common failure mode. 

Ryan Hatch: Yes.

Tom Kerwin: [laughs] For, For this sort of thing. And, uh, I have seen that even, even with, um, using pivot triggers, it's it's not perfect.

And sometimes people just wanna do their idea. I think you mentioned this to me, Ryan, someone who said, I don't care if this costs me six months of time, I don't care if it fails, We're doing it. We, I want you to just do what I tell you and that's it. And and this sometimes happens. And in fact, there were, there was I'd set a, uh, set, some pivot triggers with one team.

Well, the team set pivot triggers with me guiding them and then uh, played it back to the stakeholder. And on this case in this company, the stakeholder actually said, "N- n- no, there is no situation in which we wouldn't wanna keep investing in this because there's something you're missing about why we're doing this.

And So, yeah, no, we're just gonna do it. Um, but one thing that was nice, the plan that was laid out was still useful. So even though we were never gonna Cale or pivot that significantly, the, the sequence of the work still made sense and the team could use it to carry on. Um, so that was quite cool.

Um, the, yeah, I think that sort of covers it off really. 

Ryan Hatch: Where have you seen it work well where you've gotten those stakeholders to, kinda buy into this, being the roadmap, this being the plan of attack? 

Tom Kerwin: What I've seen it work well is really when the stakeholders are involved in that first postmart- pre-mortem in that experience of, oh God, but actually this could go wrong.

And I think, yeah, that, that really is the magic part of the process. As far as I can tell. Um, And I, might be wrong. That's probably other things that are really important and I suspect it takes, it takes an organization that is ready to learn. I think it also probably takes a team that has the cognitive overhead available to do the continual learning that this takes.

Um, one of the things, this really is. a, It's like training wheels for iteration, proper iteration, not incremental building where just stacking bricks, but actually re-imagining what the thing is that you thought you were building in the first place on a frequent basis and rebuilding your model of the world.

And I noticed that even with a team that was like incredible at this and so really high psychological safety, really, really uh, capable, it takes a lot of emotional energy to do that relearning on a frequent basis. Um, so I think that's probably of outsized importance and, uh, we'll see whether that's, whether that bears out in the future as, as more and more teams try it.

Ryan Hatch: Yeah. 

Tom Kerwin: I did notice a really good question from uh, YouTube. Um, somebody saying we love Miro and our team, use it all day. I'm interested to know when the success failure framework is used only at the beginning of initiative, or could it be done after some work is already completed?" And I, the the ideal is definitely at the beginning of the initiative, uh, ideally before anyone's really committed to what the idea is.

Um, so you talked about this earlier in the goal. There's that some cost issue. um, once you've started something, it gets much, much harder to grapple with the idea that it might fail. Um, however, I, I think, I mean, It's like that thing of planting a tree, the best time to plant a tree was 20 years ago. The second best time is right now.

So, you know, it's, it's worth trying. 

Rob Kaminski: Does the, do you find yourself using this, like, I guess tied into this question, is there a point where it's gotten so big or the momentum has gone where maybe. you're, You haven't hit any of your pivot triggers. So you've gone through some of the initial behaviors and signals, um, that are pointing in the right direction.

Is there a time, time where this sort of gets abandoned or at least gets de prioritize and how you use it day to day? Or how does that look like in your experience? 

Tom Kerwin: That is an excellent question. Nobody's asked that, but yes. um, Yeah, it makes sense. So I'm thinking about the team that used this heavily, uh, which was last year.

And after awhile, we iterated every week, uh, using pivot triggers for, I think it was about 12 weeks. And what we noticed was the rate of change. The rate of learning slowed down to the point where things were quite stable and we had eaten most of the uncertainty and we knew that we were, yeah, we're probably not in the bad world. It's, It's probably gonna be fine. 

Rob Kaminski: Yeah. 

Tom Kerwin: And we'd also forced ourselves to learn enough about reality and about the reality of our customers, that we could actually answer the questions about what mattered to them. And one of the signals, one of the things it's just occurring to me now, one of the things that didn't happen were lengthy debates about what we thought the customers would want.

Rob Kaminski: Hmm. 

Tom Kerwin: We'd already done it once we'd built to learn. And now we could build to earn in the old classic thing. 

Rob Kaminski: Yeah. 

Tom Kerwin: So we we built it really rough and ready and delivered value. We were still using that awful Google sheet. By the end of that 12 weeks, we'd done loads of adapt adaptation to the data science model behind.

It was really cool. And we had some good, um, good. data in there that we weren't expecting we'd need to include. we've been able to hook all of that up and make it work. What this meant was when it came to building it high quality, pristine into the product properly, we knew exactly what to do. And there were, there was no debate.

There was no question. You you could just make the right choices- 

Rob Kaminski: Yeah. 

Tom Kerwin: upfront and [crosstalk 00:50:45]

Rob Kaminski: Yeah. I could see that happening. That's great. 

Tom Kerwin: Yeah, thanks for the question. That was a really good one. I think. Yeah. You would not use this to build a house. There's no point, you know, if, if you understand everything about the customer's world and everything about your products capabilities completely well, you can just make a plan and do a waterfall Work that that's absolutely fine. 

Rob Kaminski: Yeah. 

Tom Kerwin: This is really for when you're working in complexity and when you have high uncertainty and there are big risks and you don't wanna, you don't wanna build another [inaudible 00:51:14] feature. 

Rob Kaminski: No. One, one of the last questions I have, Tom, it's a bit more tactical is as you're using this, it seems like collaborations really key upfront with these two workshops and how you get started connecting.

The triggers that you wanna be observing through these signals in the early phase and tying that to the project plan. I think it's really key in what you shared and how you tie it to what the stakeholders are looking for. What does it look like? I guess who owns this? Once it goes, is there a session that you have every week or two with the team?

Does it fall to the product owner or manager to keep this up to date and, tweak it? Um, and and more specifically continuing to iterate. because I know that the, aim is to keep updates on the signals that are being seen and how close we are to triggers. This is more, how do you take what you did in the first, no the second workshop and keep that going?

Is that a solo work or are you doing that with [crosstalk 00:52:05]?

Tom Kerwin: Uh, That's a great question. Ideally, it's with the team. Um, and, in the best cases it's been a, uh, yeah, in the best cases it's been a team effort and. and everybody's started to use the language and started to talk about these signals that we're looking for in the in the experience we're doing and all probes we're doing, and everyone's been involved in them.

Rob Kaminski: Yeah. 

Tom Kerwin: Um, and so when you've, when I've seen that level of buy-in, it shifts the energy from, we need to build this perfectly first time. 'cause we're never gonna go back and look at it again. to, We need to do the thinnest slice of work possible to learn the most possible. And we're working that way. And I know it's rough and ready, and I know it's not gonna be perfect forever, but that's how we're gonna work. In other cases, I think it does, 

Rob Kaminski: Yeah. 

Tom Kerwin: honestly, it probably is the, the PM. Who's gonna bear the brunt of this product manager or the project manager. Um, they tend to be the ones who own these updates, this style of update, and to run the project. plan. Um, so I think, yeah, it does fall to them. One of the things that I've noticed works very well 

Rob Kaminski: Yeah.

Tom Kerwin: ...is when you really socialize this and use it a lot. So the, the PM on the team I worked with last year absolutely nailed it. He was awesome. Jason, from cubit nailed it. He was so good. He put the pivot triggers into all the presentations. So every time he did a demo, every time there was a retro, every time there's a sprint planning. it all came out, When the, the leadership team was seeing the, the overall work of all the different product teams, he had his pivot triggers in there and was talking about what progress we're making on those fronts.

And that just made it so everybody knew what we were working towards. And it was, it was not we were working towards something in six months time that is distant and vague. It was we're working towards something that we're gonna have for you by the end of the week, by the end of the month, et cetera, et cetera. Um, so that, that was really excellent. 

Rob Kaminski: Yeah that's brilliant. 

Ryan Hatch: Yeah, it it really feels like there's really two loops going on. Right? There's this there's delivery that's operating on a sprint plan, looks different in, in just pure delivery where we're just, we know [crosstalk 00:54:14]. 

Tom Kerwin: [crosstalk 00:54:14]. We're just getting done, yeah.

Ryan Hatch: ...but this is really, yeah. Uh, this discovery is like, we have our learning loop. That, That's what it is like the, what we're running through is really like a series of questions, right. It's like, how can I get this learning the fa- as fast as possible? And I can see how how do you, how do you kinda see those two blending together? Like when, you know, when you're doing, when you're working with a team cross-functional team, you know, when do you pull up this?

Do you run sprint plannings, you know, together, do you ...just interested in how that intersects. Like I know it'll start with. like, Probably starts with really discovery heavy and like the delivery is only servicing just those questions. 

Tom Kerwin: Yeah. 

Ryan Hatch: Right. Let's just get it out there when you get, when you get a little further along, how do you see that those, Those blending together? 

Tom Kerwin: [inaudible 00:54:59] also a really good question. And I think it's, ...that relates to another slide in one of my presentations, which sort of ends up tacked on the end, but, uh, yeah, really that idea of iteration and, and working in a loop ish way, I think is is the important thing here. And It it depends on that reduction in uncertainty as we go down, uh, go along the way, reduction in risk as we get along the way, when you've got super high uncertainty and you don't know what you need to build, it doesn't make much sense to invest all of your time in delivering stuff.

It makes sense to invest a lot of your time and discovering stuff. And if you wanna deliver something, but, uh, as the project goes on and as you've learned more and as you've updated your, uh, your view of the world and what you're doing. Then you don't need to do as much discovery. You've found out most things.

It becomes more and more little details and inconsequential things. And then you can shift gears into delivery. Uh, and that's, that's what I saw on, on the team again. Last year was everybody got involved in discovery. The engineers were working on discovery too. They were building just enough code that we could learn.

And knowing that they'd refactor it, throw it away later. And then as we learned more, it, it shifted gears. So the first, the first, uh, spreadsheet that we made in that instance, it took everyone on the team. Four days over the co going, the, following iterations as we got more and more certain, they automated more and more parts of that until by the end of it, it was hit one button, five minutes later, it was finished. Uh, at that point, it's not that hard to build it into the product so That was quite cool 

Ryan Hatch: Right.

Tom Kerwin: And be a really good question. I think you're right. A lot of people see it as we do a load of discovery upfront, and then we pass it over to delivery and we deliver what we discovered and that's that's still only one learning loop that is, we think the world is like this.

We put a thing out there and then we learn whether we were right or wrong. So, You're always wrong. [laughs] All models are wrong, some are useful, but we're always wrong in that first guess. 

Ryan Hatch: [laughs] 

Tom Kerwin: Do we wanna wait three months or if we could discover and deliver and do it loads loads of times really quickly, would we rather do that? It's not free. As I said, there's that cost of having to update your model 

Ryan Hatch: Yeah. 

Tom Kerwin: and there's a cost of having to work in a different way, but it does. It gets you loads of iterations very, very quickly. 

Ryan Hatch: Fantastic. Well, this is, this is been an awesome episode and I mean, Rob tons of value here, right, Tom, thank you so much for sharing all 

...this stuff.

Tom Kerwin: Amazing.

Ryan Hatch: ... with me. 

Tom Kerwin: Absolutely delighted to come on and share it. It's been really, really cool to talk about it. I'm glad I always love nerding out.

Ryan Hatch: [laughs] We do we do to, you know, 

Rob Kaminski: You've come to the right place [laughs]. 

Ryan Hatch: Yeah. As much as we hope people will find value in this, you know, uh, we love learning from, from other wonderful, you know, product people. So this is, this has been fantastic. Um, lots of takeaways, uh, here, so, Hey, where can people, Tom, find out more about you and more about pivot, pivot triggers? 

Tom Kerwin: Uh, so I have a a Substack. who doesn't have a Substack these days, but it's tomkerwin.com And that'll take you to there. And there's various articles that are about pivot triggers and the ongoing development.

If you just want pivot triggers, there is pivot-triggers.com, uh, which is uh, sort of. a little Rough website, collecting all this stuff together. Um, and then in case anyone's actually, I think it's all online, so you can probably join this. There's uh, a future London academy workshop coming up in, uh, the end of August where I'll be talking about pivot triggers again.

And I think pushing people through at least part of the workshop process, 

Ryan Hatch: right? That's an innovation masterclass your running. is that correct? 

Tom Kerwin: That is part of it is a week long innovation masterclass with loads of awesome product people. talking about All sorts of different cool ideas. 

Ryan Hatch: Awesome. 

Rob Kaminski: Terrific. 

Ryan Hatch: Well, everyone ...check Tom, check Tom out, uh, a- a- and check pivot triggers out. So there's a lot of value. here. Tom. Thanks so much for joining us super appreciated. 

Tom Kerwin: Thanks for having me. 

Ryan Hatch: and we hope you, uh, stay connected. 

Tom Kerwin: Absolutely, definitely. Delighted to be on. Thank you so much for your time.

Ryan Hatch: Thanks for joining us today. We hope this episode gave you some fresh perspectives and even some inspiration to help you on your own product journey. 

Rob Kaminski: You can access. notes, Links and resources from this episode at exploringproduct.com. If you enjoyed it, please be sure to share it with us on Twitter so that we can chat about it togethe. Until next time, keep exploring.