Getting The Most Out Of A Software Demo

Galen Low: Well, another vendor's software demo is crashing and burning. Your technology director and your DevOps lead appear to be playing buzzword bingo with every piece of marketing jargon the presenter mentions. Your general manager's eyes are glazing over as the topics get a little too technical. And even though the person presenting really seems to know their stuff, no one seems to be addressing the elephant in the room: the fact that they're actually your CEO's cousin.

Isn't there a way to make this a better use of everyone's time?

If lackluster demos have been an obstacle to making an informed decision around new software for your organization, keep listening.

We're going to be breaking down some of the most effective strategies for making the demo phase of your software selection process heaps more relevant to your organization's specific needs.

Hey folks, thanks for tuning in! My name is Galen Low with the Digital Project Manager. We are a community of digital professionals on a mission to help each other get skilled, get confident, and get connected so that we can elevate the value of project management in a digital world. If you want to hear more about that, head over to thedigitalprojectmanager.com.

All right. Today, we are talking about the process of selecting new software for your project teams, and specifically how to cut through the sales pitch of a software demo and make sure that the tool is not only the best tool, but also the right tool for the way that your teams work. With me today is Olivia Montgomery, an Associate Principal Analyst at Capterra and former IT PMO leader who now advises small and medium businesses about software selection.

Hello, Olivia!

Olivia Montgomery: Hi, Galen! Thanks for having me.

Galen Low: It was great to have you back on the show. How have things been for you in Austin?

Olivia Montgomery: Things are pretty exciting. We have South by Southwest going on, and it is back in person since, first time since 2020. So, the city is vibrant and there's so many events and talks and all sorts of things going on. It's, it's pretty exciting this week.

Galen Low: Back at it, back at it. South by Southwest, I'm super excited about it. It's like all in my social media feed and everything like that. I've never made it down to Austin, but such a vibrant, like musical community. There's so many things happening, uh, there's so many conferences and ways, ways for people to get together and have a good time.

So now, it sounds to me like I have no excuses anymore. This might be the year. This might be the year.

Olivia Montgomery: And it's pretty cool. It definitely having tech tracks. So I've been able to attend talks from supply chain leaders, the CEO of Amtrak, uh, John Deere is here really driving, you know, their tech focus. There's VR experiences that are really focusing on storytelling in new ways, ranging from really heavy, serious topics to lighthearted entertainment and just how we can interact with content and each other differently.

Galen Low: Very cool. What's been your favorite so far?

Olivia Montgomery: So, aside from all the like actual tech and innovation we're seeing, I got to see the new Batmobile in person. And I have to say that's a highlight for me. I would film buff. I actually, I don't watch trailers, so I haven't seen any trailer. I haven't seen the Batman yet, because of South by, but I saw the Batmobile for the first time in person. So, that's pretty cool.

Galen Low: I think that's probably better than a trailer.

Olivia Montgomery: I think so. I think so. I'm like, Okay, I'm, I get it, I see the vibe. I'm really excited to see the movie now.

Galen Low: That's what I'm looking forward to more of, as things opened up again, right? Just like running into the unexpected, the things you didn't plan to do are back in front of you.

Awesome. All right. Let's get into it.

Let's, let's talk about the most pressurized and doubt inflicting part of the software selection process, which is the demo. So, let me just start by setting the scene.

Let's say you've been tasked with finding a new tool for your organization, and you're either the sole decision maker or you're chairing the selection committee. You've put in countless hours to finding use cases. You've been soliciting requirements from the team. You've been establishing budgets with your leadership, and you've been doing a whole bunch of research online. And finally, you're at a point where you've got your shortlist of tools that seem to fit the bill, and now you want to see them in action.

So, why is this pressurized and doubt inflicting? Well, I think it's mostly because this is the literal culmination of all your efforts to date. You are wrangling all the stakeholders. You're trying to communicate all the requirements effectively to your software vendors. You're being asked to absorb a ton of information and react on the spot and it all takes place in the most pressurized environment of all — a meeting.

So, I thought maybe, with all that said, let's start with the naysayer arguments. Like should a team, or an organization, should they even bother with a demo? Isn't it just a sales pitch?

Olivia Montgomery: Do demos, please, please. I understand. I have sat through my fair share of demos that are very sales heavy. And that's a lot of when I started percolating the ideas of how to make these better. I do believe in the process of demos and there's a lot of prep and a lot of work, especially the project manager, that you can do, to make them one of the more important pieces of the selection process.

So yes, do demos, but they're not quite as easy as, you know, following the vendor's lead. I think that's where some people can go wrong. Uh, you really want to drive and set the requirements essentially for what you want to see in the demo.

Galen Low: I mean, I think it's all fair. Also, I mean, these tools have a lot of depth to them, right? And a lot of cases we're picking a tool to do a lot of different things and be a lot of things to different people. But is it even possible to cover that much ground in a demo beyond just like skimming the surface in terms of like basic features of a software? Or what strategies should an organization consider to make sure that they get the answers that they need?

Olivia Montgomery: Yes. Great questions.

So, first off, no, you can't cover everything in a software demo. That's where your previous research online a, you know, was going to come in. And if you do like, uh, the RFP process, or when you get down to your final two or three, you know, vendors that you're looking at, have them send you all that information, in a document. The demo isn't exactly the best place to go through all the functionality that you need.

You want to hone in specifically on what the user interface looks like and the couple of key user stories that, your team needs to follow. Like what business processes need to be supported by this tool? And honestly, you know, do the users like the way it looks and feels? This might be their first time getting into a tool, and seeing the interface.

And so though, the look and feel is going to be more key here. The list of actual functionality should be written out, you know, before you sign a contract.

Galen Low: I think that's really important. And thank you for that because I kind of framed it earlier as this culmination, right? All these roads lead towards this demo, but actually the demo just kind of fills in some of the gaps.

It's actually part of that research process. You, there are things that you just, you, you need to see them in action in order to get a feel for it, like you said, the user interface, right, certain use cases. It's not going to be, Hey, I did all this research and I can do all these things. I've looked at the sort of comparison chart and now show me every feature, in a two hour period. That would probably not go well.

It's actually, great, I think we know all of this. We've seen all of this on paper, you've sent, additional details to me. And now let's, let's schedule a demo where we can actually dig into a few things that matter.

Olivia Montgomery: Exactly. And I actually strongly recommend scheduling two demos.

I like a business facing demo and then a technical demo. Combining them would make it a very long, complicated meeting. And half of the stakeholders wouldn't be interested in the tech side and the other half wouldn't be interested in the business side. So, I recommend splitting them, for respect of your stakeholders' times.

And also you need different people in those meetings. It's not the same people. So, I strongly agree that it is kind of another, it's a vetting step, but also still more requirements gathering from the vendor. And yeah, so schedule two — we're not discussing is it a project management tool, it's an accounting tool, a new ERP system, whatever software tool you're looking at. I like to follow kind of the same demo structure.

So yeah, the first one is business facing. Have your business owner there. That's probably a little more obvious, but you also want to include your power user. Often times it's a supervisor, you know, in the role, a lead in the role, the person who uses it on a daily basis. Purpose being this will ensure that you are looking at the tool from both like the strategic long-term point of view.

The business owner can, you know, decide like, yes, this tool will support our growth. This tool will support where we're going, but you also need the power user there to see their particular user story. They want to know that their key function that they have to do, can be done and what that looks like and what it feels like.

Sometimes, I've heard from PMs that they can't always get the buy-in to have a power user in either one of the demos. Sometimes it's too early, a myriad of excuses, but the, your power users gonna follow through the rest of the project too. They're likely end up being your QA person during the actual implementation.

They're your main source of user requirements. They're the team that's going to use it. They gotta be happy with the look and feel, which again, the demo is really a vetting of the look and feel. So you want to make sure both of those people, are in that demo.

And another way to help set that up is gather user stories from the team, and at least one or two from that power user and send them to the vendor long before. So at least multiple days before the scheduled demos so that they can set up the system to walk you through exactly that user story. You want to be able to say, as an accountant, I need to run the month end close report within a 24 hour time period, so that I can, you know, close out the accounting month. You need to actually see that in the tool happen.

Having a user story won't be defining exactly, Oh, I want to see this, this particular function in the tool. It opens it up that maybe this tool has innovated a way to achieve, you know, or execute a function that maybe you haven't seen before. So you don't want to, you don't want to pigeonhole them by saying like, only show me this, but you also do, I don't know. You got to be open.

But the general sense you need, the power user needs to be able to see their key function. And your business owners also gonna have requirements that they want to see, typically the executive dashboard. What does it look like?

What does the tool look like when I log in, as the mander, the leader, the CEO. How do I pull my reports on my dashboard? That's again, another what those look like and feel like, you need to see those and ask the vendor to enter in dummy data for you. It typically will send a, set up a sandbox for your demo. Have them preload, you know, some dummy data so you can actually get a look and feel for the reporting structure.

Galen Low: That's really cool. I love this notion of kind of splitting the pack. It seems kind of counter-intuitive sometimes when it comes to like a decision-making process, but I like the idea that there's two different demos, because there's two different conversations that need to be had about this tool.

And it actually comes down to the best use of people's time. And I think what you were saying about that, sometimes we get pushback. It's like, no, we don't want to involve, you know, our accounting power user at this stage, because it's too early in the process and yada, yada, yada, but actually, you know, a) that is part of the conversation in terms of assessing, like you mentioned, right? The long-term use of the tool, but also the depth of use of that tool.

And frankly just vetting and, and making that person, uh, the power user, like a bit of a champion of the selection process, because they were involved and they were there. And if they weren't, then actually they might be the loudest voice in the room saying that it was a bad decision.

I liked what you said about a sandbox and, for some people, I think, at least the processes I've gone through for software selection, it's kind of like do the work and then demo, you kind of sit back and watch. And then you have like, you're, okay, give me a hands-on environment where I can take the things that you just showed me in the demo and try them out for myself.

But is there a world where actually, people are trying it out for themselves in a sandbox environment during the demo, like a hands-on demo?

Olivia Montgomery: I'm sure power users out there would love the chance to do that, because the, some of the pitfall is you want to pick, you want to pick the software tool that's right for your current state processes and future state. And sometimes a demo just will focus on future state.

And sometimes the idea is, Oh, when we get this tool, our processes are going to evolve dramatically and mature automatically with all this automation that we're going to be able to get, when that's not the reality. And your power user is going to be able to know, Okay, from day one, when we use this, what is, it doesn't satisfy our current state, but also support future state.

So, I might think getting into the sandbox ahead of time could, could help, you know, your power user know, like, yeah, I could use this today and my job will still get done. And we're not going to turn it on and lose all functionality.

Yeah. Alright, I don't know if your technical teams, would really appreciate or care or have the time to go in and do that. Again, I know I am a big proponent of being very transparent, being very forthcoming. Talk to your stakeholders, talk to the vendor about what you need and what their ideas are. So you are cutting through the sales pitch by not letting them drive the demo fully, but you do also want, you know, they know the solution.

Maybe they have some innovative ways to approach a problem that you hadn't thought about before. So you want to be open to that too.

Galen Low: I love that. Let's talk a bit more about preparing for these demos, these two demos. Do you have any great stories or examples about like use cases that a team should ask to run through to make sure the software works that they do? Or even just like the process of gathering those user stories? Like you said, uh, in a way that is not, sort of constraining it too much.

So it's not like, Oh, just show me this one thing, but if the software approaches it a different way, that's something that, uh, the team can have a look at.

Olivia Montgomery: Yeah, absolutely. So, if you're lucky enough to have, you know, an IT business analyst, I can go in and gather all your requirements for you. That's fantastic. That's not common. And even if you do have that person, your power user might not always, they might not have ever given you a user story before or given requirements before they need some help.

Something that I've done in the past that I kind of like is, I've sent out a survey to, the team, the power user, the leads of the department of who's going to be using it. And I give them kind of a Mad Libs style, to give them the structure. Your user story is as, as the, and then your job function. I need to be able to, and then your job function. So that I can, and then the reason that you're doing it.

And it's already kind of a Mad Lib and it helps them really quickly be like, Okay, yeah, I can, I can fill these blanks in, and it's quick, it's easy for them. They understand, you know, what's going on. And then you're able to take that exact user story and send it to the vendor to be like, this is exactly what I want to see.

So yeah, I kind of like a Mad Libs approach.

Galen Low: I love that. I'm a huge fan of Mad Libs and user stories, and here they are combined. I think that's a great, a great way to look at it. That kind of leaves the door open, right, for the vendor, especially to shape a demo that shows how their software will address that while also still making sure that that's a thing that, you know, the team can get some affirmation and confirmation on that it will do this thing. And here's how you go about doing that. That's very cool.

Olivia Montgomery: Seeing it and knowing that you like the look and the feel is again, you know, the big driver of the business facing demo. I also talked about the second demo, being a technical facing one. This is where your user stories can be, can include, like how often do these jobs run? Do we have to wait?

Can we schedule the jobs at 1:00 AM. It's gonna run for an hour. And is that acceptable to the business? Do we need it more often, less often? Do we have the capabilities for it? That's where you want to make sure that your technical leads and the vendor's technical lead is on the demo for that side.

You need to know, your sysadmin needs to know, you know, how this tool functions, whoever is kind of owning and running your IT architecture. Whoever your, your engineer is, they need to know where is data being stored. Is the vendor hosting it? Are you okay with that? Are you hosting it? Do you know where? There's so many very technical details that if you wait until you're signing a contract to have those conversations, you can have some very unhappy surprises come off.

So you need to have that part of the vetting process and having the two technical teams together, answering their questions and showing, you know, exactly what it looks like and discussing capability. They can ask questions like, Oh, is this, is this going to require custom code? Is this functionality, you know, out of the box?

These are the kind of questions your business owner and your power user aren't going to know. And sometimes as a project manager, unless you're a technical project manager, you might not know, either. So you need to ask these, these questions. And, you also want to know about integrations. You want to ask, do any of your current existing clients have an integration with X and Y, your specific tools?

You know, don't be afraid to name the tools that you're using and ask, do you currently have integrations with these tools? Again, when you're going for a bigger, you know, name company that might not, you know, might be obvious. You might have to ask a little more probing questions, but for smaller companies you want to know.

And again, it can, if they don't have an existing or an out of the box integration, is that a deal breaker for you? You want to know, Okay, do we have to build out this custom integration? Do we have the resources for that? Is the vendor willing to do that for us? Do we have the time, do we need all those kinds of conversations can happen?

And it does sound, maybe like it would be a bloated conversation, but I promise you, it's really not, if you've kind of done your prep. Your technical teams know what they need to ask. And you're saving yourself a whole lot of headache if you wait any longer in the process, honestly.

Galen Low: I think that's a huge point, right? That there's so much to cover. It could be a bloated conversation. We do want to sort of zero in on certain use cases, but even still like a piece of software, you know, we're talking about a CRM or ERP system, content management system. Uh, it's gonna need to do a lot of different things in terms of those use cases and, and, and that will take time to, to sort of demonstrate and have a discussion around.

So, just wondering, to keep it from becoming a bloated conversation, how do you select and prioritize the use cases and the questions that you want your vendor to address?

Olivia Montgomery: Yeah. This is huge question. And I say you got to rely on your team leads, you know, your business leaders, your, your power user. You've got to talk to them before the demo and say, you know, okay, what are your main concerns? What do you want to make sure that we cover?

What them tell you what's most important, again, if you're a very technical PM, you can drive those conversations and that's great, but a lot of us aren't. So rely on them, ask them, be like, Hey, what are the two or three things you've got to see?

And I'm going to make sure that during the actual demo we cover them and we stay on track, because yeah, you need to facilitate during the demo as well. If someone wants to get too deep into something, you can say, like, Hey, you know, for, vendors are typically very good at this, you know, at time management. They've done a lot of demos of their products.

They can help with that. But also don't be afraid to chime in and just be like, Hey, I want to make sure that we cover, you know, the three points. But in order for that to work, you do have to have your prep talk with the people who are going to be in the demo and say, Okay, these are the three things. That way they're not surprised, that way, like they don't feel cutoff.

They don't feel like you're interrupting them or you're dismissing them. They know they're like, Oh, yeah, okay, back on track. We'd already agreed that this is the track we're seeing on, and it, it helps, yeah, not feel dismissed.

Galen Low: I mean, I think it's a really important point that, in the role of a project manager, if you're managing this process, there are areas where you, you know, have, have control, and want to, are responsible for setting this up.

But there are some decisions that really do lie in the hands of, even for a software selection project. Well, you will have your sponsor, right? They, the decision makers, as you said, the leadership team, and we need to rely on that decision-making process, like that leadership team to prioritize. What it is that we want to get out of, especially something like a demo, and still, you know, have that process of going through and gathering input from all the various teams that might need to use it.

So that they're filling in, you know, they're filling in the, you know that the Mad Lib, and they're having their use case sort of logged, and documented, and ultimately have a voice in the process.

Olivia Montgomery: Right. Yeah. And you always, you can remind, remind people, too, that in the demo you are trying to cover, you know, the very focused, you know, three, maybe four user stories that you, again, present to the vendor so that they, they know and they're ready.

And anything past that you can ask to be written out in your RFP, or in your, you know, whatever, there's all kinds of terms out there for that. But you can always say, yes, we, we need documentation on this. What's, what's the cost?'Cause you also want from the vendor after the demo a breakdown of, okay, this is custom code.

You asked for this specific functionality. This is how much we think it will cost. We will build it for you or you'll have to. You know, you can have a front end developer, your sysadmin, software in general is moving toward low code so that your front end, you know, sysadmin, even your business, your business high and your power user can set up a lot of workflows in these low code solutions.

But you need to know that. Do they have to write it? Do you have to do it, same with integrations? So to stay out of the weeds too much went in the demo, you know, remind everybody like, Yeah, let's get this in documentation, the details and move along with the actual watching the software work.

Galen Low: I really like that. I want us to come back to something. I want us to come back to this notion that we've kind of split the pack, in a way, right? We've said, set up two demos. One is more of a business focus, right? Get your power user in there. Look at the UI. The other one is more of an IT focus, have the conversations about like the infrastructure, the integrations, you know, the Cron jobs and the scheduling that you might need to have running in the background.

But then at some point, this needs to come back together, again, as well. It's what the pack and they're having these conversations, and they're evaluating based on different conversations. And I'm just wondering, should the entirety of the selection committee build a scorecard of some sort, just so that they have like Apples to Apples comparisons between a shortlisted products, like a way to kind of document their thinking so that when they come back together as a group, they are able to sort of quantify and evaluate, their thoughts and opinions and feelings about, about the software?

Or is that kinda maybe an outdated way of looking at it? Because as you mentioned, right, products are so different these days, software is so different and a lot of them might even be, I don't know, maybe some of them are too complex to compare using a rubric, what's your thoughts on that?

Olivia Montgomery: I love that you brought a vendor scorecards. I am a huge fan. I think they absolutely still have a place not outdated at all. Anytime that you can take out and reduce emotional reactions, which is a pitfall of sales pitch, heavy demos. You need to be able to take out the emotions and remember what you're, why you're getting software in the first place.

And it helps you be like, okay, the problem we're solving for and you have it written out and then you can compare how each vendor. So you're like, oh, this vendor was really cool. But when you come back to your scorecard, that has, you know, questions like, does it meet our functionality? And, you know, the business side and the technical side, because if it doesn't meet both, it's not a great fit.

Even if your business owner loves this one particular tool for probably an emotional reason without a scorecard or some kind of standardized rubric, you don't have a paper trail of how you're making the decision. And they're useful during decision-making discussions. And also as a project manager, I highly recommend storing and saving those every single time, because you never know when the project's going to get audited.

You never know when people are going to come up and ask questions. You don't know if they'll want to replace the software in a couple of years time. And you want to know, like, okay, why didn't we go with this vendor the first time? And you know, it'd be like, oh, it's cause they scored a one on the integration scale.

Okay, when we circle back to talk to them, do they have these integrations built out now? Great. Then you have an informed, educated decision making process. So, I fully support, endorse and love vendor scorecards. And it is a way to take, as you said, this desperate information and combine it together, you know that, and everybody can see each other's scores, whether you do one big vendor scorecard or each key stakeholder does one and then you, collate them all together.

That's going to depend on the size of your business, the size and complexity of the system. Your project management tool, you're probably going to have a lot of different leaders fill out their own scorecard.

And then as a project manager, you'll bring them together and yes, save them. Save them too, because another instance I've personally experienced. We, I was at a business and our CTO changed shortly after we had a new ERP system. And of course she came in, she wanted to know like, why do we use this tool?

How did we pick this vendor? I, whatever her, you know, emotional response was to it, I was able to go through and be like, no, this was a very informed, educated decision that we made. This is why and not only are you, that helps like relationship building with that new stakeholder. It also helps them understand their business.

They just came into a new company, you know, and they understand like, Oh, okay, yeah, this is where we're at. So yeah, you never know when they're going to come up and if you're working for a publicly company, publicly traded company, especially the auditors are going to go through, they need to know that you're not just like giving your business to your brother software company.

They need to know that you went through and made this decision, you know, in a measured, standardized way.

Galen Low: There's three things I wanted to, to dig out in what you just said, 'cause there was so many good things. One, what I'm hearing is that, you know, I think some of us think of scorecard, vendor scorecard as like being very generic, right?

It's like, UI score on a scale from 1 to 10. And that might be useful, but then I think what you're saying is actually put that specificity in. What are you trying to achieve? Why are you getting a new piece of software? That is the rubric. It's like, will this allow us to, you know, on a scale of 1 to 10, I don't know, I'm just making this up.

On a scale of 1 to 10, how well will this allow us to, whatever, plug into the rest of our software ecosystem and integrate to create fluids streamlined, automated processes. And maybe even more specific than that, I'm just kind of riffing, but that is the question on the scorecard.

Olivia Montgomery: I actually like them, some vendors scorecard's a little less specific.

So like for the sale, for instance, 1 to 10, can be a lot. I typically recommend bringing them down to just 1 to 5 and defining what each, what does a 1 mean. Is that, it can even be as basic as meets expectations, doesn't meet expectations, exceeds expectations, or they don't even have it. You know, then there, it's a zero.

Galen Low: Not existent. Yeah.

Olivia Montgomery: So I do, I do like 1 through 5. I think it's a little more clear. You get people, you force people to be a little more definite in their answer. It makes the math more simple, especially, you know, if you wait the criteria individually. Yeah, keep a 1 through 5 is kind of more---.

Galen Low: I really liked that. I'm thinking through all of those, like net promoter score surveys that I get. How likely are you to recommend this tool to a friend? 0 through 10. And yeah, I'm like scratching my head, they're going like, I wonder what a seven means?

Olivia Montgomery: Great. I was just going to say, you pick a seven. If you like it, if you like whatever they're asking you, you're going to pick seven. And if you don't like it, you're going to pick three. That's what everybody does. And that's not exactly useful.

So yeah, the 1 through 5 simplifies and kind of forces them, you know, to pick a side, and define it. You can have it, whether you use a tool, a piece of paper, a spreadsheet, whatever you're using for your scorecard, let them know exactly what this means. Again, you can even keep it basic. Meets expectations, fails to meet expectations, and exceeds.

Galen Low: I like that. The other thing I think I'm picking up in there, but tell me if I'm right or wrong. We talked about splitting the pack and there's different conversations.

And I was saying, right, like your scorecard, a lot of people think of this as very generic. And a lot of folks think that this is kind of a apples to apples situation where we want to have, yeah, the same, every individual helping to make that decision has the same sort of criteria by which they are evaluating a piece of software.

But I also, feel like, you had kind of hinted at this notion that maybe not everyone has the same scorecard between one another, but just has a consistent scorecard for themselves across a selection of software that they're evaluating. A) is that what you had meant and, and b) either way, do you think it's a good idea or bad, just so that it can reflect, for example, that the like technical IT team is evaluating based on these things and the business focus team is evaluating on these things.

As long as it going into those demos, with that consistent evaluation across tools, then when they come back to the table, they can have that conversation, but not necessarily be grading something on something that's frankly outside of their purview or outside of what they think is important.

Olivia Montgomery: Right. So, generally, I recommend having one scorecard template that has your technical and your business requirements on it. And then yes, being very clear as to who owns air quotes, hope for listeners I'm air quoting owns, that column or that row. You do want the technical team filling out, the integrations requirements, you know, cybersecurity and, the technical side.

And not, you know, the reporting dashboard, if you want your executives to be grading that. But I do want all that information shared across everyone. So whether you gray out the columns, you know, when you send it to some people to fill out or their version, you gray out the columns, but they see that it's there.

You want the same scorecard for everyone, because a lot of different reasons for this. It does help everybody to understand that there's more voices going on than just theirs. And sometimes you'll have interesting things come up of like, you know, your, your head of IT will be like, oh yeah, the integrations, when we, you know, did our technical demo, they don't actually have any current customers.

So they, they're not meeting expectations where I'm concerned. But then your business owner may be, you know, has talked to the sales person or whatever the instance of they're like, oh no, they told me they can do it. It can, but scorecard helps facilitate that conversation. Okay, what does that mean?

The sales person said that they can do it, you know, your director of IT, you know, your head of IT said that yes, they agree they can do it, but it would be a custom integration. It's not out of the box. It's not tested. No other current, you know, other clients of theirs currently has it. There's a lot of risks.

Maybe you're not willing to take those risks. That conversation needs to happen. And that's a pretty good way for it to come about, because you will have a lot of like, Oh, they said, they'll do that. You know, you could have anything from, like your shadow IT department came in and they're like, this is, we already talked, this is, we already vetted the solution.

This is what we want. You'd be like, okay, cool. We have a vendor scorecard. This is, you know, standardized what we use every time. Maybe you already filled it out depending on, you know, your relationship with them, or let's fill it out together. Same with if you have an executive who came from a conference and they saw this new tool and they're very excited about it.

Having one singular scorecard helps get everyone on the same page, feel involved and heard, and can help facilitate really key discussions that need to happen before you sign a contract, ideally.

Galen Low: That's it. That's a huge point that I think maybe gets overlooked a bit, right? Especially when we're talking about scorecard, especially when we're talking about, you know, numerical values on things. I think someone might look at that and go, okay, cool, we just put it into the computer and it tells us what software to choose, but actually it's the driver of a conversation. It's like, where is there disagreement where we need to dig in and go, okay, well, what did you mean by that?

I'm like, the geeky project manager in me is like, you know, planning poker, right? And it's like, Okay, you thought it was a 3, like it was super easy and somebody else said it was, you know, a 21. Let's get those people to talk, because clearly there is this huge range there.

There's like a lot of ground in the middle and let's, let's, let's talk about it to make sure we understand it.

Olivia Montgomery: Yeah, that's exactly it. It's, yeah, a basis for conversation and it also keeps those conversations from getting heated or getting emotional. You can share your information. You get down to numbers and not a 'he said', 'she said' you know, kind of conversation.

So yeah. Absolutely.

Galen Low: And speaking of nerdiness, I wonder if maybe we can double click on this light collation. Well, I guess the collection and collation process, you mentioned something earlier, you said maybe we have weighted scores. Everyone's kind of filling this in. We need some place to store it.

I wonder if you could kind of walk me through, you know, what you would do if you were an organization, like what, would you use a tool? What kind of meetings would you set up after all of the demos are done? And how do you put that information back in front of people? Kind of run your calculations on it and sort out where we need to have conversations then who needs to be involved in those conversations?

Olivia Montgomery: Man, Galen, that's a huge question.

Galen Low: I know, I'm sorry.

Olivia Montgomery: A lot of variability, so, because I definitely believe in always having a standardized yet customized approach. Every stakeholder is different. Every business is different. The maturity of your environment, the maturity of your business processes, the, the money you have, the time you have. Do you need a software system now? Or is this going to be a six month replacement?

You need to, as the PM, you need to have a lot of tools in your toolbox to be able to like flex and accommodate what's going on. And to start that, you need to know what the expectations are. So, hopefully before you've scheduled a demo, when, you know, someone's come to you and been like, Hey, we need you to be the project manager for getting a new software tool.

Whatever stage the project is at that point, hopefully you can take a bit of time and get an understanding of, okay, who's the actual final decision makers. It can range from, you know, a larger company might have a steering committee and IT steering committee that you need to present your findings to. Perhaps it's a very, a smaller company or, you know, a less visible tool and you just have the director of that, you know, your head of IT and the director of the business unit need to align and come into agreement. You need to know all that information before your scheduling, hopefully before even talking to vendors. So that's a lot, and I hope that that kind of answers with that into your question.

Galen Low: I mean, I think so. I think it's a, it's a, it's a big question. I think every organization is going to be different, but I think that foresight, I think my takeaway was having the foresight to do the preparation, to have a plan, especially if you're the project manager leading this process to reach into your arsenal. And reach into those, those skills that you have, the soft skills to navigate, you know, complex kind of political conversations. To be able to say, okay, here is, here's our rubric, here's our scorecard.

Here's where we're going to be storing them. Here's how we're prepping for our sessions. Here's what I'm telling vendors and asking them to show us based on the user stories that folks have submitted and the way we've prioritized. And after these demos, I have a plan to bring all this data together, resurface it in a way that we can have a conversation about, and also bring the right people to the table.

Vis-à-vis, that decision-making layer, we've gotten inputs along the way, but when it comes down to, that final decision, it's about getting the decision makers, the business owners, and the, the, the technical leads into a room to have that conversation and make a final decision. And still create an auditable trail, either a) because you might get audited, right?

Or b) because sometimes you get a new CTO and they want to know, well, why did we pick this? Why are we saying that this is the right path? Uh, and instead of going through that process all over again, it's a way to kind of build that bridge and build that relationship with that new person to say, listen, here was our process.

Here's the thought we put into it. Here's the data. And if, if they want to pivot, obviously they can pivot, but at least I'll have that information available to them.

Olivia Montgomery: Yeah. Yeah. One thing if, if you're having a tough time getting buy-in on a vendor scorecard or people aren't, if you decided to go the way where you send all the key individuals, their own scorecard, and then you're going to collate them together.

If that's a, a tough sell, or maybe your business isn't ready for that, maybe you have people who have never gone through a more mature project manager, focused, structured project that all can seem like a lot. At that point you can do the vendor's scorecard together. Say, okay, after our demos, you know, this is what we're going to be filling out.

I'm going to schedule 45 minutes and hopefully it's only 45 minutes, and let's fill this out together. That way it is, the, the, the true value of it being a conversation, and decision-making tool, it's like more active feeling. And a lot of people do, especially business leaders do like to hash things out together.

So whether you're in the same room, whether you're in different rooms, fill them out together and be like, Okay, we're, we're grading these vendors that we all saw. These are the three vendors that we watch. We're going to have a complete, at the end of this meeting we're completing a vendor scorecard, and we're all in consensus, hopefully, at the end of this meeting.

So doing one altogether can be a solution at that point.

Galen Low: I like that. It's like, it's, it's not a quiz and they are not Olympic judges. You know, we can do this collaboratively. It's, it's a discussion. It is a discussion. So I really liked, I really liked that idea, for sure.

Listen, when it's all said and done, some people might say that, you know what, when it comes to software selection, you can't make everyone happy. So how do you manage expectations along the way so that the folks within your company who did provide input, who do have stake in this decision, how do you manage their expectations so that they don't become disgruntled about the final decision?

Olivia Montgomery: Yeah. So here's another reason I like vendor scorecards, especially if you wait the criteria. So you want to, you want to keep in mind why you need a tool? Not just what tool are you getting, and like, don't name your project X software implementation. Please, don't do that, like don't name any vendor. That's another thing I see sometimes that happens.

Don't do that. It's your selection process project. Keep in mind why you need the software, because that can sometimes be forgotten or it can, you can lose sight of it. Especially when you have a lot of, when people are ready to make a decision, they're like, no, we want this software. Just buy it, turn it on, let's use it on Monday.

Always be asking, okay, is what the vendor said, does this, you know, does their proposal, does this contract, did the demo, everything, all the information you have, is it solving the business problem we're solving or we're solving for? So keep that in mind and you can use the vendor scorecard, again, as a rubric, a standardized, non-emotional exercise to keep that focus.

So you're not going into, like, okay, well, but they did, you know, X, Y, and Z functions really, really, really well. And then you're like, yeah, but they can't integrate with our email system and that's a must have. So, I know so-and-so you really liked it, because you loved how they did their reporting dashboard, but, you know.

So, yeah, waiting, bringing the conversation back to the vendor scorecard and always, always remembering to ask, does this solve the business problem? Not, does this have fancy new features? Yeah.

Galen Low: I love that. And I, what I love is, in my head, I took a note and it said scorecard equals transparency. I mean, not necessarily, not intrinsically, but that's the use of it.

It's weird being transparent about the decision that we're making, you know, like you mentioned so that we have an audit trail, whether it's by auditors or by people who are like, why did we make that decision? But for folks going through the process, it provides that clarity, right? It provides that rigor. Here is what we're trying to achieve, sharing that vision with everybody involved in the selection process so that they understand why they're being reminded, why the scorecard reinforces, why.

And then at the end, I think what's implicit here is that, you know, there is this, the spirit of sharing back, right? Uh, of, of communicating back to everyone involved, we've made this decision and here's why, and, you know, thank you for your participation. This is how it's scored. And, and just being so transparent about the process.

Is that going to make everybody happy with the, with the choice? Yeah, probably not, to your point, right? There's going to be, the folks who were like, ah, but that one had that really cool integration feature. I really wanted to use that. How dare we choose this other tool? But at least you've got this rationale to back it up and you have had the right involvement throughout so that the disgruntled, this is going to be about, Oh, I really wanted to try the natural language, uh, you know, automation interface and not that was the wrong tool for what we wanted to. And now I hate my job.

Olivia Montgomery: Yeah, you want respect. You want respect for the decision. You don't have to be happy with the decision, but you want everybody to respect the decision and they can only do that when they have honest and transparent discussions about it when they can see like, okay, yeah, we're not just going with this one, 'cause so-and-so's brother works there. I don't know.

I mean, I, I, not to be flippant, that is an honest, you know, problem in a lot of businesses. It happens. But yeah, it, it can help the, yeah, the decision has to be respected, not loved.

Galen Low: I really like that. Olivia, it's always great having you on the show. There's so much good stuff in our conversation today. I think the thing that really is going to stick with me is just like, you know, this notion of having two demos. I don't know, it kind of blew my mind a little bit because I think intuitively I want to keep this group together.

So we're making a group decision, but it doesn't mean that the demo itself needs to be the same demo that everyone sees. In fact, it's probably not a good idea because there's so much ground to cover and the conversations are so different. But you can tie it back together with something like a scorecard and have a collaborative conversation afterwards, so that you're still involving everybody, but not wasting people's time. I think that is such a cool piece of advice.

Olivia Montgomery: Yeah, 'cause your power user doesn't need to be there when you're talking about how often jobs are going to run, you know, and exactly what the API looks like. And, you know, vice-versa is to be really respectful of everyone's time, and not gloss over anything, and not to wait too long in the process for the business and IT to come together in agreement.

That shouldn't happen during implementation.

Galen Low: Last question. Last question. Trick question.

Olivia Montgomery: Another one?

Galen Low: Yeah, another trick question, just to close us out.

Let's say you are a project manager and you're working in an organization where you see that a software selection process is not being given that rigor, maybe even is, oh, that's my brother's software company, we're going to get a good deal.

What can you do? What can you say to, like maybe put some rigor into that process? Maybe take the bias away from that selection process? What are the conversations that you might need to have?

Olivia Montgomery: You want to make sure you have a relationship, a respectful relationship with whoever hat is holding that opinion.

You want to verify that they are responsible for the final decision. Sometimes, they're not and you kind of have to go to the person who is making the decision and be like, Hey, I need your help. You know, I need to, I need to leverage your authority. I need you to help me out. If they are the person making the decision, talk to them, hear them out, but ask, can you do a vendor scorecard together?

Let them know that you just, you know, you want to make sure that all the bases are covered, you know, that you are really excited about the tool. Don't be dismissive, work on your relationship. You might work with this person in the future for a long time going forward. You never know what's going to happen.

Maybe you do the vendor scorecard together, and you let them kind of, then you're using like a Socratic method of like, okay, you know what, tell me about what you learned about the integrations. And they're like, oh, we didn't talk about that. Cool, let's do that. Let's talk to them about it.

So yeah, it's kind of a Socratic method of what them kind of learn by unfolding requirements like, oh, okay, maybe this isn't the best, you know, tool. Maybe it is the best and you're like, this is great. Thank you so much for this information. You know, you're, you're helping, but we also still need to vet, let's compare it to one other option, at least.

Again, I keep coming back to, even the last time we spoke and probably every time I speak to anybody. Just be really forthcoming and transparent, let them know like I, this has to work for IT. This has to work for the business unit. This has to work for current state and for future state. And yeah, do it together.

Galen Low: Awesome. I love it.

Olivia, so nice to have you. I think we've done a really deep dive into the software selection process, especially these demos and the decision-making process afterwards.

So, I hope our listeners got a lot of value and looking forward to having you back on the show again, because I know that there is a lot of ground that we can cover and more trick questions for me to ask you.

Olivia Montgomery: Thank you so much for having me. I'm excited. I hope people did learn something, maybe they're thinking about, you know, selection process, different thinking about demos. Maybe not dreading them, not letting them be bulldozed, decisions being bulldozed over. I really hope this was helpful. I always love talking with you. Galen, you're great. Thank you so much.

Thank you!

Galen Low: So, what do you think is a software demo worth your team's time? Or is it a bit of a dog and pony show that sidesteps a more practical deep dive in evaluation period. Leave your thoughts and stories in the comments below.

And if you want to hone your skills as a strategic project leader, come and join our collective. Head over to thedigitalprojectmanager.com/membership to get access to a supportive community that shares knowledge, solves complex challenges, and shape the future of our craft — together.

From robust templates and monthly training sessions that save you time and energy, to the peer support offered through our discussion forum, community events and mastermind groups, being a member of our community means having over a thousand people in your corner as you navigate your career in digital project delivery.

And if you liked what you heard today, please subscribe and stay in touch on thedigitalprojectmanager.com.

Until next time. Thanks for listening.