From Concept to Catastrophe in Instructional Design: Understanding Why Ideas Fail

Jul 3

Jul 3 From Concept to Catastrophe in Instructional Design: Understanding Why Ideas Fail

Have you ever been captivated by an idea and thought, “I want to immediately try this for my organization.” Maybe this was a type of learning strategy, technique, or piece of technology. You ran the pros and cons list through your head and surprisingly, there were far more pros than cons. You talked to a few colleagues who have instilled the same idea at their institutions, and all signs pointed to yes. Eagerly, you apply the new idea into your learning experiences and wait for the results. A few weeks later, the results are in and it was a mega flop. Not just a normal flop, but a historically bad one. What on earth happened?

Ah my friend, welcome to the real-world. This has happened to me many times where I’ve been all in on a kind of learning strategy, delivery method, or software and then it blows up in my face and I’m scrambling to figure out where things went wrong. Whenever this happens and I sift through the smoldering ashes, it all leads back to preparation and execution. I was so caught up in the moment that I was looking through rose-colored glasses and never saw the downsides. I also naively thought that everyone was on the same page as me, but if I had just taken a few extra moments to listen, perhaps I wouldn’t have been in such a bad spot.

Allow me to share with you some real-world examples of this and to highlight what to do and what not to do.

Scenario-Based Learning

I love scenario-based learning (SBL). If you have been following my work or have taken any of my courses, you’ll hear me praise SBL. Now imagine my surprise when talking with a team member from a major institution who hated, and I mean hated, SBL. I was caught off guard when I was chit chatting with her about different learning strategies that have worked well in the online space, and then when I brought up SBL, it was scoffed at. I’m used to some level of reaction around diverse kinds of learning strategies, but I wasn’t prepared for complete and utter resentment.

So, curiosity got the best of me, and I asked why such a strong negative reaction? She went on to explain that the university tried SBL and the results were mediocre at best. The faculty didn’t enjoy creating these kinds of assessments and the feedback from students was just as poor. I asked if she could share what their SBL problem types looked like to see if I could lend a hand in figuring out what happened.

After I saw the first example, I didn’t need to see more. Why did the students hate them? Well, that’s a simple answer: because they were awful. The problem type I read was the most uninspiring scenario I’ve ever seen. It went against everything that makes SBL great, but they still called it SBL. You might be asking what does something like this look like?

Here is basically what the scenario read like:

You’re Jim. A manager at ABC company. You’re upset that your team isn’t listening to your feedback. What should you do about it?

I kid you not. This was one of their scenarios. Upon hearing this, I think back to when I was trained on how to write SBL problem types, and it goes against every principle I know. I hope the alarm bells are going off inside of your heads too.

There are at least 4 things wrong with that scenario:

Where on earth is the creativity? ABC or XYZ company is what someone picks when they don’t have time to think of something.
You should never tell someone how they “feel.” It’s entirely possible that the students will have the opposite reaction to the scenario. Even if you think that 99% will relate to the character’s feelings, I promise you, not everyone will be on the same page.
This whole scenario is far too short. It’s lacking details, feels incomplete, and feels rushed as a whole.
Adding in names into an SBL problem is tricky, and I try to avoid this if at all possible. Not everyone is going to identify as a Jim. You can create characters names, but I wouldn’t then have the students pretend to be that character.

So, here’s my question for you. Did the university technically create SBL problem types? Yes, it was a 0/10 for a score, but it’s technically an SBL problem. Now, I wasn’t at this institution and I can’t say for certain what happened, but this is my best educated guess. A high ranked member of the team heard about the merits of SBL and asked the appropriate team members to apply these techniques into their learning experiences.

With such a short problem type, I can make a few assumptions with what happened behind the scenes. Given that the first SBL problem she shared with me was 4 sentences, I’ll assume that this was hurriedly designed to make this boss happy. When you do things in a hurry, training, testing, piloting, iterating, collecting feedback, and any common sense goes out the window. You try to make the stakeholder happy by cobbling something together and hoping it works. If the internal team members didn’t receive training, I highly doubt the instructors received any kind of training as well, which has the trickle-down effect to the students. Then, the students’ feedback becomes weaponized to say, this didn’t work, let’s move on to the strategy we actually wanted to use in the first place.

All of this approach was when a person with power within the organization wants to deploy a new idea. What about when an instructional designer wants to try something new and it backfires? Well, let’s talk about this!

Podcasts

Another example of the idea was right, but the execution was a swing and a miss is podcasts for courses. 500 million people listened to podcasts in 2024, so I think it’s safe to say that podcasts are a great option for delivering content. It’s an easy point of entry for the students and it’s incredibly convenient. I love using podcasts in my courses and they’ve had a great level of success from a student satisfaction perspective. The more I heard from students about how much they enjoyed podcasts, the more I shared with other designers and encouraged them to do the same. This advice certainly didn’t work for everyone though.

In my haste of sharing about how podcasts are awesome, I had to take a step back to say that not everyone is a podcaster. Absolutely anyone can start a podcast today, but trying to create a well-polished podcast is a different story. Having listened to high quality podcasts over the years have raised the bar for consumer expectations. It’s not enough to take a piece of media and “flip” it into a podcast. There has to be intention. Students are expecting the podcasts to be legitimate.

What does that look like? Well, you need a host of the show, quality microphones, scripts, and most importantly, an editor. Podcasts can be incredible, but they are only as good as you make them. This means experimenting with different techniques and seeing what works well for all parties involved. I’ve had universities send me their podcasts and ask for my reviews of them, and while the content is good, the quality is missing. Maybe this means the host and the interviewees lack chemistry. Maybe the host has a fantastic microphone, but the guest is speaking into a tin can. Maybe there is a ton of dead space or background noise that should have been removed. Any of these elements will impact the results of adding podcasts into courses.

Badges

Truth be told, I still don’t know how to feel about badges. In some cases where I’ve used them, they’ve done well and generated a positive buzz. In other cases, they fell flat and not a single student shared them.

For one project I worked on, we were told that this was going to be gamified and the students would have badges to show for their effort. At first, I was excited. The students were going to have not only badges, but a competition board where they could compete with one another. I hadn’t yet experimented with badges in this kind of way and I thought it had real potential. After the first course was finished with the newly added badges, I refreshed LinkedIn like crazy to see when the students were going to share their accomplishments. And then I waited…and waited.. and waited…

Ah, well, that didn’t work, but why? What happened? To figure this out, I went to the source and connected with the students. I asked about the badges, how they perceived them, and what I could have done differently. Here’s what I found out:

Some students had no idea what I was talking about. There was not enough hype, marketing, and awareness built around badges or the competition board. I could’ve done more with announcements, emails, examples, tutorials, etc.
It was the wrong audience for badges. This was a professional development course with mostly senior level managers. Some of them viewed the badges as childish and didn’t want to share them. For reference, the badges did look heavily gamified and not like a professional accomplishment.
They weren’t easy to find and share. The badges were buried within the LMS and then to share them on social media, it required multiple steps. All and all, it was not convenient to do.

Now is this to say that all badging is ineffective? I highly doubt that. Others have had a massive success with badges, but my past projects didn’t come to that same conclusion. I’m still open to trying badges again in the future if it made sense to do so and as you just heard, I have certainly learned from my mistakes. If I were to do this again, there would be more awareness around badges, they would redesigned to match the audience’s expectations, and everything would be more user friendly.

Competency-Based Education

I had a dream of bringing competency-based education to one of my universities. I’ve always thought that CBE makes complete sense. Let’s measure our students’ skills by what they can do, not just by how much time they spend in their seats. The unit of “time” has never made sense to me so I’m all for CBE. I did my homework, connected with other institutions who were utilizing CBE, spoke with researchers in the field, and shared the idea back with my team. They were all onboard, so off I went to make this a reality.

I spent months on this project with rethinking our curriculum, how we were going to pass feedback back and forth, changing the LMS to meet the CBE needs, etc. And then one day, I had a conversation with a friend who had been working in the CBE space for a few years. I shared my excitement around how I was going to be trying out this model, and he then asked me something that no one else had mentioned. “Do you have the infrastructure in place to handle CBE efforts? If you don’t, it’s going to become a cost center.”

I paused and asked for more information. He shared that while CBE absolutely makes sense to do from the students’ perspectives, his university was losing money keeping the operation running. In all of my efforts with thinking about CBE, I was always thinking like an instructional designer. I prioritized the learning experience first, and hadn’t thought about it from a business model perspective. Is this viable? Is it sustainable? Is this in the best interest of the department? Does this financially make sense to do?

We were understaffed and I would absolutely need to hire a new team. I calculated what it would cost for this project to run smoothly, and forecasted what the next few years would look like. It blew my budget out of the water. Not great, Bob. Not great. And with that, my CBE idea was dead. I had to then go back to my team members and leadership to explain what I had uncovered. While it was obviously a letdown, everyone was glad that I had come to this conclusion sooner rather than later.

The Takeaway

What I have learned over the years is that research is significant to see where our field is heading, but it’s only the first step when implementing a new idea. Research will not be enough to sway the key decision makers in your org. You have to think about what’s important from their perspective. This can also take weeks to months to be able to win someone over depending upon the size of the project.

I also can’t stress enough the urgency of finding someone who is already deploying your idea and asking about the highs and lows. You want the full perspective to understand all the pros and cons. It’s easy to become biased with an idea you truly want to work, but you have to have the answers for when people push back. It’s also crucial to pivot and iterate on ideas. It’s so rare for everything to work out perfectly. People want to highlight their incredible efforts and not reveal the struggles along the way. However, this paints an unrealistic picture. I would love to be writing a post right now about how I brought CBE into a major institution, and instead, I’m sharing about how it flopped, and I had to go back to the drawing board.

If you are trying to lead a new initiative at your organization, I hope this blog helps you to see a real perspective. Mistakes happen. Things take time. That’s the real-world. Good luck on your next idea!

A huge thank you to our sponsors! Consider supporting those who support our show!