Khan Academy has come a long way from when I first joined waaaaay back in 2010. The company and product have both matured. The content and experience that we offer the world has grown and changed dramatically, but this is the slow steady change of iterative improvement. And even though we’ve added over 1,000 videos and hundreds of interactive exercises to the site, that change isn’t always obvious.

It’s going to be hard to miss the changes we’re releasing today.

Current:

New:

Today, we’re launching the first piece of what will eventually be a major redesign of Khan Academy. This first step is part visual refresh, part re-focusing of the interface that folks use every day, and part foundation for future changes. This change should help to clarify exactly how our content is organized and what order we recommend proceeding through tutorials while fixing a host of other small bugs (like progress indicators not showing up or not being properly aligned). It also includes user experience improvements like the menu improvement that Ben Kamens blogged about.

For the curious, we’re planning a more in-depth look at the ideas and process behind these changes to be written by our amazing design intern, Tabitha Yong. Keep an eye out here  and on the official KA blog for an announcement when that’s ready.

We’ve been testing these changes with a subset of our students and teachers around the world for the last month or so, and lots of folks have reached out to us, excited about the changes. But change isn’t automatically good and it’s definitely not always easy to deal with. We know folks will have questions and concerns. I invite you to leave your comments and questions here, or get in touch with the team via Twitter, and DEFINITELY file bug reports for things that are getting in your way. Your feedback is critical to helping us understand how to improve. Please don’t be discouraged if we can’t respond to you individually! We are reading/listening.

Love,

Team Athena (Ben Komalo, Marcia, Marcos, Jason, Tabitha)

Or Brit Club? (This is Brit, he’s from Canada!)

As a product designer, I thrive on high quality feedback about my work. My designs would be worse without it. It’s that simple.

Giving and receiving high-quality feedback is challenging. Sometimes feedback sessions can go sideways. You find yourself heading into the second hour of an intense feedback session discussing things that are clearly important but only tangentially related to the design you put up for feedback. Frustration and defensiveness start to appear in the tone of the discussion, which is changing quickly and organically. You start to worry that you’re not getting the specific feedback you need to improve the proposed design.

Given that feedback is critical, how can designers create an environment that helps them get the feedback that they need while ensuring that important but unrelated feedback gets attention it deserves. And how can you do this in a way that encourages even more and better feedback?

How about a simple rule?

The First (and only) Rule of Crit Club

Welcome to Crit Club. We only have one rule: Critique the work that’s in front of you. Be specific.

Really? One rule?

Well, I guess you could say we have two rules. Rule the Second: be civil. I’ve found it easier to enforce this rule because most organizations recognize a lack of civility as unhelpful. So let’s focus on that first rule.

A somewhat real-life example

Shit just got real.

I am working on a project that includes changes to my product’s homepage. The project doesn’t change the scope or functionality of the homepage significantly, but it does re-prioritize and change the visual treatment of certain elements. I present the design to a group of my colleagues walking them through the scope and rationale for the design. I explicitly invite folks to offer feedback about the design. Here are some samples of the kinds of feedback I get:

Feedback that obeys the first rule

• This seems less readable to me.
• I don’t understand why we’ve changed the placement of element “X”
• I really like having more color on the homepage!

Important but not specific-to-the-current-design feedback

• I don’t think the homepage is showing the right information. We should change the homepage to show “Y” instead.

And herein lies our challenge. The non-specific feedback is really important, and comes in the form “I don’t think we’re solving the right problem.” Because the suggestion is (almost always) an incomplete idea, the conversation becomes asymmetric. Without sketches, user flows, or use cases clearly defined, everyone starts to compare their idealized version of the new suggestion to my higher resolution idea with all its warts clearly visible. Folks start jumping in with agreements or disagreements. And suddenly I’m debating the purpose of the homepage. And it’s important, but it’s out of scope for the proposed design change.

Invoking the first rule

The only way invoking the first rule works is if, when invoked, it also means that you take responsibility for following up on the feedback that prompted you to invoke it in the first place.

In this example, I would offer an exchange: a future conversation in which I agree to address, in detail, the concern and proposed solution in exchange for the ability to keep the current conversation focused on specific feedback. In addition, I’d commit to documenting the results of that follow-on conversation and sharing it with the participants of the original discussion.

This is a bunch of extra effort, but that’s OK. The extra effort helps to ensure that the first rule isn’t invoked casually and provides clear accountability and increased transparency for the follow-up decisions. That additional accountability and transparency creates an environment that encourages more and better feedback. A virtuous cycle is formed. Peace and harmony run rampant through the team.

Peace, harmony, and naps

We’re trying this out at KA now, and for obvious reasons YMMV. One thing we’re still figuring out is exactly how to handle the kind of feedback that would make you reconsider the entire design. We’re thinking that the follow-up feedback loop will catch these problems and we’ll handle them on a case-by-case basis for now. Keep an eye out for a future post on a more systematic approach!

The one where my friends, Ben and Rebecca, decide to share dessert:

The one where, moments later, Rebecca discovers that Ben has eaten the entire thing while she was telling a story: