Design experiments are more important than ever, and they don’t require a lab

I’m about to say something that would irritate my old academic advisors and colleagues.

You don’t need nearly as much scientific rigor as you think to run a design experiment.

That’s something I’ve learned after talking with 260+ designers.

Experimentation is increasingly becoming a bigger part of what design does, whether that’s running AI experiments in your process or laying out a hypothesis to get buy-in.

But the word “experiment” makes a lot of designers freeze up. So if the word “experiment” makes you tense up, just call it a bet.

That’s what many design leaders do to successfully communicate with executives.

Why experiments carry a stigma

I understand the freeze, because I went to grad school for design and teach designers how to work with data. I’ve watched whole classrooms shut down the moment A/B testing comes up.

Minimum detectable effect. Statistical power. Sampling bias. A/B versus multivariate testing. P-values. These are terms most…