How to Keep Your Prompts from Getting Boring

Researchers at Stanford University — Zhang, Yang, and Xie (2025) — recently published How to Mitigate Mode Collapse and Unlock LLM Diversity.

Their study highlights a subtle but essential problem: as large language models are fine-tuned and “aligned” through human feedback, their responses can become tedious, predictable, and repetitive.

They start playing the same note over and over — polite, safe, and uninspired. That’s what researchers call mode collapse — when an AI’s responses lose their variety. To fix it, the authors introduce a technique called Verbalized Sampling (VS).

Instead of asking the model for one answer, you prompt it to generate several possible responses, each with an estimated probability.

For example:

“Generate five different short stories about a bear, and give each a probability estimate.”

You can then pick the most interesting one — or intentionally choose a less likely (and often more creative) option.

In experiments, this approach nearly doubled the diversity of model outputs without reducing quality.

---

💡 Why this matters

Think of this less as a guardrail and more as an amplifier. Guardrails keep the model from wandering off a cliff. Amplifiers encourage it to explore new terrain.

Prompting this way reminds the model — and you — that creativity lives in probability space, not just in the top 1%.

---

⚙️ Try it yourself

“Give me five possible answers to the following prompt, and assign each a probability from 1–100%. Then present them from most to least likely.”

Use the higher-probability outputs when you need reliability, and dip into the lower ones when you want surprise.

That balance — between coherence and diversity — is where true prompt craftsmanship lives.