Prompt engineering for text generation: Difference between revisions

A general recommendation is to maintain a diverse selection of examples relevant to the test sample and present them in random order to avoid majority label bias and recency bias. Increasing model sizes or including more training examples does not necessarily reduce variance among different permutations of in-context examples. The exact order may work well for one model but poorly for another.

When the validation set is limited, Lu et al. (2022) suggested choosing the order such that the model does not produce extremely unbalanced predictions or exhibit overconfidence in its predictions.<ref name="”117”">Lu et al. (2022) Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity https://arxiv.org/abs/2104.08786</ref>