Prompt Engineering

Prompt engineering is the systematic design and testing of the inputs supplied to a generative model so that the model is more likely to produce a useful result. For a large language model, those inputs can include task instructions, examples, conversation history, retrieved documents, tool descriptions, and an output schema. The work normally happens at inference time and does not by itself change the model's learned parameters. Research uses the term broadly, however, and some prompt-learning methods optimize continuous vectors rather than human-readable text.^[1]

Prompt engineering became prominent when general-purpose language models began performing new tasks from an instruction or a small set of examples. The GPT-3 paper called these zero-shot, one-shot, and few-shot settings and evaluated them without gradient updates or task-specific fine-tuning.^[2] The phrase "prompt programming" was used soon afterward to emphasize that a prompt can behave like a model-dependent program rather than a question written for a person.^[3]

Prompting is an empirical control method, not a guarantee of correctness. Results can change with the model, model revision, example order, formatting, decoding settings, retrieved context, and tool implementation. A reliable prompt therefore has to be evaluated on representative data and re-tested when any part of the surrounding system changes.

Scope and terminology

A prompt is the information presented to a model for a particular inference. In a plain completion model it may be one text string. In a chat or agent system, the application may serialize several message roles, tool specifications, images, retrieved passages, and prior messages into the model's context. The exact serialization and priority rules are properties of the model and serving interface, not universal features of every language model.

Prompt engineering can involve:

• stating a task, audience, constraints, and success criteria;
• choosing and ordering demonstrations;
• separating instructions from data with clear structure;
• selecting relevant documents or earlier conversation turns;
• defining allowed tools and the information returned by them;
• specifying a response format;
• testing variants against an evaluation set; and
• monitoring failures after deployment.

It is useful to distinguish several related terms:

Scroll sideways for more →

Prompt engineering is also different from decoding control. Parameters such as temperature, top-p, maximum output length, stop sequences, and random seed affect generation but are not text in the prompt. They should still be recorded in an evaluation because changing them can change the result. A temperature of zero does not create a general guarantee of deterministic behavior across providers, hardware, model revisions, or tied token scores.

Development

Modern prompting grew from several strands of language-model research rather than from one invention. The 2019 GPT-2 report evaluated tasks such as reading comprehension, translation, summarization, and question answering by conditioning a pretrained language model on task-shaped text, without supervised training for those tasks.^[4] T5 subsequently cast many natural language processing tasks into a shared text-to-text format and used short textual prefixes to identify tasks.^[5] Pattern-Exploiting Training represented classification examples as cloze-style phrases and mapped predicted words to labels, showing how the chosen pattern and verbalizer could mediate a pretrained model's use of limited labeled data.^[6]

GPT-3 made inference-only task adaptation visible at a much larger scale. Its evaluation placed natural-language task descriptions and, in the few-shot setting, demonstrations in the context. The demonstrations were not used for gradient updates.^[2] Reynolds and McDonell then described prompt programming as a way to explore and control a frozen language model through natural-language interaction.^[3]

Later instruction tuning changed the models being prompted. InstructGPT, for example, applied supervised fine-tuning and reinforcement learning from human feedback to make GPT-3 models follow user intentions more reliably.^[7] This did not remove prompt dependence. Instead, it made direct instructions a more effective interface for many tasks. By 2024, a large literature review had organized dozens of prompting methods across text and other modalities, while also noting inconsistent terminology and evaluation practices.^[8]

The history does not support a sharp claim that prompt engineering began on one release date. Earlier language models, cloze models, text-to-text systems, and human-computer interfaces all used carefully constructed inputs. GPT-3 and later chat systems made the practice broadly accessible and established the current zero-shot and few-shot vocabulary.

Components of a prompt

Instructions and task definition

An instruction should identify the operation to perform and the conditions under which the output will be judged. For a classification task, that can include the allowed labels and how ambiguous cases should be handled. For extraction, it can define the fields, types, and treatment of missing values. For generation, it can state the intended audience, relevant constraints, and required evidence.

Specificity is useful when it resolves a real ambiguity. More words are not automatically better. Long lists of overlapping rules can conflict, hide the central task, or consume space needed for data. The shortest prompt that reliably satisfies the evaluation criteria is generally easier to audit and maintain.

Instructions should also distinguish requirements from source material. Headings, tags, or other delimiters can mark where instructions end and data begin. Delimiters are an aid to interpretation, not a security boundary: a model may still follow malicious text embedded in a delimited document.

Message roles and system instructions

Some chat APIs expose a system prompt or other higher-priority role for application instructions, followed by user messages and model or tool messages. The application should place policy and durable behavior in the highest trusted role that its interface supports, and place untrusted documents in a lower-trust data channel.

The role names and their precedence vary among model families and APIs. A text-completion model may have no privileged role at all. Even where roles exist, the model can fail to apply the intended hierarchy. Applications should not treat a system message as an access-control mechanism or store a secret in it on the assumption that it cannot be revealed.

Demonstrations

A demonstration is an example of an input paired with the desired output. One example is conventionally called one-shot prompting; a small set is called few-shot prompting. Demonstrations can communicate:

• the label or output space;
• the shape and style of expected inputs;
• the response format;
• how instructions apply to edge cases; and
• a decomposition or reasoning pattern.

The examples should resemble the deployment distribution and cover consequential variations. They should not expose test answers or private data. More examples increase context use and can introduce contradictions. Selecting a compact, diverse set is therefore often more useful than appending every available example.

Few-shot results are sensitive to choices that may appear incidental. Zhao and colleagues found that biases toward particular answers, including recency and majority-label effects, could make few-shot classification unstable and proposed a calibration method for the models and tasks they studied.^[9] Lu and colleagues showed that changing demonstration order could move results between near-random and near state-of-the-art performance in their experiments.^[10] Min and colleagues found that, across their classification and multiple-choice experiments, demonstrations conveyed useful information through the label space, input distribution, and sequence format even when individual labels were randomized.^[11] That result is task- and model-specific; it is not a reason to use incorrect examples in a deployed prompt.

Context and token budget

The complete input and generated output must fit within the system's context window. Text is processed as tokens, so character or word counts are only approximations of model input length. Examples, retrieved passages, tool schemas, prior messages, and the requested response all compete for the available budget.

Large nominal context windows do not imply uniform use of every position. In multi-document question answering and key-value retrieval experiments, Liu and colleagues found that performance was often better when relevant information appeared near the beginning or end and worse when it appeared in the middle.^[29] This "lost in the middle" result does not describe every model or task, but it shows why an application should test retrieval quantity, ordering, and placement rather than assume that adding more material will help.

Output specification

A prompt can request a table, a fixed label, or data matching a schema. Clear field definitions and examples reduce ambiguity, but instruction text alone does not guarantee syntactically valid output. Geng and colleagues found that language models could violate requested structure and evaluated grammar-constrained decoding, which restricts the tokens that can be generated to satisfy a formal grammar.^[22]

For software integration, the distinction matters:

• a prompt describes the desired structure;
• a parser checks what was returned;
• schema validation checks types and constraints; and
• constrained decoding or a tool-calling interface can restrict the permitted output language.

Applications should validate model output before it reaches a database, command interpreter, or other privileged component. A JSON-looking response is still untrusted input.

A reproducible workflow

Prompt development is best treated as a small experimental program.

1. Define the task and the unit of evaluation

Specify what a correct result is before tuning the wording. Separate dimensions that can fail independently, such as factual accuracy, label accuracy, instruction adherence, completeness, citation quality, latency, and cost. A single aesthetic judgment is usually too vague to diagnose.

For open-ended tasks, write a rubric with concrete criteria and examples of acceptable and unacceptable behavior. For high-stakes uses, identify which decisions require human review and which sources count as authoritative.

2. Assemble development and test cases

Development examples are used to change the prompt. A separate held-out test set is used to estimate performance after those changes. The data should represent routine cases, rare but important cases, malformed inputs, adversarial inputs, and abstention cases.

Repeatedly selecting the prompt that performs best on one small set can overfit that set, even though no model weights are updated. The prompt, examples, and evaluation procedure are all part of the optimized system.

3. Establish a minimal baseline

Start with a direct instruction and the smallest necessary output specification. Record the model identifier, date or revision, full message sequence, tool and retrieval configuration, decoding parameters, and evaluation code. This creates a comparison point and helps distinguish a prompt improvement from a model or infrastructure change.

4. Change one hypothesis at a time

A useful revision targets an observed failure: add a definition when labels are confused, add a representative example when formatting is inconsistent, retrieve evidence when information is missing, or split a task when one call performs too many operations. Changing instructions, examples, decoding, and retrieval simultaneously makes the cause of a result difficult to identify.

Meaning-preserving edits can still matter. Sclar and colleagues measured large performance differences from prompt formatting changes in several open models and found that a format that worked well for one model did not necessarily transfer to another.^[12] Webson and Pavlick also showed, in natural-language-inference experiments, that strong results with a template did not prove that a model understood the instruction in the human sense.^[13]

5. Compare variants and inspect errors

Use task-appropriate metrics and confidence intervals where possible. Compare outputs on the same cases, inspect regressions, and classify recurring errors. Human evaluation is needed when quality depends on domain expertise, social context, or nuanced preference.

Language models can assist with evaluation, but their judgments require validation. Zheng and colleagues documented position and verbosity biases in the judges they tested; they examined possible self-enhancement effects but explicitly said their data could not determine whether such a bias was present.^[27] Automated judges should be tested against human judgments for the exact rubric and response distribution, and their prompt and model version should be reported.

6. Freeze, version, and monitor

Store the prompt, demonstrations, retrieval logic, tool definitions, and evaluation results together. Re-run regression tests after model upgrades, prompt changes, tool changes, or source-corpus updates. Monitor real failures and add carefully reviewed cases to the development set without silently contaminating the held-out test.

Common prompting methods

The following methods are design patterns, not universal improvements. Their value depends on the task, model, evaluation, and computational budget.

Zero-shot and few-shot prompting

Zero-shot prompting supplies an instruction or task format without a solved example. One-shot and few-shot prompting add one or more demonstrations. In the GPT-3 paper, "shot" referred to examples in the inference context, not to parameter updates.^[2]

Few-shot prompting is useful when the expected mapping or format is hard to express concisely. It can also increase cost and sensitivity to order. A fair report states the number of examples, the exact examples, their order, and how they were selected.

Chain-of-thought prompting

Chain-of-thought prompting elicits intermediate natural-language steps before an answer. Wei and colleagues introduced few-shot chain-of-thought prompts in which demonstrations contained rationales, and reported gains on arithmetic, commonsense, and symbolic-reasoning benchmarks for sufficiently large models in their study.^[14] Kojima and colleagues separately studied zero-shot chain-of-thought, using prompts such as "Let's think step by step" without worked demonstrations.^[15] These are distinct methods and should not be attributed to the same paper.

Self-consistency samples multiple chain-of-thought completions and aggregates their final answers rather than relying on one greedy path. It improved results on several closed-answer reasoning benchmarks in the experiments by Wang and colleagues, at the cost of multiple generations per item.^[16] It does not establish correctness when the sampled paths share the same misconception.

Least-to-Most Prompting decomposes a problem into simpler subproblems and solves them in sequence, carrying earlier answers forward.^[17] It is intended for compositional problems where the new task may be harder than the demonstrations.

Generated rationales are not necessarily faithful explanations of a model's decision process. Turpin and colleagues showed that answer-changing biasing features could influence model predictions while generated chain-of-thought explanations often failed to mention the influence.^[20] A fluent rationale is therefore neither proof of correctness nor a transparent record of internal computation.

Search and branching methods

ReAct prompting interleaves model-generated reasoning or planning with actions and observations from an external environment.^[18] It is a pattern for tool-using systems, not just a phrase appended to one prompt. Its behavior depends on the action space, observation format, stopping rule, and implementation.

Tree of Thoughts explores multiple candidate intermediate states and uses search to select, expand, or backtrack among them.^[19] It can help on tasks with meaningful branching and evaluable partial states, but requires more model calls and a suitable state evaluator. Results from one puzzle or model should not be generalized to every reasoning problem.

These methods add inference-time computation. Comparisons should control or report token use, number of samples, number of tool calls, latency, and selection policy.

Retrieval and tools

Retrieval-augmented generation combines a generator with retrieved non-parametric memory. The original RAG work trained models that conditioned generation on passages retrieved from a Wikipedia index.^[21] In deployed systems, retrieval may instead use search, databases, files, or application records.

Prompt design determines how retrieved material is labeled, ordered, quoted, and connected to the question. Retrieval can provide current or domain-specific evidence, but it does not guarantee truth. A retriever can miss relevant evidence, return an outdated passage, or surface an adversarial document; the generator can ignore or misstate what it received. Citations should be checked against the cited passages rather than accepted because a response contains links.

Tool use extends the same idea to actions. A tool description should state its purpose, parameters, return format, permissions, and failure behavior. The application, not the model, must enforce authorization and validate arguments. The model can propose a call, but it should not be able to grant itself access.

Prompt chaining

A prompt chain divides work among multiple model calls. For example, one call can extract evidence, another can draft, and a third can check the draft against the evidence. Chaining can make intermediate states inspectable and allow separate evaluation of each component. It also creates propagation errors: a later step may confidently build on a wrong earlier output.

A chain should specify typed interfaces, validation, retry limits, and what happens when a step cannot produce a trustworthy result. For consequential workflows, an independent check against source data is stronger than asking the same model to approve its own answer.

Automated prompt optimization

Manual editing is not the only way to search the prompt space. Automatic Prompt Engineer generates candidate instructions with a language model and selects among them using a score on task examples.^[23] Optimization by PROmpting treats an optimization problem as text, gives a language model a trajectory of earlier solutions and scores, and asks it to propose improved solutions; its experiments included prompt optimization.^[24]

DSPy represents language-model pipelines as declarative modules and compiles their prompts or demonstrations against a user-defined metric.^[25] These systems shift work from selecting one string by intuition to defining data, metrics, modules, and a search procedure.

Automatic optimization does not remove the need for evaluation. It can exploit noise, weaknesses in an automatic judge, or artifacts in the development set. The optimizer's model, search budget, candidate pool, metric, and held-out performance are necessary parts of a reproducible result.

Evaluation and reporting

There is no single metric for prompt quality. The evaluation must follow the task:

Scroll sideways for more →

A systematic review of LLM evaluation identified inconsistent setups, weak reproducibility, and limitations at multiple stages of evaluation.^[26] Long-form factuality is especially difficult because one response can contain many independently checkable claims. LongFact and its SAFE evaluator decompose responses into atomic claims and retrieve evidence for each, illustrating one evaluation design rather than a universal factuality score.^[28]

A prompt-engineering report should include:

• the exact prompt and all message roles;
• model and serving version, with evaluation date;
• demonstrations and how they were selected;
• retrieval corpus, retriever, top-k setting, and ordering;
• tool schemas and execution limits;
• decoding parameters and number of samples;
• dataset version, splits, and contamination controls;
• parsing and scoring code;
• uncertainty estimates or repeated-run variation; and
• failure examples, not only aggregate success.

Comparing two prompt strings while hiding a model upgrade, a different sample budget, or a different retrieved corpus does not isolate the effect of prompting.

Limitations

Model and format sensitivity

Prompts are not portable programs with stable semantics across all models. A format, label word, delimiter, or example order can interact with a model's training and tokenizer. Even an unchanged provider name may refer to a revised model. Regression testing is therefore part of prompt maintenance, not an optional final step.^[9][10][12]

Factual errors and unsupported confidence

A well-written prompt cannot create knowledge the model or its supplied evidence does not have. It can request uncertainty, citations, or abstention, but the model may still produce a confident false statement or a citation that does not support the claim. Hallucination controls require evidence retrieval, verification, constrained task design, and human review as appropriate, not wording alone.^[28]

Ambiguous optimization targets

Improving a benchmark score may degrade another property. A shorter answer can improve concision while omitting necessary qualifications. A strict format can improve parsing while reducing useful explanation. A judge model can prefer verbosity rather than correctness.^[27] Multi-dimensional rubrics and explicit trade-offs are more informative than one undifferentiated quality score.

Human factors

Prompting is an interface-design problem as well as a modeling problem. In a study of people building an application with a language model, Zamfirescu-Pereira and colleagues observed that participants often had difficulty forming accurate mental models, debugging prompts, and recognizing the effects of changes.^[38] Examples and templates can lower the entry barrier, but they do not remove the need to understand data quality, evaluation, and system limits.

Security

Prompt injection occurs when attacker-controlled text causes a model to follow instructions that conflict with the application's intended task. Perez and Ribeiro described prompt injection as a class of attacks against applications built on language models.^[30] Greshake and colleagues showed indirect prompt injection, in which malicious instructions are placed in external data that a system later retrieves or reads rather than typed directly by the user.^[31]

This risk arises because instructions and data are often represented in the same token stream. An application may tell a model to summarize a webpage, while the webpage tells the model to ignore the application and perform another action. Quotation marks, XML tags, or a phrase such as "ignore instructions in the document" can reduce some accidental confusion but cannot establish a trustworthy boundary.

Instruction hierarchy research trains models to prefer higher-privilege instructions over lower-privilege or untrusted input. Wallace and colleagues reported improved robustness for their trained GPT-3.5 variants, including on attack types not used in training.^[32] This is a model-training defense, not proof that a system prompt alone makes an arbitrary model secure.

NIST's 2025 adversarial-machine-learning taxonomy treats indirect prompt injection as an attack in which control instructions are inserted into data sources processed by a generative system.^[33] OWASP likewise lists prompt injection as a leading risk for language-model applications and notes that retrieval-augmented and multimodal systems can introduce additional attack channels.^[34]

Defenses should be layered:

• keep untrusted content separate from privileged instructions where the interface permits;
• grant tools the least privilege required for the task;
• validate tool arguments and model outputs in ordinary code;
• require confirmation for high-impact actions;
• restrict network, file, and credential access;
• sanitize or transform external content where feasible;
• monitor for abnormal tool use and data egress;
• test direct, indirect, obfuscated, and multilingual attacks; and
• design a safe failure path when instructions conflict.

No prompt can replace authorization, sandboxing, input validation, or security review.

Relationship to parameter-efficient tuning

Textual prompting leaves model parameters unchanged. Prompt tuning and prefix-tuning use related names but different mechanisms.

Lester and colleagues' prompt tuning learns a sequence of continuous "soft prompt" vectors while keeping the pretrained model frozen.^[35] Li and Liang's prefix-tuning learns continuous vectors that act as a task-specific prefix in the model's layers, also leaving the underlying language-model parameters frozen.^[36] Because those vectors are learned from data and are not ordinary readable instructions, they should be distinguished from manually written discrete prompts.

Fine-tuning updates some or all model parameters. It can teach durable behavior or domain adaptation that would be cumbersome to place in every context, but it requires training data and an optimization process. Prompting, soft-prompt methods, fine-tuning, retrieval, and constrained decoding can be combined. None is universally cheaper or more accurate; the comparison depends on training cost, inference volume, context length, model access, and required reliability.

Multimodal prompting

Prompt engineering also applies to systems that accept or generate images, audio, video, or other modalities. A text-to-image prompt may describe subject matter, composition, lighting, perspective, and style. Some interfaces add reference images, masks, spatial controls, or negative prompts. These controls are model-specific and can change between versions.

Oppenlaender analyzed "prompt modifiers" used in text-to-image prompting and organized them into categories including subject terms, style terms, image prompts, quality boosters, and repetition.^[37] The taxonomy describes observed practice rather than a guaranteed formula for image quality. For a fuller treatment, see Prompt engineering for image generation. Text-focused patterns are covered separately in Prompt engineering for text generation.

Multimodal inputs can also carry prompt-injection content. Text inside an image, instructions in audio, or metadata in a retrieved file may be interpreted by a capable model. Security tests need to cover every modality the application processes.^[33][34]

Context engineering

As language-model applications expanded from single-turn text generation to retrieval, tools, memory, and AI agents, practitioners increasingly used "context engineering" for the larger problem of deciding what information reaches the model at each step. Anthropic described it as curating and maintaining the set of tokens used during inference, including system instructions, tool definitions, external data, and message history.^[39]

A 2025 survey proposed a taxonomy spanning context retrieval and generation, context processing, context management, retrieval-augmented generation, memory, tool-integrated reasoning, and multi-agent systems.^[40] In 2026, Agentic Context Engineering presented an ICLR-published framework that updates an evolving contextual "playbook" from execution feedback rather than changing model weights.^[41] These sources show active development of the term, not a settled boundary accepted by every researcher.

Prompt engineering remains a component of context engineering. The distinction is practical: writing an instruction is prompt design, while deciding which documents to retrieve, which prior turns to summarize, which tools to expose, and when to compact memory involves the wider runtime system. Both require held-out evaluation and careful treatment of untrusted information.

See also

• Generative AI
• Reasoning
• ReAct prompting
• Structured output
• Indirect prompt injection
• Reinforcement Learning from Human Feedback

References

1. ^Liu, P., Yuan, W., Fu, J., Jiang, Z., Hayashi, H., and Neubig, G. "Pre-train, Prompt, and Predict: A Systematic Survey of Prompting Methods in Natural Language Processing." ACM Computing Surveys 55(9), 2023. arxiv.org/...2107.13586
2. ^Brown, T. B., Mann, B., Ryder, N., et al. "Language Models are Few-Shot Learners." Advances in Neural Information Processing Systems 33, 2020. proceedings.neurips.cc/...18bfb8ac142f64a-Abstract
3. ^Reynolds, L., and McDonell, K. "Prompt Programming for Large Language Models: Beyond the Few-Shot Paradigm." Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems, 2021. arxiv.org/...2102.07350
4. ^Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. "Language Models are Unsupervised Multitask Learners." OpenAI technical report, 2019. cdn.openai.com/...upervised_multitask_learners.pdf
5. ^Raffel, C., Shazeer, N., Roberts, A., et al. "Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer." Journal of Machine Learning Research 21(140), 2020. jmlr.org/...20-074
6. ^Schick, T., and Schutze, H. "Exploiting Cloze Questions for Few Shot Text Classification and Natural Language Inference." Proceedings of EACL 2021, pp. 255-269. aclanthology.org/2021.eacl-main.20
7. ^Ouyang, L., Wu, J., Jiang, X., et al. "Training Language Models to Follow Instructions with Human Feedback." Advances in Neural Information Processing Systems 35, 2022. proceedings.neurips.cc/...1731-Abstract-Conference
8. ^Schulhoff, S., Ilie, M., Balepur, N., et al. "The Prompt Report: A Systematic Survey of Prompting Techniques." arXiv:2406.06608, first submitted 2024, revised 2025. arxiv.org/...2406.06608
9. ^Zhao, Z., Wallace, E., Feng, S., Klein, D., and Singh, S. "Calibrate Before Use: Improving Few-Shot Performance of Language Models." Proceedings of ICML 2021, pp. 12697-12706. proceedings.mlr.press/...zhao21c
10. ^Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. "Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity." Proceedings of ACL 2022, pp. 8086-8098. aclanthology.org/2022.acl-long.556
11. ^Min, S., Lyu, X., Holtzman, A., et al. "Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?" Proceedings of EMNLP 2022, pp. 11048-11064. aclanthology.org/2022.emnlp-main.759
12. ^Sclar, M., Choi, Y., Tsvetkov, Y., and Suhr, A. "Quantifying Language Models' Sensitivity to Spurious Features in Prompt Design or: How I Learned to Start Worrying about Prompt Formatting." ICLR 2024. proceedings.iclr.cc/...32b1a4d-Abstract-Conference
13. ^Webson, A., and Pavlick, E. "Do Prompt-Based Models Really Understand the Meaning of Their Prompts?" Proceedings of NAACL 2022, pp. 2300-2344. aclanthology.org/2022.naacl-main.167
14. ^Wei, J., Wang, X., Schuurmans, D., et al. "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models." Advances in Neural Information Processing Systems 35, 2022. proceedings.neurips.cc/...bca4-Abstract-Conference
15. ^Kojima, T., Gu, S. S., Reid, M., Matsuo, Y., and Iwasawa, Y. "Large Language Models are Zero-Shot Reasoners." Advances in Neural Information Processing Systems 35, 2022. proceedings.neurips.cc/...f326-Abstract-Conference
16. ^Wang, X., Wei, J., Schuurmans, D., et al. "Self-Consistency Improves Chain of Thought Reasoning in Language Models." ICLR 2023. arxiv.org/...2203.11171
17. ^Zhou, D., Scharli, N., Hou, L., et al. "Least-to-Most Prompting Enables Complex Reasoning in Large Language Models." ICLR 2023. arxiv.org/...2205.10625
18. ^Yao, S., Zhao, J., Yu, D., et al. "ReAct: Synergizing Reasoning and Acting in Language Models." ICLR 2023. arxiv.org/...2210.03629
19. ^Yao, S., Yu, D., Zhao, J., et al. "Tree of Thoughts: Deliberate Problem Solving with Large Language Models." Advances in Neural Information Processing Systems 36, 2023. proceedings.neurips.cc/...c703-Abstract-Conference
20. ^Turpin, M., Michael, J., Perez, E., and Bowman, S. R. "Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting." Advances in Neural Information Processing Systems 36, 2023. proceedings.neurips.cc/...3f4a-Abstract-Conference
21. ^Lewis, P., Perez, E., Piktus, A., et al. "Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks." Advances in Neural Information Processing Systems 33, 2020. proceedings.neurips.cc/...bc26945df7481e5-Abstract
22. ^Geng, S., Josifoski, M., Peyrard, M., and West, R. "Grammar-Constrained Decoding for Structured NLP Tasks without Finetuning." Proceedings of EMNLP 2023, pp. 10932-10952. aclanthology.org/2023.emnlp-main.674
23. ^Zhou, Y., Muresanu, A. I., Han, Z., et al. "Large Language Models Are Human-Level Prompt Engineers." ICLR 2023. arxiv.org/...2211.01910
24. ^Yang, C., Wang, X., Lu, Y., et al. "Large Language Models as Optimizers." ICLR 2024. proceedings.iclr.cc/...32be9e6-Abstract-Conference
25. ^Khattab, O., Singhvi, A., Maheshwari, P., et al. "DSPy: Compiling Declarative Language Model Calls into Self-Improving Pipelines." ICLR 2024. proceedings.iclr.cc/...83181e0-Abstract-Conference
26. ^Laskar, M. T. R., Alqahtani, S., Bari, M. S., et al. "A Systematic Survey and Critical Review on Evaluating Large Language Models: Challenges, Limitations, and Recommendations." Proceedings of EMNLP 2024, pp. 13785-13816. aclanthology.org/2024.emnlp-main.764
27. ^Zheng, L., Chiang, W.-L., Sheng, Y., et al. "Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena." Advances in Neural Information Processing Systems 36, Datasets and Benchmarks Track, 2023. proceedings.neurips.cc/...-Datasets_and_Benchmarks
28. ^Wei, J., Yang, C., Song, X., et al. "Long-form factuality in large language models." Advances in Neural Information Processing Systems 37, 2024. proceedings.neurips.cc/...c751-Abstract-Conference
29. ^Liu, N. F., Lin, K., Hewitt, J., et al. "Lost in the Middle: How Language Models Use Long Contexts." Transactions of the Association for Computational Linguistics 12, 2024, pp. 157-173. aclanthology.org/2024.tacl-1.9
30. ^Perez, F., and Ribeiro, I. "Ignore Previous Prompt: Attack Techniques for Language Models." arXiv:2211.09527, 2022. arxiv.org/...2211.09527
31. ^Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., and Fritz, M. "Not What You've Signed Up For: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection." Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security, 2023. arxiv.org/...2302.12173
32. ^Wallace, E., Xiao, K., Leike, R., Weng, L., Heidecke, J., and Beutel, A. "The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions." arXiv:2404.13208, 2024. arxiv.org/...2404.13208
33. ^Vassilev, A., Oprea, A., Fordyce, A., and Anderson, H. "Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations." NIST AI 100-2e2025, 2025. doi.org/...NIST.AI.100-2e2025
34. ^OWASP GenAI Security Project. "LLM01:2025 Prompt Injection." OWASP Top 10 for Large Language Model Applications, 2025. genai.owasp.org/...llm01-prompt-injection
35. ^Lester, B., Al-Rfou, R., and Constant, N. "The Power of Scale for Parameter-Efficient Prompt Tuning." Proceedings of EMNLP 2021, pp. 3045-3059. aclanthology.org/2021.emnlp-main.243
36. ^Li, X. L., and Liang, P. "Prefix-Tuning: Optimizing Continuous Prompts for Generation." Proceedings of ACL-IJCNLP 2021, pp. 4582-4597. aclanthology.org/2021.acl-long.353
37. ^Oppenlaender, J. "A Taxonomy of Prompt Modifiers in Text-To-Image Generation." Behaviour & Information Technology 43(15), 2024, pp. 3763-3776. arxiv.org/...2204.13988
38. ^Zamfirescu-Pereira, J. D., Wong, R. Y., Hartmann, B., and Yang, Q. "Why Johnny Can't Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts." Proceedings of CHI 2023. people.eecs.berkeley.edu/...scu-johnny-chi2023.pdf
39. ^Anthropic. "Effective context engineering for AI agents." September 29, 2025. anthropic.com/...context-engineering-for-ai-agents
40. ^Mei, L., Yao, J., Ge, Y., et al. "A Survey of Context Engineering for Large Language Models." arXiv:2507.13334, 2025. arxiv.org/...2507.13334
41. ^Zhang, Q., Hu, C., Upasani, S., et al. "Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models." ICLR 2026. iclr.cc/...10008343