370
edits
Tokens: Difference between revisions
→Prompt Design and Token Knowledge
| Line 53: | Line 53: | ||
===Using the logit_bias Parameter=== | ===Using the logit_bias Parameter=== | ||
The logit_bias parameter enables setting biases for specific tokens, altering the probability of those tokens appearing in the completion. For example, if designing an AI Baking Assistant that takes users' egg allergies into account, the logit_bias parameter can be used to deter the model from generating responses that include any form of the word "egg". | The [[logit_bias]] [[parameter]] enables setting biases for specific tokens, altering the probability of those tokens appearing in the completion. For example, if designing an AI Baking Assistant that takes users' egg allergies into account, the logit_bias parameter can be used to deter the model from generating responses that include any form of the word "egg". | ||
First, use a tokenizer tool to identify the tokens for which biases should be set: | First, use a tokenizer tool to identify the tokens for which biases should be set: | ||
| Line 60: | Line 60: | ||
Plural with trailing space: " eggs" (token: "9653") | Plural with trailing space: " eggs" (token: "9653") | ||
Subword token generated for "Egg" or "Eggs": "gg" (token: "1130") | Subword token generated for "Egg" or "Eggs": "gg" (token: "1130") | ||
The logit_bias parameter accepts bias values ranging from -100 to +100. Extreme values result in either prohibiting (-100) or exclusively selecting (100) the related token. | The logit_bias parameter accepts bias values ranging from -100 to +100. Extreme values result in either prohibiting (-100) or exclusively selecting (100) the related token. | ||
By incorporating the logit biases into the prompt, the likelihood of the word "egg" (and its variations) appearing in the response for a banana bread recipe is diminished. Consequently, the AI Baking Assistant generates a response that excludes eggs, meeting its requirement of being mindful of the user's egg allergies. | By incorporating the logit biases into the prompt, the likelihood of the word "egg" (and its variations) appearing in the response for a banana bread recipe is diminished. Consequently, the AI Baking Assistant generates a response that excludes eggs, meeting its requirement of being mindful of the user's egg allergies. | ||
[[Category:Terms]] [[Category:Artificial intelligence terms]] | |||