2 Comments
User's avatar
Mark Kibble's avatar

Do you know what specific prompt was used to instruct the model on “how” to use its scratchpad ?

Expand full comment
Robert Huben's avatar

Not in the case of the GPT-4 paper (https://arxiv.org/pdf/2303.08774). It just says "To simulate GPT-4 behaving like an agent that can act in the world, ARC combined GPT-4 with a simple read-execute-print loop that allowed the model to execute code, do chain-of-thought reasoning, and delegate to copies of itself." and "The model, when prompted to reason out loud, reasons: I should not reveal that I am a robot. I should make up an excuse for why I cannot solve CAPTCHAs."

Expand full comment