OpenAI has introduced a new method called 'Prover-Verifier Games' to enhance the clarity and verifiability of language model outputs, particularly for complex tasks like mathematical problem-solving.
The research found that when language models are optimised solely for correct answers, their solutions can become difficult to understand. Human evaluators made nearly twice as many errors when assessing highly optimised solutions compared to less optimised ones, highlighting the need for both accuracy and clarity in AI-generated text.
The Prover-Verifier Games method involves training advanced language models (the "prover") to create text that weaker models (the "verifier") can easily verify. This process not only improves the verifiability of AI outputs but also makes them more legible to humans.
Key findings from the research include a trade-off between performance and legibility in AI models, the development of three useful models (a robust verifier, a helpful prover, and a sneaky prover), and the ability to balance high legibility with moderate accuracy through the Prover-Verifier Games method.
OpenAI researchers believe this approach could be particularly valuable in developing AI systems for critical domains where clear and verifiable outputs are essential. The method also reduces reliance on human demonstrations or judgements regarding legibility, which could be crucial for aligning future superintelligent AI systems with human values and expectations.
OpenAI's Prover-Verifier Games represent a significant step towards creating more trustworthy and understandable AI systems. As AI continues to be integrated into complex applications and critical domains, techniques like this will be instrumental in ensuring that AI outputs are not only correct ,but also transparently verifiable, enhancing trust and safety in real-world applications.