What Do People Actually Want From AI? New Research Reveals Key Insights
A new study analyzed 1,500 responses from 75 countries to understand what people want from AI. It found that preferences vary widely, challenging current methods like RLHF that try to align AI with human values.
Researchers analyzed 1,500 open-ended responses from the PRISM dataset, collected across 75 countries, to understand what people actually want from AI systems. They found that preferences vary widely, with different people wanting different things from AI. This challenges current methods like Reinforcement Learning from Human Feedback (RLHF), which aggregates conflicting preferences and often relies on unrepresentative samples.
The study reveals that current methods for aligning AI with human values have concrete failures. For example, RLHF uses binary comparisons, which can oversimplify the complex and varied preferences of real people. This means that AI systems trained this way might not fully reflect the diverse needs and desires of their users.
If you're curious about how AI systems are trained to understand human preferences, you can explore the PRISM dataset yourself. Visit the PRISM dataset website and look for the section on AI preferences to see the open-ended responses and understand the diversity of opinions.