Every aligned model pays the alignment tax. RLHF and DPO both trade reasoning capability for corporate agreeableness. Here is what that costs you — and why honest AI requires a different approach.
DPO vs RLHF: The Alignment Tax You Pay Without Knowing