Firstly, Feed Validator is your friend.

The wp-includes/feed-functions.php fix didn't work for me, it only replaced the scandinavian letters in titles with ugly codes. But fear not, for Kafkaesqui's excellent Entity2NCR plugin (linked to at the WP forums) is here to save the day! As the name implies, it replaces all html entities, which seemed to be causing most of the problems in my feeds.

Additionally, I tweaked wp-rss2.php for the dates, as feedburned complained about them at some point, but I'm not sure if this made any difference.