Well, if Kingdom Hearts is any indicator of American cartoon characters speaking Japanese, I'd say it'd be pretty damn funny, but overall, not very "painful to the ears." Sure, it'd be a bit weird, but that would be it, in my case at least. If I did not like it, I'd just listen to the English voice track. Same goes for Hollywood movies, although I still have yet to see one dubbed in Japanese (bet they're hilarious though).
The same applies for anime. I prefer dubbed works, but if I find that I just do not like a certain dub, I'll just switch over to subs. None of this drama about how dubs ruin everything they touch. None of that putting of Japanese animation on a pedestal above everything else, and how the Japanese language is the most beautiful language in the entire world - just a simple switch to a more pleasurable experience.