The barbarians are a real problem. They are really good at slaughtering your workers. In the game I'm playing now I got a report that there was a barbarian uprising, and I thought "cool, I hadn't seen that yet", then waited a minute or so as horseman after horseman moved onto my screen, must have been about 30 or more . . . seriously
I can't remember what I set it on though, may have been the highest, or even only the second highest difficulty setting.
But anyway, I like them anyhow, they do pose a good bit of a challenge, though a few more ancient units would be cool for them, and maybe make some better than conscript.