I've yet to find a way to upgrade units in an army, or to unload them out of it, either.
It really needs to be patched to allow units to come into, and out of armies. Tonight, I was moving a swordsman into my army producing city, and instead of clicking on the disband button, I accidentally loaded it into the empty army. On the turn that I discovered motorized transport, no less! I was so pissed, I disbanded the army, to get a jump on the next one...