I expand all the time
if there's a reason. In my current game I conquered the French and the English, who were sharing a small continent, for the sole reason of gaining a monopoly on ivory and silk. While under a democracy, the only city there of any intrinsic worth was Paris (closest to my major continent) with about 75% corruption, it's not an issue.
If you are under a democracy, you can afford to rush build the basics: temple, cathedral, marketplace, and maybe a granary and library. You don't really need more than that for a colony city under democracy since it's not going to produce any merchandise or gold to justify the upkeep. Ship in the military from your mainland production centers.
If you are under a communism, then these cities can be every bit as productive (relatively speaking
) as your mainland cities. Key upgrades are the temple, cathedral, and marketplace to keep people happy plus the granary for producing extra people/shields ("Yes, get out my whip!"). If you want, build the courthouse and police station; they make a measurable difference under communism.
A city is always worth something because:
1. It secures territory which buys you respect from the AI (if you've got the military to secure it) and bumps your score if you care about that.
2. It secures territory which keeps the AI from getting any use out of it and might potentially yield a resource down the road if there isn't already one.
3. It produces citisens irregardless of corruption which can be used for making workers and/or boosting the population in more productive cities.
4. More happy citisens means a higher score if you care about such things.
You can't just think of a city as a production center, Civ3 makes them a lot more than that. They're also culture and power generating centers. Alot depends on your ultimate strategy for victory. If you're going for cultural, conquest, or domination then by all means, expand anywhere or everywhere so long as you have the military to defend and the money to afford development of the new cities. If your goal is to win by diplomatic or space race, then you need to evaluate the benefits/costs more carefully. Depending on your strategy, those "worthless" cities aren't so worthless and might even be key to winning the game.
) other civs. I need to expand further even if I take the whole of that island. 
) 1! I have rush bought a harbour, temple and courthouse but will it ever produce anything at all? If I could just get it to 2 shields I could build the FP and it would all be fine.
