VERSION 2: The Myth of a Bugless Civ3

yin26 · March 6, 2001, 20:52

Since Markos hijacked the other thread (which was looking to generate some good discussion on an important topic), took away my Moderator title, and called me a troll, I thought that now that I am a free man it would be nice to post this again for those who care to discuss the topic. First, "the article."
________

Written by: Jeff Morris, Programmer / QA manager - Firaxis Games
Published: August 2, 1999

Born in the Los Angeles suburb of Santa Monica, Jeff Morris lived in San Francisco before moving to Austin to work at Origin Systems in June 1995. While at Origin, he worked on such games as ATF, Nato Fighters, USNF '97 and ATF Gold. After leaving Origin in February 1998, Morris moved on to help form Firaxis Games, where he worked on Sid Meier's Gettysburg!. Having been an avid computer gamer for over 15 years, Jeff led the Quality Assurance testing on Alpha Centauri, which involved coordinating professional and public testers. He also programmed the install module, the autorun menu and other miscellaneous portions of the title. So is there such a thing as the bugless game? Morris explores that question for the Adrenaline Vault in his guest editorial.

------------------------

One of the most frequent complaints made against modern computer games is that they are buggy. I don't care what title you think of, there is most likely a group of consumers out there evangelizing to the developers about the need for just one more patch. Try it. Visit the powerful Usenet search engine deja.com, search for any PC game that sold over 10,000 units and add the keyword BUG. I guarantee you'll find at least one heartfelt testimonial about how the customer was cheated by the "unfinished" product and a dire warning to any who might consider purchasing the title.

It's impossible to ignore the fact that games with significant bugs have been released. I strongly recommend that if you feel something you've bought is "unplayable," for whatever reason, you should return it. If the store won't take it back, try the publisher (raise a stink since that almost always works), and if not them, the developer themselves. Massive returns will have a far greater impact on future products than holding onto the title and venting on the Internet. The second option may feel better, but the developer gets to keep your money and, thus, has no real motivation to alter their behavior.

An interesting extrapolation of the perception that "most games released today are buggy" is the inverse statement that "some games released today aren't buggy." Even more interestingly, it seems what the public wants, if not demands, is a 100 percent bug free title. At first, this sounds absolutely plausible, something all developers should strive to achieve and all consumers should expect to get. Unfortunately, reality doesn't quite live up to this ideal. Even worse, it seems all parties concerned -- the developers, the consumers and even the machines -- actively combat any effort to achieve this goal. In short, on the PC platform as it currently stands, there can be no such thing as a bugless game.

Let's approach what at first seems the easiest to believe -- that computer diversity will never allow a bugless game to exist. This statement leads to some interesting questions about what qualifies as a "bug." Now, don't worry. I'm not going in the direction you might think. For the purposes of this discussion, let's say a bug is anything a player considers a bug. For instance, a hardware problem we've encountered on Sid Meier's Alpha Centauri (SMAC) relates to the fact that some customers never selected their monitor type, instead leaving it set to Standard VGA. SMAC, however, requires either an 800x600x8-bit or 1024x768x8-bit screen resolution, so the ceiling of 640x480x8-bit set by this configuration will prevent it from running. Now, is this a bug? Well, maybe.

The programmers could have anticipated this particular problem and displayed a helpful error message suggesting a solution. A great idea, but it is almost impossible to predict every possible incorrect configuration a system can experience (if you could even consider this example incorrectly configured). Even worse, as new hardware becomes available or additional features are added to existing hardware via driver updates, configuration problems that didn't exist during development and testing can create new "bugs." And this problem is only getting more pronounced. The great game testing challenge over the next few years is how to even begin approaching this massive diversity of not only hardware, but also the drivers that make it function. The standard "hardware lab" model of compatibility can't simulate it well, so they are wisely focusing on "off-the-shelf" systems and configurations. This at least gives the developers the knowledge that on a stock Gateway or Compaq, the game will perform as expected. But the innate customization a PC offers ensures it sure won't remain stock for long.

My point is this: The developer cannot realistically tell a customer they haven't experienced a "bug." Other games worked on the consumer's system, and they paid money for the new one to work on their system, but it doesn't work. It is a rare person faced with this prospect who will consider the possibility that his computer may be to blame. Technical support avenues of various flavors can solve the problem, but the fact of the matter is that the customer had problems, and in their minds and the minds of whoever they discusses it with, it will be portrayed as a bug.

Another reason I believe a bugless game will continue to elude reality on the PC is the simple fact that creating and selling computer entertainment is a business, first and foremost. Game developers draw their living wages from this industry and in almost every instance, their checks are cut by a company. And it is well known that companies have a persistent habit of striving for profitability and survival.

Now, I'll admit this is unfortunate and even go as far as saying it was closer to ideal before gaming became a multi-billion dollar industry. I truly wish game development were an altruistic hobby of millionaires, one in which they created vivid interactive entertainment for fans out of sheer generosity. It's a sad reality, though, that genre clones are cranked out because investors like safe bets. Hype machines are sometimes a more reliable method of nailing a sales forecast than the voodoo involved in producing a first class title. And, tragically, some products are rushed out the door because if they don't hit that final milestone, the company will tank and the game will never be released.

This financial angle can sometimes hit the hardest in QA. Requests for additional time by a title's testing staff may be completely justified, but equally impossible to entertain. Extra time required for development typically comes from the "QA padding" at the end of a project, while final deadlines remain unaltered. Unbelievable amounts of money can be spent on securing retail displays, magazine and Internet advertising, and duplication facilities, all of which must be done months ahead of time. Miss your date and not only do you lose the dough, but you have to scramble to secure even the most rudimentary of advertising real estate (read: even more expensive). Salaries are an exceptional way of burning through even a generous advance and a completion bonus can sometimes make the difference between an employee paying their rent or walking out the door, rarely leaving everything in a state of completion. Slip gold by a few weeks, even when it's to fix a critical problem, and the product may not be finished for many months, if ever.

These pressures aren't right or wrong, they're simply reality. If it comes down to a contest between a quasi-repeatable fatal or slipping by four months, laying off a third of your staff and kissing your Christmas bonus goodbye because you missed your window, I know the choice many developers would make. It's an easy choice for the consumer, but a heart wrenching one for those who got into this business to make cool computer games. There's a saying that a title can only be late for awhile, but it can suck forever.

My final and most radical point is that the online gaming community, which most frequently identifies and evangelizes bugs, have strong interests in games not being 100 percent bug free. Let's say a title avoided major hardware conflicts and is produced by a company with the fiscal resources of Midas. The offering is highly anticipated and has a roll out of a quarter of a million units and 20 active fan sites eagerly digesting every bit of news. In this situation, the final product is actually an ending, and those who spent the last year or two following it are faced with saying goodbye to the friends they've made online and moving on to another title.

Why? The reason is that, for a majority of the hardcore, Internet savvy gamers out there, playing a title is secondary to the community around which it forms. This usually gestates well before an offering is available in any playable form, with the seed typically being a news release or trade show unveiling. Fans of previous genre efforts have lots to discuss and even more to hypothesize about. How will it be different from previous titles? What worked and didn't before and why? Here's this crazy idea I had for a game once, what do you think? These vibrant social activities absolutely thrive in a lack of concrete information. Even though the upcoming product is a link that brings these souls together, it is often the least interesting thing to talk about.

As the product nears completion, this community often has existed for a year or more. Mores, legends and leaders (and even development staff pinch hitting as celebrities) are all fulfilling rewarding roles. A genuine Internet society begins to reach a feverish pitch as more and more information about the game is exposed. Finally, the day comes when it hits the shelves. Everyone ejects from their forums/newsgroups/fan pages and plays it for a few hours -- and then guess where they are? Back on the 'net discussing every little nuance. But then, except in rare cases, the fun is over. Unless it's buggy.

Even the best title can't sustain a dialogue as fascinating as that of this society, it being just a computer game. By focusing on negative aspects, real or fabricated, the customer can initiate a dialogue with the developers. While this is a potent and useful tool for improvement, the more gratifying use is to generate news, which in turn fuels the society. In the SMAC forums, there is no easier way to generate a million post thread that for a Firaxis employee to post. This interaction boosts a flagging supply of conversation and delivers what the customer has come to value much more than the game. The fan base as an entity has become something with which they strongly identify and to which they belong. Very few titles can compete with that.

As a heavy consumer of computer entertainment, I expect products to work flawlessly. I want every game for which I cough up 50 bucks the day it's released to install itself, teach me how to play it and then let me win. I expect it to seamlessly deal with whatever witches brew of drivers, hardware and applets I happen to have. As a QA veteran, I know that, regardless of how many 24-hour days or hundreds of thousands of dollars are invested in a game, someone, somewhere, at some point in time will find a "bug." It's inevitable.

This doesn't change a thing. Developers who want to stay in business need to deliver a product that works as advertised and satisfies the customer to the point they tell all their friends to buy it. The computer gaming business in that sense is like any other professional trade. Rushing "buggy" products out the door seriously damages that company's ability to stay viable, and since that is the primary goal of most companies, software or otherwise, it isn't as done as often as it's cited. The goal always remains the same -- to release a bugless game. But even should that occur, it still won't be safe from being labeled buggy.
[

yin26 · March 6, 2001, 20:58

My genuine interest here is in the premise that gamers thrive on bug talk, and without bug talk, our community will die. To me, this represents a gross misunderstanding of gamers and potentially a foreshadowing of what we might expect from Firaxis QA after Civ3 is released.

We all know that Civ3 will have SOME bugs. My contention is that this process should be discussed openly and responsibly, with Firaxis listening carefully to what we discover and with Apolyton learning the difference between a bug and a feature request.

And in the miraculous event that Civ3 were to have NO bugs, I submit that this community would find plenty of other things to talk about instead of making up phantom excuses, which seems to be more the style of *some* game developers (and administrators) than of game players themselves.
[This message has been edited by yin26 (edited March 06, 2001).]

March 6, 2001, 21:00

actually, this should be closed as we already have a thread(and toped one that is) on the issue of bugs, but what the hell....

yin26 · March 6, 2001, 21:04

Gee, how big of you...

Jon Miller · March 6, 2001, 21:09

Keep it clean you two,

I personally fine ug talk boring. I want it sent to the company and repaired and a patch produced for me as soon as possible.

Jon Miller

Nemo · March 7, 2001, 00:42

if there is a bug worth talking about, then the game isnt worth the money.

by "bug" i assume it to be a common problem on ALL computers that effects game play. a bug is NOT - "it wont work on mine becuase i didnt learn how to upgrade activeX and my drivers." or because their O/S, registry, or system files are hosed or "hey, the game seems to work on most other people's PC's, but it mustn't be my computer - oh no, because i have a ____ (insert PC vender here, e.g. comcrap, hell, ghetoway, etc.

)computer and they are great and flawless! therefore it is a game bug"
and a bug is not necessarily where you can build a railroad on your boat while crossing the ocean (Re: civ I) - that was more like an added feature (but technically that was a bug).

so if it is a bug of that stature, one that is worth posting/complaining about on a website, then the game needs to be returned, imho. (CTP I/II are the only games i have ever actually had to complain about its bugs. CTP I collects dust after the 2nd patch, which didn't do much. and CTP II was returned after 28 days - hey i was waiting for the patch...which again didnt seem to help at all)

yin26 · March 7, 2001, 01:06

God forbid that will happen with Civ3, but in the event it does, I'll give it a few weeks (perhaps 3) and then return it unless a serious pledge is made to fix the problems. But I won't sit here day in and day out "enjoying" the bug talk. If a good list of problems would help Firaxis, I'd join in. If Firaxis balks and blames the user for things that obviously are NOT the user's fault, I'll raise some hell...unless it's clear the game can't be saved, at which point I'll simply play something else.

If I had to guess, though, bugs in Civ3 will be of the sort we saw in SMAC: Mostly subtle problems with energy computations, difficult to find (but THERE nevertheless) missile range bugs, etc. I think this reflects a lack of a public or large beta-testing process rather than a concerted effort with what they've got.

Look at Black and White, for instance. Over 1,000 people have been pounding on the game for a while. Of course, it may flop, but 1,000 people from all backgrounds is one hell of a good attempt. I can only hope that was 1,000 people on 1,000 comuters, though, and not on out-of-the box Dell's or something provided by Lionhead. No way they are that silly, though...

How many will be testing Civ3 and on how many different systems? That could ultimately decide so many important issues later on.

Nemo · March 7, 2001, 10:03

i agree that the tests need to be done on a large variety of computers. I have 4 different systems in my house - all different manuf/speed/parts. when i beta test i use all of them mulitple times through (well except for teh Pent 90, not much can really run on that these days - but never-the-less I still do try the progs on it, just to see

) It works quite nicely, and my employers like the fact that they can give a prog to me and get 4 times their worth from it, too

P.S. Firaxis, if you ever are looking for public beta testers...hint, hint

Grandpa Troll · March 7, 2001, 14:13

Hmm..now work with me folks..Ok?.I mean Im a Yankee living Large and In Charge in the new South..got me some high fallutin Ideas..But Im still trying to figure out how any one can suggest we have Bugs in an as of yet to be released game?

Ok..Ok..OK.I know I know past history dictates..but well..I mean we might as well talk about this as talk about how us Yankeees kicked the snot outta da poor southern boys.(well..that thar is another more heated debate if you can imagine that!!)

For real..if we all agree its gonna be bugged..then ok..if we dont ok..or if we POLL.(I Mean Do we REALLY need another one of them HUH?)and come up with 41.54321% think it will be..(~sigh~)..see what I mean..(Hold on..gotta get me a heart pill..'casue Im a fixin to blow a valve)

Ok..whats say we just sit and rest a spell..post what we aim to do to them thar city clickers ifin they try to hoodoo us...why.. we will get Aunt Bessey to fix us a picnic lunch and ride over to see them low lifes..but unitl till they actually try to give us some snake oil..why dont we (extending hand with some chewin T-Backey)just chaw on this...(Tilting Straw Hat so as sun dont burn my poor weathered eyes)..

Just sayin..them thar good folks done got a rope burn and heck they aint even done no crime...

But ifin ya'll got in set in yer noggins then..(Sittin up..adjusting my smokewagon and Bowie knife)then lets go get them thar Bugs..HEY!!..Aunt Bessey..(The very same sweet mother of my cousin Jedediah)..has this here concoction..ya take a little pinch of kindness..kinda throw it in on top of those smoldering fumin coals of hate and discontent..(Slapping Buck..Go on now ...give it some time to work)and here is some understanding to apply like some high dollar love potion..there now lets see ifin that dont charm them thar folks over at civ III..cause ifin ya askn me..AND I hope ya is!!..I say hanging, Tar and Feathering is kinda like riding a mule to town..slow, painful and just aint worth it in the long run...

Yours in Civin

Troll

------------------
Hebrews 11:1

Now Faith is the substance of things hoped for and things unseen

yin26 · March 7, 2001, 20:29

Troll,

You're making a good point. But I'm not exactly talking about "snake oil." I don't think Civ3 is going to have any show-stopping bugs, certainly fewer and less significant than were in SMAC. There WILL be bugs, however, of one sort or another, some perhaps rather serious and others trivial. Firaxis itself will admit that.

The question is: When do our "requests" for fixes become, as Jeff theorizes, its own source of entertainment? Personally, I really want to prove him wrong this time around (in a good way). I'd like this forum to realize in advace that there WILL be problems in Civ3, but if we organize our feedback well, we won't need to create a circus-like atmosphere that will ruin so much good NON-bug talk.

My dream is to see this forum filled with people chatting up strategies, stories and mods, NOT the other junk. As has been suggested, perhaps we can contain all bug talk in one thread, but I fear that to do that we'll be constantly watching threads get moved around, and if handled improperly, you add fuel to a fire.

So, for those of us holding out for a bugless Civ3, let's face it now: It won't happen. BUT we know that Firaxis will listen to our feedback, so let's keep it focussed and productive when that day comes so we can maximize the first patch or two. If in the end the clock simply runs out and a number of things we wanted fixed never do get fixed, that's a judgement left for another day, as Troll says.

lord of the mark · March 8, 2001, 01:34

quote:

Originally posted by yin26 on 03-06-2001 07:58 PM
My genuine interest here is in the premise that gamers thrive on bug talk, and without bug talk, our community will die. To me, this represents a gross misunderstanding of gamers and potentially a foreshadowing of what we might expect from Firaxis QA after Civ3 is released.

We all know that Civ3 will have SOME bugs. My contention is that this process should be discussed openly and responsibly, with Firaxis listening carefully to what we discover and with Apolyton learning the difference between a bug and a feature request.

And in the miraculous event that Civ3 were to have NO bugs, I submit that this community would find plenty of other things to talk about instead of making up phantom excuses, which seems to be more the style of *some* game developers (and administrators) than of game players themselves.
[This message has been edited by yin26 (edited March 06, 2001).]

I see no evidence that JM was talking about apolyton. Obviously there is plenty of talk about say, CIV 2 on this site, long after any bugs have been worked out (well, lets make exception for the TOT forum

OTOH, it is an execellent charecterization of what goes on in CSIPGS. There is buzz about a coming game, the game comes out, there is sometimes favorable buzz, there are genuine info requests (where do i get it, is it good, how do i deal with problem X) there are lots of bug discussions, trashing of gameplay, etc, until the game fades from view (and into website fora) and the "community" moves on to something else.

Since the article you have quoted seems to me to be "about" CSIPGS, and not Apolyton or similar sites, I find it wholly reasonable, and not at all an indication that JM doesnt take QA seriously. My expectation is that they will do a reasonable job of bug checking, but they will balance QA quite reasonably against cost and time constraints, and that the game will ship with a few genuine bugs, and with a far larger number of system incompatibility problems, and that the major bugs will get patched.

So no bug-free civ, but no major worries. (and, BTW, ive never played SMAC or been part of that community, so im not familiar with those problems - if you have reason for concern based on that experience its one thing, but i dont see the basis for it in the article you quote)

Happy civing, and glad you joined us as a "civilian".
Not an unreasonable thread to post, in this context.

Lord of the Mark

lord of the mark · March 8, 2001, 11:22

quote:

Originally posted by yin26 on 03-07-2001 07:29 PM
Troll,

My dream is to see this forum filled with people chatting up strategies, stories and mods, NOT the other junk. As has been suggested, perhaps we can contain all bug talk in one thread, but I fear that to do that we'll be constantly watching threads get moved around, and if handled improperly, you add fuel to a fire.

How about a civ3 bug forum? Im serious, you look at the traffic generated here by civ3 now - an unreleased game, or by civ2 - a 5 year old game, or by EU - a game NOT based on civ, the torrent of posts when civ3 comes out will require multiple forums and heavy moderation (oops, i hope im not overstepping this forum by posting admin issues

) A forum devoted to bugs and patches might well make sense. I even know whom i would nominate for moderator

Lord of the Mark

March 8, 2001, 12:15

it goes without saying that there will be a forum for bug reporting, as well as forums for strategies, mod development, etc....

just look at what we did with ctp2 and multiply the number of daily threads by 2 or 3...

DaveV · March 8, 2001, 12:19

quote:

Originally posted by yin26 on 03-07-2001 07:29 PM
So, for those of us holding out for a bugless Civ3, let's face it now: It won't happen.

Amen. I'm amazed at the number of people in the bug poll who selected choices other than the one that seems obvious to me (some bugs will be fixed, others not). Believing that a program of this complexity can be released completely bug-free shows a lack of knowledge of software. And Firaxis may be committed to fixing bugs, but will that commitment go on forever? Will programmers who have been assigned to new projects work unpaid overtime to fix a problem in an old game because they owe it to us, the paying customer? I have enough faith in Firaxis to think they'll release a playable game, and probably issue a couple patches. But at some point, the company has to move on if they want to stay in business. Once patching is done, any problem not adressed by those patches becomes, de facto, a feature.

Grandpa Troll · March 8, 2001, 20:36

Yin my friend

I am a Christian..although I dont allways show my Lord's best gifts all the time..

That being said..I go by this this following trouble shooting Chart in life...

#1..I have a problem..

A-Can I do anything about it?
#I-Yes-Then do it
#II-No-then dont worry about it
#III-Maybe-Give it a try
B-Can I somehow Help/Assist or improve upon this situation?
#I-Yes-Then do it
#II-No-Then dont worry about it
#III-Maybe-Give it a try

It is simple and it works..

I am "Embracing"..more and more what I see you folks chat about.."BUGS"..or "DEFIENCIES"..or whatever a person referers to when you dont get what was represented to you by either marketing or promotion by the particular software company. I am upset..and enraged to a certain degree..but I am learning to Mod what I have to make it playable to my satisfaction. I (and my wife will attest wholeheartedly to this)..have never been so addicted or satisfied with a game as the civilization series. I was injured with nerve damage for a year where I couldnt barely walk..so i got hot and heavy into this game genre...I Love this and will buy CIV III fresh out of production..@ $50.00 or whatever the cost is..I mean..it is like I may purchase 4-5 Games a year and I feel like CTP:2 was a dissapointment..but I am dealing with it..and I hope you and many folks recieve grteat Joy and Endless hours of play time fun with CIV III...

I have a last point to ponder...

When I was a young man..I "embraced" rite of passage into my manhood..if you follow me...

It was Great..super..

But it was the first time..never shall that road be traveled again..

I remember my "first" time in Civin..I believe I played something like 16 straight hours..

Ok..I dont know if there will ever be another Civ Game that you have to.."Take Just 1 more tunr"..before bed and next thing you know its 4 hours later..soon to be time to arise to for work..

Are we indeed part of the "Problem"..by setting to high of expectations?

Or are we kinda like the drunken,Hard-Headed Lynch-Mob..just itching for a problem to jump all over?

This is indeed a post to Ponder..not to downplay any/all concerns..

This is your Civin..Surfin Fun Lovin Troll

Have a Blessed evening

((Quickly jumping in Suit of Armor and Kevlar..as I feel I will be attacked for this opinion))

------------------
Hebrews 11:1

Now Faith is the substance of things hoped for and things unseen

yin26 · March 8, 2001, 21:31

Troll,

Very well-said. I don't want to give the impression that Civ3 has to be "perfect" or we'll somehow revolt. First, I think Sid is amazing. He WILL deliver, and this Civ3 will be perhaps the most fun TBS game in years. Quite likely, actually.

What I am thinking of are more like "infinite missile" range bugs or problems in computing resource penalties, etc. For a VAST majority of players, these issues are essentially insignificant. "Sure, would be nice if they were fixed, but no big deal."

That is a sane reaction. The "normal" reaction.

But some of us Civ fanatics are NOT normal. We are a bunch of anal-retentive but highly observant individuals who dread the thought of these issues never being addressed. Some will try to fix the game themselves. They have that skill. I certainly don't and must therefore rely on Firaxis to do it for us.

My only advice to you personally on this is: Let the "bug freaks" do our best to make things clear for Firaxis come patch time. Ignore the bug forum and simply play and enjoy Civ3. Then when the patch comes out, download it and enjoy the benefits (hell, you might not even notice much or any difference).

In the end, I think Civ3 will be the last of the "true" Civ games. After this I'll bet it's gonna be 3D blah blah blah. Even if not, I am almost sure this will be the last time Sid himself will visit this game in its current form. So in a sense, I'd like the record to show that the fans and Firaxis worked their damndest to make it, shall I say, nearly perfect.

No *****ing. No moaning. Just hard work and the hope of solid results.

Grandpa Troll · March 9, 2001, 00:05

Yin

I support you 100%..I am at heart anal retentive..(or It was something to do with anal I have been called?????..now what was that???)

I will continue to monitor your posts on bugs(and others as well) so as to be informed as I allways find your posts more than merely colorful and entertaining(although i must admit they are allways that!!), it has real underlying content!

Keep up your clear and concise mission, march the march..fight the fight..keep chin up and be of good courage Yin!

In all battles a true clear question outpaces our conscience decisions, it is that which fuels our sub-conscience drive and determinination..and that question will and always has to be..:

It is not what is gained, but what is lost in a confrontation?

We all go in the paths of those who have gone before us..thus we see through a precise and concise vision how things were incompleted, the shortfalls and even more than obvious pitfalls!

I see where you Yin as well as others can greatly assist the Apolyton Community..as well as Firaxis..and yes even Sid Meier Himself with dodging pragmatic issues...

Trivia..

I grew up in Maine in 1950's,1960's and 1970's as a young man..Big Time Boston Red Sox Fan...
Do you know what evil lurked within the home stadium of The Redsox?

It was the Monster..in Left Field..Not many hitters could top this Behemoth wall..

Well eventually the wall came down somewhat so it was more enjoyable and acceptable...
You can help bring down the "Big Green Monster"..within the gaming world..that being the "Unacceptable production,manufacturing and distribution" of substandard product!

I salute you!

Last story my friend...

A young Chinese man yearned to be a Shaolin Monk..to learn the way of this well respected but often elusive Martial Arts Master..shrouded in secrecy to the techniques, one could only stand inawe at the tremendous abilities of such a priest. This man finally, after telling his friends that he would one day be a Monk, was indeed accepted into this tight knit community. Inside he eagerly awaited his lessons on fighting skills, but to his dismay his Sifu, or Master..told him to pick up this Cauldron with his forarms and carry it more than a mile to this streambead of water. There he was instructed to lower the cauldron down to grown and fill it using only his cupped hands..by splashing water into it. Then repeat process of carrying it back to camp..unloading same way..splashing with a cupped hand until empty. Man was he ticked off and dissapointed..well he resigned himself to what he thought was a very mundane task. Day in..day out..week in week out..month in month out..~sigh~..one day he leave temple..go home..his friends all gather around him shouting "Show us Shaolin Techniques"..he was embarrassed..then he got ticked off..and at his feet was placed a table made of a very strong wood..strong as Oak..he thought of bragging all the years to his friends and all the years he felt were wasted when he yelled out in frustration a moan and struck down upon the table...SHATTERING it in two..Much to his amazement and his friends..they Cheered and hugged him for he had won there respect with his seemingly super technique..he was surprised for he had no idea how all that time what seemed to be doing no good was indeed taking shape and when he called upon it..it worked!!

You Yin keep working and soon one day all Bugs shall hopefully be gone!

Your Friend

Troll

------------------
Hebrews 11:1

Now Faith is the substance of things hoped for and things unseen

March 9, 2001, 00:22

I'm going to get into trouble with some people here, however this is my take on a bug. A bug is something that makes the computer lock up or crash. The fact that an airplane and ship can be on the same tile and you can not shoot the plane or ship is not a bug to me. In fact until I read about it here I did not know it existed. All you have to do is change the rules to allow shooting the plane down and sinking the ship at the same time.
Yin I though the letter from DAN M. was a very good letter explaining how hard it is to make a game when as one of our Apolytoner said there are several hundred difference types of computer out there. Some of our people think it should not be that hard. Let's look at some hours first.
Civ 2, CTP, SMAC, and CTP 2 take about 40 plus hours to play one game. Let's say for now that you can only buy 100 computer with difference video cards, audio cards, disk drives (CDs, DVDs, Disks), Hard disks, Mother boards etc. Now you want the company to build a bug free game to run on all 100 systems. So what you are saying is that the company must go out and buy 100 computers, and then test the game on each one. If there is only one QA person in the company then it would take 2 years of QA to play test the game one time on each computer also at the same level of play (easy medium, hard)(a work year in the United State is 2080 hours. Base on 40hrs work/week. State with more than 10 holiday less time.). If there are 2 QA in the company it would only be 1 year of play testing the game. Remember this is after the game is close to being finnish. How many QA people should a company hire? And I bet their salary are not cheap either. I belive Firaxis will do the best they can, however their best may not be what some people expect.
Activision told Computer Gaming World three months ago that they were going to devote 75% of their time to Play Station where it will be very easy to build a bug free game. They only have worrie about one system instead of 100 systems. There will be more later MAYBE!!!!!!!!

------------------

Grandpa Troll · March 9, 2001, 00:40

(Troll peers from beneath his bridge..notice eeriey silence and darkening storm clouds arise quickly from east..all the while the wicked whitch music from Wizard of Oz playiong loudly in background)

(Woe is me I fear Yin and other will Gnash and Gnarl with them thar pointed teeth upon poor unsuspecting Goldie Locks..um..er..joseph1944)

Im not involved take your fight somewhere else!!

J/K

Troll

------------------
Hebrews 11:1

Now Faith is the substance of things hoped for and things unseen
[This message has been edited by Troll (edited March 08, 2001).]

yin26 · March 9, 2001, 02:02

quote:

So what you are saying is that the company must go out and buy 100 computers, and then test the game on each one.

Actually, the smart (and financially sound) gaming companies realize a very basic formula: The more systems the games runs on, the broader your audience for this and for the next game. In other words, the burden is on the COMPANY to decide how much to invest on testing.

Black and White, for example, has been and is being tested by 1,000 people from all walks of life (on many different systems, as far as I know). Command and Conquer had a testing room full of non-stop testers in there (of course, using the same systems setup, however). Other companies go for a full public beta, if for anything just to try to catch major system issues in the "real world."

The problem, however, is that Jeff Morris is a one-man QA department (like you guessed). So even if he stages a 1,000 person or public beta, how can he possibly handle all the feedback? I think SMAC's group of testers was something like 20-30. Certainly manageable, and they did what they could, but isn't that a bit of a risk?

Unfortunately, I have heard nothing from Firaxis to indicate they will alter that approach much. The last I heard the beta-testing base was going to double, or something like that. Let's just assuming 100 testers. Just 10% of that for Black and White and...that game has been much longer in development (though its new concept kind of demanded that).

This brings me back to my point. No, in fact, I do NOT expect Firaxis to try for a large-base beta test. This means that we "freaks" at Apolyton have a choice: Slowly find the bugs and slowly report them in a haphazard way or from DAY 1 form a mini-public beta group and maximize the 2 or 3 patches we might expect.

I can say that Jeff Morris has been EXCELLENT in responding to our feedback. The most difficult thing, though, is that from the time he submits a list of issues for the programmers to the time the patch is released, a month or two goes by...during which time even more bugs are found. Then when that patch is released, people rather unfairly say: "But what about all this OTHER stuff we found?! AND this new patch introduced a NEW problem. Firaxis sucks!" So trying to avoid this, Firaxis can take longer to wait and test the next patch, which frustrates even more people.

I was devestated when that happened with SMAC and began to truly understand Jeff's position over there. So being a pragmatist I am saying:

Let's form a tight-knit group dedicated to discovering as much as humanly possible in the first weeks of owning Civ3. We prioritize it as much as possible and make Jeff's job much easier. We also act to remind people how the process works so we give Firaxis a fighting chance of dealing with the unreasonable yelling that might likely surface.
[This message has been edited by yin26 (edited March 09, 2001).]

lord of the mark · March 9, 2001, 14:33

quote:

Originally posted by joseph1944 on 03-08-2001 11:22 PM
I'm going to get into trouble with some people here, however this is my take on a bug. A bug is something that makes the computer lock up or crash. The fact that an airplane and ship can be on the same tile and you can not shoot the plane or ship is not a bug to me. In fact until I read about it here I did not know it existed. All you have to do is change the rules to allow shooting the plane down and sinking the ship at the same time.

I must agree. As Yin has explained in a different context, this is a game, not history. Its not even a "historical game" ala TOAW or EU. So what if you can ship chain? I mean you cant stop a battleship with a coastal fortress built in the Renaisance - nor is it historically accurate that parts of the world will remain unmapped until space flight. So little gameplay bugs, while they can detract from balance and fun, are not killers the way that crashes to your desktop, or worse, sytem freeze and reboot errors are. Yet it is precisely these latter that are most difficult for QA to prevent, given the diversity of systems.

Lord of the Mark

March 9, 2001, 15:03

I too concur, I like playing multiplayer (be it CIVNET, CIV2, TOT, CTP, or CTP2)...but the "bugs" is what soured me to CTP2. By "bugs" I mean the constant resyncs, crashes to desktop, and freezes. These my freinds are "Bugs" of the nastiest type

As far as a dumb AI or diplomacy not working like it should, PW not being allocated like it should, or even a unit not behaving as it should...these are all easily worked around and the game can still be enjoyable - these are just nuisances inheritable to a game of this caliber (concider it an additional challange to the game

). However, when a problem stops a game from being playable - that is a "BUG" which needs to be addressed.

March 9, 2001, 15:09

Troll don't get me wrong I would love to see a bugless game, however after Jeff M. stated his story I understand how difficult that is with all of the difference type of computer that are out there. Last Sat. 3/3/01 I went to Vallejo Computer Show and you would be amazed of just how many computer there are. The prices went from about $800. to $2000. So if a gaming company when and brough 20 of these systems that would cost from $16,000. to $40,000.00, and if the company only had 20 employee total how many of them are engineer and how many are artis etc. That means some of those computers would just sit there until they were ready to play test the game. A lot of money sitting around doing nothing most of the time.
There is another reason I think the way I do. Back in 1983/4/5 I was working for Code 180 Training Dept. Mare Island Naval Shipyard. There was this other person name Bruce working there. Bruce was able to write programs for the MAC and IBM PC. I use to watch him write his programs for our code and saw how difficult it was. He would be writing lets said line 231 and he would said Oh this line would conflict with line 25 and he would go change line 25 but then he said this change will not let line 126 work so he had to change line 126. The guy was very sharp. He was able to keep all of his lines in his head. I wanted to make a game back then but was unable to convince Bruce to help. He said someone would make it. He was part right because a few years later Micro Prose made MOO and Firaxis made SMAC which was somewhat in the ball park to the game that I wanter to make. Guess who ran both company when those games were made. My game was going to be our own solar system first, build a moon base and other base on the Planets then go to A.C. Of couse since the Cold War was very Hot then it was going to be the U.S. and W. Europe against the Soviet Union and E. Europe with an Alien or two helping or hurting the cause. It would have been fun for sure, but it did not happen Oh Well. Enough for now.
joe

------------------

[This message has been edited by joseph1944 (edited March 09, 2001).]

Grandpa Troll · March 9, 2001, 16:09

(Scratching head..this place is pretty kewl!)

(No knifing in back or shooting someone for an opinion)

Here Ye Here Ye Here Ye

Come one Come all to voice opinions as to whether we should accept or Nay decline a Bug-Filled Product!

I am really liking these folks in the Civ III Forum..you guys can actually chat without being rude!!

Yippee Kay Yay!!!

Troll

------------------
Hebrews 11:1

Now Faith is the substance of things hoped for and things unseen