So, you'd do anything to be on the Civ 3 alpha/beta team, eh? Consider...

[yin26]

Another interesting article people here might find enlightening.

http://www.cgonline.com/features/010807-f1-f1.html

Just one sample is the following. To which category does Civ 3 belong?

Due to the ease of "patching" buggy games after they're released, many publishers ignore final test reports to get the game out on time. "Unfortunately, we all know this happens," says Terminal Reality's Russell, who doesn't feel developers or publishers knowingly want to put a buggy program out. "In some cases it is from a lack of proper testing, other times it could be from external pressures placed upon the publisher or developer."

And here's that quote again from Jeff Morris:

An open public beta was originally planned, but that window I was describing never really opened. A closed public beta wasn't an option since don't they involve a large enough 'sample' for compatibility testing and requires immense manpower to manage.

So, if you guessed BOTH "lack of proper testing" AND "external pressures," you appear to be the winner!

[yin26]

YIN'S DISCLAIMER: Of course, I firmly believe that Firaxis' small group will no doubt find all the bugs that matter and produce gameplay that will need very little feedback from us. Sure, sounds impossible, but darn it if I'm not a believer!

( )

[Rhysie]

A non-public beta test does not equal 'a lack of proper testing' - Why does it have to be a public test to be a proper test? As has already been stated by Firaxis the main benefits of a public beta are compatability issues, which are not bugs, they are compatability issues.

I also think the quoted article is a little bit stupid, as if the cause is lack of testing then they aren't _knowingly_ putting out a buggy program - The point is if they don't test properly they _don't know_ that it's buggy.

[LaRusso]

Originally posted by yin26
YIN'S DISCLAIMER: Of course, I firmly believe that Firaxis' small group will no doubt find all the bugs that matter and produce gameplay that will need very little feedback from us. Sure, sounds impossible, but darn it if I'm not a believer!
( )

Sid, if YOU are so optimistic, who are we to grumble?

[yin26]

As has already been stated by Firaxis the main benefits of a public beta are compatability issues, which are not bugs, they are compatability issues.

They say it, you believe it. Simple? Of course ONE of the benefits of large or public beta is compatibility. What Jeff willingly is misleading you on here is the fact that a large of public beta is ALSO extremely valuable in helping balance the gameplay.

He has said, in effect, we think (or have to think because we have no more time left) that our primarily in-house staff (plus a handful of testers) can do all the playbalancing. He's wrong. He knows it. But he probably wasn't given any other choice, so he gives you the throw-away line about 'compatability testing' being the only thing lost.

Interestingly, however, you caught the lie in the above quote by noting:

I also think the quoted article is a little bit stupid, as if the cause is lack of testing then they aren't _knowingly_ putting out a buggy program - The point is if they don't test properly they _don't know_ that it's buggy.

I was hoping somebody would catch that, and you did! Nice work. However, I would change something a bit: "The point is if they don't test properly, they _SHOULD ASSUME!_ that it's buggy and far from optimized for solid gameplay."

The fact is they KNOW it's buggy to one degree or another but are just praying the patch work won't be too horrid. They also KNOW that the game likely won't have good gameplay until at least the first patch since the public will, in effect, be replacing the real beta testing that should have taken place before the game's release.

Civ 3 will be no different. We will be public beta testers. However, I accept this fact and will not resist it simply because I know that Firaxis will at least be listening to our suggestions. What I found particularly interesting is that some people (you, for instance) think Firaxis is meeting its testing obligations. And I also find it interesting to realize that even if we HAD tested with Firaxis, there's a good chance a lot of that feedback would have been ignored because of a release date.

Like I said, I don't mind paying to be a beta tester after the release as long as we get results.

[Wittlich]

Yin has just summed it up in a nutshell -

I too don't mind being a "beta tester" (for lack of a better term) after I buy the game - as long as the "bugs" which we find are "properly" addressed by Firaxis.

I just have this bad taste in my mouth after the treatment we (Apolyton as a whole) received from Activision over CTP2 - once bitten, twice shy.

However, I need to stress that IN NO WAY do I equate Firaxis as being on the same level as Activision!

[Rhysie]

Originally posted by yin26

What I found particularly interesting is that some people (you, for instance) think Firaxis is meeting its testing obligations.

Yep, I do, because I don't think they have any 'obligations'. They aren't a charity or government funded organisation, the only people they really have obligations to are their shareholders or owners - If we don't like what they do then we don't buy their games, so therefore it's not an obligation for them to please us, only a desirability (Both for us and them).

And they don't have an obligation to reward us for our loyalty either, in case someone wheels that out - I feel the opposite, we have more of a responsibility to reward them for the great games they have given us already (reward in the sense of giving them a chance to prove they can do it again and not assume the worst). But even that isn't an 'obligation', only a responsibility.

I would go so far as to say any money I have paid to them or previous civ teams is nowhere near as much as I think they deserve for the enjoyment those games have given me. (Though I wouldn't pay them extra out of the goodness of my heart because I'm tight). Therefore it's me in debt to them, not the other way around.

[MarkG]

Originally posted by yin26
What I found particularly interesting is that some people (you, for instance) think Firaxis is meeting its testing obligations.

what obligations are these? i dont recall firaxis pledging to have a public beta test. what firaxis has promised is a great game with as little bugs as possible. since the game is not out i dont see how you can make safe judgements on that....

[yin26]

they really have obligations to are their shareholders or owners

Agreed. And I assume by that you mean to make them money. How is that money made? By making games people actually want to buy. How do you manage that? Well, a smart company realizes the importance of ...yep... testing their software thoroughly to try to avoid needing months of patching. Because reputations DO matter.

It so happens that Firaxis has saved its reputation by excellent service through patching after release. Now, I might not like having to jump through all those hoops, but at least they deliver eventually. Otherwise, they'd be in massive trouble by now.

So, their obligations to their shareholders and owners mean, in effect, their obligations to make us happy enough to buy their games. NOT the other way around. The money, after all, comes from the bottom of the pyramid, not the top where shareholders and owners are sitting...though in actual practice, that might be awful hard to remember.

[yin26]

what firaxis has promised is a great game with as little bugs as possible.

Actually, I don't think ANY promises have been made. Vague hype, yes. But a proper promise? No. And that's fine. But the release of this game WILL reveal beyond a shadow of a doubt a game that could have been MUCH stronger in the gameplay department with a concerted beta effort (despite the fluff that a beta is mainly for compatability testing...sure, if that's all you use it for). How do I know? Do you want proof the sun will rise tomorrow, too?

But as I have said elsewhere, I will gladly pay my $50 to beta test Firaxis and give them feedback because I think they'll pay attention, which is another one of their 'obligations' if their goal is to keep selling lots of games.

[yin26]

I should clarify something here, too: Firaxis (as with most companies) is in many ways tied to the schedules determined by the top of that pyramid. I realize and understand this. I realize and understand the very people who hope to make the most money of these games are often the very ones doing everything to make the games bomb. I realize Firaxis itself would probably love nothing more than to "blue sky" the testing phase, but they are being forced to keep a release date.

So, when I say "Firaxis' obligations" I am saying that fully realizing that things like release dates might well have been forced on them in an unrealistic and dangerous fashion. Still, it's the rare company that puts a premium on testing even without imposed release dates. I suppose it's human tendency to become too blind and too tired to your own work to see it objectively.

That's why cooler heads should prevail...

[MarkG]

Originally posted by yin26
But the release of this game WILL reveal beyond a shadow of a doubt a game that could have been MUCH stronger in the gameplay department with a concerted beta effort

SO WHAT if it does? the issue is whether or not the game will be worth it's money. the rest is fluff....

Do you want proof the sun will rise tomorrow, too?

amazing comparison.....

[yin26]

amazing comparison

Ask common sense questions get common sense answers. If you want a better answer, ask a better question. Here's a good one: Given that Firaxis can't/won't do testing any more than they did with SMAC, how can you NOT assume that the problems will be similar, or worse, since they have added so many new elements to the game?

Common sense, Markos: They upped the complexity of the game without seemingly upping the testing effort. But will they listen to us as post-release public beta testers? I am sure they will. But we should call this effort what it is: Release and Patch.

Sadly, in the end, this may be the most efficient way to do it...*sigh*...I only wish we could look the beast square in the eyes.

[MarkG]

Originally posted by yin26
Here's a good one: Given that Firaxis can't/won't do testing any more than they did with SMAC, how can you NOT assume that the problems will be similar, or worse, since they have added so many new elements to the game?

given that the developers are different, i dont think i have enough data to compare the two situations....

[yin26]

Fair enough. Boy, this is gonna be an interesting release...

[kolpo]

It's not because they have no legal obligations to do so that they don't have obligations to do so. Consummers can force companies to take obligations. This does me think about Shell they thought some time ago that they should not listen to consumers; the result was a boycot by a lot of people in Europe, there profits in Europe crashed so at the end, saw they that they had no other choose then to do what the consumers want, they increase income for workers in nicaraga and invested in alternative energy.NOT because they wanted it, but because the consumers forced them to take certain obligations(same happened with company that tried to sell genetic manupilated food in europe).

By attacking the inresponable way they beta test, do you lower there reputation=lower there sales. This is why we MUST complain if we don't like something in a game. If nobody complains are the other not informated by that fact and will they buy the game even with those lacking things, but if people complain loud then will others stop buying the game also, and hurt the company and let them reconsider there strategy. Just not buying because you don't like the way they beta test is not enough, this will not have any big influence, but if you complain and cry about it loud then will others also hear it and also maybe stop buying(if waht you says makes sense) it so can you force a company.

Pure legaly has firaxis no obligations at all, if they own the majority of there stocks have they even no real obligations to the othere share owners. There is no law that prefents firaxis to release a dos prompt game, But they won't do that because consumers will then complain and not buy the game. It is an illusion to think that companies only have to follow legal obligations. That is a quick way to bankrupcy.

So if you don't like something about a product can you best: complain about it and do it loud.

[Urban Ranger]

Originally posted by MarkG
what obligations are these? i dont recall firaxis pledging to have a public beta test.

Well no, but they are still obliged to test the game thoroughly before they sell it.

Just imagine what happens if Ford markets a new car without proper testing...

[Inverse Icarus]

or if it equips their cars with untested tires...

[Adm.Naismith]

I would add a couple of sentence about the "obligation" and "reason to complain before game is published".

I NEVER think that people at Firaxis are stupid: they very probably want to make a great game, well balanced and almost as bug free as is realistic with the current development method.

Working in (business) software development (or strictly related to it) for around 17 years now, I know how much pressure they may take from people who care of money but not of lot of others things; they aren't necessarly bad people, anyone understand they have a different role and different target to meet, at least just to keep the company alive while developers play with PC

So developers needs any help from us, the final customer, to support their argument. We have hints about "window opportunity" and Infogrames announcement about "better profitability results from selling of great games as CivIII next year".

I bet they do internal meeting and meet Infogrames as well, and may be sometime they can have some help showing some marketing statistic about "hard core fan afraid of trouble without proper beta test" or "they feel the graphics is sub-par with others game, let's tweak it another bit". Need of fresh cash ever has loud voice than ours, but still we can do our best to raise the chore loud enough.

It's not about badnaming a product before the release, is about doing our part on product development: asking and be suspicius of every vague response.
Buying the final thing or not is the last choise from a customer. It doesn't help any of the two side, either: I don't want a Firaxis broken (I know it can't happen is just for sake of example) because of customer boycott. I way prefer a Firaxis that answer any complain as best as it can, as early as possible.

No need for b/s, just a plain explanation. How could be a:

"Yes, we decided once again that map graphics must stay simple, because we had to move more budget on a more needed effort in AI improvement. We realize that none will buy the game for terrain looking, but we are sure that, after the official "post release" open tuning you'll have a game AI to die for."

[Sabre2th]

The tires weren't untested, the company just ignored a known flaw.

When the tires weren't inflated enough, the weight of the car put pressure on a metal band on the tire. The (sharp) edge of the metal band cut into the tire and eventually caused it to explode. In a way, it is partially the consumer's fault. However, companies should prepare for people's stupidity (and/or ignorance)

[SerapisIV]

Originally posted by yin26
Actually, I don't think ANY promises have been made. Vague hype, yes. But a proper promise? No.

One of the Firaxis guys was quoted as claiming that this will be a "clean release" as opposed to SMAC. That's a promise in my opinion. Not to say it will be perfect, but a heck of an improvement over the multitude of SMAC problems

[Rhysie]

Originally posted by Urban Ranger


Well no, but they are still obliged to test the game thoroughly before they sell it.

Just imagine what happens if Ford markets a new car without proper testing...

That is a completely spurious comparison, there are laws regarding the safety of cars, there are no laws regarding the bugginess of games.

I certainly have never heard of someone being seriously injured or killed as a direct result of buying a buggy computer game, as opposed to buying a car with brakes that don't work.

Therefore it's not an 'obligation', merely a 'desirability'.

With regards to the suggestion that their obligation to their shareholders/owners means they have an obligation to have an open beta, this is wrong. The obligation they have to their shareholders is to make as much money as possible.

A few bugs in a game which are subsequently patched will have minimal impact on future sales of future games, whilst a delay pushing the release back beyond the christmas period to conduct further testing would mean both a loss of sales as they would lose the christmas extras, and a loss of resources spent on conducting the test itself and maintaining the hype and advertising for the additional time. I am pretty certain that the monetary value of losses caused by the slight loss of reputation caused by a slightly buggy and then patched game would be a lot less than the monetary value of the losses caused by a delay to the release.

Therefore, the most shareholder friendly route is to get it out for christmas, and thus that is their main obligation.

[yin26]

I am pretty certain that the monetary value of losses caused by the slight loss of reputation caused by a slightly buggy and then patched game would be a lot less than the monetary value of the losses caused by a delay to the release.

Of course, you keep using the word 'slight' here. 'Slight' loss of reputation and 'slightly' buggy game. Sure, if the game is only slightly buggy and has strong gameplay, you'd be right. But this is the risk, isn't it? What if there are some major (and obviously unexpected) problems? For example, what if in the rush to meet a deadline, a last minute AI change is made that causes all enemy civs to stop building ANYTHING after 15 turns?

So, you use the word "slight" and I use the word "what if," but clearly the proper thing to do is to test the damn game carefully so as to NOT rely on patches to do the finishing work. I'm sure you agree in theory, but we can also agree that, in reality, shareholders and suits have far more faith in the Release and Patch process than we do.

I could list a bunch of games / companies that died a tragic death by playing that dangerous method with the public, but you are aware of them, I'm sure. And, frankly, I'd be stunned if Civ 3 actually had any showstoppers. No, Civ 3's biggest problems will relate to gameplay issues...which are just perfect for us to handle as post-release public beta testers.

[d_dudy]

i like yin and markos going at it


i hope there's not too many bugs because they've been working on this game forever

[Rhysie]

I say slight, because they are spending almost two months beta testing it in-house and in a department of their publishers which is dedicated to testing computer games. Therefore I think the benefits from a _more_ extensive beta test would be neglibile, and who is to say they won't extend the current beta test if there are problems?

Try giving them some credit that they will act intelligently instead of assuming that no-one but the public can test a game.

[yin26]

Oh, but the public WILL test this game, see?

[Rhysie]

I never said that the public wouldn't test the game after it is released, that's a rather obvious point. However, it doesn't have any relevance to the proposition that the game is 'not properly testing' if it is released without previous public testing, which is the proposition you previously put forward.

[yin26]

I see. So the public actually helping the company finish the game AFTER it is released is just 'obvious' these days? Gee, I wonder how it got that way...

[Rhysie]

It got that way through improvements in communications which allow a games player to contact a games producer and tell them what is wrong with the game and what they would like to be improved. I would say it's the producer helping the player rather than the other way around, as by this point the player has paid the producer already.

In the past when you bought a game, you had no way of communicating with the games producer, and so the game you bought was the game you were stuck with. Nowadays we can help ourselves by taking advantage of the modern communication.

You make this sound a bad thing, I don't see anything about it which isn't good.

Oh, and the same communications advances make open betas possible, if you lived a long way away from the producers in the past you wouldn't have had a chance to participate in a beta, and the chances are you wouldn't even know it was occurring.

[yin26]

I would say it's the producer helping the player rather than the other way around, as by this point the player has paid the producer already.

So, I pay my money. But the game is weak. No, that isn't the producer's fault, of course, since it's OBVIOUS they needed our advice to get it right in the first place. So rather than getting that advice BEFORE taking our money and making a game worth playing out of the box, they ship the weak product only later to look at the financials and decide if sales warrant any further "helping the player."

This leaves the gamer firmly in a bent-over position, sticking his hands (or other) out to get whatever comes his way.

Oh yes. They ARE helping the player. Helping the player get the game that should have come in the box. With help like that...Of course, some rare companies actually use instead of ABuse this process, but that is the exception, not the rule you paint it to be. I felt Ensemble Studios, for instance, was OUTSTANDING in delivering a game that truly evolved with the public's comments.

Firaxis? Let's just say their process needs, uh, work.

Face it: The Release and Patch process does one primary thing -- get games out the door in time for quarterly financial statements. This "helping the player" business is just a delusion and the sickest of corporate BS 90%+ of the time.