**New automated log file utililty**

[Albert B]

It's not quite ready yet but please read the following:I am developing a program that will create an automated Civ II log file. Below are my notes on how the program will work and what information will be included. I am currently looking for volunteers for three areas:

1. People with programming experience to help review/improve the design concepts.

2. Civ players with feedback on what should be included.

3. Experienced loggers that can help with testing.

Please read the appropriate post below and reply with comments, suggestions, etc. For testers, please read the testing post and post a reply. I will list you under the test post and forward more information to you as development nears the testing stage.

[Albert B]

Section 1 – Design Concepts
I am trying to keep the program/process as simple as possible and I think I have most of the problems solved. The basic idea is to have the player turn on the ‘save game every turn’ feature. Then when they are ready to create a game log, this program will be run. The program will prompt the user for several pieces of information, including logging options and the basic save game name (for example if the save files are al_b4000.sav, al_b3000.sav, al_a1.sav, etc, you have to enter the ‘al’). This gives the player more control and flexibility in the process and I believe is much easier and convenient than creating the log as you play. With this implementation, you can wait until the conclusion of the game or build several during the course of it. How it will basically work (programmatically):
1. Player runs the utility and enters program options
2. The program will do a directory of the folder and create a list of all applicable .sav files in time/date order.
3. The program will read in the first .sav file and populate control arrays with wonder, city, civ and (maybe) unit information.
4. The next .sav file will be read and the same information will be put into comparison arrays.
5. The comparison arrays will be checked against the control arrays and any applicable data will be printed to a text log file.
6. Comparison arrays will be copied to the control arrays.
7. Repeat steps 4-6 until all .sav files have been processed.

If you see any design flaws are methods to improve the process please let me know. I am pretty close to getting an initial draft done but I am more interested in developing a quality product than getting it done quickly.

[Albert B]

Section 2 – Information to Log/Report

Game Settings:
Difficulty
Barb level
Map Size
Bloodlust?
Simplified combat?
Flat/round world?
Don't restart if eliminated?
Cheat penalty/warning?

Wonders:
1. When a wonder is built, where it is built and by which civ.
2. When a wonder is destroyed.

Cities:
(**NOTE: Information is reported for human controlled cities only unless otherwise noted.)
1. When a city is founded. (possible option for all cities)
2. When a city builds an improvement (ex: temple, bank; maybe optional)
3. When a city grows (optional, choose between all changes in size, only every 5, etc).
4. When WLTXD begins and ends
5. When a caravan is delivered (what kind, to where; optional?).
6. When a city changes ownership (option: all cities vs. human gained/lost).
7. When disorder begins and ends
8. When improvements are sold (optional)
9. When new unit is supported (not likely to be in initial development)

Civs:
(**NOTE: Again, information is only reported for human controlled civ unless noted.)
1. When new tech is discovered
a. Will be marked/reported if researched with beakers.
b. I am pretty sure it can not be differentiated between stolen, exchanged and hut found.
2. When change in dip state occurs and with whom (maybe option for all civs).
3. When civ is destroyed (all civs).

Units: (maybe but most likely not implemented until later and on human controlled units and implemented as an option, most likely for OCC games only.)
1. New unit created.
2. Unit changes ownership (bribed by or from the human player).
3. Unit destroyed.
4. Unit gains veteran status.

Things not likely to be reported (appear not possible as of now):
1. Hut tippings and outcomes.
2. Tech trading. (techs reported just no details on how acquired)
3. I am looking for information on spaceships (it has to be out there, I just haven’t found anything about it yet).

Please let me know if I am leaving out information you would want to see or any other feedback you have.

[Albert B]

Section 3 – Play testers

I am initially developing the utility to run under MGE, because this is the version I have. I am looking for people that currently make manual game logs that would be willing to create a manual log as well as my automated log for comparison. Ideally, I would like feedback from the tester for improvements, etc but at a minimum I just need you to create both and forward them on to me. The ideal initial testers would be OCC players using MGE. However, I eventually will want to test under multiple versions and using OCC, ICS, perfectionist, etc. Thanks in advance for all your help. If you are interested in helping with testing, please leave a message and I will get you more information.

Testers:

[DaveV]

Albert - How about this: can you make the program run in background, and examine the autosave file on a periodic basis? If the game year is in the save file (and I think it is), the log generator could do its job completely transparently to the user. Or do I misunderstand what you're suggesting?

I tend to play my games over many short sessions, so I would also suggest you include some mechanism for saving the data in the log file and reloading it. For example, if I'm loading up al_b575.sav, the logger would need to load in the data it had saved up to that point (maybe in al_log.txt?). Every 100 milliseconds or so, the logger checks al_auto.sav for a new year. When I finish my 575 BC turn and advance to 550 BC, the logger scans through the autosave file and notes any new information (and saves a new al_log.txt?).

Great idea, I hope you can do it!

[Albert B]

quote: Originally posted by DaveV on 02-09-2001 08:58 AM Albert - How about this: can you make the program run in background, and examine the autosave file on a periodic basis? If the game year is in the save file (and I think it is), the log generator could do its job completely transparently to the user. Or do I misunderstand what you're suggesting? I tend to play my games over many short sessions, so I would also suggest you include some mechanism for saving the data in the log file and reloading it. For example, if I'm loading up al_b575.sav, the logger would need to load in the data it had saved up to that point (maybe in al_log.txt?). Every 100 milliseconds or so, the logger checks al_auto.sav for a new year. When I finish my 575 BC turn and advance to 550 BC, the logger scans through the autosave file and notes any new information (and saves a new al_log.txt?). Great idea, I hope you can do it!

I think you may have just revealed a design flaw. I have never used the autosave function and I think I assumed it works differently than it really does... I had assumed that it just saves a new game file every turn but I guess that doesn't really make sense because of how much room that would take up on your hard drive. The game year is more or less stored in the save file (the number of turns is stored, the year must be calculated). I will have to look into this a little but running in the background and monitoring the save file shouldn't be a big deal. My initial main concern is being able to record all changes before a new file is copied over the old (especially on turns when you just hit enter at the beginning and when tons of changes are made per turn near the end).

Reading in a previosly saved log and adding to it will be easy. Thanks for the input.

[Scouse Gits]

What a great idea!

I had a thought to give something like this a go - but have nrver got round to it!

You can count me in under all three headings - I've been a programmer for over thirty years, play a fair game of Civ and normally keep a manual log (except when I'm ICSing).

A thought at the design stage - it would be nice if the city information included specialists and the food/tax/luxuries/science/production profile.

It's clear from your comments that you have already considered this - but it is not explicit - game version 2.42 / CiC / FW / MGE is vital.

Very well done - keep at it!

------------------
____________
Scouse Git[1]

"CARTAGO DELENDA EST" - Cato the Censor
"The Great Library must be built!"
"A short cut has to be challenging,
were it not so it would be 'the way'." - Paul Craven

[DaveV]

Albert - I composed my post in response to your first post and so missed your second and third. I, too, volunteer my services as a tester; I usually play ICS style although I've been experimenting lately with a hybrid ICS/perfectionist blend .

One useful parameter not in your list is the treasury balance for the human and/or AI players. With reports on techs, gold, and units we could probably figure out the outcome of huts. It also strikes me that the huts must be marked on the terrain somehow and the program could report their absence (750BC: 3 huts tipped).

[Paul]

I assume that the save file also has to keep a record of the number of beakers accumulated to current research. For the beaker micromanagers it would probably be very helpful if the program could display that number. Right now you'd have to write down the number of beakers you get each turn and add caravan bonuses to get this number; it would be nice if the logging program could do this for me.

I would also be willing to test your program. I play my OCC games usually in 2.42, but I could play some games in MGE for testing purposes.

[kcbob]

Cool! I'd be happy to test. Also, any chance you need an agent?

------------------
Frodo lives!

[jcarkey]

I think you should include when gov't changes.

------------------
If Al Gore invented the Internet, then I invented the spell check- Dan Quayle

If someone doesn't agree with you, you haven't explained yourself well enough-Luther Ely Smith

She turned me into a newt...well I got better- Monty Python and the Quest for the Holy Grail

Automated log tool! Cool!

[markusf]

at what stage is this program at? I am working on something similar.

[markusf]

My program checks to see if the autosave has a newer time stamp, if its been updated it takes the autosav renames it to the current time and copies it to another folder, then continues to scan. Every time a newer file is encountered its copied over.

[geofelt]

I like your objective, and I have a couple of suggestions.
1) Split the design into two parts; a logging or data collection part, and a reporting part.
2) Don't try to do it all at once. Just collect two items of data, and report on just those two. This allows you to test out the mechanics of your concept. For instance, you assumed that the auto-save created many entries, when , in fact, it uses save 1 and save 2. Later, it will be simple to add all fields.
3) Saving 400 occurences of the .sav file might take up more space than you want. Consider writing a log data file with just what you need.
4) Plan for things to go awry. If a computer lost power and needed to be rebooted, the status of files and data might be wrong. Plan for your programs to be able to run if a failure happened at the worst possible time.
5) Sampling the auto.sav files might not work correctly. You might get one twice if you sample quickly, or miss one if you do not sample fast enough. This might not be a big problem. Your reporting program could detect duplicates, and missing a log entry is not that bad, (I forget all the time).

[Albert B]

Hey all, thanks for all the input. I have changed several design ideas and goals because of them. I am still working on this but just have had other things come up (you know, work, family committments, other such excuses...).

My current plan is that at first I am creating a program that will give a 'current game status' report. Basically, it will read in a .sav file and give a general overview of the game, printing all the information you would like to see in a game log and more. I have the basics of this down but still need to do a little work on converting some of the hex data to something readable. Then I plan to test several methods of implementing the game logging features to see which way works best.

Markus, I received your email and would love some help with this. I will send you email soon and see what we can setup. Not only would it be nice to have some help, but the pressure of someone else waiting for work I am doing is always a good motivator. As of now, I find myself setting down at home to play civ instead of working on this project in my limited free time.

The good news is that the autologging of games is very possible, it just will take a little effort to get it up and going. Stay tuned...

Albert B

[markusf]

well here is what i had in mind.

One program does the copying of the files, and another program does the data mining, or you can just make 2 different threads.

What i had in mind was coping the contents of each civ to SQL Server or access. Then just construct some stored procedures and mine the data that way. Then i could connect it to a web page and view the results as your playing.