TTYrec tools

Contents

TTYAVI - creates AVI video files from .ttyrec input
TTYCONCAT - concatenates NetHack .ttyrecs into full games, creates logs of games, and other fun things

TTYAVI

This standalone utility for Windows systems renders a TTY recording as an AVI video file. This is helpful for viewing .ttyrecs with terminal width greater than 80 characters (existing Windows .ttyrec viewers use a fixed 80-character screen width, making viewing any wider recording difficult.) It also facilitates uploading .ttyrec recordings to video sites like YouTube.

An example of TTYAVI output can be seen here. (The YouTube video is available at up to 720p, which is only slightly lower quality than the original AVI file.)

Dark gray (bold black) is supported by TTYAVI as a terminal color. DECgraphics are partly supported (line and box drawing characters in particular). IBMgraphics are not currently supported, and if used in the .ttyrec will render oddly (I have not yet found any reference indicating how IBM extended ASCII characters are escaped in .ttyrec.)

You can specify a codec and compression options. If you're using a lossy compression codec, doubling the size of the output frames may greatly improve color fidelity and reduce distortion in the output, at the cost of increased encoding time and an increase in file size (in my experience around 50-100%.)

Depending on the codec you select and the options, the AVI output can be fast or very slow. Reading the ttyrec and frame output are very fast; the speed bottleneck is in compression. Many decent codecs at high output quality will encode rather slowly.

Note that this application does not drop frames. If many frames appear in the .ttyrec in a very short period of time, the output .AVI may be longer (real-time) than the original recording. If you use a low frame rate, this is more likely to occur. At the default frame rate (30 fps) the output .AVI's length will generally be close to the original recording, though probably not exact. Using low frame rates does increase the speed of encoding, so if run-length fidelity is not important for your application, you may want to consider a lower output fps.

TTYAVI uses fixed-width bitmap fonts to render (typewriter) characters; the bitmap font used by the application is based on Courier New and can be directly edited by changing the file "font.txt". Note that TTYAVI needs to find the file "font.txt", either in the same directory or on the path.

How to use it

TTYAVI is a console application. It takes the .ttyrec filename and some options as input from the command line. The syntax is:

ttyavi <input file> (options)

Options can include:

-width (# columns)	This sets the terminal width, in number of characters.
-height (# columns)	Sets the terminal height, in number of characters.
-output (file name)	Gives the name of the output .AVI file. If a file with the same name exists, it will be overwritten. If -output is not specified, uses "output.avi".
-fps (#)	Sets the number of frames per second in the output video
-skip (# frames)	Skips the specified number of frames from the beginning of the recording. This option can be used with -stop to output an AVI in multiple parts.
-stop (# frames)	Stops after outputting the specified number of frames. This option can be used with -skip to output an AVI in multiple parts.
-double	Doubles the dimensions of the output frames (improves video quality)
-png	Output a series of .PNG files containing the frames, instead of an .AVI

When you create an .AVI file with TTYAVI, a dialog will appear from which you can select and configure the compression codec.

With the option -png, instead of creating an AVI, you can output every frame that would have been output to the video file losslessly into .PNG still images. Output .PNGs are saved in a sub-directory called frames; if this sub-directory does not exist, it will be created. 30 fps works out to 108,000 frames per hour of video, so if you do want to use this option to get still images, you may want to use the -skip and -stop options as well to limit the output to the portion of interest.

Download

You can download a Win32 binary of TTYAVI here. Note that TTYAVI needs to be able to find the file "font.txt" when creating AVI video; it does search the path, so you may want to add the directory into which you copy the files to Windows' %PATH% environment variable.

TTYCONCAT

TTYCONCAT is a tool for concatenating .ttyrecs of partial NetHack games into full games. It also includes features that will scan a recording of a NetHack game and output a wide variety of information about the game. Some of the things that TTYCONCAT can determine include:

• When, where and how wishes were made during the game, and what the wishes were
• When and where the player encountered bones of other players
• When the Sokoban prize was found, and what it was
• What weapon (or non-weapon) the player was wielding when they defeated Vlad the Impaler
• When and where various important game milestones were acheived (acquiring resistances, defeating the Wizard of Yendor, etc.)
• When and where shops were used, and when important items like magic lamps, magic markers or wands of wishing were found in shops
• When and where certain conducts were broken
• When the player changed form, and what they changed into
• Monster genocides and reverse-genocides
• Player comments (using # extended commands), and in-game mail delivery
• ...And a bunch of other stuff.

What's the concatenation good for?

Many NetHack public servers, such as NAO, exist, where players can play, find the bones of other users, and watch the games of or send mail to other users. Most public servers record the games of players and make them available for download in .ttyrec format. For example, here are my NAO .ttyrecs. Note that the .ttyrecs are named according to their timestamp, and each .ttyrec contains one session. As a game may take place over multiple sessions, it can be time-consuming to find the .ttyrecs corresponding to a specific game. Once found, multiple .ttyrecs have to be downloaded to watch the game. And, thanks to the length and complexity of most games of NetHack, finding a particular part of the game can be difficult even if it was your game.

TTYCONCAT can take a directory full of NetHack .ttyrecs and produce the full-game .ttyrecs automatically, each helpfully indexed by role/turns/conducts/cause of death/etc., making it easy to find and watch a specific game. Further, by using the "game log" function of TTYCONCAT, you can identify points of interest that occurred within the games -- when the player found bones, wished, self-polyed, made comments using the extended command (#), etc.

Here's how to concatenate games using TTYCONCAT:

1. First, download or copy the .ttyrecs into a directory. TTYCONCAT expects that files with names that come first alphabetically/numerically are earlier ttyrecs. If the ttyrecs are named with timestamps (like the ones on NAO), that is true. If they've been named something like player-11conducts-1.ttyrec, player-11conducts-2.ttyrec, etc. this should also be true.
2. If the ttyrecs are in a compressed format, uncompress them. 7-Zip is a suitable decompression tool for Windows.
3. Execute TTYCONCAT in the directory with the single option -c. This will automatically process every .ttyrec in the directory, determine which ttyrecs belong to which game, and output the concatenated full-game .ttyrecs. By default, the full-game .ttyrecs will have filenames of the form fullgame-(#number)-(gamename).ttyrec, where (gamename) is the name of the .ttyrec containing the start of the game, and (#number) is the game number, by default starting at 1.
4. When TTYCONCAT is done, there will also be a text file, games.txt, in the directory, that will include information about each full game. The information in games.txt is similar to NetHack's own game logs, and includes the character's role, the number of turns, final score, cause of death, and may include conducts kept. Here's an example games.txt, output after running TTYCONCAT in a directory with my ttyrecs (from the link above). The number for each game corresponds with the number given to the fullgame .ttyrec.

By default, TTYCONCAT looks at all existing .ttyrecs in the directory. This might not be desireable if you already have full game .ttyrecs in the directory and just want to add some more. TTYCONCAT can take wildcard input after the -c option to specify the input .ttyrecs, so ttyconcat -c 2008*.ttyrec will work.

To use a different prefix than fullgame for the output .ttyrecs, specify the command line option -prefix followed by the desired name, for example ttyconcat -c -prefix playername.

To specify a starting game number, use the command line option -start followed by the desired number. By default, output .ttyrecs will start at 1.

Game searching and logging with TTYCONCAT

Along with concatenating .ttyrecs, TTYCONCAT can also search NetHack .ttyrecs for various points of interest. The easiest way to do so is simply to specify a .ttyrec on the command line with the option -log, for example ttyconcat -log fullgame.ttyrec. Then, TTYCONCAT will scan the input and print events of interest in the game to standard output. (You can pipe the output to a file.) Here is example output of TTYCONCAT's game logging.

Options -wish and -bones specifically look for wishes made or bones found. You may also search for an arbitrary top-line message using the -find option, for example ttyconcat game.ttyrec -find "The minotaur hits!".

The option -stats outputs various turn-by-turn game statistics (dlvl, HP, Pow, score, etc.) to the file "output.csv".

Known TTYCONCAT bugs/limitations:

• Finding bones is currently determined by the grave message. Causes of death that don't leave a grave (disintegration, petrification) or that leave a monster instead (killed by a wraith, e.g.) can leave bones that are not detected by TTYCONCAT. (If the player found good loot or made comments, it still may be easy to determine when these kinds of bones were found from the log.) Some custom grafitti messages may list as bones -- although cases from Vanilla NetHack are handled by TTYCONCAT already, games played on NAO may have many more grafitti messages.
• Final conducts as listed in games.txt are taken from the in-game "Do you want to see your conduct? [yn]" listing. If conducts were not shown at the end of the game, the conducts may not be included in the log.
• Cause of death is taken from the tombstone message or the score table listing shown at the end of the game, with some exceptions (ascension, quit, escaped, e.g.) If neither the tombstone nor the score table is shown, the cause of death might not be printed in games.txt.
• The -log mode of TTYCONCAT expects a full game .ttyrec as input.
• Since TTYCONCAT determines many in-game events based on the messages in the .ttyrec, it's quite possible to fool it, with creative monster or fruit naming for example. Since TTYCONCAT doesn't have access to the full original game state, this can't be fixed 100%. However, the TTYCONCAT logging works well for games that don't go out of their way to fool it.
• Both TTYAVI and TTYCONCAT load .ttyrecs into RAM (one at a time); this may be a problem if you have extremely large recordings.

Download

You can download a Win32 binary of TTYCONCAT here.

Reporting bugs, etc.

Both TTYAVI and TTYCONCAT are early in development, and while they work with a wide variety of tty recordings, there's no guarantee that they'll work with anything you throw at them (in fact, they're certain to break if you try hard enough.) If you see issues using these programs, feel free to report them by e-mail to codehappy (--at--) codehappy.net; please include as much information as necessary to reproduce the issue (if possible, an example .ttyrec would be helpful.)