Demorecord Hosting?

[Arachnion]

I did some digging and found some fragments of my old code. It dates back to 2005 using old technologies... and looking at the source it's a bit messy. 

But I could clean it up and hand it over to anyone who's interested. It's nothing fancy, but it works. Basically it hacks the demofile into string rows analyzing it's content and making arrays of models, fx and stuff, then compares that to what's already in the database. I guess I need to check if the demofileformat have changed so I wont miss things. [Ok, just looked int the files and they still say Verion 2 so I guess they are compatible.] We should of course look fro other things to like costumes and maps and stuff.

Data is currently stored in an Access database.

So, should I clean this up? I could make it as a separate DLL that you could plugin wherever. Oh, and I code in Visual Basic in VS 2012. Sorry, but VB has been my main language since ... since... well, since they introduced it in 1991. Gosh! Now I seem old. 

You have a fantastic site there, spindisc. You should indeed clean it up.

I believe it's easy enough (at least IMO) to go through the current day piggs and pull out costumes and maps and stuff.

I guess I'll post this archive of mine I compiled yesterday then.

http://planz.basketcasesoftware.com/uploads/1354765161.rar

21 RTF files. Plaintext. From beta's bin.pigg. Contains maps, npc model names, all sorts of stuff.

When you see .minimap and .mapstats in there, you should be able to change them to .txt and then view them properly (working form) in demorecords.

Enjoy.

[The Fifth Horseman]

Have a functional demo parser now. Didn't identify 2 of 28 costume part slots, but that's something for later. Will work on the database tables to store the extracted information.

[spindisc]

Perfect Fifth. Then I don't need to upgrade my old code. 

Let me know if there's anything else I can help with.

[The Fifth Horseman]

I would like to take a look at your code as it is now anyhow, in case I missed something important. 

EDIT: For public use, here's the copy of my current extractor code. It's not storing the information anywhere yet, but it will in due time:

Code: [Select]

<?php
$temp_dir="temp\\";

function parse_demo($demo_file)
{
global $temp_dir;

$allowed_extensions=array("cohdemo");
// If RAR and ZIP extensions are installed, allow upacking and processing those files.
if (extension_loaded("rar"))
$allowed_extensions[]="rar";
if (extension_loaded("zip"))
$allowed_extensions[]="zip";


if (file_exists($demo_file))
{
$ext = strtolower(pathinfo($demo_file, PATHINFO_EXTENSION));
$file_contents=file_get_contents($demo_file);

if (!in_array($ext, $allowed_extensions))
{
Error("Not an allowed extension");
die();
}

$header=substr($file_contents, 0, 4);
if ($header=="PK\x3\x4" && $ext="zip")
{
// Open the ZIP, unpack and parse any .cohdemo found
$zip = zip_open($demo_file);
if ($zip) 
{
// Note: This potentially allows upload of multiple demos in a single ZIP
// Take into account when coding upload interface.
while ($zip_entry = zip_read($zip))
{
$file = basename(zip_entry_name($zip_entry));
$ext = strtolower(pathinfo($file, PATHINFO_EXTENSION));

// Ignore files that are not .cohdemos
if ($ext=="cohdemo" && zip_entry_open($zip, $zip_entry, "r")) 
{
$buffer = zip_entry_read($zip_entry, zip_entry_filesize($zip_entry));
zip_entry_close($zip_entry);
file_put_contents($temp_dir.basename($file), $buffer);
// Parse the unpacked file then delete it.
// Note: For actual upload script we might want to keep them or repack.
parse_demo($temp_dir.$file);
unlink($temp_dir.$file);
}
}
zip_close($zip);
}
else Error("Error reading ZIP archive!");
}
else if ($header=="Rar!" && $ext="rar")
{
Error("RAR archive support not mplemented");
// TODO: Business logic - open the RAR, unpack and parse any .cohdemo found
}
else if ($header=="1   " && $ext="cohdemo")
{
$file_contents=str_replace("\r", "", $file_contents);
$file_contents=explode("\n", $file_contents);
$base_time=0;
//$frames=array();
$costumes=array();

$extract=array('NPC','PARTSNAME','MOV','Map','FX','FXTINT', 'SEQ', 'geometry', 'textures', 'patterns');
foreach ($extract as $type)
$extracted[$type]=array();
$extracted['skin_colors']=array();
$extracted['costume_colors']=array();
$extracted['MOV_assoc']=array();
$animations=array();
$models=array();

foreach ($file_contents as $line)
{
$line=str_replace("  ", " ",$line);
$line=str_replace("  ", " ",$line);
$line=explode(" ", $line);
$time_offset=array_shift($line);
$entity=array_shift($line);
$command=array_shift($line);
$base_time+=$time_offset;

// The values are used as string keys, yes. 
// It's a little illogical until you realize we want to collect unique values.
if ($entity!='')
{
//Breakdown of operations by entity/time/command.
//$frames[$entity][$base_time][$command][]=implode(" ", $line);
switch($command)
{
// We're keeping an index of models assigned to each ID
// This is what we use to determine association of animations to models
case 'NPC':
case 'SEQ':
$models[$entity]=$line[0];
$extracted[$command][$line[0]]=1;
break;
case 'MOV':
$extracted[$command][$line[0]]=1;
if (array_key_exists($entity, $models))
$animations[$models[$entity]][$line[0]]=1;
else
$animations['Player'][$line[0]]=1;
break;
case 'DEL':
unset($models[$entity]);
break;
case 'Map':
case 'DYNLIB':
$extracted[$command][$line[0]]=1;
break;
case 'PARTSNAME':
//NOTE: parts number is variable depending on game version and model
// some NPC and pet models use same structure but different part numbers
// Solution: divide them by both indices and entities
$extracted['PARTSNAME'][$entity][$base_time][]=array($line[0],$line[1],$line[2]);
$extracted['geometry'][$line[0]]=1;
$extracted['textures'][$line[1]]=1;
$extracted['patterns'][$line[2]]=1;
$extracted['costume_colors'][$line[3]]=1;
$extracted['costume_colors'][$line[4]]=1;
break;
case 'FX':
$extracted['FX'][$line[2]]=1;
break;
case 'COSTUME':
$extracted['skin_colors'][$line[1]]=1;
break;
case 'FXTINT':
$extracted['FXTINT'][$line[0]]=1;
$extracted['FXTINT'][$line[1]]=1;
break;
//case 'Chat':
default:
break;
}
}
}
$keys_to_extract=array('NPC', 'MOV', 'Map', 'FX', 'skin_colors', 'costume_colors', 'FXTINT', 'geometry', 'textures', 'patterns');
foreach ($keys_to_extract as $key)
$extracted[$key]=array_keys ($extracted[$key]);
foreach ($animations as $id => $object)
$animations[$id]=array_keys ($object);
$extracted['MOV_assoc']=$animations;
unset($animations);
$extracted['PARTSNAME']=array_values($extracted['PARTSNAME']);
foreach ($extracted['PARTSNAME'] as $entity=> $base_time)
{
$extracted['PARTSNAME'][$entity]=array_values($extracted['PARTSNAME'][$entity]);
foreach ($base_time as $costume)
{
$indexed_costume=array();
switch (count($costume))
{
//i23 and i24 costumes have 28 slots
case 28:
$indexes=array(
0=> "Lower Body: Bottoms",
1=> "Upper Body: Chest",
2=> "Head: Face",
3=> "Upper Body: Gauntlets/Left Arm",
4=> "Lower Body: Boots",
5=> "Upper Body: Belt",
6=> "Head: Hair",
9=> "Upper Body: Chest Detail",
10=> "Upper Body: Shoulders",
11=> "Back: Backpacks",
12=> "Weapons: Left",
13=> "Head: Face detail 2",
14=> "Upper Body: Arms",
15=> "Back: Capes: Mantle",
16=> "Back: Capes: Brooch",
17=> "Back: Capes: Cape",
18=> "Aura",
19=> "Lower Body: Skirts",
20=> "Upper Body: Jackets: Jacket",
21=> "Upper Body: Jackets: Sleeves",
22=> "Head: Helmet Detail 1 / Ears",
23=> "Head: Helmet Detail 2",
24=> "Weapons: Right",
25=> "",
26=> "",
27=> "Lower Body: Tail"
// Unknown: Two-part pants?
); 
break;
default:
//echo "Costume piece count (".count($costume).") does not match known versions";
//print_r($costume);
//Error("Costume piece count (".count($costume).") does not match known versions");
break;
}
}
}
// print_r($extracted);

}
else
Error("I don't know what you uploaded, but it doesn't look like a demo.");
}
else Error("Failure when reading uploaded file");
}

function Error($msg)
{
echo "Fatal Error: ".$msg;
die;
}
?>

[Codewalker]

Here's the list of costume parts for players by internal name, might help fill in some gaps.

• Pants
• Chest
• Head
• Gloves
• Boots
• Belt
• Hair
• Face
• Eye Detail
• Chest Detail
• Shoulders
• Back
• Right Weapon
• Neck
• Right Upper Arm
• Collar
• Broach
• Cape
• Aura
• Skirt
• Top
• Sleeves
• Cranium
• Jaw
• Left Weapon
• Two Handed Left Weapon
• Two Handed Right Weapon
• Tail

[The Fifth Horseman]

Thank you, that is most helpful.

[The Fifth Horseman]

Harvesting data from demos works fine now. Not sure if the table structure for pattern-slot association makes much sense, but that's something for later.

[Arachnion]

Nice job, Horseman.

 

[The Fifth Horseman]

I went and changed the way MOVs are linked with NPC names - now it will discard underscores and numbers from the end of the NPC name to get a general "NPC class" they are a part of (there's something like 70 generic female NPCs alone) and link the MOV with that.