Author Topic: Sentinel+ Import Tool  (Read 10952 times)

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Sentinel+ Import Tool
« on: May 03, 2019, 02:42:31 PM »
Somebody asked me to make a thing, so I made a thing.

You'll need to run this from the command line, but hopefully if you've managed to set up either the dedicated or the VM image personal server, it shouldn't be too difficult.

Usage

As always back up your XML files before doing anything! This tool does not modify them, but it's strongly recommended to always have a good backup in case of accidental foot-shooting like switching the order of the input and output files.

xml2db [authid] [authname] [bin.pigg] [xmlfile] [txtfile]

authid: The ID number for the game account the character will go on. You can get this from the sql database. I think it's also possible to see it in-game by turning on self info with the GM commands.
authname: The name of the game account - same thing you type in on the login screen.
bin.pigg: Path to the I24 bin.pigg file.
xmlfile: Sentinel+ XML file to be converted.
txtfile: Output file to create.

Example:
xml2db 150 myaccount C:\I24\piggs\bin.pigg CoolGuy.xml CoolGuy.txt

Once you have the output file, you can use mapserver in dbquery mode to import it.

bin\dbquery.exe -putcharacter CoolGuy.txt

Download

https://www.dropbox.com/s/u5411ralpbnnjuu/xml2db.exe?dl=0

Source

https://github.com/cwtitan/xml2db

To get this done quickly I reused my old libcoh library from back in the day. It was an experiment with C++/Boost (never again) and can be a rather large pain to compile from source.

Notes & Caveats

Sentinel+ only exported information that the client had access to, which means a lot of stuff is missing, especially state related to mission and contact completion. This tool attempt to correct some of the bigger issues related to accolade powers and inventory sizes, but it is far from comprehensive and there may be some slight irregularities, especially related to having badges for arcs that the game doesn't think you've completed yet.

All characters processed by this tool are granted the Passport badge.
« Last Edit: May 25, 2019, 03:06:58 AM by Codewalker »

USCI

  • Underling
  • *
  • Posts: 5
Re: Sentinel+ Import Tool
« Reply #1 on: May 04, 2019, 03:51:16 AM »
You, sir, are an Elite Boss!

However....is it possible that you could draw up some instructions for the idiots to follow?  Oh, not for me....asking for a friend.

Thanks!

The Fifth Horseman

  • Elite Boss
  • *****
  • Posts: 961
  • Outside known realities.
Re: Sentinel+ Import Tool
« Reply #2 on: May 04, 2019, 08:10:09 AM »
However....is it possible that you could draw up some instructions for the idiots to follow?
Those are the instructions he already posted. It cannot be simplified any further.
We were heroes. We were villains. At the end of the world we all fought as one. It's what we did that defines us.
The end occurred pretty much as we predicted: all servers redlining until midnight... and then no servers to go around.

Somewhere beyond time and space, if you look hard you might find a flash of silver trailing crimson: a lone lost Spartan on his way home.

Gun-Nut

  • Underling
  • *
  • Posts: 8
    • Ninja's Fate
Re: Sentinel+ Import Tool
« Reply #3 on: May 04, 2019, 08:23:22 AM »
Nice, unfortunately I don't have any dumps, I feel like I would have made them but only in recent years have I started storing more data "in the cloud" and old computers and hard drives are long gone so I doubt I'd find them.

I would be curious if Sentinel+ could be made to work on current "i25" servers / future servers, while we are still in unknown territory on what will happen it would be nice to know I could make a backup of any characters, sure it can be done on the server end but I can't imagine we'll see access to backing up features, though in theory it should just be a matter of allowing "/packageent" to be used on any access level, in other words 0, no clue if this takes a hit on the database or if your game instance is already holding all your data once you login.
aka DarkSynopsis

Tivomaniac

  • Minion
  • **
  • Posts: 35
Re: Sentinel+ Import Tool
« Reply #4 on: May 04, 2019, 02:49:54 PM »
It would be great if they could tie this into icon in game. But you’d need a database of imported checksums to keep people from importing and then taking influence and the deleting and reloading it over and over. It would need to be one import per character.
--
You may know me as @PricklyDarkness from Triumph

USCI

  • Underling
  • *
  • Posts: 5
Re: Sentinel+ Import Tool
« Reply #5 on: May 05, 2019, 06:44:50 PM »
Those are the instructions he already posted. It cannot be simplified any further.

This is where I get lost...

Quote
authid: The ID number for the game account the character will go on. You can get this from the sql database. I think it's also possible to see it in-game by turning on self info with the GM commands.

I don't know how to access the sql database nor do I know any of the GM commands to find the authid.

Canine

  • Lieutenant
  • ***
  • Posts: 78
Re: Sentinel+ Import Tool
« Reply #6 on: May 05, 2019, 08:10:49 PM »
This is where I get lost...

I don't know how to access the sql database nor do I know any of the GM commands to find the authid.

Unless you are running your own server, you won't have access to the commands or the databases necessary.

You run the commands on the server, not in the client.

At least that's how my non-specialist brain is deciphering the instructions.

JordanYen

  • Minion
  • **
  • Posts: 33
Re: Sentinel+ Import Tool
« Reply #7 on: May 06, 2019, 12:41:05 AM »
Somebody asked me to make a thing, so I made a thing.
Example:
xml2db 150 myaccount C:\I24\piggs\bin.pigg CoolGuy.xml CoolGuy.txt



Dies for me at LoadingBodyParts.bin. Well, actually it just gets stuck. Doesn't complete :P
Love the concept, but not working at the moment for me :(
« Last Edit: May 06, 2019, 12:54:14 AM by Jordan Yen »

CraZboy

  • Minion
  • **
  • Posts: 42
Re: Sentinel+ Import Tool
« Reply #8 on: May 06, 2019, 10:50:03 AM »
For me, starts up, right after "Indexing powers.bin...", it gets Error: std::bad_alloc
Any thoughts?

CraZboy

  • Minion
  • **
  • Posts: 42
Re: Sentinel+ Import Tool
« Reply #9 on: May 08, 2019, 12:02:11 AM »
Still getting
Error: std::bad_alloc
anyone have any luck with this?
Please let me know.

Felderburg

  • Ask me how I got this title!
  • Elite Boss
  • *****
  • Posts: 1,600
  • Personal text? What's that?
Re: Sentinel+ Import Tool
« Reply #10 on: May 08, 2019, 04:22:23 PM »
This is where I get lost...

I don't know how to access the sql database nor do I know any of the GM commands to find the authid.

As Canine mentioned, this is for people running a server. However, you can use the Senticon converter to extract costumes: https://cit.cohtitan.com/senticon/index.php (Although I would wait until the browser-based nicer version is out; you can see posts about its progress here: https://www.cohtitan.com/forum/index.php?topic=11121.0).

It would be great if they could tie this into icon in game. But you’d need a database of imported checksums to keep people from importing and then taking influence and the deleting and reloading it over and over. It would need to be one import per character.

Ideally, a Sentinel+ uploader will be contained with any "private server package" so that people running their own CoH servers can decide on their own whetehr their server will or will not allow Sentinel+ uploads.
I used CIT before they even joined the Titan network! But then I left for a long ol' time, and came back. Now I edit the wiki.

I'm working on sorting the Lore AMAs so that questions are easily found and linked: http://paragonwiki.com/wiki/Lore_AMA/Sorted Tell me what you think!

Pinnacle: The only server that faceplants before a fight! Member of the Pinnacle RP Congress (People's Elf of the CCCP); formerly @The Holy Flame

MREBoy

  • Underling
  • *
  • Posts: 5
  • EMPATHY 4 LYFE !
Re: Sentinel+ Import Tool
« Reply #11 on: May 09, 2019, 04:36:38 AM »
I asked a server operator about implementing this and one item they brought up was people altering an input XML file for personal gain (ie putting a few extra 0s in the Inf line). So what I am asking is are the exported XMLs protected from this ? Is the VKEY line at the very end a file hash or checksum ?


therain93

  • Elite Boss
  • *****
  • Posts: 570
Re: Sentinel+ Import Tool
« Reply #12 on: May 09, 2019, 01:13:00 PM »
I asked a server operator about implementing this and one item they brought up was people altering an input XML file for personal gain (ie putting a few extra 0s in the Inf line). So what I am asking is are the exported XMLs protected from this ? Is the VKEY line at the very end a file hash or checksum ?


Yes.  See post 19 here:
https://www.cohtitan.com/forum/index.php?topic=4890.msg44659#msg44659
@Texarkana - March 5, 2004 - December 1, 2012 -- Imageshack |-| Youtube
---------------------------------------------------------------------------------------

You don't know what it's like.... |-| Book One. Chapter one...

tatmia

  • Underling
  • *
  • Posts: 3
Re: Sentinel+ Import Tool
« Reply #13 on: May 15, 2019, 12:12:53 PM »
This is where I get lost...

I don't know how to access the sql database nor do I know any of the GM commands to find the authid.

I’m in the same boat.   Not certain if my post was deleted because I shared a screenshot of my personal VM or if I messed up the post  - but I just don’t know where to find it on the SQL database.   

You can find a list of GM commands on the ouro wiki, but the self does not give an authid.   

While I prefer playing with people, I saved 75 XMLs and would love the opportunity to occasionally take some of my favorite characters through a few missions again.   

Sinistar

  • Elite Boss
  • *****
  • Posts: 1,045
Re: Sentinel+ Import Tool
« Reply #14 on: May 16, 2019, 10:17:00 PM »
Somebody asked me to make a thing, so I made a thing.

You'll need to run this from the command line, but hopefully if you've managed to set up either the dedicated or the VM image personal server, it shouldn't be too difficult.

Usage

As always back up your XML files before doing anything! This tool does not modify them, but it's strongly recommended to always have a good backup in case of accidental foot-shooting like switching the order of the input and output files.

xml2db [authid] [authname] [bin.pigg] [xmlfile] [txtfile]

authid: The ID number for the game account the character will go on. You can get this from the sql database. I think it's also possible to see it in-game by turning on self info with the GM commands.
authname: The name of the game account - same thing you type in on the login screen.
bin.pigg: Path to the I24 bin.pigg file.
xmlfile: Sentinel+ XML file to be converted.
txtfile: Output file to create.

Example:
xml2db 150 myaccount C:\I24\piggs\bin.pigg CoolGuy.xml CoolGuy.txt

Once you have the output file, you can use mapserver in dbquery mode to import it.

bin\mapserver.exe -putcharacter CoolGuy.txt

Download

https://www.dropbox.com/s/u5411ralpbnnjuu/xml2db.exe?dl=0

Source

https://github.com/cwtitan/xml2db

To get this done quickly I reused my old libcoh library from back in the day. It was an experiment with C++/Boost (never again) and can be a rather large pain to compile from source.

Notes & Caveats

Sentinel+ only exported information that the client had access to, which means a lot of stuff is missing, especially state related to mission and contact completion. This tool attempt to correct some of the bigger issues related to accolade powers and inventory sizes, but it is far from comprehensive and there may be some slight irregularities, especially related to having badges for arcs that the game doesn't think you've completed yet.

All characters processed by this tool are granted the Passport badge.

ah Codewalker, I cannot begin to thank you enough for this, however I do seem to be at a mild impasse and I hope or someone in the know can assist.

I seem able to make the output file but how does one run mapserver in dbquery mode for the import?

I'm probably missing something pretty basic and silly but it is late for me, plus medication.....oy. 

Anyway, help would be appreciated.  Thank you.
In fearful COH-less days
In Raging COH-less nights
With Strong Hearts Full, we shall UNITE!
When all seems lost in the effort to bring CoH back to life,
Look to Cyberspace, where HOPE burns bright!

Leandro

  • Elite Boss
  • *****
  • Posts: 310
Re: Sentinel+ Import Tool
« Reply #15 on: May 17, 2019, 04:01:13 PM »
Open a command line in your server root folder (where the bin and data folders are) and, with your exported .txt on the same folder,  run bin\dbquery.exe -putcharacter yourfile.txt

If you have multiple to import, use for %a in (*.txt) do bin\dbquery.exe -putcharacter %a

Sinistar

  • Elite Boss
  • *****
  • Posts: 1,045
Re: Sentinel+ Import Tool
« Reply #16 on: May 19, 2019, 11:19:28 PM »
Open a command line in your server root folder (where the bin and data folders are) and, with your exported .txt on the same folder,  run bin\dbquery.exe -putcharacter yourfile.txt

If you have multiple to import, use for %a in (*.txt) do bin\dbquery.exe -putcharacter %a

Still having trouble
In fearful COH-less days
In Raging COH-less nights
With Strong Hearts Full, we shall UNITE!
When all seems lost in the effort to bring CoH back to life,
Look to Cyberspace, where HOPE burns bright!

CraZboy

  • Minion
  • **
  • Posts: 42
Re: Sentinel+ Import Tool
« Reply #17 on: May 20, 2019, 04:36:40 AM »
Same here ..
I follow everything to the letter ..
When I enter in command line:  bin\dbquery.exe -putcharacter yourfile.txt
I get a failed to connect to DbServer
I run it when server is up ..
I get a file not found ..
Am I missing something?

The Fifth Horseman

  • Elite Boss
  • *****
  • Posts: 961
  • Outside known realities.
Re: Sentinel+ Import Tool
« Reply #18 on: May 21, 2019, 04:33:43 PM »
Same here ..
I follow everything to the letter ..
When I enter in command line:  bin\dbquery.exe -putcharacter yourfile.txt
I get a failed to connect to DbServer
I run it when server is up ..
I get a file not found ..
Am I missing something?
1. You are supposed to run it when the server is up.
2. Is the file you're trying to load is actually named yourfile.txt, or did you follow the example without considering that part ?
We were heroes. We were villains. At the end of the world we all fought as one. It's what we did that defines us.
The end occurred pretty much as we predicted: all servers redlining until midnight... and then no servers to go around.

Somewhere beyond time and space, if you look hard you might find a flash of silver trailing crimson: a lone lost Spartan on his way home.

CraZboy

  • Minion
  • **
  • Posts: 42
Re: Sentinel+ Import Tool
« Reply #19 on: May 21, 2019, 08:47:44 PM »
Hi,
No, the file I am trying to load is mychractersname.txt .. so many of them ..

mychractersname.txt = any of my character's individual  file names ..