Author Topic: Testing the Waters/Interest  (Read 21635 times)

ukaserex

  • Elite Boss
  • *****
  • Posts: 500
Re: Testing the Waters/Interest
« Reply #20 on: June 26, 2015, 11:57:02 AM »
I think, for the first time, I'm looking forward to waiting how long it will take for the little blue bar to fill as the game loads. A very well timed hold over while the games in development begin to make some good progress towards the end goal.

Thank you Codewalker for paying attention during Calculus. (I'm assuming you've got some CS or Math background and didn't teach yourself to code by getting a book called "Coding for people who can't get past Calculus.)
Those who have no idea what they are doing genuinely have no idea that they don't know what they're doing. - John Cleese

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Testing the Waters/Interest
« Reply #21 on: June 26, 2015, 05:56:38 PM »
I did teach myself programming, starting with a pair of 8086/80386 assembly language reference books (one of which I even got out of the closet, dusted off, and used during Icon development). I find it immensely helpful when writing in higher level languages to have a good feel for what the compiler is actually generating behind the scenes, and which language features map to actual hardware features versus which ones are purely synthetic constructs.

I took Calculus later and mostly paid attention. It's fairly rare that I use Calculus in the course of developing something, though. A couple instances come to mind of calculating the total price for buying a virtual resource in bulk whose individual vendor price is variable based on the supply available. Basic area-under-a-curve problem. That's perhaps 2 or 3 times in the last 5 years.

Now, Trig on the other hand, I use that a lot. Linear algebra also factors heavily into things like engine work. I'd say that Calculus might be useful for physics engines, but all of the interesting formulas have already been derived/integrated for you and it's easy to look them up, so most people don't even know calculus is involved.

Victoria Victrix

  • Team Wildcard
  • Elite Boss
  • *****
  • Posts: 1,886
  • If you don't try, you have failed.
    • Mercedes Lackey
Re: Testing the Waters/Interest
« Reply #22 on: June 27, 2015, 03:54:50 AM »
I did teach myself programming, starting with a pair of 8086/80386 assembly language reference books (one of which I even got out of the closet, dusted off, and used during Icon development).

I wish I had known that Assembly was useful in this!  I'd have dusted off my machine code/Assembly expertise and tried to lend a hand. (Coded Assembly Language from 1979 to 1992 when I quit American Airlines).
I will go down with this ship.  I won't put my hands up in surrender.  There will be no white flag above my door.  I'm in love, and always will be.  Dido

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Testing the Waters/Interest
« Reply #23 on: June 27, 2015, 04:16:21 AM »
I wish I had known that Assembly was useful in this!  I'd have dusted off my machine code/Assembly expertise and tried to lend a hand. (Coded Assembly Language from 1979 to 1992 when I quit American Airlines).

It was in Icon, though you're right that Icon was closer to machine code since it was hand assembled and then put into an array to be pushed into client memory, albeit with some semi-automatic relocations.

It wasn't so much in Paragon Chat, as that would have been far too massive to write in assembly directly.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Testing the Waters/Interest
« Reply #24 on: June 27, 2015, 07:19:59 AM »
I wish I had known that Assembly was useful in this!  I'd have dusted off my machine code/Assembly expertise and tried to lend a hand. (Coded Assembly Language from 1979 to 1992 when I quit American Airlines).

Technically, assembly is always useful when it comes to hacking programs you don't have the source code for, since you're often required to read or modify individual binary instructions, and the best you're going to get is disassembly (reverse source decompilers are notoriously awful).  The catch is (as I'm sure you're aware) that assembly isn't so much a language as it is a representation of CPU opcodes, and depending on what target cityofheroes.exe was compiled for, it would be more important to understand the CPU architecture and how compilers generally create compiled code than assembly syntax itself.

I'm guessing Codewalker used a debug tracer and memory/stack inspector, and whipped out his Art of Assembly and MASM manuals for reference, and set about cataloging which memory locations corresponded to which parts of the game client, then found some code space to inject his own byte by byte instructions to alter the client's behavior to make Icon.  But I suspect there wasn't actually a lot of actual programming in assembly.

Victoria Victrix

  • Team Wildcard
  • Elite Boss
  • *****
  • Posts: 1,886
  • If you don't try, you have failed.
    • Mercedes Lackey
Re: Testing the Waters/Interest
« Reply #25 on: June 27, 2015, 09:54:13 AM »
Technically, assembly is always useful when it comes to hacking programs you don't have the source code for, since you're often required to read or modify individual binary instructions, and the best you're going to get is disassembly (reverse source decompilers are notoriously awful).  The catch is (as I'm sure you're aware) that assembly isn't so much a language as it is a representation of CPU opcodes, and depending on what target cityofheroes.exe was compiled for, it would be more important to understand the CPU architecture and how compilers generally create compiled code than assembly syntax itself.

I'm guessing Codewalker used a debug tracer and memory/stack inspector, and whipped out his Art of Assembly and MASM manuals for reference, and set about cataloging which memory locations corresponded to which parts of the game client, then found some code space to inject his own byte by byte instructions to alter the client's behavior to make Icon.  But I suspect there wasn't actually a lot of actual programming in assembly.

Well now I don't feel so bad.

Someday I will have you and CW together in a bar and explain how airline reservation programming works. 
I will go down with this ship.  I won't put my hands up in surrender.  There will be no white flag above my door.  I'm in love, and always will be.  Dido

FloatingFatMan

  • An Offal
  • Elite Boss
  • *****
  • Posts: 1,178
  • Kheldian's Forever!
Re: Testing the Waters/Interest
« Reply #26 on: June 27, 2015, 10:59:34 AM »
Well now I don't feel so bad.

Someday I will have you and CW together in a bar and explain how airline reservation programming works.

NO one understands those things!

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Testing the Waters/Interest
« Reply #27 on: June 27, 2015, 05:22:32 PM »
then found some code space to inject his own byte by byte instructions to alter the client's behavior to make Icon.  But I suspect there wasn't actually a lot of actual programming in assembly.

The very first version did that. It was a tight squeeze and kind of annoying as I had to find dead code to overwrite.

The second version remotely allocated extra memory in the process (VirtualAllocEx) and copied as much code and data as it wanted into those segments. It then inserted jumps in a few key places to hook the existing code to branch to the Icon functions. There were still a number of modifications to bypass certain checks, but all of the things added by Icon like NPC spawning and extra commands were located elsewhere, with much more room to grow.

So in that version and onward there was quite a bit of assembly involved, though mostly I just used the debugger's built-in assembler to do it and copied the resulting bytecode, replacing absolute addresses with special markers so that Icon's loader can do its relocation depending on the client version, as well as fix up calls that use relative addresses.

If you look at the Icon repo, the patching done by patch.c is quite small compared to the amount of new assembled code added by code.c

I'm guessing Codewalker used a debug tracer and memory/stack inspector,

OllyDbg has been very helpful in the reverse engineering efforts to date. It has a clean UI, good code analysis features*, support for custom annotations in the disassembly that are saved, struct display for common Win32 APIs, and most critically a persistent database that remembers the symbolic names you've given to specific addresses and shows them in the disassembly in place of that address.

* automatic loop/jump detection with arrows in the disassembly to show the program flow, automatic local variable detection, switch() recognition, function call recognition that shows the parameters pushed to the stack, and support for several other compiler constructs.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Testing the Waters/Interest
« Reply #28 on: June 27, 2015, 07:40:48 PM »
The very first version did that. It was a tight squeeze and kind of annoying as I had to find dead code to overwrite.

The second version remotely allocated extra memory in the process (VirtualAllocEx) and copied as much code and data as it wanted into those segments. It then inserted jumps in a few key places to hook the existing code to branch to the Icon functions. There were still a number of modifications to bypass certain checks, but all of the things added by Icon like NPC spawning and extra commands were located elsewhere, with much more room to grow.

I wonder if ASLR is going to eventually make this a lot more difficult.  I suspect they either switched it off or their compilation target did it automatically in CoH, but now that XP has gone the way of the dodo unless you happen to be in the US Navy, I wonder if game devs are going to start using it more to prevent memory mods.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Testing the Waters/Interest
« Reply #29 on: June 27, 2015, 07:55:14 PM »
Well now I don't feel so bad.

Someday I will have you and CW together in a bar and explain how airline reservation programming works.

I'm not sure it always works.  In principle, airline reservation systems are just big databases.  But in practice, their highly specialized requirements tend to make them extremely complex customized beasts, and in the age of integrated systems airline reservation systems don't just make flight reservations and seat assignments, they also perform fare analysis, advertise information on federated systems, try to figure out how to squeeze Joe out of a couple more bucks for that flight to Wichita to visit grandma.  Sometimes I think it would be better if we booked flights by logging into World of Warcraft and just camped the spawn point for our seat assignment.

American Airlines, your old stomping grounds, I believe worked with IBM to make the first digital reservation system, the predecessor to the current SABRE system sometimes in the 1960s.  I actually thought if you were writing assembly for AA in the 1970s, you might have touched this system at some point.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Testing the Waters/Interest
« Reply #30 on: June 27, 2015, 09:38:36 PM »
I wonder if ASLR is going to eventually make this a lot more difficult.  I suspect they either switched it off or their compilation target did it automatically in CoH, but now that XP has gone the way of the dodo unless you happen to be in the US Navy, I wonder if game devs are going to start using it more to prevent memory mods.

While ASLR would make simple things like Cheat Engine presets not work, it probably wouldn't affect Icon much. ASLR operates on a per-module or per-segment basis. Icon already reads the PE header to map out the segments and get the base address of each. That's combined with the relative addresses that it's looking for, then written into Icon's program code in the appropriate places before it's inserted into the cityofheroes process.

If the layout within the code segment was somehow randomized, I have another technique that I've used before that would probably do the trick. It involves scanning the read-only data section (which contains the string tables), getting the addresses of a few known strings, then scanning the code for instructions that reference those known strings and examining the surrounding area to find the targeted procedure.

I used that method on my OpenGL launcher/patcher, which hot-patched the client to fix several bugs so that it could run under VirtualBox's OpenGL passthrough driver without crashing. It successfully applied the fixes across about 8 issue releases without needing to be altered or even recompiled. I had planned to use the same technique for Icon 2.0 until it became apparent that I was pushing the limits of what I could accomplish using hand-assembled code in a reasonable timeframe and needed to look at a different approach.

So while ASLR would have implications for simple mods that don't have the equivalent of a poor man's runtime linker built in, it's only a minor roadblock and I'm sure toolkits for dealing with it would appear in short order.

Victoria Victrix

  • Team Wildcard
  • Elite Boss
  • *****
  • Posts: 1,886
  • If you don't try, you have failed.
    • Mercedes Lackey
Re: Testing the Waters/Interest
« Reply #31 on: June 28, 2015, 09:10:34 AM »
I'm not sure it always works.  In principle, airline reservation systems are just big databases.  But in practice, their highly specialized requirements tend to make them extremely complex customized beasts, and in the age of integrated systems airline reservation systems don't just make flight reservations and seat assignments, they also perform fare analysis, advertise information on federated systems, try to figure out how to squeeze Joe out of a couple more bucks for that flight to Wichita to visit grandma.  Sometimes I think it would be better if we booked flights by logging into World of Warcraft and just camped the spawn point for our seat assignment.

American Airlines, your old stomping grounds, I believe worked with IBM to make the first digital reservation system, the predecessor to the current SABRE system sometimes in the 1960s.  I actually thought if you were writing assembly for AA in the 1970s, you might have touched this system at some point.

I did touch that system, in the form that Associates Finance used it, the earlier and more primitive version of the one AA was using when I went to work there in 1982.

The baggage/weight and balance system at AA still used it.  Probably still does.

Every airline reservation goes through several hundred thousand passes in the system per minuteas it calculates the lowest possible fares for the itinerary.  The peculiarities of how it does that would probably make your eyes cross.  Basically, it's still (or it was when I quite) maxed-out 1960s code that only works because of a rigid hierarchy of rules the programmers must follow. 

When I quit in 1995, it was still running on IBM 370s chained together.  No, I am not making that up.  There were plans to replace it, but as long as I was there, those plans never came to fruition.  I think they were afraid to blow on that particular house of profitable cards.
I will go down with this ship.  I won't put my hands up in surrender.  There will be no white flag above my door.  I'm in love, and always will be.  Dido

FloatingFatMan

  • An Offal
  • Elite Boss
  • *****
  • Posts: 1,178
  • Kheldian's Forever!
Re: Testing the Waters/Interest
« Reply #32 on: June 28, 2015, 09:18:39 AM »
When I quit in 1995, it was still running on IBM 370s chained together.  No, I am not making that up.  There were plans to replace it, but as long as I was there, those plans never came to fruition.  I think they were afraid to blow on that particular house of profitable cards.

Well, y'know... If it ain't broke, don't fix it!

Styrj

  • Boss
  • ****
  • Posts: 154
Re: Testing the Waters/Interest
« Reply #33 on: June 28, 2015, 10:56:07 AM »
Well, y'know... If it ain't broke, don't fix it!

I second that! :D
If it ain't broke, don't fix it!  Infinity Server...

LaughingAlex

  • Giggling like an
  • Elite Boss
  • *****
  • Posts: 2,019
Re: Testing the Waters/Interest
« Reply #34 on: June 28, 2015, 11:07:06 AM »
Well, y'know... If it ain't broke, don't fix it!

Until someone hacks the planes like that one guy recently did.  Old code technically "breaks" as new technologies and people figure out it's various bugs and exploits, sadly.
Currently; Not doing any streaming, found myself with less time available recently.  Still playing starbound periodically, though I am thinking of trying other games.  Don't tell me to play mmohtg's though please :).  Getting back into participating in VO and the successors again to.

Rotten Luck

  • Elite Boss
  • *****
  • Posts: 766
  • "I'd rather go out a hero than a coward."
Re: Testing the Waters/Interest
« Reply #35 on: June 28, 2015, 12:45:18 PM »
I be game for some Gaming!  I been Rping Free form in the City if Titan's forums.  A PnPRPG in Paragon right up my alleyway. 

I use this dice roller in a pinch.  http://www.wizards.com/dnd/dice/dice.htm

Technically I haven't done any PnP gaming just haven't found any players to sit down at a table.  So I be a Newbie to any set of rules. 

Good game night for me is tuesday.  Just seems that's a day I been getting off the most.
One way or another... Heroes will fly again!

Nyghtshade

  • HERC Advisor
  • Elite Boss
  • *****
  • Posts: 881
Re: Testing the Waters/Interest
« Reply #36 on: June 29, 2015, 03:36:31 AM »
*Waves to Rotten*. 

I too would love me some CoH RP.  Once we're in, feel free to look up Nyghtshade, Celtic Honor, Ice Kat, some of my mains.   :)

Lycantropus

  • Elite Boss
  • *****
  • Posts: 255
Re: Testing the Waters/Interest
« Reply #37 on: June 29, 2015, 08:56:41 PM »
Well there's been talk of all kinds of bots and scripts and wingdings that I have no knowledge about, but look forward to using the heck out of if it makes some form of tabletop play even easier through the Paragon Chat system! :)

Because of all the 'up in the air' about what will be possible for sure, I don't want to commit to how many members or such yet (depends if it shapes up more like a LARP group with player heroes and villains interacting to create their own stories and plots, or a small group of heroes (or villains) where I create scenarios for them to run through, and summon up NPC's for them to 'fight' (and I delete once they're defeated)... Again, not sure what's going to be completely possible in PC versus what we can do in Icon... and of course all the clever folks that are already starting to figure out all kinds of ways to work the system to do even more :)

Better yet, for those interested in something like this, which would you prefer?

Players doing actual heroes and villains creating their own plots (probably villain oriented, making goals for bank robberies and world domination and such that the heroes react to and foil, while dealing with their own lives and stuff- a-la Justice League versus Legion of Doom or X-Men versus Brotherhood of Evil Mutants kind of thing) working more like a 2 team co-op LARP (live action role playing group)

Pros: Can include a LOT more players. (that's a big plus in my book, for so many folks wanting to fight or commit crimes in Paragon again)
Cons: Large combats/events can be slow, particuarly at first while people get used to the rules, especially if arbitration is needed. (though I could get more 'arbiters' to help me with that if needed)

or

Small group of players being heroes (or villains) and I GM up stories for them, more like a tabletop group style but with visuals?
Pros: More concise story and direction, and everyone's their own star of the story.
Cons: VERY limited group size. Everyone would have to agree on a side (hero or villain)

I can't plan around all the details, but I'm pretty sure something like either of the above is possible regardless of any limitations, and can only be made better with any enhancements (pardon the pun) the clever technical folks around here can do.

Again, I'm trying to work availability for a few hours, Sunday or Monday evenings, Central US time.

Of course, anyone else wanting to do actually spearhead anything like this and lead on other nights, shout out here too! I may join your events as well, if I can! :D

Lyc~

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Testing the Waters/Interest
« Reply #38 on: June 29, 2015, 09:28:03 PM »
There won't be any NPC summoning at first. Probably not until we implement the custom environment support, then whoever sets up the private session will be the designed GM and has the ability to spawn NPCs that show up to players who join it.

Lycantropus

  • Elite Boss
  • *****
  • Posts: 255
Re: Testing the Waters/Interest
« Reply #39 on: July 02, 2015, 12:07:49 AM »
Good to know. Thanks! I'll do my best to be patient. :)

I'm sure I'll have more questions, but I'm willing to wait and see what we have to work with first. Thanks so much for putting in all the time and effort for this! I'm sure there's a ton of things everyone wants and expects from this, and may, in time. For now I'm just happy we're getting a chance to share the City again in whatever state it's in. (Rhode Island, I know... ;) )

Just as importantly, thanks to TonyV and the team who've kept all this going, so I'd have a place to lurk and hear about this to begin with!  ;D

Lyc~