Author Topic: Technical side discussion  (Read 164662 times)

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #260 on: August 02, 2015, 01:08:11 AM »
A note about protocol.  I just started using 0.98b which now supports diagnosing invisible characters.  While testing with this, I discovered something that wasn't explicitly specified (as far as I can recall).  If you change Presence in any way, you MUST immediately send a new <u />.  It seems if you change presence Paragon Chat essentially considers you a new entity, and that means it won't know where you are until you send position information.  I'm not sure if this is new to 0.98b, or if this behavior was ill-defined in previous versions of PC but I noticed that my costume-changing code started to glitch out in 0.98a, and 0.98b told me that the problem was no position information.  I changed my code so that whenever a costume change occurred I immediately send pc:u after presence, and the problem went away.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #261 on: August 02, 2015, 01:18:34 AM »
That sounds like a bug. Presence should apply the new costume to the current entity. It doesn't reset the 'has received initial position' flag that was added in 0.98b. The only other thing I can think of is it's failing to find the current entity and sending a new one to the client, but that shouldn't be happening either.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #262 on: August 02, 2015, 01:21:54 AM »
Its also vulnerable to poisoning: if I know your bot does that, I can write a bot that specifically interacts with your bot and then sends it to the moon.

We're already breaking the "never trust the client rule" six ways from Sunday anyway. There's no point in someone maliciously doing that because it doesn't (yet) benefit them in any way.

For the rest of what you're saying, I actually do agree with you but, I have this nagging idea that something about that question "Where do you think I am?" is interesting but I can't quite formulate what it is yet.

Maybe it's just a bad idea. I'm never one for tossing away an idea just because it sounds bad, at least not until the people who know better confirm that it really is something better let drop.


Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #263 on: August 02, 2015, 01:26:54 AM »
That sounds like a bug. Presence should apply the new costume to the current entity. It doesn't reset the 'has received initial position' flag that was added in 0.98b. The only other thing I can think of is it's failing to find the current entity and sending a new one to the client, but that shouldn't be happening either.

If I use my "resize" command on my bot in 0.98b, it disappears.  If I then move the bot any amount in any direction, it reappears.  If I change my code to immediately resend position after sending updated presence, the bot never disappears in the first place.  When I did a /whoall, the bot showed up as not received initial position after sending (updated) presence but before resending position.  Just FYI.


Also, I just did an informal test of altitude by commanding my bot to move exactly 6 units upward.  The bot moves slightly more than its own height upward, which is consistent with the height of the costume and one unit equaling one foot.

I too am getting the same "dance, bot, dance" feeling slickriptide got when he got his bot to obey commands.  Its weirdly empowering.  /t ArcanaBot, /comehere.  /t ArcanaBot, /moveto 942.2,-32,-714.3.  /t ArcanaBot, /experiment fall 300.  /t ArcanaBot, dance like a chicken.

When the robot revolution happens, my head is going to be one of the first ones on a pike.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #264 on: August 02, 2015, 01:56:05 AM »
Quote
When the robot revolution happens, my head is going to be one of the first ones on a pike.

LMAO

Okay, so here's a question for everyone; in fact, it's one that even @Dyne is in a position to weigh in on.

It's this - How much of what we're doing is something that should be available to anyone as a service they can interface with and avoid having to repeat the basic implementation themselves?

Take the sequencer data, for instance.

Maybe, instead of five people handling that data five different ways in five different formats, it would instead be useful to create a repository of sequencer data that can be asked for by XMPP feature discovery and queried with IQ queries in a well defined namespace. Then everyone who wants to use the data is using the same data, and everyone who wants to talk about it is speaking the same language. The service could be provided by a standalone server that registers with a nearby XMPP server to advertise its services, or it could be a plugin to a modular server like Openfire. The advantage to the plugin route being that when someone sets up their own personal server, they just request the plugin to be installed and there's no need for any further management of the service by that person.

I guess that what I'm suggesting is that maybe there are some functions that can be designated as being functions that every bot/paragon-related-server needs to be able to handle, and codify those functions into a form that makes it so that future programs don't have to reinvent their own version of the wheel in order to interface with Paragon Chat.

.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #265 on: August 02, 2015, 04:39:47 AM »
Here's my take on that.

Right now, this is all experimental, but I do intend to document everything I do in a sort of shortest-path to making a bot.  Its a bit fluid because I'm not 100% there yet and because I'm aiming at a moving target with Codewalker obviously improving and enhancing Paragon Chat itself.

Codewalker and his team have a concern and I believe its a valid one: can you make the software in a way that avoids as much legal entanglement as possible.  I'm not operating under the same limitations they are, but I'm also cognizant of the same issues.  I believe the solution is to implement in a similar manner: never distribute data, only code.  For our purposes, that means while I'm happy to include elements of information in the bot like specific MOV names, I'm not prepared to distribute entire sequence databases with the bot itself.  But in the long run, that won't be necessary for two reasons.  One: Paragon Chat's architecture may make that redundant.  Two, if it doesn't, then at some point I'll simply add the ability to read that information within the bot itself.  That way, any bot writer that emulates or just plain rips what my bot does will have access to the same information mine does - provided they also have a copy of the I24 client to run Paragon Chat in the first place.

The irony here is that if you take Codewalker's sequencer data that he posted a while ago and stick it in your bot, no one will care.  However, the more organized you attempt to make the sharing of that data the more you become an attractive target and a failure point of the system.  So if you decide to make a public sequencer or map data internet accessible service that is designed to be used by people trying to replicate the CoH environment, you potentially become a take down target.  And when you're taken down, everyone who is depending on you goes down with you.

Philosophically, I believe (one of) Paragon Chat's development principles is to avoid that situation, and even though nothing I'm doing will itself ever rise to a level that mandates it, I'm tending to follow that principle as well.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #266 on: August 02, 2015, 05:23:51 AM »
How not to measure gravity in City of Heroes:

1.  Command an entity to change its location to a point XX feet above the ground.
2.  Immediately start timer.
3.  Wait for entity to hit the ground.
4.  Stop timer.

Why this doesn't work:

1.  When you command an entity to change its location, that *doesn't* instantly teleport the entity to that location.  Instead, the game performs an interesting "warp" to that position, and more interestingly the game always tries to perform this act in about the same amount of time: about a quarter of a second.  Moreover, there's a visual artifact: you actually *appear* to move to that new location at very high speed.  Because of this, there's a correction factor of about 0.25 seconds before the entity actually starts falling.

2.  If you do this while standing next to the entity, odds are it will actually fall *on* you.  And that means there is a momentary collision which deflects the falling entity in one direction.  At the falling speeds being tested, this has a minimal but measurable effect on the falling trajectory.

I discovered this with my "high precision" method of testing gravity, which is to analyze demorecords of the bot falling which require Paragon Chat (city of heroes, technically) to record the path of the bot with timecodes.  Demorecords are still not perfectly precise and have timing glitches, but those can be averaged out.  Given what I now know about motion** and observer collisions*** I will perform a more detailed analysis probably tomorrow.


** I actually knew about the space-warping thing because I had witnessed effects related to that in the past, but knowing and consciously factoring in are two different things.

*** Its interesting that it is possible to be standing next to an entity and obviously not colliding with them, and yet when they move in a mathematically precise manner straight up they can land almost on top of you.  I'm not sure if that means CoH movement calculations have a lot of round off errors, or if gravity itself isn't 100% perpendicular to the Y-axis on Paragon Earth.  I had a brief thought of making a gravity map of Paragon, but it took only three swift forehead strikes with a hammer to dispel that notion.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #267 on: August 02, 2015, 09:01:23 AM »
Assuming I'm not screwing up the math completely, and assuming the usual convention of 1 world unit = 1 foot, that comes out to around 17.8 m/s^2 when rising and a whopping 26.7 m/s^2 when falling. That doesn't seem right to me as it doesn't seem like it's that much higher than real world gravity, so it's possible I'm missing a conversion factor somewhere.

Analyzing demorecords and dropping entities from heights of 100 feet to 500 feet and only analyzing the segments of the demorecords when the entities are definitively tracking the free-fall portions of their descent, I calculate gravitational acceleration during downward falling as -87.3 f/s^2 or -26.6 m/s^2, plus or minus 0.3 m/s^2 (variations in demorecord timing).  That basically confirms your calculations, and means gravity is about 2.7 times stronger in Paragon City than on Earth (at least when falling).

Also, on the general subject of speed caps, I can also confirm there appears to be no enforced speed cap in the Y direction when falling, at least no cap within the region of my testing.  Maximum downward velocity measured was 286 fps from a drop height of 500 feet.  That is 195 mph. Coincidentally, that's about terminal velocity for a human being falling while in roughly the standing position, and its achievable on Paragon Earth in about three seconds of free fall.  By comparison, on Earth Earth it takes about three seconds to hit the pavement jumping out of a fifth story window, and you hit the ground moving at about 60 miles per hour - fast enough to probably kill you.  In Paragon City, jumping out of a fifth story window causes you to hit the ground in a blink-and-you-miss-it one second, hitting the ground at about 90 miles per hour - fast enough to absolutely kill you if you were a normal Earth person.  In fact, its fortunate Paragon City citizens are practically indestructible because just tripping on the pavement would cause you to hit the ground at speeds of 25 mph or better - significantly higher than the minimum collision speeds necessary to cause life-threatening injuries in auto vs pedestrian accidents (which I'm a lot more familiar with than I was two years ago).

The next interesting question is why gravity is so much stronger in City of Heroes than real world gravity, and why stronger gravity "feels" correct even though its enormously stronger than real world acceleration.  I have suspicions, but there's probably more testing necessary to confirm them.

I'm probably just ducking making space trees.

Dyne

  • Lieutenant
  • ***
  • Posts: 54
  • Gluer of Gears
Re: Technical side discussion
« Reply #268 on: August 02, 2015, 01:36:18 PM »
Okay, so here's a question for everyone; in fact, it's one that even @Dyne is in a position to weigh in on.

It's this - How much of what we're doing is something that should be available to anyone as a service they can interface with and avoid having to repeat the basic implementation themselves?

I'm of two minds.

On the one hand,  I'm a big fan of not reinventing the wheel if you don't need to.

On the other hand I am, in fact, re-implementing a lot of the same stuff (probably in stupid ways) that you and Arcana have already done.  Often even bits for which a working solution was already posted (like loading a costume file).  Mainly that's because I am more interested in writing my own bot than in copy-pasting someone else's (if for no other reason than because I'll understand it better that way, at least in theory.  Which I'll probably need in order to make it do what I want.)** 

Quote
Maybe, instead of five people handling that data five different ways in five different formats, it would instead be useful to create a repository of sequencer data that can be asked for by XMPP feature discovery and queried with IQ queries in a well defined namespace. Then everyone who wants to use the data is using the same data, and everyone who wants to talk about it is speaking the same language. The service could be provided by a standalone server that registers with a nearby XMPP server to advertise its services, or it could be a plugin to a modular server like Openfire. The advantage to the plugin route being that when someone sets up their own personal server, they just request the plugin to be installed and there's no need for any further management of the service by that person.

It's not a bad idea in theory, but I'd tend to agree with what Arcana said earlier.  It might be better to simply distribute the code for loading such data, written as a generic module that others can use like a black box (as long as they are using python, anyway).  They just import it in their own bot or a server plugin or whatever they want.  That way you'd still get most of the benefits but less legal risk.  And, while it may not be a huge issue, that way there's no need for a dozen bots to start eating bandwidth to look up information that they theoretically already have locally.


** Not that it means I won't welcome a guide or look at a solution to each problem before I start, and not that I haven't borrowed from the code posted earlier (I took the basic clevel structure from Arcana's costume loader, for example, but most of the actual actions it takes have been heavily modified because I'm stuffing the data into my own Costume class).

I expect I'll have to do more of this for sleekxmpp, because hoo boy did that library give me a headache when I first started fooling with it (registerHandler vs add_event_handler?  stanza interface vs. stanza plugins, and stanza plugins vs. XEP-plugins? etc.)  I did get to the point of successfully logging into my Openfire server (once I turned openfire's SSL off, because getting self-signed certificates working are SO not what I'm interested in right now), but it's not a particularly impressive feat considering that most of that was pretty much in the Echobot demo.

As seems to be par for the course, I more-or-less understand what needs to be done, but understanding how to get the library to do it is a different question.
« Last Edit: August 02, 2015, 05:26:25 PM by Dyne »
-= Virtue Server - 2004-06-13 to 2012-11-30 =-
hortis publicis gemini in aeternum

Usurper Dyne, Still Heart, River Elemental, Saul Invictus, Grim Grinner

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #269 on: August 02, 2015, 04:31:42 PM »
My bot can walk around Atlas Plaza when told explicitly to "Walk this way. No, *this* way. Walk east at velocity 0.07. Turn right. No, *YOUR* right. Switch velocity vector, you're walking sideways!"

Yay me.

All of which just means that if it was following a script that it would be walking around like a champ. If it waved hello when it walked by you, it would be just like a real CoH street NPC.

So, there's a scripting interpreter that needs to be written.

There's also the thing I really want the bot to do, that's turning out to be a bitch to figure out a "right" way to do it - following a player. Plus, if I want to continue to do most of my coding at the ErrBot level rather than at the SleekXMPP level, then I'm going to have to bite the bullet and make a ParagonChat backend to ErrBot that is basically the existing XMPP backend modified to also recognizes all of the Paragon Chat data stanzas and pass them up to ErrBot along with the standard presence and message data structures.

Basically, a whole bunch of bot functionality tends to boil down to the bot having to know who is in the zone and how far away they are. Mostly, that's so that the bot can determine the answer to "Which players are within 20 feet of me?" or "How far away am I from player X and in what direction?" Even cheating on the "follow player X" bit by just replaying the target's movement stream at a bit of an offset, position-wise, means having to track that player's <U> footprints, which means, once again, making sure that the bot has access to those footprints in the first place.

So, basically, once again diving into a whole mound of foundational stuff in order to make one seemingly simple, but vital, effect possible.

I'm getting a better appreciation for what real game designers have to deal with when building these sorts of systems.


Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #270 on: August 02, 2015, 07:22:36 PM »
There's also the thing I really want the bot to do, that's turning out to be a bitch to figure out a "right" way to do it - following a player.

Last week I talked about refactoring my code to change it from a remote controlled drone using keyboard to a more goal-executing bot.  That was specifically to implement /followme.  Hypothetically speaking, you can rough-cut commands like "/faceme" and "/comehere" and "/emote XXX" by sending instantaneous commands: change orientation to this and then send it, for example.  But /followme cannot work that way for obvious reasons.  So that meant I could no longer have keyboard control and command execution coexisting as it did before.  So I tossed that code and rewrote it to execute goals.

Now, keyboard control only sets movement flags.  A separate goal-executing function checks those flags and decides what to do with them.  In a sense, the goal exec only replicates what the original keyboard-brain-surgery function did: edit the bots positional/velocity/orientation values and send them on the wire.  But it now *also* checks a goal queue.  If there is at least one goal on the queue, it executes the first one.  Also, whenever a goal needs to "move" the bot, instead of using brain-surgery to edit the velocity of the bot, it actually sets the same movement flags as keyboard control does.  In a sense, the bot presses its own keys.  Damn I'm clever (or would be, if that isn't basically how a lot of bot software does the same thing).

When I send "ArcanaBot: /faceme" to the bot, the message handler adds a goal to the goal queue: ['faceme','position'] where the first item in the goal entry is the command and the second is its parameters, in this case the location of the entity that sent the command.  When the goal exec sees that, it figures out which direction to turn, sets 'CLOCKWISE' or 'COUNTERCLOCKWISE' and lets the normal movement routines do their thing (or rather, it mostly does that because I haven't really thought through the best way to "coast" to a final orientation quite yet, so there's a hacky thing where the rotational code will "force" a final orientation at the end of a turn).

The queue lets me do something I couldn't do before quite as easily: I can take actions to satisfy a goal that don't actually complete the goal.  Instead, I can do something, check to see if the goal is complete, and if not I don't delete the goal.  On the next clock tick the bot will check the queue again, see the goal is still there, do what its code tells it to do, check to see if done, and if not leave it on the queue.  Rinse, repeat.

Now, the ['followme','entity'] goal can work like this: on each tick of the clock, if you are within five feet of 'entity' stop moving.  If you are not, accelerate towards entity and face entity.  Continue doing this indefinitely.  A different command, ['unfollowme','entity'] deletes that goal from the queue.

Its not quite working yet because I got distracted by gravity, and also because I want to add some sophistication to the actual follow tracking algorithm.  For example, if the target entity is within about 45 degrees of current facing angle, just swing towards it.  But if its, say, suddenly behind the bot, I want the bot to do a more visually appealing stop, turn around, and start moving again in the opposite direction.

I also have "/orbitme radius", "/mimicme", and "/knockback" in the works.

All of this is prelude to the holy grail of this project: "/fightme"


Quote
Basically, a whole bunch of bot functionality tends to boil down to the bot having to know who is in the zone and how far away they are.

My presence and <u /> handling code keeps a pcRoster.  Whenever I get a presence stanza, I check to see if I already have them.  If I don't, I add them to the pcRoster dictionary.  Whenever I get a <u /> I add those fields to the appropriate pcRoster entry if it exists.  That allows me to do this:

Code: [Select]
    def add_command(self,cmdstr,sender):
        for rosterEntry in self.pcRoster:
            if self.pcRoster[rosterEntry]['pc_jid'] == sender:
                rosterID = rosterEntry
                print "Debug: sender " + str(sender) + " converted to rosterID " + str(rosterID)
        cmd = cmdstr.split()[0]

        if cmd == 'faceme':
            position = self.pcRoster[rosterID]['position']
            self.cmdQueue.append(['faceme',position])
 

Yeah, there's no error checking.  If I zone the bot in and execute that command too quickly, it gets there before the roster entry is created and it bombs.  That's my failsafe for when the robot revolution happens.  But whenever someone sends my bot a command, I can look them up in the roster and figure out things like where they are.  That allows me to easily execute commands like "/faceme" which get converted to the command ['faceme',position] where position is the p= entry from the last <u /> I got from that entity.  Technically, I should apply (velocity) prediction to that, but I'm not there yet.  Mostly because I stupidly forgot to timecode my roster entries.

Also, why didn't I just key my roster entries to jid?  That's complicated.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #271 on: August 02, 2015, 08:53:14 PM »
Its not quite working yet because I got distracted by gravity, and also because I want to add some sophistication to the actual follow tracking algorithm.  For example, if the target entity is within about 45 degrees of current facing angle, just swing towards it.  But if its, say, suddenly behind the bot, I want the bot to do a more visually appealing stop, turn around, and start moving again in the opposite direction.

I think this one is sufficiently interesting to bot writers it was worth amplifying.  Here's a video of my current following algorithm:

https://www.youtube.com/watch?v=VFxbyLM4OTA

Notice it actually works not too bad at first.  I was smart enough to create a following distance: once the bot reaches this distance it stops trying to get any closer.  That prevents bots from following you right up your nose.  And it turns and moves forward.  You can also notice what I'm not doing yet: without motion prediction the bot is actually lagging behind and following where the character *was* rather than where it is.  Which can seem oddly almost more sophisticated of a follow.

Then note what I do at about 40 seconds in: I start running in circles, and I specifically get to a very specific distance and orientation from the bot.  Its tricky to do, but when I do it something remarkable, or at least remarkably weird happens.  The bot begins to orbit my character (remember my previous post when I said I was working on an "/orbitme" command?  Well, I didn't just decide to do that for no reason).  Basically, because the follow me algorithm just keeps trying to point towards the character and accelerate forward in that direction, there exists a magic distance from the target where turn + accelerate = orbit, very much analogously to how actual objects orbit gravitational bodies.  In this case, my following distance is 5 feet, and my algorithms intrinsic orbiting distance is about 10 feet, so it can just about work.

There's a couple of things you need to do to prevent this from happening.  First, you need to make sure your bot isn't a rocketship: because I do not implement friction, my movement commands are accelerating the bot in a certain direction while it keeps all of its current velocity.  Discounting the y-axis, in effect I'm ice skating on the XZ plane.  Second, you need to make sure that when your direction of acceleration diverges from your actual velocity vector by a high enough angle, you stop the bot and have it make a turn, rather than try to accelerate at an obtuse angle (which real people can't do when not sitting on curling stone).

And then, of course, you need to figure out how to reproduce the effect within those parameters so you can actually do it on command, when you actually want to.

Also, notice that as long as you are on relatively level ground, map geometry is not a huge problem.  Because the bot is "following"  the character, it is always trying to move towards the character's ground level, which itself honors the ground surface.  So long as your bot doesn't try to get too involved with sudden changes in altitude and interacts with players that are themselves standing on the ground, ground collision is not a mandatory requirement to work reasonably well.

I'm still going to do it, because I ultimately will need it.  But for other bot writers, consider the fact that this one can, in many (but not all) circumstances, be ignored and still mostly work.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #272 on: August 02, 2015, 09:33:41 PM »
One more word to aspiring bot writers.  It may seem like all of this is daunting, and its not exactly easy, but its worth noting that in my experience to date, development goes in cycles where first you are climbing what seems to be a very steep and unrewarding learning and coding curve, and then there's a payoff in being suddenly able to do a lot of things with the results of that curve.  Followed by another curve.

First, you have to learn your tools.  The language you're using, the frameworks you're using, enough to be able to write any meaningful code at all.  Initially, it will seem like its difficult just to "Hello World" yourself out of a box.  But once you get into the flow of things, your production rate will start to rise substantially.

The second hurdle to overcome is to understand and implement enough of XMPP to be able to connect to servers and send and receive messages.  Until you get there, its difficult to get any feedback as to whether your code is actually doing anything or not.  But once you can connect and send and receive messages, your ability to process those messages and do things will speed up dramatically.  If you want to make a conversational bot, or a bot that interacts with the world but doesn't itself need to appear in that world, you can do a lot just from here.  For example, the hypothetical "ski slope bot" that was discussed in another thread could have been written entirely from this point.

The third hurdle is visibility.  It takes a lot of things to work simultaneously to make a bot visible to other players, and any one of them failing will make you invisible.  Until you can see your bot, its difficult to really know if you can make it do anything at all beyond the chatting and other non-visible tasks discussed above.  But once you make your bot visible, all sorts of things open up.  You can get hacky things working like making the bot move to a spot, spin around, and emote without too much difficulty.  Smooth locomotion - i.e. running, jumping, flying, walking - is trickier, but once your bot is visible, its a lot easier to experiment with this.

The fourth hurdle is to realize that your bot needs a way to execute a sequence of commands.  Depending on how you've written your bot and what language and tools you're using, this could be easier or harder, but you will probably at some point need to put in at least some work here.  And its entirely possible that the way you wrote your bot up to this point makes this difficult, because you didn't take this fully into account.  My strongest recommendation here is, like you should have been doing up to now, keep every version and iteration of your bot.  When you start refactoring code, make sure you never, ever, EVER delete any previous version, particularly versions that worked even if they didn't do exactly what you wanted.  Nothing is more disheartening than "breaking" a working bot, and discovering you forgot how it used to work in the first place.

Also, there's two kinds of "sequences" you need to consider, and you need to consider if you're going to address them in the same way or different ways.  First, there are animation sequences.  When animating a visible bot, animations need to flow in certain orders.  Some Paragon Chat will do for you automatically, some not.  You need to learn how to do that, and how to write your bot so it can understand what state it is in, and what it should do next when it comes to animations.  Separately, you will probably one day want to give your bot a script or sequence of commands, and each command may or may not require many different actions by your bot to perform them.  Don't be afraid to stop and think about this problem carefully, taking a break from coding the bot if you need to, until you have a solution you like.  This is probably a critical part of your bot development because it will influence how you interact with the bot from now on.  My recommendation is to try to disconnect what the bot literally does from moment to moment and the higher level tasks it has to perform, if for no other reason than it becomes easier to try out ideas, and if they don't work to discard them and try something else without having to make major changes to the bot.

Above all, remember that if you break up the task into manageable components, each one with a payoff, you can better manage your own expectations and keep development moving without getting bogged down in trying to do too much at once.  If you keep trying to swing for the fences, you'll never get on base.

Here's a partial list of my intermediate bot goals as I was developing the bot (much of which has been documented in this thread):

- Log into XMPP correctly, see my chat in pidgin.
- Send and receive chat on the Atlas Park broadcast channel between bot and a player
- Receive command by tell, interpret as a query to City of Data website, send response to sender
- Connect to meta channel, see other player data
- Become visible using costume of logged in player, doesn't require costume inquiry code
- Add keyboard controls so can move bot, turn bot.
- Implement walk, run, fly animations in conjunction with keyboard controls
- Implement costume code, allow bot to appear as any costume
- Implement grow, shrink with costume code
- Implement basic in-game command processor to send commands to bot, implement "comehere" and "faceme"
- Refactor controls, add goal queues and basic goal processing
- Implement basic "followme"

Small steps, each one with an achievable and demonstrable goal.  And remember that from the very beginning, the goal was to make a shadow boxing combat bot.  These little goals are all achievable and fun in and of themselves, but they are also heading in that ultimate direction.

At the moment my bot is 990 lines of very organic (i.e. messy, ugly, ill-documented, poorly error checked, highly unoptimized) python code with an additional 320 lines of costume stanza library code.  Technically speaking, I started actually writing code about June 30th, which means at this moment in time I've been working on this, not every single day, for about 63 days.  I've been averaging 21 lines of code a day.  In other words, on an average day, I sit at my computer, think about the bot, write this:

Code: [Select]
        elif self.cmdQueue[0][0] == 'comehere':

            distance = 5.0 # how close to get
           
            x1 = self.position[0]
            y1 = self.position[1]
            z1 = self.position[2]
            x2 = float(self.cmdQueue[0][1].split()[0])
            y2 = float(self.cmdQueue[0][1].split()[1])
            z2 = float(self.cmdQueue[0][1].split()[2])

            xx = x2-x1
            yy = y2-y1
            zz = z2-z1

            k2 = math.sqrt(xx*xx + yy*yy + zz*zz)

            xk = xx/k2 * distance
            yk = yy/k2 * distance
            zk = zz/k2 * distance

            self.position[0] = x2 - xk
            self.position[1] = y2 - yk
            self.position[2] = z2 - zk

            print "Debug: popping comehere, moving to " + str(self.position)
            self.send_pcu()

            del self.cmdQueue[0]

and then I go out and see Ant-Man.  I'm not saying everyone should be able to code, or everyone should want to do this, but if you can code, and you want to do this, I'm here to tell you its doable, and doable without a herculean level of effort (although everyone has different skill levels with code, I'm talking relatively speaking).

Go.  CODE.  MAKE BOTS.

Dyne

  • Lieutenant
  • ***
  • Posts: 54
  • Gluer of Gears
Re: Technical side discussion
« Reply #273 on: August 02, 2015, 11:39:18 PM »
My strongest recommendation here is, like you should have been doing up to now, keep every version and iteration of your bot.  When you start refactoring code, make sure you never, ever, EVER delete any previous version, particularly versions that worked even if they didn't do exactly what you wanted.  Nothing is more disheartening than "breaking" a working bot, and discovering you forgot how it used to work in the first place.

I would highly recommend using a source code repository for this reason.  Depending on your preferences, you might like Mercurial, Git, or Subversion better  (leaving aside things like Perforce).  Each has different strengths and weaknesses.  On Windows, you can grab the Tortoise version (TortoiseHg, TortoiseGit, and TortoiseSVN respectively) to integrate into explorer and get a nice GUI, not only for the repository itself but also to tools like Diff and Merge. 

I have all three, but I mainly use Mercurial, because that's what I am used to.  It's what I use for other projects (including UE4 game dev and non-programming projects like 3D models).

In fact, all of my COH costume files, keybinds, screenshots, chatlogs, Sentinel+ files, and demos are also kept in mercurial (it's primarily for version control, so it's not strictly useful for some of these files unless I want to see if another copy is different, but e.g. character mugshots sit alongside my character backgrounds and archived wiki pages, so why not).  So are my Paragon Chat config files and database (I copy the current database over to the repository working folder only after making major changes, because I don't feel the need to make a commit every single time I change coordinates or zones).

For certain repositories, I keep a copy on Google Drive or Dropbox for offsite backup; you can easily push or pull changes from one repository to another to keep them in sync if your primary repository is not in the local Google Drive or Dropbox folder.

My biggest problems are 1) remembering to commit and 2) remembering enough of what I changed to make a good commit message.  My repositories are littered with commits to binary files (so I can't easily see diff) with comments like "Probably nothing important" or "Vaguely remember X, or maybe Y"
-= Virtue Server - 2004-06-13 to 2012-11-30 =-
hortis publicis gemini in aeternum

Usurper Dyne, Still Heart, River Elemental, Saul Invictus, Grim Grinner

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #274 on: August 03, 2015, 12:54:28 AM »
Technically speaking, I started actually writing code about June 30th, which means at this moment in time I've been working on this, not every single day, for about 63 days.

Oh wait, I forgot that in odd numbered years July has 31 days, not 61.  That should read 33 days, not 63 days.  Wow, that means I've been writing 40 lines of code per day.  I'm awesome.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #275 on: August 03, 2015, 12:59:07 AM »
I would highly recommend using a source code repository for this reason.

I started using git a short while ago, but I'm not a frequent git user so I can't comment yet on its efficacy firsthand.  I find checkin-checkout to be a little more overhead than might be reasonable for such a small and single developer project like this, but I wanted to see if there was some reason for doing it anyway that might compensate for the overhead.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #276 on: August 03, 2015, 03:08:50 AM »
Ooooh, this is a nasty one.  For those using python and sleekxmpp, I noticed what appears to be a reproducible and horrible bug.  I was trying to extract channel information from message stanzas, because I only want the bot to obey commands sent as tells, as opposed to things it might overhear on local, say.  Long ago, I wrote this:

Code: [Select]
class pcChannelStanza(ElementBase):
    namespace = 'pc:message'
    name = 'channel'
    plugin_attrib = "channel"
    interfaces = set(('id'))

registerStanzaPlugin(Message,pcChannelStanza)

However, when I try to extract the channel id, nothing happens:

Code: [Select]
print msg['channel']['id']
u''

I couldn't figure out what was going on, until I dumped the value dictionary out of the msg stanza.  Incredibly, if I just do a x=pcChannelStanza() and then x.values, I get this:

Code: [Select]
[u'lang':'', u'i':'', u'd':'']
That's right, the interface name "id" was being exploded into its constituent characters, and the stanza was being defined as having two attributes, "i" and "d".  Apparently, this happens whenever you define a stanza with exactly one interface.  I'm not quite sure why yet.  But this fixes it:

Code: [Select]
class pcChannelStanza(ElementBase):
    namespace = 'pc:message'
    name = 'channel'
    plugin_attrib = "channel"
    interfaces = set(('id','pcfoo'))    # there is a weird bug where if there is only one interface, its name
                                        # gets exploded into its individual characters as interfaces

The extra unused attribute fixes the problem.  Maddenly insane.  Fortunately, <channel /> is the only stanza that has one and only one attribute (at the moment) so its the only attribute affected by this (so far as I've discovered so far).

Oh, and also, this fix won't work until you delete your pyc files.  Because if you radically change your code python updates those, but apparently if you change this one definition the optimizer knows you don't use that extra attribute ever, and doesn't update the pycs.  Only took me three hours to figure this out.



Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #277 on: August 03, 2015, 03:18:57 AM »
I started using git a short while ago, but I'm not a frequent git user so I can't comment yet on its efficacy firsthand.  I find checkin-checkout to be a little more overhead than might be reasonable for such a small and single developer project like this, but I wanted to see if there was some reason for doing it anyway that might compensate for the overhead.

If awesome were a goose, Git would be the sauce.

It's light-years ahead of Subversion. In theory, Mercurial has feature parity, but I never could get the feel for it. IMO Mercurial tries to simplify things too much for the average user, and makes it difficult to do interesting things with, like rebasing (rewrite history on your local branch before merging it back to master). The git ideology of branch early, branch often, merge often I've found works well in small groups of people. Mercurial doesn't offer lightweight branching out of the box (all branches in Hg are permanent), though IIRC there are addons to do it.

Like many awesome things, it's targeted at programmers, so there is a steeper learning curve than most SCMs before you figure out how to make it do awesome things. There comes a moment with Git where the architecture behind it just 'clicks' and you realize that while its primary interface is a version control system, it's not just a version control system. It's a content-addressable filesystem with mutable pointers that represent the head of a branch. Once you come to that realization, the reason why it works the way it does and how to leverage that makes much more sense.

But most importantly, it's damn fast when it comes to large projects, and it doesn't need a whole ton of dependencies to install. No scripting language interpreters or Webdav libraries or whatever required.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #278 on: August 03, 2015, 07:56:23 AM »
There comes a moment with Git where the architecture behind it just 'clicks' and you realize that while its primary interface is a version control system, it's not just a version control system. It's a content-addressable filesystem with mutable pointers that represent the head of a branch. Once you come to that realization, the reason why it works the way it does and how to leverage that makes much more sense.

It'll probably take a bit more usage for me to reach that point with git.  I can kind of see what you're describing already, but the live-wire connection hasn't been made yet.

Actually, I've heard ZFS described in terms very similar to how you describe git, and there are strong analogies between the two technologies, some probably not entirely coincidental.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #279 on: August 03, 2015, 08:09:33 AM »
In theory, Mercurial has feature parity

As I said, I'm not an expert in either, but I am passingly familiar with both.  One thing I did after sticking my sources into git was realize I didn't need to keep the long names around anymore: i.e. pcbot1.02-refactor-controls.py.  I could just call that pcbot.py and let git handle version comments in commits.  And *then* I realized I wanted to import my entire previous source history, from pcbot-0.1.py to the latest versions.  So I just recreated my repository and threw those files into it one at a time, scanning and committing as I went along**.  Then deleted the last one and added the latest pcbot.py source.  Bingo, automatic history with automatic name tracking.

I don't think Mercurial does that in quite the same way.  Which is to say, Mercurial has its own way of tracking such renaming histories, and because reasons, I'm pretty sure it takes more work (philosophical difference in design, as I understand git's history).

That's a totally unfair comparison point, but life's not fair.


** Even that's not really the most efficient way to do that in git, but I'm on Windows so sue me.