Author Topic: Technical side discussion  (Read 164645 times)

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #60 on: July 13, 2015, 01:53:28 AM »
Do you have the latest update, Arcana?

Versions prior to 0.97c had a bug in the order of processing where the metadata channel presence callback would be called before the main presence handler (that parses <character>). So if the first presence received was for the meta channel, it would fail to create the proxy entity, making position updates meaningless.

This was masked by the separate issue that PC sends <character> in every presence stanza, including those to the broadcast channel and globals (i.e. Paragon Chat). So in most cases it had already seen <character> first. Once I fixed that issue the processing order bug became readily apparent. But your bot might very well be joining the metadata room first and bumping into that.

I tested with 0.97c today, although amusingly enough I was going to PM you to ask if it was possible that Paragon Chat needed to see me in the normal room first before it would accept metadata updates, because *I* originally had the bot signing into the two rooms with different names.  Switching to identical handles didn't fix the problem, so I assumed the bug probably wasn't that.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #61 on: July 13, 2015, 02:44:46 AM »
It did used to need that because of the bug, but it should be fixed now. The design intent is that a bot should be able to join the metadata channel and not have to join the broadcast channel unless it wants to participate in zone broadcast messages.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #62 on: July 13, 2015, 06:57:09 AM »
I had the notion that there was a possibility there was a subtle Paragon Chat bug where two people with exactly identical costumes might not see each other, but then I realized I could easily test for that myself by just running Paragon Chat twice, and as it turns out you can have as many exact clones of yourself as you want.  So identical costume hashing is not the issue.  I think the next step is to trace two Paragon Chat clients talking to each other, and stanza for stanza compare that to a Paragon Chat bot to see what's different, if anything.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #63 on: July 13, 2015, 07:29:40 AM »
One point of confusion that I don't think is central to the problem, but has turned up in my stanza by stanza tracing.  Your spec says Paragon Chat advertises jid in the <pc /> stanza, explicitly stated to be user@domain/resource.  However, what I see coming from Paragon Chat is user@domain/CharacterName, which if I'm understanding XMPP correctly (I might not be) isn't the assigned Resource from the server.  The user@domain/resource JID I get assigned to me from Openfire is more like arcanabot@core3770/f57daa0e.  arcanabot@core3770/ArcanaBot is the bot, and direct messages sent to that address work, as do arcanabot@core3770/f57daa0e.  But does Paragon Chat require something specific, like user@domain/CharacterName to work correctly with <pc /> presence substanzas?  Or does it not matter.  Maybe I'm just confused on terminology: I've only been speaking XMPP for two weeks.  But I thought "f57daa0e" was the resource, and "ArcanaBot" (character name) was the nickname.

Honestly, I've tried it both ways and neither fix invisibility, but resolving this ambiguity would mean one less thing to have to check for.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #64 on: July 13, 2015, 01:19:10 PM »
The resource is only auto-assigned by the server if you don't specify one when you log in. The resource is normally up to the client to decide how it wants to identify itself. For example, you might set Pidgin on your work laptop to use a resource of 'work', so your full JID would be user@domain/work

Paragon Chat uses the character name you log into as your resource. It does not have any specific requirements or even care what the resources of other XMPP users are, other than remembering them in order to distinguish separate clients using the same base JID. The only place it gets character name from is the character stanza in the presence.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #65 on: July 13, 2015, 02:39:12 PM »
I think the next step is to trace two Paragon Chat clients talking to each other, and stanza for stanza compare that to a Paragon Chat bot to see what's different, if anything.

If it helps, the version of 0.97d uploaded a few minutes ago (link) has a -debug command line option that causes it to show all XMPP sent and received in a console window.

It's newer than the 0.97d in Tequila and that was uploaded yesterday, but didn't see a reason to bump the version number just for that.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #66 on: July 13, 2015, 03:43:49 PM »
And then... nothing.  I keep sending, PC keeps sending, but I don't see the bot in Paragon Chat.  Not sure why.  I engineered it so the bot has the same costume hash as the Paragon Chat client, so I do not need to support Iq costume, but even if that was wrong I should have then seen an Iq request for costume, which I don't.  Paragon Chat doesn't think I'm "there" yet, and I'm still trying to hunt down why.

Does Paragon Chat store the costume of its own avatar in its cache?

Is it possible that PC is recognizing the hash as "known", preventing it from requesting a costume stanza from the bot? If that was the case and PC did NOT have a cached copy of its own avatar, it would have nothing to display for the bot.

On a different note: Are possible values of the "map=" attribute documented someplace, or discoverable by pigg diving?

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #67 on: July 13, 2015, 03:53:57 PM »
Does Paragon Chat store the costume of its own avatar in its cache?

Is it possible that PC is recognizing the hash as "known", preventing it from requesting a costume stanza from the bot? If that was the case and PC did NOT have a cached copy of its own avatar, it would have nothing to display for the bot.

Yes, and it serves costumes requests from peers out of the cache. It periodically refreshes the player's costume in the cache at intervals half that of the cache expiration to ensure that it is always available to serve responses to costume requests with.

Quote
On a different note: Are possible values of the "map=" attribute documented someplace, or discoverable by pigg diving?

It could be pretty much any map file (even ones that don't exist), though right now only the ones in zone.cfg are meaningful. The corresponding names for the zone maps can be looked up in clientmessages-en.bin. Mission maps don't have translations there, so if somebody adds a mission map to their zone.cfg, global friends will see the map filename in the list.

KummerWolfe

  • Underling
  • *
  • Posts: 1
Re: Technical side discussion
« Reply #68 on: July 13, 2015, 04:34:44 PM »
I have a technical / "has anyone asked this" type of question. ( I have the "has anyone asked this" cause I couldn't find a post on it ).

Naturally this would be a "after costumes and powers and such are working", but has the idea been tossed around about what it would take to get Architect Entertainment up and running? I know there is no server at the moment to back it up, but would it be theoretically possible that a chat server could store the saved maps/meta data to serve up when requested by 1 to n users?

I would think the clients would handle the actual "play" of the scripts and assets transmitted, yes?

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #69 on: July 13, 2015, 06:58:01 PM »
Does Paragon Chat store the costume of its own avatar in its cache?

Is it possible that PC is recognizing the hash as "known", preventing it from requesting a costume stanza from the bot? If that was the case and PC did NOT have a cached copy of its own avatar, it would have nothing to display for the bot.

It slipped my mind this might not have been discussed in the thread, in a private question I asked Codewalker, he stated that PC caches its own costume, which is what made me realize that someone who was writing a bot and had not yet perfected costume stanzas could still make itself visible by cloning another character if it knew their has (which it would, because it would know everyone's hashes from their Presence stanzas).

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #70 on: July 13, 2015, 07:15:28 PM »
I have a technical / "has anyone asked this" type of question. ( I have the "has anyone asked this" cause I couldn't find a post on it ).

Naturally this would be a "after costumes and powers and such are working", but has the idea been tossed around about what it would take to get Architect Entertainment up and running?

Discussed?  Yes.  On a technical level? No.  Only Codewalker would really know how much effort it would take to do this, although I suspect its more than people think for several technical reasons, not the least of which would be:

Quote
I know there is no server at the moment to back it up, but would it be theoretically possible that a chat server could store the saved maps/meta data to serve up when requested by 1 to n users?

Theoretically, anything is possible.  Openfire has a plugin architecture that allows anyone to write special extensions to the chat server.  You could write one that created an architect mission database, accepted messages to store them, and served Iq requests to retrieve them.  However, I should point out that in practice, what you are asking is comparable to asking whether, in the year 1215, it would be theoretically possible to recreate the spark plug.  The answer is yes, it would take a lot of effort to do so, and most importantly alone it would be practically useless.  Making an architect mission server would be the last step in a very long list of things necessary to get player generated mission content working.  We don't even have a way to get NPCs to move without ghosting through walls or falling through the floor, or really at all yet, because NPC AI was a server-side feature: clients have no capability of doing that on their own (which is part of the whole reason behind thinking about bots, which could pretend to be NPC and have their intelligence within the bot and not require anything special from the player clients).

Quote
I would think the clients would handle the actual "play" of the scripts and assets transmitted, yes?

In a perfect world, yes.  But that would require adding an enormous amount of logic to Paragon Chat itself, because a lot of things City of Heroes did was only done on the servers and sent to clients.  It would be difficult to make a server plugin that became an entire mapserver (it would be easier for Codewalker to literally write a mapserver that talked directly to the game clients).  It would be almost as difficult to make Paragon Chat assume those capabilities.  And beyond all of that, there's the question of whether doing all of that subverts the original idea behind Paragon Chat, namely to be a community platform.

The way I see Paragon Chat, I don't see it recreating the game, or even wanting to.  Codewalker could wake up tomorrow and decide that's what he wants to do, but I think what Paragon Chat is supposed to be is a platform where Codewalker can continue to experiment with how the game worked, and in the process we get a set of tools to make our own things happen in a community environment.  Its most important feature is that its an open and (relatively) simple platform.  So far, I've spent very little time figuring out how Paragon Chat works, and most of my time figuring out how XMPP and sleekxmpp work, because Paragon Chat is not difficult to work with.  It opens the door for everyone from popmenu writers to bot writers to literal dice rollers to play around in the environment.  I think adding more support for more things CoH used to be able to do is a good thing, but its best if its done in a way that continues to allow people to play with the environment more than literally replicate the entire game.  If Codewalker wants to give players a literal Architect server to run missions in, I have to believe it would be far easier and faster if he implements an actual mapserver than tries to retrofit one into Paragon Chat.

I haven't even touched on the legal sticky wickets involved in going from a server that does nothing to a server that does something.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #71 on: July 13, 2015, 08:07:48 PM »
It slipped my mind this might not have been discussed in the thread, in a private question I asked Codewalker, he stated that PC caches its own costume, which is what made me realize that someone who was writing a bot and had not yet perfected costume stanzas could still make itself visible by cloning another character if it knew their has (which it would, because it would know everyone's hashes from their Presence stanzas).

I had expected something of the sort, really, but the tester in me tries to come up with all of the edge cases, heh. I actually think that serving the avatar's costume out of the cache is a rather clever bit of design.

So, as a tester, here's what I'd consider next - To eliminate any potential bugs related to duplicate hashing (or alternatively, narrow it down to that) I'd login as PC-A, then while PC-A is online, login PC-B. Once you can be sure that PC-B ought to have cached PC-A's costume hash (whether by time passed or by sniffing the stanza stream), logout PC-A, then login ArcanaBot with PC-A's costume hash. This should still cause PC-B to pull the costume from its cache but without a "hash collision" between the PC and ArcanaBot.

The worst case is that it all produces identical results, in which case you're no closer to a solution but you're definitively eliminated one of the edge cases. The best case is that now it works and you've narrowed it down to something in the "hash collision" being the culprit. Yes, you've previously proven that two instances of Paragon Chat can withstand a "hash collision" but that's not a definitive statement about ArcanaBot, given that ArcanaBot is what is being tested. (One reason being that both instances of PC-A and PC-A-prime have the costume in their cache while ArcanaBot does not have a cache at all.)

Most likely it will be much ado about nothing but hey, that's unit testing for you. You don't pick and choose based on expected results, you test everything, unless your goal is to only test the things that achieve the results you expected. ;-)
« Last Edit: July 13, 2015, 08:26:59 PM by slickriptide »

Ironwolf

  • Stubborn as a
  • Elite Boss
  • *****
  • Posts: 1,503
Re: Technical side discussion
« Reply #72 on: July 13, 2015, 08:35:47 PM »
I have been reading through the xmpp theory and trying to get an overview.

I have a question, could powers be given a name linked to each character as I saw this in the basic theory and was trying to figure a way to activate powers.

You can login with multiple locations for instance as test@titan.com and then have your powers logged in as test@titan.com/power1 and the conversation your powers have activate the animation? I am trying to get my head around this it would seem the ability of XMPP to allow multiple concurrent logins would help if the powers were run from instanced servers?

If I am way off base a simple sorry won't do that is good :)



Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #73 on: July 13, 2015, 08:56:19 PM »
So, as a tester, here's what I'd consider next - To eliminate any potential bugs related to duplicate hashing (or alternatively, narrow it down to that) I'd login as PC-A, then while PC-A is online, login PC-B. Once you can be sure that PC-B ought to have cached PC-A's costume hash (whether by time passed or by sniffing the stanza stream), logout PC-A, then login ArcanaBot with PC-A's costume hash. This should still cause PC-B to pull the costume from its cache but without a "hash collision" between the PC and ArcanaBot.

My strong suspicion now is that something is subtly wrong with the way I am constructing Presence stanzas, and Codewalker already confirmed that in at least one respect I was probably treating something not quite right, namely conflating resources and nicks.  Until now, it was difficult to see without using extremely verbose server logs what Paragon Chat was seeing coming from me (what you say and what they see are not exactly the same thing, because of XMPP semantics).  -debug will make that a lot easier, but it will have to wait until I get home from work tonight.  If I brought my bot code to work I'd never get anything done.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #74 on: July 13, 2015, 09:42:26 PM »
I have been reading through the xmpp theory and trying to get an overview.

I have a question, could powers be given a name linked to each character as I saw this in the basic theory and was trying to figure a way to activate powers.

You can login with multiple locations for instance as test@titan.com and then have your powers logged in as test@titan.com/power1 and the conversation your powers have activate the animation? I am trying to get my head around this it would seem the ability of XMPP to allow multiple concurrent logins would help if the powers were run from instanced servers?

If I am way off base a simple sorry won't do that is good :)

*If* I understand what you're saying, that's not really a good idea.  First of all, the problem with activating powers is simply that power activation was a server-side thing, and therefore Codewalker has to program Paragon Chat to understand the fundamentals of how to do that.  PC currently has no idea how to spawn power bolt projectiles, for example, so there's no way for any player or bot to do that.

I think what you are thinking is that the problem is literally activation; that the powers would work if we could only figure out how to tell the game to do that.  But that's not the case.

In a sense, think of XMPP and more precisely Paragon Chat's implementation of XMPP as a translator.  Its a CoH to Kind-of-English translator, but with a very limited dictionary.  When something happens in your game client, that is sent to Paragon Chat.  Paragon Chat converts the CoH network message into ParagonChat-ese, and that's sent to the XMPP server, which sends that to everyone else connected to that server.  Their Paragon Chat programs convert the ParagonChat-ese into CoH network messages and send those to their game clients, and they see what you did.  And vice versa.

Because ParagonChat-ese is XMPP and in Kind-of-English, its possible to inject messages into it without even running Paragon Chat, vis-a-vis bots.  But the bots can only talk in ParagonChat-ese.  ParagonChat-ese has no words for "Please Execute Energy Blast's Power Bolt."  You can't say it, so you can't tell Paragon Chat to do it.  To make that happen, Codewalker has to add those words into Paragon Chat's dictionary, and have the appropriate conversion into CoH network message protocol.

Now, *IF* Codewalker figures out how to add the appropriate words to Paragon Chat's XMPP message dictionary, *THEN* there are certain advantages bots might have.  Separate from being able to do something in theory, for a player to do it you need some kind of control to make it happen.  For example, consider power animations.  Right now, if you wanted your character to look like they were firing invisible Sniper Blasts, you'd need to play two animations: the Sniper Blast Windup, and then the Sniper Blast shot.  You'd have to do that yourself, somehow, by typing in the two commands back to back with a pause between, or use macro buttons.  Not convenient.  Bots don't have that problem, because bots don't have game controls.  A bot can simply be programmed to send the first message, pause one second, and then send the second one.  Bots can be scripted, in ways players currently cannot be (although Codewalker is thinking about embedding LUA, a separate discussion topic).  So *WHEN* Paragon Chat has more "power stuff" in its dictionary, bots might be able to do certain things better than players might be able to do, and that opens the door to bots potentially being player-helpers as well.


Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #75 on: July 14, 2015, 04:59:50 AM »
The plot thickens.  I added resources to my XMPP connections, and reran with -debug.  According to sleekxmpp, this is what I'm sending out as presence:

<presence from="arcanacoh@core3770" xml:lang="en"><pc xmlns="pc:presence" jid="arcanabot@core3770/ArcanaBot_meta" protocol="5" /><character xmlns="pc:character" origin="Magic" map="maps/City_Zones/City_01_01/City_01_01.txt" costume="Lxdw1TQl8g/dEYfAJNh7LE7oe6c=" name="ArcanaBot" class="Class_Blaster" /></presence>

This is what Paragon Chat records at the exact same moment:

xmpp DEBUG RECV: <presence lang="en" to="arcanacoh@core3770/Arcana" from="atlaspark_meta@conference.core3770/ArcanaBot_meta"><x xmlns="http://jabber.org/protocol/muc#user"><item role="participant" jid="arcanabot@core3770/ArcanaBot_meta" affiliation="none"/></x></presence>
xmpp DEBUG RECV: <presence lang="en" to="arcanacoh@core3770/Arcana" from="atlaspark@conference.core3770/ArcanaBot"><x xmlns="http://jabber.org/protocol/muc#user"><item role="participant" affiliation="none"/></x></presence>
xmpp DEBUG RECV: <message lang="en" type="groupchat" to="arcanacoh@core3770/Arcana" from="atlaspark_meta@conference.core3770/ArcanaBot_meta"><u p="146.4 0.3 -98.4" xmlns="pc:u" m="READY" v="0.0 0.0 0.0" o="0.0 -2.83 0.0"/></message>
xmpp DEBUG RECV: <message lang="en" type="groupchat" to="arcanacoh@core3770/Arcana" from="atlaspark@conference.core3770/ArcanaBot"><body>Hello, participant Arcana</body></message>

Somehow, my pc namespace substanzas are not making it to Paragon Chat, so Paragon Chat doesn't know I exist.  It must have something to do with Presence stanza handling I'm unaware of yet.  Time to RTFM.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #76 on: July 14, 2015, 06:16:48 AM »
Okay, I see two different things going on above.

1. You're sending presence to the server, I assume right after login, with the pc:presence and pc:character namespaced attributes. This is good. However, be aware that this presence gets sent only to people who have a presence subscription -- i.e. friends on your roster.

2. I see incoming presence from the two rooms the bot joined. I don't see outgoing directed presence on the bot side, so I assume your xmpp library is handling that for you. That's the important presence, though.

In XMPP MUC, the directed presence that you send to the room in order to join it is the presence that other members of the room receive. They do not get the presence that you sent to the server at the start. It sounds weird at first, but makes sense when you think about it. Hopefully your library has the ability for you to add custom elements to the presence it's sending to the room in order to join it.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #77 on: July 14, 2015, 06:46:57 AM »
In XMPP MUC, the directed presence that you send to the room in order to join it is the presence that other members of the room receive.

Feh**.  I *originally* sent presence to the room, but when I ran into a separate problem with server initiation, I accidentally deleted that code.  Then I realized I did that and put it back.  Except in the round trip, I put it back wrong: I accidentally sent presence to atlastpark_meta@core3770.  When I correctly send presence to atlaspark_meta@conference.core3770, it works.  Thanks for pointing in that direction.  Also, I had no idea what the two Presence messages were for, I just copied sample code.  That makes a lot of sense now.

Incidentally, why do you apparently send presence to atlaspark_meta@ConferenceServer.FQDN/Resource when sending to atlaspark_meta@ConferenceServer.FQDN seems to work just as well?


So, I'm now visible:



I can start cleaning things up.  At the moment, a lot of stuff is just plain hard coded, just to get things working.  The bot only logs into Atlas Park, has hardcoded login information, has a hard coded costume hash, doesn't really do anything except say hi, is currently two pixels embedded into the sidewalk, but it does work.  Proof of concept.  But for that one stupid typo, I've been banging on this for a couple days.

I need to add local chat support, multiuser chat support (for local chat), and some better means of sending the bot commands besides hard coded functions.  Maybe I'll have it accept JSON just to mess with FFM's head.  And once the costume algorithm settles down, some other way of setting appearance not involving cloning experiments.

Insert appropriate "squee" here.


** See also:

FloatingFatMan

  • An Offal
  • Elite Boss
  • *****
  • Posts: 1,178
  • Kheldian's Forever!
Re: Technical side discussion
« Reply #78 on: July 14, 2015, 07:01:45 AM »
^ I have two things to say.

1.  Congratulations on getting your bot visible! :)
2.  Oi!! No messing with my head! :P


slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #79 on: July 14, 2015, 01:15:54 PM »
Well done, @Arcana!

I am taking a longer way around. I am working on a paragon chat plugin to ErrBot, so I'm having to learn those ins and outs as well as the general business of Python data structures.