Author Topic: Technical side discussion  (Read 164701 times)

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #20 on: July 09, 2015, 08:05:51 PM »
Simple question - Is the protocol documented someplace such that a bot could simulate a client when talking to other clients?

If it's not obvious, what I want is to make a chatbot that has an avatar in the game and the ability to "speak" or emote as if it was a player. In other words, a kind of XMPP-based NPC.

As I understand things, at the current time a "player" is a collection of costume bits and a set of map coordinates.

Is that about the size of things?

Well, your chat bot would need to understand XMPP and be able to do XMPP things.  I would start there, in the programming language of your choice.

Then, to the best extent I've figured out the process, you have to:

Login into the Titan XMPP server
Find and join a room or rooms.  You would probably want to join a room that represents a City of Heroes zone, like "atlaspark"
You would need to join its related meta data channel, in this case atlaspark_meta
If your bot wants to send zone broadcast messages, you would then send zone broadcast chat as a muc (multi user chat message) in the zone room, aka atlaspark
To actually appear to other players, you would need to announce yourself with a Presence message in the meta data room.  This would have among other things your character data - name, origin, costume hash ID, etc.
You would also need to send position data in another different XMPP message.
When Paragon Chat players got within visual range of your bot (based on position) they will send you an inquiry for what you look like.  You have to respond with a proper message with your costume data.
When your bot moves, or performs an animation, or both, you have to send that as a message with some combination of position, orientation, movement velocity, and animation sequence name.  Paragon Chat clients will see that message and animate your bot's visual avatar on their screens appropriately.
Rinse and Repeat.

Bingo: you're visible.

Edit: I posted the code I was playing around with for a simple invisible bot; a bot that doesn't actually appear to Paragon Chat players (because I didn't know how to do that yet, and I'm still working on it) in this post: http://www.cohtitan.com/forum/index.php/topic,10947.msg185173.html#msg185173.  That should give you an idea, however, of how to get off the ground.  It can log into an XMPP server, talk, reply to other players, and look up stuff in City of Data (because, of course it does).  Becoming visible, however, is an order of magnitude harder.  Still not hard-hard, but it takes knowledge of the protocol to get it right and I've been trying to figure it out without bothering Codewalker too much.  But if he's willing to give up the stanza schemas, I'll write them up in something less byzantine and then go back to banging on bots.

AmberOfDzu

  • Boss
  • ****
  • Posts: 179
Re: Technical side discussion
« Reply #21 on: July 09, 2015, 08:34:49 PM »
If it's not obvious, what I want is to make a chatbot that has an avatar in the game and the ability to "speak" or emote as if it was a player. In other words, a kind of XMPP-based NPC.
I immediately think of Eliza. This could actually be pretty cool.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #22 on: July 09, 2015, 09:09:16 PM »
If you have a schema or something, I could take a swing at it.

I don't have a DTD or anything, but can sketch out the structure:

Presence schema:
<presence>
 (normal XMPP stuff)
 <pc xmlns="pc:presence" protocol="5" jid="user@domain/Resource" />
 <character xmlns="pc:character" name="Somebody" class="Class_Blaster" origin="Science" map="maps/City_Zones/City_01_01/City_01_01.txt" instance="2" costume="{hash}" />
</presence>

The "pc" stanza indicates to other Paragon Chat clients that you are running PC. The protocol attribute is the protocol version, and is incremented when incompatible changes are made, or at least changes that other clients need to be aware of to maintain compatibility. The current protocol version is 5. The jid attribute is just a copy of your JID, needed to make things work in anonymous chatrooms where only you room alias would otherwise be visible. I suppose I should probably disclose this comment:

Code: [Select]
// CW NOTE: We are explicitly disclosing the full JID in the presence data.
// This is a big no-no according to the MUC spec, as anonymous chatrooms go
// out of their way to hide the real JID of the participant. However, we
// really do need it to keep identities straight and associated with the
// right character in the game world. So we assume that anyone using this
// program will not be joining anonymous chatrooms with it, or if they do,
// they consent to making their real JID be known to the others in it...
xmpp_stanza_set_attribute(pc, "jid", makeJid(xl->node, xl->domain, xl->resource));

The character stanza includes character information, used both to create the entities of other players in the same zone, as well as to show info in the global friends list. It really should only be sent to your roster as well as the meta channel, but currently is sent to all channels you join. I'll fix it to not disclose your location to global channels at some point.

The instance attribute is normally omitted, but contains the instance number if you're not in the first instance. It's used only to show the correct zone name in the global friends list for other people. The costume attribute is a Base64 encoded SHA1 hash of your costume data; I'll post the exact algorithm for generating it later when I have some more time (and when I'm sure it's not still causing people to be invisible, heh).  When you switch costumes, PC broadcasts a new presence stanza with a different hash.



The description protocol is used to fetch the character description (bio) when someone clicks info. It is an IQ request.

<iq to="user@domain/resource" id="id12345" type="get"><description xmlns="pc:description" /></iq>

Upon receiving this iq, Paragon Chat will respond with an iq that differs only in that type="result", and the description element contains the text of the description, properly escaped (or in a CDATA) of course.



Metadata updates:

These are groupchat messages sent to the metadata channel, i.e. kingsrow_meta. They are sent when you are moving according to an algorithm designed to minimize updates where possible and batch them together. When someone joins the metadata channel, Paragon Chat starts an internal timer with a random value from 1 to 9 seconds in the future. After the timer expires, they send a full update with all of the attributes filled out so that the joining player knows where you are standing. Otherwise they might not see stationary players for quite some time. If someone else joins before the timer expires, a second update will not be sent; it assumes that a single update will cover them both.

<message type="groupchat" to="zone_meta@conference.xmpp.server">
<u xmlns="pc:u" v="1.0 2.0 3.0" p="1.0 2.0 3.0" o="1.0 2.0 3.0" m="mov" />
</message>

p is the absolute world position, 3 floating point numbers separated by a space. PC rounds to the nearest tenth to keep the message short.

v is the velocity. This is in world units per tick, where a tick is 1/30th of a second. PC rounds this to the nearest hundredth.

o is the orientation. This is pitch, yaw, roll, in radians. Note that Paragon Chat, like COH, applies the transformations in the order Yaw, Pitch, Roll despite calling it a "PYR" and the numbers being in PYR order. PC rounds to the nearest hundredth of a radian.

m is the current animation that is playing. This is the same MOV names that are used in demorecords. Paragon chat optimizes this by only sending a new MOV if the client couldn't reasonably predict it by using the "next move" and cycle info from the sequencers.



Standard messages.

There are a few special XML tags that might show up in direct and groupchat messages.

<message type="groupchat">
<body>(standard XMPP body)</body>
<channel xmlns="pc:message" id="request" />
</message>

The <channel> element is sometimes inserted into groupchat messages in the zone broadcast room. Currently the only valid attribute is id="request", which causes Paragon Chat to send the message using the Request channel rather than Broadcast.

For direct messages, the message type is chat. Normally these are handled as private tells. However, if they include a <channel> element with the id of "local", the message is handled as local chat instead. The id can also be "emote" for the emote text channel. Local and emote chat do not go to the zone channel, but are sent as direct messages to everyone who is in range. They use the XMPP multicast extension if possible (XEP-0033), otherwise they fall back to sending a bunch of individual messages.

Code: [Select]
if (xaddr) {
        xmpp_send(xl->conn, message);
} else {
        int i;
        // have to do this the hard way...
        for (i = 0; i < listsize(to); ++i) {
                XMPPResource *res = to[i];
                xmpp_stanza_set_attribute(message, "to", xmppJidFromResource(res));
                xmpp_send(xl->conn, message);           // hope we don't get banned
        }
}

The color, bgcolor, and bordercolor pseudo-html tags that are allowed in COH chat messages for the chat bubble colors are converted to XML elements in the "pc:attr" namespace and inserted into the message alongside the body, so that they don't show up as ugly inline tags to people using standard XMPP clients.



Costume Protocol

I'm out of time so I'll have to do this later. It's easily the most complex thing PC does with XMPP.

Didn't have time to check for typos, so please quote anything that looks suspicious and I'll correct if needed.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #23 on: July 09, 2015, 10:20:12 PM »
Extremely quick summary for validation purposes:

Message stanzas/namespaces you need to support to write a visible Paragon Chat bot:

pc:presence
pc:character
pc:u
pc:description
pc:message
pc:attr


A visible bot MUST announce Presence using <pc /> and <character /> stanzas in the pc:presence and pc:character namespaces respectively, when you login, when you zone, and when you change costumes.  Presence will be automatically sent to new Paragon Chat users when they first login to Paragon Chat: you do not need to periodically announce Presence (pro forma XMPP, but mentioned here for completeness sake).  You should do this on the zone meta data channel and the bot's friends roster if any.  Paragon Chat currently broadcasts to all channels a user is a member of but this is considered a bug: bots do not need to do this.  A bot that fails to implement this protocol will be effectively invisible.

A visible bot MUST send the bot's position with groupchat messages using <u /> stanzas in the pc:u namespace to the metadata channel of the zone you're in.  You can send these as often as desired, but Paragon Chat uses throttles to send this only periodically when moving, and only at periodic intervals when someone new joins the zone room if not moving.  If you are not moving, not playing an animation, and no one has joined the zone room recently, Paragon Chat doesn't send positional updates.  This is important if the bot has to track where players are.  A bot that fails to implement this protocol will not visibly move or animate.

If a visible bot wants to talk, it can use standard XMPP <message /> message stanzas with type "groupchat" or "chat."  If the type is "groupchat" and the target is a room corresponding to a City of Heroes zone, that message plays in the broadcast channel of that zone unless there is a <channel /> substanza.  If so, it plays on the City of Heroes channel in that zone, although the only currently supported channel id is "Request."  If the message type is "chat" and the target is a Paragon Chat user that plays as a private tell if there is no <channel /> substanza.  If there is a channel substanza then if it has an id of "local" or "emote" that plays as a local chat message or an emote message to its target.  Paragon Chat calculates how far every player is from its own (player's) location, and sends this message to all players within range.  A bot would have to do this calculation itself: a bot cannot send a "local" message and have it automatically play to everyone closeby, because in Paragon Chat protocol local messages are send to individual users that the Paragon Chat client itself deems needs to hear it.  <Channel /> substanzas are in the pc:message namespace.

A visible bot MAY implement color tags in the pc:attr namespace and use them in chat messages.  The supported tags are <color />, <bgcolor />, and <bordercolor />.  Using these tags in the pc:attr namespace will prevent them from showing up in chat clients that do not understand them: Paragon Chat will interpret them and color the chat bubbles on screen appropriately.  Using them in the default namespace will still work, but chat clients will display the raw tags in chat messages cluttering text messages for non-Paragon Chat users.

A visible bot MAY implement handlers to read and process XMPP messages similar to the above, to see what Paragon Chat players are saying.

Note: a bot that implements the chat message protocols but none of the other ones will be invisible, but able to communicate with other Paragon Chat users.

A visible bot SHOULD implement a reply to handle Iq requests for description.  If a player tries to view the description field of the bot, Paragon Chat will send a request to fetch that description using an XMPP Iq get message for <description /> in the pc:description namespace.  A bot SHOULD choose to implement a response.  A bot that fails to implement this protocol will not have a readable description in character information.

A visible bot MUST implement a reply to handle the Iq request to fetch costume data.  When a Paragon Chat client detects that the bot is within visual range, it will send an Iq get request to the bot for <costume /> in the pc:costume namespace.  The bot must respond with the appropriate data, or the Paragon Chat client will be unable to render the bot's appearance (note: costume data is cached and indexes with a hash key; if the Paragon Chat client already has a costume with that key cached, it will not request the same costume data again).  The details of costume encoding are not currently published.  If the bot fails to respond or responds with invalid data, the bot will be invisible.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #24 on: July 09, 2015, 10:30:29 PM »
That is way more detail than I expected to get without having to dig for it somewhere. Thanks loads, @Codewalker and @Arcana.




Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #25 on: July 09, 2015, 11:06:33 PM »
That is way more detail than I expected to get without having to dig for it somewhere. Thanks loads, @Codewalker and @Arcana.

Technically speaking, I think its enough to make a bot, period.  Until Codewalker gives us the costume computation protocol, we can't create random bots on the fly.  But a technically savvy person wouldn't need to.  You set up your own Openfire server**, log Paragon Chat into it, and log your bot into it.  Create a character that looks like what you want the bot to look like.  Send the Iq character inquiry from the bot.  Paragon Chat will respond with the correct character stanza.  Have your bot record that.  Now add code to the bot to play that literally when it receives Iq character requests, with the appropriate to/from/other stuff.  You've now created something I earlier called a "mirror bot***."  You could make a bot that "steals" appearances from other players, and then plays them back.  It would be visible without needing to know the precise details of the character stanza structure.  It would be limited, but functional.

With the above information and a dev server and a bot that can stanza-trace, I think its now possible.  Technically, it was possible before Codewalker posted his info, which is why I was already working on it.  Its just that I couldn't write a guide with any sharpness without a lot more work on determining the technical requirements, as opposed to just what happens to work.


** This step is not strictly speaking necessary, but I'm going to keep harping on any prospective bot writers to test their creations on their own servers first, rather than make a radical coding error on a live but still early beta server that real people are using.  Be responsible.

*** I was going to wait until I could compute costume data before continuing Bot development, but I realized yesterday I could vampirically steal costumes from other players, including ones I create.  At that moment, ArcanaBot0.1 became MirrorBot0.2.  MirrorBot is probably a few dozen lines of python away from working.


Edit: although what I've described is enough to make a visible bot, its not the end of the road for bot makers by any means.  There are issues like pathing and how velocity interacts with client side prediction to create smooth motion that would make bots better.  Clocking, threading, zoning, object interaction, map/geometry interaction, sequence following - the sky's the limit for making a bot behave like you want it to.

Edit2: what I wrote above is *not* the documentation I intend to write to document the technical information necessary to understand how Paragon Chat works the way it works and how you might use that information to write a bot.  Its a very technical summary.  I will write a more detailed set of guides after I actually confirm my understanding is good enough to actually make a bot work.  Codewalker's post and my post above should allow anyone with sufficient programming skill and knowledge of XML/XMPP to get started.  But the more reasonable documentation will have to wait a tiny bit.  I would hate to write a guide that doesn't actually work.  Hopefully, I'm in striking distance of having a functional bot that provides proof of concept for the technical information above within a day or so, real life permitting.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #26 on: July 10, 2015, 03:09:36 PM »
Sidebar: XEP-0033

I briefly mentioned this in the description of local and text emote chat. XEP-0033, or Extended Stanza Addressing allows you to send a single message to multiple recipients. If the server supports XEP-0033 natively, it will include <feature var='http://jabber.org/protocol/address'/> in its feature discovery response (disco#info). In that case, you can format the outgoing message like this:
<message type="chat" to="domain">
<addresses xmlns="http://jabber.org/protocol/address">
<address type="bcc" jid="user1@domain/Resource"/>
<address type="bcc" jid="user2@domain/Resource"/>
<address type="bcc" jid="user3@domain/Resource"/>
</addresses>
<body>Hello, people!</body>
<channel xmlns="pc:message" id="local"/>
</message>

Paragon Chat uses bcc as the address type, as it is not necessary nor desirable for each recipient of a local chat message to see the entire list of who it was sent to.

Some servers do not support multicast natively, but instead as a separate service under a subdomain. Paragon Chat does not yet implement that style of XEP-0033, and will fall back to sending to each recipient individually if the server does not support it internally.

FloatingFatMan

  • An Offal
  • Elite Boss
  • *****
  • Posts: 1,178
  • Kheldian's Forever!
Re: Technical side discussion
« Reply #27 on: July 10, 2015, 03:15:12 PM »
It's a shame the XMPP message format is in XML... So much wasted space spent on the message schema itself instead of the more important data.  It'd be better in JSON format!

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #28 on: July 10, 2015, 03:44:17 PM »
It's a shame the XMPP message format is in XML... So much wasted space spent on the message schema itself instead of the more important data.  It'd be better in JSON format!

Perhaps, but it *is* a chat server. They're specifically designed to promote communication rather than efficiency, particularly between live people and automated responders. If you were building a game server from the ground up, you'd probably end up with something like the CoH server, compressing as much information as possible.

In any case, if part of your goal is to make it so easy to run a server that anyone can do it then you are going to prefer something comprehendable over something obfuscated in the name of efficiency.

I'll be setting up an Openfire server this weekend and doing some initial botting experiments. I really appreciate all of the info being shared here about the workings of Paragon Chat.

I had a thought out of the blue this morning, that sounds completely impractical but that sort of illustrates where things could go, eventually. For no particular reason (other than having been thinking about bots and such) I began musing about the MOO I used to help administer many, many years ago. That's when some part of my brain asked, "What if each of those NPC's in a MOO 'room' was a chat client? What if the UI to a MOO or MUSH or MUD was not a text parser but a chat server gateway?"

Mind you, I'm not suggesting that you would literally take an existing MUD and tack a XMPP gateway onto it. You'd pretty much have to build something new from scratch to make the whole thing work, but existing MUD-style game engines could be used as models for how to build a simple Architect Entertainment style of scripted mission processor.


FloatingFatMan

  • An Offal
  • Elite Boss
  • *****
  • Posts: 1,178
  • Kheldian's Forever!
Re: Technical side discussion
« Reply #29 on: July 10, 2015, 03:52:31 PM »
Perhaps, but it *is* a chat server. They're specifically designed to promote communication rather than efficiency, particularly between live people and automated responders. If you were building a game server from the ground up, you'd probably end up with something like the CoH server, compressing as much information as possible.

Sure, but PC needs speed above all else.  Personally, and this is just me and not a suggestion to anyone at all, I'd use the minimum possible XML message stanza and for the actual CoH related data itself, I'd use JSON or something even smaller (but not compressed or you'd have to decompress it), to get the data across as fast as possible.  Infact, JSON's also easier on the dev to process than clunky old XML is. :p

Mind you, for all we know, CW's doing something along those lines for the costume data...

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #30 on: July 10, 2015, 04:07:49 PM »
When working in non-managed languages, binary data is an order of magnitude more efficient and easier on the programmer than JSON is. In C, JSON is no easier to deal with than XML is.

If maximum performance was a design goal of Paragon Chat, it would certainly be using a handrolled binary protocol rather than XMPP.

But of course that wouldn't be easy for the community to plug their own stuff into.

FloatingFatMan

  • An Offal
  • Elite Boss
  • *****
  • Posts: 1,178
  • Kheldian's Forever!
Re: Technical side discussion
« Reply #31 on: July 10, 2015, 04:15:31 PM »
When working in non-managed languages, binary data is an order of magnitude more efficient and easier on the programmer than JSON is. In C, JSON is no easier to deal with than XML is.

If maximum performance was a design goal of Paragon Chat, it would certainly be using a handrolled binary protocol rather than XMPP.

But of course that wouldn't be easy for the community to plug their own stuff into.

TBH, I was primarily thinking of the unwieldy verbosity of XML over JSON. Obviously, binary is better, but not human readable. JSON is!  (Not that I'm evangelising JSON.  I'd be inclined to just go with comma separated data if I was that worried about brevity + readability).

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #32 on: July 10, 2015, 04:45:47 PM »
And if there were a bunch of premade servers for routing JSON-based messages between multiple peers, then it might be an option. :P

Embedding JSON inside XMPP seems even sillier as then you'd need two separate parsers.

In practice, the difference in verbosity is really a non-issue. XMPP very strongly pushes TLS for connection privacy; and TLS, at least in the configuration we're using it in, uses DEFLATE compression. So all of those redundant end tags and repeated attributes are replaced by tiny binary backreference tokens, taking up no more space than JSON's "}".

Yes, there is additional CPU time wasted on compression, but far less than the amount you're spending to encrypt the data.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #33 on: July 10, 2015, 05:24:42 PM »
Embedding JSON inside XMPP seems even sillier as then you'd need two separate parsers.

Technically speaking, I had to write an attribute parser to yank values out of <u /> blocks, since those aren't technically "XMLified" (i.e. <pitch>0.1</pitch>, or even <u xmlns="pc:u" pitch="0.1" roll="0.0" yaw="0.3" ... etc).  Bad, bad monkey.   :P

But no, I would not want to embed JSON inside of XML sent as XMPP.  If you were going to go JSON, you'd go all the way and make Paragon Chat push JSON to a central web server, and fetch JSON from the same web server using state pages to echo dynamic state to the clients.  Its possible (JSON push "I'm here"; JSON fetch "who's here"'; JSON push "send message to zone: 'yo!'"; JSON fetch "latest messages from zone X") but you'd have to write that central server.  And the latency would probably be horrible.  Instead of City of Heroes as chat system, it would effectively be City of Heroes as message board.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #34 on: July 10, 2015, 06:41:59 PM »
Okay, here it is, the costume exchange protocol.

First, some definitions.

COLOR: For the purposes of this document, COLOR is an 8-digit hexadecimal representation of a color. The order of the digits is RRGGBBAA. The numbers a-f must be represented in lowercase (important for hashing). The alpha channel MUST be ff in all cases; Paragon Chat enforces this by ORing the alpha with 0xff before hashing or sending the costume.

The alpha value is unused by the game, but the client sometimes sticks random numbers there that have no meaning. In hindsight, using 6 digits for the protocol would have been better, but it's too late now to change it without breaking things.

COSTUME HASH: A costume hash is a short (~28 byte) string that uniquely identifies a costume. These hashes are sent in presence stanzas. When a client needs a costume, it can use the hash to request a peer who owns that costume to send a copy. Once a client has received a costume, it should cache the costume locally for a reasonable length of time and associate it with the corresponding hash, so that it does not need to be requested again.

To compute the costume hash, first concatenate together the following into a single byte array or string*:

BodyType + SkinColor + Scale + BoneScale + HeadScale + ShoulderScale + ChestScale + WaistScale + HipScale + LegScale + Head3DScale + Brow3DScale + Cheek3DScale + Chin3DScale + Cranium3DScale + Jaw3DScale + Nose3DScale + Parts[]

BodyType is a string that is "0" for male, "1" for female, or "4" for huge.

SkinColor is a COLOR.

Each *Scale is a string representation of a floating point number for each of the scale sliders; as found in a costume file. They should be truncated to at most 4 decimal places before hashing and trailing zeroes removed. Scales which have a value of 0 are skipped and not concatenated.

Each *3DScale is an 8-digit hexadecimal number representing the encoded version of a 3-dimensional floating point vector. How to calculate this will be covered below. Encoded scales which have a value of 00000000 are skipped and not concatenated.

For each costume part, numbered 0 to 27 in the same order they appear in a saved costume file or demorecord, examine the Geo and Fx of the part. If they are both empty or "None", skip this costume part and do not hash it. Otherwise, concatenate the decimal string representation of the number (0-27) of the part to the string to be hashed. Then concatenate the Geometry, Texture1, Texture2, and Fx strings, in that order, if they are not empty or 'None'.

For each Color1, Color2, Color3, or Color4 of a costume part that is not skipped, and where the color is not black (RGB = 000000), first concatenate either a "1", "2", "3", or "4" depending on which color it is, then the hex COLOR representation.

Once all of the valid parts are concatenated, take the SHA-1 of the combined string. Base64 encode the resulting hash. The Base64 result is the costume hash.

* Or use incremental SHA-1 updates, which is what PC does, but saying to concatenate it all is easier to understand for documentation purposes and has the same end result.



Costume request protocol:

Upon encountering an entity whose costume hash is known, but no valid costume is cached for, Paragon Chat sends a costume request to the peer in this form:
<iq to="user@domain/resource" id="id12345" type="get"><costume xmlns="pc:costume" hash="ZAqyuuB77cTBY/Z5p0b3q3+10fo="/></iq>

When such a request is received, if the hash corresponds to a known costume, Paragon Chat will reply with a copy of the costume in the following format:

<iq to="user@domain/resource" id="id12345" type="result">
<costume xmlns="pc:costume">
  <appearance bodytype="1" skincolor="123456ff"
    scale="2.1" bone="0.5" head="0.5"
    shoulder="0.5" chest="0.5" waist="0.5" hip="0.5" leg="0.5"
    head3d="123456ab" brow3d="123456ab" cheek3d="123456ab"
    chin3d="123456ab" cranium3d="123456ab" jaw3d="123456ab"
    nose3d="123456ab" />
  <part n="0" geom="Leather_02"
    tex1="Leather_03a" tex2="Leather_03b"
    fx="Auras/Female/Gaseous/GasEyes.fx"
    color1="123456ff" color2="123456ff"
    color3="123456ff" color4="123456ff" />
  <part n="1" ... />
  ...
</costume>

The rules for composing the stanza are identical to the rules for computing the hash. Scales with a value of 0.0 may be omitted from the appearance element. Encoded 3D scales with a value of 00000000 may also be omitted.

Costume parts that have neither a geom nor an fx (or where they are 'None') may be omitted. The "n" attribute is the decimal index of the costume part, from 0 to 27, so that the order can be reconstructed even if parts are omitted. geom, tex1, tex2, and fx attributes should be omitted if they are empty or 'None'. color* attributes should only be included if they have a value other than 000000ff.



3D scale encoding:

The three dimensional face scales are encoded in an optimized form comprised of a single 32-bit integer value rather that represents a 3-element floating point vector. This form comes from the COH network protocol and is how costumes are transmitted over the wire. It is only applicable to three-element vectors with each component having a value between -1.0 and 1.0.

The 32-bit integer is divided into 3 8-bit sections:

31       23       15         7       0
 |        |        |         |       |
 00000000 Szzzzzzz Syyyyyyyy Sxxxxxxxx

First, the value of each component (X,Y,Z) of the vector should be clamped to the range -1.0 to 1.0. It is multiplied by 100 and truncated, then encoded as an 8-bit integer with the 'S' acting as a sign bit. Bits 24-31 are unused and must be 0.

This encoding ensures that a vector of 0,0,0 is represented by an integer 0 so that it can be optimized out of transmission.
« Last Edit: July 15, 2015, 09:54:24 PM by Codewalker »


Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #36 on: July 10, 2015, 07:32:19 PM »


Believe it or not, there are very good reasons for most of that... For example the skipping of empty or 'None' parts and geos is because the client treats them as interchangeable, so a costume you send to the COH client with a NULL Geometry might very well come back as "None" later and hash to something different.

Technically speaking, I had to write an attribute parser to yank values out of <u /> blocks, since those aren't technically "XMLified" (i.e. <pitch>0.1</pitch>, or even <u xmlns="pc:u" pitch="0.1" roll="0.0" yaw="0.3" ... etc).  Bad, bad monkey.   :P

There are two schools of thought with XML. One is to make everything into an element and character data inside the element. I find that makes things annoying to parse more often than it doesn't, and leads to ambiguities that are difficult to resolve.

I'm a follower of the second, which is to use elements only for things that should either be able to contain other elements, or that it makes sense to have more than one of (because elements can be repeated). Attributes should be used for simple and unique data. It wouldn't make much sense for a single update message to have multiple orientations, after all.

Or are you complaining about shoving the X/Y/Z and P/Y/R values into a single attribute? I suppose a valid argument can be made there that they should be separate. I was just thinking along the lines of it being a single value of type Vector rather than 3 individual floats.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #37 on: July 10, 2015, 09:13:48 PM »
Or are you complaining about shoving the X/Y/Z and P/Y/R values into a single attribute? I suppose a valid argument can be made there that they should be separate. I was just thinking along the lines of it being a single value of type Vector rather than 3 individual floats.

That was one of the two examples I gave for better "XMLization" - either tag everything, or if using attributes break up the values into separate attributes instead of structured data that required separate parsing.

I was joking more than complaining: since I'm writing in python I can pretend those are actually string tokenized vectors.  I can almost treat them as actual vectors with something like:

def message_insert_position(self, pmsg):
   pmsg['u']['p'] = ' '.join([str(x) for x in self.position])
   pmsg['u']['o'] = ' '.join([str(x) for x in self.orientation])
   pmsg['u']['v'] = ' '.join([str(x) for x in self.velocity])

and

def bot_update_position(self, pmsg):
   self.position = [float(x) for x in pmsg['u']['p'].split('  ')]
   self.orientation = [float(x) for x in pmsg['u']['o'].split('  ')]
   self.velocity = [float(x) for x in pmsg['u']['v'].split('  ')]

Its *barely* parsing.  The question is when I get to motion tracking, will I treat those as vectors with vector calculations, or will I loop through Cartesian directions?  That'll have to wait until I can troubleshoot an odd problem with some of my presence stanzas apparently going out as quoted escaped bodies of message stanzas.  I didn't have a lot of time last night to work on it.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #38 on: July 10, 2015, 09:29:18 PM »
I was joking more than complaining

In case it wasn't obvious in text form, my word choice itself was intended as a joke.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #39 on: July 11, 2015, 01:11:46 AM »
My dreams of adapting an existing bot with a PC plugin are starting to look like so much star dust, lol.

I guess I'm going to have to dive into the nuts and bolts of XMPP to do this right.