Author Topic: Technical side discussion  (Read 164636 times)

FloatingFatMan

  • An Offal
  • Elite Boss
  • *****
  • Posts: 1,178
  • Kheldian's Forever!
Re: Technical side discussion
« Reply #380 on: May 29, 2016, 04:59:17 PM »
Personally, I'd rather CW -not- waste time implementing a feature anyone can do just by logging in with a regular xmpp client, and use it to bring more functionality to PC.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #381 on: May 29, 2016, 07:33:47 PM »
Putting aside my normal semi-unrealistic wish list -- A person who uses Paragon Chat solely as a chat client will occasionally want to "lurk" and hear what's going on even though she isn't actively participating. That's why text-only clients have options like "away", "do not disturb", and "extended away" instead of just forcing the person to disconnect.

A person in "lurk mode" may prefer that his avatar be hidden to avoid the appearance of inviting conversation, or of ignoring someone who doesn't realize they are AFK.

It's a valid answer to say, "Then they should just use Pidgin for that" but you ordinarily don't want to tell someone that they have to use two different clients to do one job.

Likewise, putting the responsibility of typing "/afk" on the user is a valid answer. I'd just suggest that sometimes a better answer is to remove the illusion of presence entirely. At least, the option would give a user choice in how to make themselves be "afk".

Something for the feature suggestion list.

You may need to wait until Codewalker implements stealth and perception attributes, whenever that might happen, or whenever he implements teleport which requires implementing an alternate method of becoming temporarily invisible the client supports.

Separately, there are ethical concerns regarding lurking in a graphical chat environment.  The reasoning is that in normal XMPP chat there's really usually just the notion of broadcast chat (across the entire chat room) and private chat.  But Paragon Chat by virtue of supporting other kinds of chat like emotes has a theoretical way of "localizing" chat to only nearby participants.  There's no analog to "nearby" in text-based communications.  Is it ethical to allow someone to become invisible and yet listen to group chat intended only for avatars within a certain distance (and presumptively visible) from the participants?

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #382 on: May 30, 2016, 02:24:34 AM »
A person in "lurk mode" may prefer that his avatar be hidden to avoid the appearance of inviting conversation, or of ignoring someone who doesn't realize they are AFK.

The easiest way to do that is to hide in some far-off corner, or use a custom zone config to park in a uniquely-named zone.

Players going invisible while still interacting normally isn't going to happen; and even the invisible costume issue may get changed to use a default so that the player name is at least visible. The local chat system is built on the expectation that if someone is close enough to hear your chat, you should be able to see them to know someone is listening.

The closest you can come is to send someone the "I am ignoring you" IQ. That serves as a hint that you have removed someone from your game universe and are not interested in interacting with them in any way, so they might as well remove you from theirs as well. Once you do that, you will also no longer receive their local chat, since it basically asks them to put you on (non-persistent) ignore.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #383 on: June 01, 2016, 09:46:38 PM »

Players going invisible while still interacting normally isn't going to happen;

Roger that. If I want a bot to "fade out" then it'll be the hard way, then.

For the record, I'm pretty much always aware of the fact that I'm asking for things that outside the bounds of Paragon Chat's primary function as a graphical chat room. My interests are in pushing it beyond the limits of the vision and seeing just how far it can be bent that way. That is not a criticism of anything to do with Paragon Chat or the vision driving it. I just want to see what we can do with it if it happens that this is the most that is ever done in the way of a CoH "environment".

Yes, I'm aware that other things are happening, including the hinted-at scripting interface. I'm also aware that projects end without notice all the time due to life events, changing interests, and people simply getting tired of working together. While I hope that there will be much more coming in the next year or two, my own vision is based on what's in front of me at the moment.

Here's an example of the sorts of things I'm talking about:

The first time I put a velocity on a bot and discovered that it "jumped", I immediately considered the idea of making a "Duck Hunt" game. One bot would listen for the command "Pull!" and launch a "duck" in a semi-randomly determined arc. Another bot would listen for command "Punt!" and "punt" a Red Hat into the air in whatever direction the player avatar was facing. Since actual targeting is not available at the moment, the command would probably be more like "Punt X" where X is a force between 1 and 10 or something. In any event, if the "duck" and the "projectile" intersect then a third bot scores a point for the player. The scorekeeper would keep track of individual scores as well as a leaderboard of the ten highest scores or something similar.

These three bots might all be one bot, or specialized individuals.

I'm a ways off from doing anything of the sort - I am still finding plenty of things I need to change about my current bot to make it handle player tracking properly, and I'm just now reaching that point where I can say I really understand XMPP, SleekXMPP, ErrBot, and CoH fundamentals and how each plays with the others. The last thing I need right now is to get sidetracked into learning cannon physics.

But maybe that gives you an idea why I'm constantly asking for things that appear to have nothing to do with chatting with people, heh.

Diary time:

Having got as far as I am, I'd recommend that anyone considering this pursuit do two things. One is choose a coding environment that you are familiar with. The other is start with the fundamentals of XMPP, master that, and then move up the "stack" to your XMPP API/Library and then to Paragon Chat and the internals of the CoH client.

I started down this road knowing zero about any of it, and assuming that I could take a "top down" approach to learning just whatever bits and pieces I'd need to get a chatbot up and running and then let the chatbot do the fundamentals while I did the stuff I actually cared about doing.

Arcana was both right and wrong about the follies of taking this approach. She was right, from the standpoint that despite my hopes and desires, there was simply no way to actually accomplish interactivity, especially if it was done in anything like a correct manner, without first learning the fundamentals of how communication in XMPP happens at all.

That meant that saddling myself with a chatbot framework from the outset was just adding yet another thing to the stack of things that I didn't know anything about and would have to learn thoroughly before I could use it effectively. I ended up handicapping myself to a certain extent, especially since I was forcing myself to struggle with data structures in the chatbot framework that were never intended do what I wanted them to do.

My unfamiliarity with Python only aggravated the problem. I didn't know how inheritance worked in Python, and even simple tasks required not only education in Python methods and modules but also in how SleekXMPP implemented certain tasks and then how Errbot implemented other things on top of those layers.

Has it paid off in the end? Yes, to some extent. Example - When I was working out how to correctly handle channels (and then working out that my understanding of channels was somewhat flawed) I was able to rewrite part of the bot and command Errbot to reload the affected module on the fly and test it immediately. When I wanted to write a "voice command" for the bot to respond to, it was a simple task using Errbot's existing interfaces.

On the other hand, it also cost me some pain. With no real knowledge of Python, I mistakenly approached it as if it was C or C++ and treated everything about it as if it was "sacrosanct", meaning that data structures were immutable and functions very strongly typed. I ended up spinning my wheels rather a long time trying to accomplish things that turned out to require no special accomplishment at all, because I was following examples written by other people who treated Python as if it was strongly typed and assuming that meant that it must BE so. It took some input from the Errbot people to point out how I was making things hard on myself to realize that I was, in fact, making things much harder than they needed to be.

As a beginner in all of this, I'd have been better off if I'd started out writing in C and using the Libstrophe library or something like it to learn to communicate in XMPP. Once I had that down, then moving on to interacting with Paragon Chat and expanding to other things once that was mastered.

I'm far from a "master" of all of this but I've reached the point where I can say I understand each of the pieces and how they interface, and where to place the new interfaces between Errbot and Paragon Chat so that the back end of Errbot can deal with the metadata and the front end can query and manipulate the metadata without having to understand where it came from.

It'll be worth it in the end but I made it a lot harder row to hoe than it needed to be, at least from the perspective of just learning to communicate with Paragon Chat in the first place.

If you're thinking of making a bot, then start with the fundamentals and work up from there. Like most other activities in life, short cuts aren't all they appear to be unless you're already so well versed in the subject that you didn't really need the short cut in the first place.






Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #384 on: June 01, 2016, 09:55:36 PM »
For the record, I'm pretty much always aware of the fact that I'm asking for things that outside the bounds of Paragon Chat's primary function as a graphical chat room. My interests are in pushing it beyond the limits of the vision and seeing just how far it can be bent that way. That is not a criticism of anything to do with Paragon Chat or the vision driving it. I just want to see what we can do with it if it happens that this is the most that is ever done in the way of a CoH "environment".

I get that, and it's fine. I'm not trying to jump on you for experimenting or coming up with crazy ideas -- just being honest about what's possible and what the goals/limitations of the project are.

Ideally I want to encourage creative uses of the platform, but as you said time is limited so some things need to be prioritized.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #385 on: October 29, 2017, 02:12:26 PM »
At the risk of being a thread necromancer, I'm resurrecting this thread because the answers to questions can be found here.

A year ago, roughly, I lost my hard drive and my bot code along with it. Rather than start over, I threw up my hands and went and played Hearthstone.

A couple of months ago, I ran across a backup of the costume code from the old bot, and was inspired to have a go at it again.

The break was good, as it turned out. This time I was in a learning frame of mind rather than a shortcutting frame of mind.

Anyway, my bot learned how to walk around this week so I thought I'd share a video of the bot doing some tricks, to celebrate.

https://youtu.be/xOhfslGTZ08

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #386 on: November 02, 2017, 12:38:10 AM »
So, velocity vectors. I'm having some difficulties wrapping my head around how to figure out what they should be.

My initial implementation of bot avatar movement was to emulate Arcana's approach and send a position every tick. That worked okay - The bot moved where it was supposed to at approximately the speed I wanted it to move. It was a little herky-jerky but I expected it to need some fine tuning. One thing I noticed was that Arcana wasn't computing any velocity for her "come here" command. Out of curiosity, I had Rover stop sending velocity and just send position and orientation. He started running smooth as silk. It didn't matter that his velocity between ticks was zero because the updates were coming in so fast that he was effectively moving at desired velocity just from the constant position updates.

So, I was getting the velocity wrong; hardly a surprise. Unfortunately, when I took a look at the debug logs I could see that Rover was spamming the Heck out of the metadata channel. It was good that he was running around on a private server because if there was more than a couple of people in the zone with him he'd be DDOS'ing the server!

No matter how accurate it might be, it was an unacceptable amount of traffic to be hitting the server with(or would be if there were multiple Rovers in the zone).

Then I took a good look at Paragon Chat and affirmed that it only sends position and velocity updates when something changes. Otherwise, client prediction handles things.

Which brings me back to velocity vectors.

I've got the math down for moving from point A to point B. Rover can respond to "come here" or "face me" or "follow me", but somehow when he reaches his goal, he always either over shoots or under shoots and gets rubber banded to the point he was trying to reach.

That is, he does this:

Compute one tick of movement.
If the any of the new velocity values are different from the current velocity values, update current and send a u stanza
If computed PYR is different from current than update current and send a u stanza.
If distance is less than 5 then set velocity to zero, update current and send a u stanza.

So, it sends the bare minimum number of messages, which is great, but it always overshoots or undershoots when it reaches its goal position.

The 64-million-dollar question, then - When I set my avatar to "walk" and I turn 45 degrees and press the "forward" key, how does the client and/or Paragon Chat figure out that the velocity vector is "-.01 0 -.12" and why does that add up to a different vector length than, say, a vector like "0, 0, .15"?

What are the velocity values for "walk", "run", and "sprint" and how does the client apply them to calculate the position in time from any given unit vector?


Surelle

  • Guest
Re: Technical side discussion
« Reply #387 on: November 02, 2017, 04:52:38 PM »
At the risk of being a thread necromancer, I'm resurrecting this thread because the answers to questions can be found here.

A year ago, roughly, I lost my hard drive and my bot code along with it. Rather than start over, I threw up my hands and went and played Hearthstone.

A couple of months ago, I ran across a backup of the costume code from the old bot, and was inspired to have a go at it again.

The break was good, as it turned out. This time I was in a learning frame of mind rather than a shortcutting frame of mind.

Anyway, my bot learned how to walk around this week so I thought I'd share a video of the bot doing some tricks, to celebrate.

https://youtu.be/xOhfslGTZ08

Nice, Slick!  Are you aiming to get mobs into the game with this, is that the purpose?

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #388 on: November 02, 2017, 07:54:47 PM »
Nice, Slick!  Are you aiming to get mobs into the game with this, is that the purpose?

That's the end-goal, yes. OTOH, Codewalker's "ideal world" goals for Paragon Chat include eventually building a scripting interface that would do a better job of pretending to be mobs in the game than a bot that is essentially a player pretending to be a mob in the game, so for now Rover is mostly experimental.

I'll decide what to do with him next after I've solved the "move somewhere without flooding the server with move commands" problem.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #389 on: November 02, 2017, 11:32:20 PM »
As a consequence of logging my movement computations, I confirmed not only that my math was correct, but also that my clock is slow.

Whether it's the overhead of Errbot or the overhead the movement computations, the clock is running at 21 ticks per second instead of the 30 ticks it's specced for. The game client is happily running along at an actual thirty ticks per second and so it's sending the bot further along than the bot believes itself to be.

So, in the short term I need to scale things between the standard and the actual clock and in the long term I need to have the scaling to be adjustable and have some sort of profiling that can adjust it for the performance of the machine running the bot.

Ohioknight

  • Celebrating Columbus Day
  • Elite Boss
  • *****
  • Posts: 736
  • 65 years old
Re: Technical side discussion
« Reply #390 on: November 03, 2017, 01:58:17 AM »
That's the end-goal, yes. OTOH, Codewalker's "ideal world" goals for Paragon Chat include eventually building a scripting interface that would do a better job of pretending to be mobs in the game than a bot that is essentially a player pretending to be a mob in the game, so for now Rover is mostly experimental.

I'll decide what to do with him next after I've solved the "move somewhere without flooding the server with move commands" problem.

Cool, what happens if you toss him a frisbee while he's moving?
"Wow, a fat, sarcastic, Star Trek fan, you must be a devil with the ladies"

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #391 on: November 03, 2017, 04:17:59 AM »
Nothing right now. That is, Paragon Chat might show Rover catching it but Rover wouldn't know it. Maybe I'll see if I can trick or treat up some candy canes to buy a frisbee power and figure out how to throw it.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #392 on: November 03, 2017, 03:23:30 PM »
So, here's an unexpected interaction/challenge that, in retrospect, ought to have been totally expected.

Once I got Rover's velocity scaled properly for his actual clock, he started arriving at his destination spot on the money. The next issue to deal with was a "bug" that caused him to frequently sit in one spot for a period of time, before taking off and following the current position of his target.

It was while I was watching the live debug logs playing on both Rover and Paragon Chat that the light bulb went off.

Rover didn't have a bug. He was doing exactly what he was programmed to do.

The issue is Paragon Chat's economy of traffic.

If I ran my character backwards in a straight line, the client didn't send any motion stanzas beyond the initial stanza that tells nearby people that Slickriptide is at position x,y,z with velocity a,b,c. As long as I maintained a constant velocity, Rover was blind to my current position. As far as he was concerned, I was still standing at x,y,z. Meanwhile, from my standpoint and from the standpoint of anyone watching via their own Paragon Chat client, my avatar was sliding away from x,y,z at whatever my run velocity was set to.

Once I changed velocity (by slowing or by curving in a new direction) then my client sent a position update and Rover realized, "Hey! He's really way over there!" and started running towards that new position.

That also explained why Rover seemed to always  be chasing my back trail instead of chasing my current movements. His idea of "now" is based on the last position update he received from his target. That could be reasonably accurate if I was making a lot of turns and shifts or terribly inaccurate if I was moving in a straight line at a constant velocity (and therefore sending few or zero position updates).

That means that if Rover is going to follow someone that he needs to change from a "follow" behavior to a "pursuit" behavior, where he tries to anticipate where his target will be instead of going to the last place his target actually was. The biggest challenge with such a behavior being that he can't know how accurate he is until the target finally relents and sends a position update.

Basically, Rover is a victim of General Uncertainty, heh.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #393 on: November 03, 2017, 06:35:23 PM »
Well, you probably don't want to anticipate too much. If you're too accurate, the bot will try to occupy the same space as the player and trigger collision + pushing. Ideally you want to be directly behind the player.

What it doesn't sound like you're currently doing: the bot needs to know how to figure out where the player is right now, rather than where the player was when the last update was sent. In order to do this, you'll need to retain one more piece of information - the timestamp of the last velocity data that was received.

From there it's as simple as scaling the velocity vector by time elapsed and adding it to the last received position:

posCur = posLast + velLast * dT

Where dT (? Time) is the time elapsed in ticks. There are 30 ticks per second, so if your time interval is measured in seconds, just multiply by 30.

Paragon Chat does this exact calculation in order to determine if it needs to send an update or not. It compares the position that everyone else will predict (based on the last sent position and velocity) with the player's actual position. If it's within a reasonable margin of error, it knows it doesn't need to send an update. That's why it doesn't send anything at all when you're running in a straight line.

There are a couple things that will help. The first is that when PC sends a velocity change, it always sends a position update in the same stanza. That means you don't need to track the last received value/time for position and velocity separately.

The second is that while PC has a minimum threshold for when things have changed "enough" to warrant sending an update -- and may delay sending for a fraction of a second to try and batch up multiple changes -- the player coming to a complete stop (velocity becomes 0 after previously having a value) always triggers an immediate send.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #394 on: November 03, 2017, 07:18:49 PM »
The base run speed is 0.7 world units/tick (or 0.525 when moving backwards).

The following multipliers are applied by travel powers:

Sprint: 2.0
Super Speed: 6.46
Walk: 0.216
Ninja Run & Beast Run: 2.4

For walk, 0.7 * 0.216 = 0.1512, which is pretty consistent with the vector length that you're seeing.

There are a couple things that may cause the velocity vectors seen on the wire to not always match that exactly.
  • When you press a movement key, velocity doesn't immediately jump to the maximum. There is a ramp-up for the first second or so that you hold the key down before you hit the max.

  • The velocities above are the input velocity that get fed into the physics engine (which is a poor ripoff close-enough reproduction of the client's physics engine to not cause rubber-banding). Other factors like surface friction or becoming airborne from elevation changes can change the final velocity. It's the output from the physics that is ultimately sent over xmpp.

Are you sure you were seeing -0.01 0 0.12 for a 45 degree angle? It should be closer to -0.11 0 0.11.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #395 on: November 03, 2017, 07:35:05 PM »
Are you sure you were seeing -0.01 0 0.12 for a 45 degree angle? It should be closer to -0.11 0 0.11.

Oh, no, sorry, that was pulled out of memory and not from an actual log. I'm sure it probably was more like -0.11, 0, 0.12 (since the 45 degrees was eyeballing).

Thanks for all the movement info. That will help a lot.

I'm not too concerned with being deadly accurate. I chose a 5-world-unit stopping zone, like Arcana did to avoid the kinds of collisions you mentioned. I've also been reading some interesting articles about steering forces steering forces that I'm going to try to incorporate into Rover to make his following/escaping behaviors more interesting.

Of course, now Rover doesn't walk through walls any more, heh. He gets hung up on terrain until he decides it's time to update his own position. I'm going to have to revisit the whole conversation about beacons and paths before all is said and done.

Codewalker

  • Hero of the City
  • Titan Network Admin
  • Elite Boss
  • *****
  • Posts: 2,740
  • Moar Dots!
Re: Technical side discussion
« Reply #396 on: November 03, 2017, 08:20:24 PM »
Eventually some sort of bot API on the paragon chat client itself rather than speaking XMPP directly will be the way to go, for two reasons:

  • At some point in the medium term I'm going to be reworking how movement works. Meta channel position broadcasts will be reduced to only happen every few minutes - or after moving a large distance - just enough for everyone to get a very rough idea where other players are.

    A compressed binary stream containing timecoded movement data will be sent to a private pubsub node. Clients are expected to subscribe to the pubsub node of other players that they care about getting fine-grained movement/animation data for (i.e. are in visual range).

    Not only does playing back timecoded data result in much smoother movement than the velocity prediction system, it avoids everyone spamming the meta channel with their position 1-2 times per second. No reason to waste bandwidth getting movement updates for someone on the other side of the zone you can't see anyway.

    For at least a few versions the old protocol will still be supported as a fallback if clients older than a certain protocol version are in the zone.

    That's not impossible for a bot to learn to speak, but a lot more work. Using an API would free the bot author from having to deal with it.

  • It would be a lot easier to write bots that move around convincingly if collision info was available. An API would allow bots to get a feel for their surroundings, using either complex methods like pulling geometry, or simple ones like a call to cast a ray and see where it collides with the world.

That's still some distance off, so for the moment what you're doing may not be the best way - it's just the only way. :)

Oh, no, sorry, that was pulled out of memory and not from an actual log. I'm sure it probably was more like -0.11, 0, 0.12 (since the 45 degrees was eyeballing).

That sounds about right. -0.11 0 0.12 is a vector with length of 0.162 (a2 = x2 + y2 + z2), which is within the expected error range since the XMPP values are rounded to hundredths.

Since you said you got the velocity otherwise working I'm assuming you've got the math for moving around in a plane down (cos(a), 0, sin(a)) and just needed the right velocity for different types of movement.

Arcana

  • Sultaness of Stats
  • Elite Boss
  • *****
  • Posts: 3,672
Re: Technical side discussion
« Reply #397 on: November 03, 2017, 10:03:06 PM »
I'm happy to see this project is still alive.  It seems my brain can only hold one dev project at a time without exploding these days, and I've been continuously distracted by work stuff which actually pays money so I can make a big pile of it and retire. 

At some point I would like to return to this stuff, but it might not be until after Codewalker releases an API for it.  But keep on keeping on: I will be following along, vicariously part of the robot revolution.

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #398 on: November 04, 2017, 05:15:00 PM »
Since you said you got the velocity otherwise working I'm assuming you've got the math for moving around in a plane down (cos(a), 0, sin(a)) and just needed the right velocity for different types of movement.

So far, so good. We'll see what happens the day Rover straps on a jingle jet. :-p

slickriptide

  • Elite Boss
  • *****
  • Posts: 356
Re: Technical side discussion
« Reply #399 on: November 08, 2017, 07:18:13 PM »
Rover is following nicely now, though he's still a little jumpy at high speeds and occasionally he forgets to turn off his velocity when he comes to a stop. Not sure what that last is about, yet. The scaling between Rover's clock and the real clock is probably part of the issue. I might have to look into what's causing the Errbot clock to run slow and see if I can correct it for my needs. Errbot already has thread management in place and I'd rather leverage that than have to roll my own to handle timer interrupts.

Next step is to take all of my ad-hoc movement code and turn it into methods on my avatar class. That will make it easier to assign movement instructions and follow/flee targets to individual avatars, and it will put movement processing where it really belongs - on the avatar that's doing the moving. When that's done, I'll add in methods to implement steering behaviors. The end game here is being able to assign Rover a seek target AND a flee target, so that he moves towards one thing while avoiding another thing (presumably a player avatar) as part of a single movement computation.

Looking towards the future - Navigation is the big problem to be solved.

Do we have schemas for the city maps, like city_01_01.bin? Never mind - that's geobin.xml, yes?

I'm conflicted about having to roll my own virtual file system to read the map and costume files. On the one hand, there are open source pigg viewers that can provide examples of what to do, but ultimately the question is whether it matters? There'll never be an issue 25. Short of modding the game client, the data in the pigg files is never going to change, so why not just extract the data I care about and put it in a SQL database where I can deal with it on my own terms?

Courtesy of bindump, I find the following info in city_01_01.bin:

294 groups named 'Omni/_NAV'. Each of them has beacon data embedded that parses out of bindump like so:
Code: [Select]
    Beacon =
        -------- Beacon 0 --------
        u080 = BasicBeacon
        u081 = 35.6909
I have no idea what that u081 number represents. Codewalker? Arcana? Any ideas? Distance to another beacon, maybe, or some bit-coded data? None of them has position data listed for the beacon; these are just flags that say that the beacon exists (and whatever u081 represents).

Defs 415 seems to be the list of paths and beacons. It's a list of nearly 1400 "groups" that looks like this:
Code: [Select]
    -------- Defs 415 --------
    Name = grp3953
    Group =
        -------- Group 0 --------
        Name = Omni/NAV
        Pos =
            X = -909.5
            Y = -0.447678
            Z = 1235.75
        PYR =
            Pitch = 0
            Yaw = 0
            Roll = 0
        Flags = 0
        -------- Group 1 --------
        Name = grp105167
        Pos =
            X = 644
            Y = 41.4434
            Z = -746.75
        PYR =
            Pitch = 0
            Yaw = 0
            Roll = 0
        Flags = 0
        -------- Group 2 --------
        Name = Omni/NAV
        Pos =
            X = 643.5
            Y = 41.4434
            Z = -773.75
        PYR =
            Pitch = 0
            Yaw = 0
            Roll = 0
        Flags = 0
        -------- Group 3 --------
        Name = Omni/NAV
        Pos =
            X = 633.5
            Y = 41.5527
            Z = -803.75
        PYR =
            Pitch = 0
            Yaw = 0
            Roll = 0
        Flags = 0

"grp105167" is a basic beacon. "Omni/NAV" is, presumably, a point on a path from one beacon to another. How the paths run between beacons isn't immediately obvious. Individual basic beacons are sometimes referenced multiple times at different absolute positions, so it seems that those can't actually be "absolute" positions for any given beacon.

Any hints on how to make sense of the paths is appreciated. Did I read someplace that there's a dev command in Icon that will make the paths visible?