Checking in

Improvements, Problems, or Questions on the Forum format or software go here ( not about the game )
Post Reply
User avatar
Tim Riker
Maintainer
Maintainer
Posts: 15
Joined: Thu Dec 02, 2004 5:33 pm
Location: USA, Utah, Salt Lake City
Contact:

Checking in

Post by Tim Riker »

Blast from the past here,

How are things in the BZFlag world? What's happened with the bzflag list server lately? The json and lua versions of the list say they are text/plain instead of their respective Content-Types. Where is the db source these days?

What happened with the bzauthd project? Was OAuth2 considered? This could issue a client jwt which would then not need to be revalidated for each server connection. Of course, that makes it reusable, so that might not be a step forward.

Is @blast still running the list server and such? It looks like I don't have server admin access anymore. @blast, are you still happy to keep hosting things? Do we have a backup plan?

Looks like phpBB could use an update from 3.3.2 to 3.3.9. I'd be happy to work on that.

It looks like MediaWiki is locked. I can't login there. It could also use an update from 1.26.3 to 1.39.1 which could add the Visual Editor. The home page still points to 2.4.10 as the download version.

https://wiki.bzflag.org/Main_Page

I created a Google Translate plugin for both phpBB and MediaWiki that I could install here. It translates the current page to the user's selected language. I have no clue how many non-english users or posts we are getting currently. Is that of interest? It looks like the Forum only has British English installed.

Did anyone add IPv6 support? I'd like to do that at some point. Server operators should be able to run on a machine that only has IPv6 connectivity. Some thought needed here. How does the server register? Does it list all it's IPs? what happens if it has more than one IPv4 or IPv6 address? Do we validate the dns name given? The list server should probably not give IPv6 servers out to ipv4 only clients, but it probably should give ipv4 servers out to ipv6 clients. Should bans for ipv6 clients assume some larger mask? /64 at least? Do we show ::ffff:<ipv4> versions of client addresses in the list, or the native ipv4 everyone is used to?

I have server space I could host things on. I'll probably setup a server again just to get more involved again.

I wonder how much work these projects would be:
  • BZFlag in Unity for mobile, etc.
  • BZFlag in WebVR, perhaps in A-Frame or something similar.
There used to be a web interface to preview maps around, wasn't there? Is there still? I version on the main website that would list current servers and load and display all the maps would be very cool.

I hopped on to IRC. I'll try to be there sometimes. Looks like I have no perms there anymore since the move off FreeNode years ago.

Do we have a Discord server? I really like having a Minecraft admin channel tied to discord so I can promote/demote/kick/ban without having to join the server or ssh in.

Has crs32 been seen in recent history? I think he's been at Pixar for over 20 years now.

https://www.imdb.com/name/nm1674196/
Tim Riker - http://rikers.org/ - Tim@Rikers.org
BZFlag maintainer - http://BZFlag.org/ - for fun!
User avatar
blast
General
General
Posts: 4900
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Checking in

Post by blast »

bzfls

I don't have the current changes to bzfls in a repo at the moment. We have an archive of an older version in the bzflag-web repo on GitHub. Not too much has changed, but I should still get the changes published somewhere. Currently the lua format is being used by Strayer's BZStats and the JSON format is being used by The Noah's site (probably for BZList).

I have at least the start of a design for a list server replacement and I documented the behavior of the current server list.


Authentication

I've looked at OpenID Connect (which is based on OAuth 2) to replace our weblogin system. For that, I experimented with using the Hydra product from Ory, and might look into some of their other products. I don't know if OpenID Connect would work for in-game authentcation since it's mostly focused on web browsers.

For the client I think that would change to having a login process that would generate a session ID, which would be stored in the config instead of the password. That could also be invalidated by the user.


Forum and Wiki

I can work on the phpBB upgrade this week. The wiki is in read-only mode since we're trying to put documention and information into the main site instead of a wiki. As for the translation plugin, I don't really see much of a user for that. Web browsers can translate pages themselves (though Firefox may have it disabled by default).


IPv6

Nope, no IPv6 support yet. Currently the authentication system restricts a token to an IP address, so that doesn't mix well with IPv6 or CGNAT. A simple-ish replacement for that for 2.6 would be to bind a token to a specific game server instance instead of the player IP. That would have the same benefit, which is preventing a rogue server operator from stealing a player token and using it to impersonate the player on another server.

The servers running our web services all have IPv6 addresses. I've made some attempts at implementing IPv6 support in the game. It's been a while since that so I can't recall how far I got on it.

I think some of the areas that needed to be updated, beyond the normal listen and connect socket stuff, were:
  • Local server discovery (Ping Packet?) - I was looking at replacing our multicast discovery with mDNS/DNS-SD
  • Bans - Having IPv6 bans in the ban file, updating commands, and ideally having some UI for creating and managing bans so that we do not have to hand-type IPv6 addresses... And yes, the default ban would probably have a
  • ServerId - We currently have a ServerId type that contains an IPv4 address and a port number. In one of my attempts, at least, I was either getting rid of ServerId or switching it to use a hostname instead of an IP.
  • Server List - The current server list format includes an IPv4 address in the response. As far as I know, this isn't actually used anyway.
I have some utility functions created for converting a string to an IP, an IP to a string, and checking if a subnet contains an IP, and these should work with both v4 and v6 addresses.


Misc

There was some work a while ago by macsforme/Constitution to port the graphics to OpenGL ES so that it could run on mobile devices.
https://github.com/macsforme/bzflag-embedded/

I was also experimenting with an UI library called RmlUi that uses an HTML and CSS like markup to define the UI. I managed to get a mockup of the menu and the in-game UI working, including having the menu work with mouse, keyboard, and gamepad support.
Mouse/keyboard support
Gamepad support
Keyboard support
Chat message examples
Scoreboard rendering and text outline on the chat message names:
Image

As for rendering map thumbnails, there is bzflag-rendering.php by allejo that can handle some maps. My proposed central services API also includes the world hash as part of the server list response, so either by a hash lookup or by adding additional fields, we could show a map thumbnail and/or map screenshots in the server list.

Edit: There's also a web-based map editor being created (Web-BZEdit by allejo) that could also be modified or repurposed to render 3D previews of worlds.

Chat networks

IRC is still the main development discussions. We also have a Matrix bridge set up for both of our IRC channels, linked from our site.

There is a Discord server for the league, Leagues United.
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image
User avatar
tainn
Private First Class
Private First Class
Posts: 273
Joined: Sun Nov 18, 2018 7:25 pm
Location: zone of the phantoms

Re: Checking in

Post by tainn »

To expand on the Discord server segment, it has indeed started off as a Leagues United server, but is now considered a general hub for anything and everything BZ-related, minus the core dev that's still on the IRC. Perhaps we ought to create a bridge between the two at one point. Albeit I'm unsure if everyone on the IRC will be okay with that, considering Discord does not provide end-to-end encryption and has other privacy-related concerns.

The invite link is currently not passed around publicly, and I would recommend messaging kierra for it.
User avatar
Tim Riker
Maintainer
Maintainer
Posts: 15
Joined: Thu Dec 02, 2004 5:33 pm
Location: USA, Utah, Salt Lake City
Contact:

Re: Checking in

Post by Tim Riker »

I think it would be nice to have a server list and other stats right on the website where people could select a server and see the map online. allejo's map viewer or something similar might be a good start.

If we're updating the list server, delivering json instead of plain text sounds like a good move. The json there, however, does not pass validation.

curl -s https://my.bzflag.org/db/?action=LIST\& ... at=json|jq .
parse error: Invalid escape at line 21, column 149

The "example's server" is rendered as "example\'s server" which is not valid json.

Also, if moving to json, splitting up the hexcode into fields probably makes sense so that portion is human readable. We might consider some defaults so that players:0 does not need to be sent for all the empty servers. Does the client request compression for the server list? Web browsers can do gzip as well as probably more effective br compression.

The raw server list is around 25k, with gzip compression it's around 5k. I wonder what it would be in json with all fields expanded and br or gzip compressed? Check the list server code in someplace, and I'll experiment.

The list login process gets a token at the same time as the client is requesting the list of servers. This is why I didn't make the token server specific. We don't yet know what server the user wants to connect to. If the token were made server specific, there would be at least one more web request. I guess that's not the end of the world.

This means that if the user requests a server on the command line when starting the app, we still get the server list before connecting so that we can get a token.

In Oauth2 client flow, the site gives the user a nonce as well as it's clientid. The client then contacts the oauth provider with that nonce and clientid, logs in and gets a token signed for that specific clientid. We could do the same. Starting up bzfs might have to somehow generate a clientid (for the server) and confirm it's secret with the oauth provider so that the jwt issued by the server could be validated by that bzfs clientid.

I wonder how much overhead adding SSL/TLS to server connections would be.

It would be interesting to implement the client/server connection as http2 with QUIC and see what overhead that introduces. More requests would get through firewalls, but the https overhead would add up even in binary http2. http2 support could use server push to push out server events.

With all that implemented, it's very likely that site outbound traffic would block the QUIC and probably block server side http2 pushes too.

My MRTG polling script was broken back in August. I think I've fixed it now.

https://list.bzflag.org/bzflag.html

I think it would be nice to merge stats into the main site too if the developers of the stats pages are willing.

http://bzstats.strayer.de/ et. al.

Server, player and league lists with links to the phpBB user for users, etc.

Looks like his site still links to svn. Would be nice to link to git now.
Tim Riker - http://rikers.org/ - Tim@Rikers.org
BZFlag maintainer - http://BZFlag.org/ - for fun!
User avatar
Zehra
Private First Class
Private First Class
Posts: 811
Joined: Sun Oct 18, 2015 3:36 pm
Location: Arctic
Contact:

Re: Checking in

Post by Zehra »

To expand on IPv6/authentication, Wiki, and also preview of maps on the list server.

On IPv6/authentication, myself and a few others have noticed it being where on mobile carriers/mobile hotspots, there is some issues with authentication. (It's gotten different results according to different players, but I'd say it's due to the varying speed of the rollout.)

On the Wiki, I can summarize most of what the pages are related to in terms of overall content. BZFS API, BZW related or specific, Map Editor(s), Plugin(s), BZFlag version release. All other information generally relates to either a map, project idea/concept, todo list, tutorial, league. Mostly everything can be entirely summarized via official documentation, similar to the BZFS API documentation: www.bzflag.org/documentation/developer/bzfs_api, hence there not being much of a reason to keep it in the long run.

On the preview of maps on list server, BZList.net created by The Noah does something similar on some maps, by having screenshots manually taken of some maps. So it's easy to imagine something similar providing radar views of maps as well.

--

Relating to map previews and similar, since it seems either way, for a full map preview which isn't simply an image, the map representation would need to be sent, it seems a plug-in similar to Fastmap could be used for the purpose of providing a preview world.(This could be a trick till it could be made an actual feature too.)

If we would see a return of an official stats site, could we possibly see some APIs for it?
(For reference: I use bzstats.strayer.de/stuff/AkosToth2.php and bzstats.strayer.de/stuff/LePoulpe303.php for player info and related.)

-Zehra
Those who are critical of me, I'll likely be the same of them. ~Zehra
The decisions we make are the ones we look forward too and the ones we regret. ~Zehra
There's a difference between knowing my name and knowing me, one shows respect to my name and the other is to who I am. ~Zehra

See where I've last been active at Strayers.
Visit BZList.net for a modern HTML5 server stats site.

Click here to view the 101 Leaderboard & Score Summaries Last updated 2021-01-12 (YYYY-MM-DD)
Latest 101 thread
User avatar
blast
General
General
Posts: 4900
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Checking in

Post by blast »

Yeah, I don't like either of the alternative list formats we have right now. I'd go for a proper JSON encoding with named values for each server. I'm not yet sure I want to move away from the hexcode game info. I suppose if we did, we could have the game server send that as a JSON object and then store that as a jsonb column in Postgresql. That would further inflate the data size of the server list response though.
Zehra wrote: Tue Jan 17, 2023 12:47 am On IPv6/authentication, myself and a few others have noticed it being where on mobile carriers/mobile hotspots, there is some issues with authentication. (It's gotten different results according to different players, but I'd say it's due to the varying speed of the rollout.)
With mobile connections, I think most of them put you behind CGNAT (Carrier Grade Network Address Translation), where a smaller pool of addresses is shared among many customers. The address used to make a request to one internet host can be different than when accessing another internet host. So that causes problems since your client might access the server list from one address and the game server from another address.
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image
User avatar
Tim Riker
Maintainer
Maintainer
Posts: 15
Joined: Thu Dec 02, 2004 5:33 pm
Location: USA, Utah, Salt Lake City
Contact:

Re: Checking in

Post by Tim Riker »

Re NAT
At my office we have 4 outbound routers doing NAT, so any given tcp/udp connection might come from any of the four IPs when it hits an external server. We try to pin a client to one external IP so that sites that don't handle the IP changing work, but the pinning not guaranteed.

The client could send IPs to add to the list, but the client often does not know the ips. It knows the rfc-1918 address for IPv4, which the site/server won't see. The token request could be over either ipv4 or ipv6 so the issued token would only match one of those.

One way around this is what blast references. Require the user to request a token prior to connecting to any server, then issue a server specific token to the user.

That's extra overhead on the list server to be sure, but we might look at logs for that to track server usage? There are options. Alas, that means even more overhead on the list server.

I'd have to test fielded json with compression vs hexcode and see what we end up with. I think we are likely to optimize the server fields in the db so that server lists are easier to generate and send. I don't recall if we do any caching there now. In theory we could cache the entire list and only update the cached json once every 30 seconds or so. Or update the cached data only for a server that checks in, when that server checks in. We could then update the entire cached json once in a while in case the list server misses a change.
Not sure where we'd store the cached data. memcached? or one sql blob? etc. We could play with options.
Zehra wrote: Tue Jan 17, 2023 12:47 am a plug-in similar to Fastmap could be used
What does the Fastmap plugin do?
Tim Riker - http://rikers.org/ - Tim@Rikers.org
BZFlag maintainer - http://BZFlag.org/ - for fun!
User avatar
Zehra
Private First Class
Private First Class
Posts: 811
Joined: Sun Oct 18, 2015 3:36 pm
Location: Arctic
Contact:

Re: Checking in

Post by Zehra »

Tim Riker wrote: Tue Jan 17, 2023 3:30 am What does the Fastmap plugin do?
It serves a cache of a map via HTTP.

Readme has more technical details: github.com/BZFlag-Dev/bzflag/blob/2.4/plugins/fastmap/README.fastmap.txt

I have the idea that maybe for a quick proof of concept of map previews, a server runs something similar to Fastmap, but it's sole purpose is for providing/giving map previews. Instead of serving a cache, the plug-in hosts two files it generates from the actual map, one being a simple thumbnail of the radar view, while the other is a slightly stripped down version of map solely meant for 3D previews of the world.(Or for 3D preview, it could do something similar to Fastmap, but serve a plain-text bzw without zones/comments..etc.)

A basic outline of how it could work:

Code: Select all

Plugin generates thumbnail and minified 3D map preview.
Plugin makes request to list server (or map preview listings) ->
List server or map preview listings makes request to server for map thumbnail file and 3D map file preview <-
List server or map preview listings now has thumbnail of preview and URL for 3D preview.
It would be simply a proof of concept/prototype.

-Zehra
Those who are critical of me, I'll likely be the same of them. ~Zehra
The decisions we make are the ones we look forward too and the ones we regret. ~Zehra
There's a difference between knowing my name and knowing me, one shows respect to my name and the other is to who I am. ~Zehra

See where I've last been active at Strayers.
Visit BZList.net for a modern HTML5 server stats site.

Click here to view the 101 Leaderboard & Score Summaries Last updated 2021-01-12 (YYYY-MM-DD)
Latest 101 thread
User avatar
blast
General
General
Posts: 4900
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Checking in

Post by blast »

I wouldn't really worry about the overhead too much. We actually have extra overhead right now with the current implementation because we don't actually use the GETTOKEN action in the client. The client always requests the whole server list even if they're just asking for a new token.

The protocol in 2.4 allows bzfs to act either as a game server or a simple HTTP server. So fastmap is a plugin that serves the map cache via the built-in HTTP server. Note that we've removed fastmap (and HTTP server support) in the master branch since we resolved the issue that caused slow map downloads. In 2.4 and earlier releases, bzfs sends a 1KB chunk of the map and then waits for the client to acknowledge that it received that. That meant that the higher the latency to the server, the slower the map would download.
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image
User avatar
Tim Riker
Maintainer
Maintainer
Posts: 15
Joined: Thu Dec 02, 2004 5:33 pm
Location: USA, Utah, Salt Lake City
Contact:

Re: Checking in

Post by Tim Riker »

So is the mini-http server there or gone? It might be nice to deliver a web page that has the current scoreboard etc.

When it was delivering the map, what format is/was the map in?
Tim Riker - http://rikers.org/ - Tim@Rikers.org
BZFlag maintainer - http://BZFlag.org/ - for fun!
User avatar
Tim Riker
Maintainer
Maintainer
Posts: 15
Joined: Thu Dec 02, 2004 5:33 pm
Location: USA, Utah, Salt Lake City
Contact:

Re: Checking in

Post by Tim Riker »

The Getting Started page is a great start.

https://www.bzflag.org/getting-started/

Are there more things to include here? There are other online references, Google finds this:

http://www.kirkanddonna.com/kirk/bzflag/BZFlagKeys.htm

It would be nice to capture all that kind of information and refer to it in the Getting Started page. The key assignments area in the app shows many more keys. It would be nice to explain each of those and list the default assignments.
Tim Riker - http://rikers.org/ - Tim@Rikers.org
BZFlag maintainer - http://BZFlag.org/ - for fun!
User avatar
blast
General
General
Posts: 4900
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Checking in

Post by blast »

The mini-http server is gone in master. Fastmap served the same binary version of the map that bzfs sends in-protocol, which is also what the -cacheout option uses (which can be used to write out a cache file that can be placed on an HTTP server and referenced with -cache).

The getting started guide was intended to cover the essentials. If there are some essential keys or other useful information missing we could add them. I could also see having a page that covered all the default mappings. A new file could be added under the user documentation section for that.

Regarding moving wiki information to the main site, we do have a GitHub project for that: https://github.com/BZFlag-Dev/bzflag.org/projects/1
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image
Post Reply