Getting frame buffer data

Questions or HOWTOs about the above? Post 'em here...
Post Reply
User avatar
SpringTank
Private First Class
Private First Class
Posts: 38
Joined: Sun Nov 02, 2008 11:11 pm
Location: Texas
Contact:

Getting frame buffer data

Post by SpringTank »

I've been tossing around the idea of a neural network based AI for bzflag lately for private servers, machine learning experiments, and so some players have a ruthless challenge to practice against. I have a few other experiments I've tried in the past using neural nets and genetic algos to some success. https://github.com/RingingResonance/GeneticAlg

I've searched and searched through the clients source code and I can't seem to figure out where the frame buffer is accessed. I tried looking at the source for the screen shot portion of the code and can't seem to figure that out either. My question is, how would I be able to get frame buffer data for bzflag without using an external application? The idea being that the AI will only use screen data and can only control the client through basic movement controls.
User avatar
JeffM
Staff Sergeant
Staff Sergeant
Posts: 5193
Joined: Fri Dec 13, 2002 4:11 am
Location: https://github.com/OmniTanks
Contact:

Re: Getting frame buffer data

Post by JeffM »

There isn't an easy way. The game uses OpenGL, OpenGL does all the drawing directly on the video card where it will use one or more buffers. The game code itself doesn't actually access the frame buffer for drawing.

You are going to have a very hard time getting direct frame buffer access in the current BZFlag codebase. The screenshot code is VERY slow as it calls glReadPixels, you should not expect to be able to do that at anything like the runtime frame rate. This happens in clientCommands.cxx:cmdScreenshot. the call to glReadPixels(0, 0, w, h, GL_RGB, GL_UNSIGNED_BYTE, ssdata->pixels); makes a copy of the frame data into the pixels buffer. The actual framebuffer will be on the video card and you don't have direct access to it.

Frame buffer access from the CPU in modern hardware accelerated systems is painfully slow. For thing that do need that kind of frame buffer access on modern (like last 10 years) the best thing to do is to have your rendering system render to a texture/buffer, then have a thread peel off that texture data while the rest of the game runs.
ImageJeffM
User avatar
SpringTank
Private First Class
Private First Class
Posts: 38
Joined: Sun Nov 02, 2008 11:11 pm
Location: Texas
Contact:

Re: Getting frame buffer data

Post by SpringTank »

That's what I was afraid of. Do you think the current rendering pipeline in bzflag could render to a texture buffer? I can imagine it would probably require a rewrite of the rendering engine with modern opengl. If it could, it would open up some options for a few other ideas that I have that are mostly unrelated to my AI project.
User avatar
JeffM
Staff Sergeant
Staff Sergeant
Posts: 5193
Joined: Fri Dec 13, 2002 4:11 am
Location: https://github.com/OmniTanks
Contact:

Re: Getting frame buffer data

Post by JeffM »

The rendering pipeline needs a rewritten anyway. Some people are working on updating it to slightly more modern standards, but no mater what you are talking about some decent changes and an upgrade in the required OpenGL version. I'm sure it's possible but it'll be a bit of work.
ImageJeffM
User avatar
SpringTank
Private First Class
Private First Class
Posts: 38
Joined: Sun Nov 02, 2008 11:11 pm
Location: Texas
Contact:

Re: Getting frame buffer data

Post by SpringTank »

Rewriting the render pipeline is something I've always wanted to help with but I've never had any experience with render pipelines or opengl before. I guess a good first step would be to remove all current opengl calls in the client and get it to a state where it will compile and run without any graphics? Other than that I have no idea where to even start.
User avatar
macsforme
General
General
Posts: 2033
Joined: Wed Mar 01, 2006 5:43 am

Re: Getting frame buffer data

Post by macsforme »

Look for where the game renders to an MSAA-enabled render buffer for anti-aliasing. You could convert that to render to a texture pretty easily (although the OpenGL requirements would be higher). I think only the 3D view, not the HUD, get passed through there, though.
User avatar
JeffM
Staff Sergeant
Staff Sergeant
Posts: 5193
Joined: Fri Dec 13, 2002 4:11 am
Location: https://github.com/OmniTanks
Contact:

Re: Getting frame buffer data

Post by JeffM »

TheFish wrote: Sun Aug 30, 2020 10:39 pm Rewriting the render pipeline is something I've always wanted to help with but I've never had any experience with render pipelines or opengl before. I guess a good first step would be to remove all current opengl calls in the client and get it to a state where it will compile and run without any graphics? Other than that I have no idea where to even start.
The OpenGL code is spread out all over the codebase, removing it will be a chore, macsforme's suggestion will be the shorter path, but will still involve adding new GL code.
ImageJeffM
User avatar
SpringTank
Private First Class
Private First Class
Posts: 38
Joined: Sun Nov 02, 2008 11:11 pm
Location: Texas
Contact:

Re: Getting frame buffer data

Post by SpringTank »

That's a good idea. I suppose I could also recreate the radar using basic software rendering as it's just simple shapes and doesn't have to be that high fidelity. Thanks.
Post Reply