Lagstats in replays

Discussion for GU League Players
User avatar
blast
General
General
Posts: 4732
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Lagstats in replays

Post by blast » Tue Feb 17, 2009 7:44 am

Replays basically record the raw network data. Having a replay on the client side would not be useful because you can't trust the client, and thus defeats the purpose of this.
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image

User avatar
FiringSquad
Sergeant
Sergeant
Posts: 847
Joined: Thu Jan 26, 2006 5:53 pm
Location: Ireland

Re: Lagstats in replays

Post by FiringSquad » Wed Feb 18, 2009 2:36 pm

Of course anything is possible, but faking believable raw network data would be very difficult.
Take for example the simple cheat where the tank dimensions are altered in the client to make it slightly smaller than a normal tank.
If someone replays this match they would record a hit for times that the cheating client records a miss.
In order to get around this they would either have to move their tank in a physically possible way or alter the timing of data packets in a way that would make the shots miss.
Either way this would be very difficult to do, in a way that is not detectable.

User avatar
blast
General
General
Posts: 4732
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Lagstats in replays

Post by blast » Wed Feb 18, 2009 3:59 pm

It would not be hard at all. You would just have to change the values that were packed into the network messages. So it's one of those changes that only works for 'clean players', and thus defeats the purpose and would be wasted effort.
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image

User avatar
FiringSquad
Sergeant
Sergeant
Posts: 847
Joined: Thu Jan 26, 2006 5:53 pm
Location: Ireland

Re: Lagstats in replays

Post by FiringSquad » Wed Feb 18, 2009 5:26 pm

Think about it. If you squeeze in between two bullets that no normal tank could fit through, what packets would you change so that this could not be detected?

Remember the altered values would not match the server's values for these packets either.

I know the standard mantra is "client code is open source so not cheat detection is possible", but this is nonsense. It's only enough to make cheating sufficiently complex that it becomes too difficult to contemplate. I think a client log that just records tank position, speed & orientation and shot speed, direction & location would even be enough. This would be a far easier log file to manipulate and yet even this would be difficult to do in an undetectable way.

What would make it even more difficult to circumvent is a CRC value for the current log that is sent along with other data to the server. That way the file could not be cleaned up afterwards, instead it would have to be created on-the-fly with modified values within it.

I really think this might work.

ts
Dev Monkey
Dev Monkey
Posts: 970
Joined: Fri Jan 14, 2005 6:26 pm

Re: Lagstats in replays

Post by ts » Wed Feb 18, 2009 5:48 pm

blast wrote:So it's one of those changes that only works for 'clean players', and thus defeats the purpose and would be wasted effort.
I wouldn't say it is wasted because then one could get a more accurate understanding of the influence of lag by simply looking at the different worlds (each player is in its own world because each could see different versions of the same events) and thus have a nice visualisation of the differences. Of course only in case each player is clean.

The downsides would be bigger changes of the replay system, more than double the network usage and it can obviously be manipulated. For instance the shots are done using plain UDP, a single lost packet can force the client not to see a bullet. From my point of view the shots need some protocol yet to be specified (there are reserved values for protocol but the protocols are not specified) or we modify the game logic to confirm such a packet is received and resent it in case it was lost. This would mean the bullets may be fired late in the opponent's view. Another way to deal with it is double or triple (or even more redundant) sending UDP messages. That is a way to deal with lost packages borrowed from the quake engine. However don't get it wrong: Quake engine computes kills server side.


Pro:
bzlogview for every server that supports it
better understanding of influences of lag
better graphics than in bzlogview, always correct map visible
may make cheats more obvious

Cons:
can be forged
would more than double the network usage
big changes in replay system needed



FiringSquad wrote:What would make it even more difficult to circumvent is a CRC value for the current log that is sent along with other data to the server. That way the file could not be cleaned up afterwards, instead it would have to be created on-the-fly with modified values within it.
A CRC value would need to be computed by the client and thus modifications could access that data or even change it to fit what was broadcast.
GU league: http://www.guleague.org/
An introduction to TCP: http://www.lafkon.net/tc/

User avatar
blast
General
General
Posts: 4732
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Lagstats in replays

Post by blast » Wed Feb 18, 2009 10:11 pm

FiringSquad wrote:Remember the altered values would not match the server's values for these packets either.
Of course they would. The client is the one sending them to the server, after all.
FiringSquad wrote:I know the standard mantra is "client code is open source so not cheat detection is possible", but this is nonsense. It's only enough to make cheating sufficiently complex that it becomes too difficult to contemplate. I think a client log that just records tank position, speed & orientation and shot speed, direction & location would even be enough. This would be a far easier log file to manipulate and yet even this would be difficult to do in an undetectable way.
As much work as it would take to implement your idea, it would be circumvented in a matter of minutes or hours.

And the CRC idea is just as pointless. You're trusting the client to send a non-faked CRC.


If you guys want to get an idea of how lag works from a client view, you're free to write your own system to record the data or to use something like WireShark, but it will likely not be something that will be developed into BZFlag officially.
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image

dexter
Private First Class
Private First Class
Posts: 212
Joined: Sun Apr 30, 2006 7:36 am
Location: Germany :o)

Re: Lagstats in replays

Post by dexter » Thu Feb 19, 2009 9:44 pm

ts wrote:
slime wrote:A great trait our admins have is COMMON SENSE.
Honestly you have no chance to know. Most decisions are kept secret and when a ban is issued for example it may not be announced or the exact reasons are not revealed to the public. I have a very different opinion regarding the definition of a cheat. Anyway, banning based on suspicions alone is something I don't recommend, either.
So we have no idea of knowing that the gu admins have common sense? Speak for yourself, but the ones I know do posses common sense and quite a bit of it, for that matter. And thanks for taking the thread off-topic..

I heard that snick wrote a little patch that would show if lag was high or low at specific intervals in replays. Not the exact values but something like an estimate or just some form of "high" or "low". That way we would at least have an estimate of what the lag is.
I wanted to thank him for investing time in this problem and coming to some sort of solution, I'd be eager to test it out if I got the code. :p

smoooth
Private First Class
Private First Class
Posts: 81
Joined: Sun Aug 05, 2007 6:13 pm

Re: Lagstats in replays

Post by smoooth » Fri Feb 20, 2009 12:15 am

Well these are all very good ideas.

However we have to get one thing straight. This is basic logic.

Either the replays represent what happened for they don't. If they do not then I can walk thru every bullet and no one can say anything. If the replays do represent what happened then thats where statistics can come in to play.

I don't know about the rest of you but typically what I see in replay is pretty damn close to what actually occurred and in most cases its dead on. Hence when you see someone type "?" or "How'd that miss" you can go back and look and see the purported thru.

Now take that data and start compiling information about players. I've done it to a certain degree. I am telling you what there is a "normal" amount of thrus. Then there are some players who are astronomically higher -- even after you correlate lag and jitter.

This can be an effective means of recognizing potential cheaters. First everyone has to agree that replays are very close (not 100%) to what happened. You can't have it both way. Either replays are realistic data or they aren't -- period.

smoooth

ts
Dev Monkey
Dev Monkey
Posts: 970
Joined: Fri Jan 14, 2005 6:26 pm

Re: Lagstats in replays

Post by ts » Fri Feb 20, 2009 12:53 am

smoooth wrote:Either the replays represent what happened for they don't. If they do not then I can walk thru every bullet and no one can say anything. If the replays do represent what happened then thats where statistics can come in to play.
The replays do not represent what happened exactly. If you "walk thru every bullet" conclusions depend on how the bullets go through you and which lag you and your opponent had. Also one would rather expect a cheat in handy situations.
GU league: http://www.guleague.org/
An introduction to TCP: http://www.lafkon.net/tc/

smoooth
Private First Class
Private First Class
Posts: 81
Joined: Sun Aug 05, 2007 6:13 pm

Re: Lagstats in replays

Post by smoooth » Fri Feb 20, 2009 3:16 am

ts - you're proving my point exactly.

How do you "suspect a handy cheat" when the replay doesn't represent what actually happened.

You can't say

1. Tank locations and shots in replays are always false.

and then say

2. If a tank walks thru all shots on replay then they are cheating.

Like saying
A is B
B is C
but A is not C

This is impossible.

So one of the two has to give way. Either the replays are, to some degree, accurate representations of the events, or they are not. If they are not then nothing can be proven with them. (Not even a player who appears to walk thru every single bullet)

I can see valid arguements from either side but the problem is, one can't pick and chose when to validate replay information. It is valid -- or it isn't.

smoooth

User avatar
Mucho Maas
Private First Class
Private First Class
Posts: 515
Joined: Tue Sep 21, 2004 5:14 pm

Re: Lagstats in replays

Post by Mucho Maas » Fri Feb 20, 2009 11:04 am

smoooth wrote:I don't know about the rest of you but typically what I see in replay is pretty damn close to what actually occurred and in most cases its dead on. Hence when you see someone type "?" or "How'd that miss" you can go back and look and see the purported thru.
My experience is that of 30 "?" and "How'd that miss", 10 were actually through a player while the player was moving away (so most likely to have been lag) and 1 was a shot through with an unmoving player (or a player not moving away from the bullet). So that's 1 out of 30 situations with meaningful information. That I don't hold as statistical significant. If you really get to really high values for a specific player, I would like to know about it.
Probably you are doing it already, but it would suggest to note of these really obvious shot through a record of which replay (replay name and server) it was, and at what time in the game they occurred. And then present that list. If it results into one situation every 3rd match for that player, we would deem it as not significant. If there are a couple of such situations per match, it will be really worth going through the matter.

For this scenario, I would say the replays are very helpful. And that's how we learned we can use replays.
"meet the new fo0 , same as the old f0o ... no no no .. don't get fo0'ed again ... " - The Who

User avatar
Saturos
Art Master General
Art Master General
Posts: 1111
Joined: Mon Apr 19, 2004 2:48 pm
Location: Berlin/Germany
Contact:

Re: Lagstats in replays

Post by Saturos » Fri Feb 20, 2009 11:51 am

I agree with you smooth that replays can be useful to review information and that most of the time the recording is "pretty damn close" to what actually happened. However, you're conveying that cheats are the only explanation for straight shoot throughs... sadly its not that easy. We all tend to jump to the cheat-conclusion if something weird happens on a replay, but we have to always remind ourselves that there might be other explanations.
I'm in the lucky situation that I know a lot of players in reallife and can therefore say that in all likelihood they are not cheating (I've played with most of them in the same room, you get the idea). And still from time to time a shoot-through occurs that makes me immediately think "cheats?". One has to understand that there are so many reasons for throughs that cheating is just another possible explanation, but by far not the most likeliest. I've seen "from front to back"-throughs occur with players that sat right next to me, I've seen shots appear out of nowhere, I've seen jumps that arent possible... but with more experiece one realizes that the game has bugs, network quality has a tremendous impact on tank behaviour (not even talking about lag), your computer might mess things up (like the system clock realigning, I learned recently), etc etc. Some of those can even explain frequent shoot throughs (most likely the network quality)... so to come to a conclusion: Yes, replays can provide valuable information, but the key is how to interprete the data. If you found out some people have lots of throughs and you can provide pointers to the corresponding recordings, I'm happy to take a look.

User avatar
joevano
General
General
Posts: 1863
Joined: Sat Jun 18, 2005 1:08 pm
Location: South Bend, Indiana, USA

Re: Lagstats in replays

Post by joevano » Fri Feb 20, 2009 12:55 pm

Saturos, thank you that was exactly what I was thinking, but could not figure out how to put into words in such a good way.
There is nothing worse than aggressive stupidity. -- Johann Wolfgang von Goethe
"How many legs does a dog have if you call his tail a leg? Four. Calling a tail a leg doesn't make it a leg." -- Abraham Lincoln

smoooth
Private First Class
Private First Class
Posts: 81
Joined: Sun Aug 05, 2007 6:13 pm

Re: Lagstats in replays

Post by smoooth » Fri Feb 20, 2009 11:47 pm

Saturos/Foo -- I completely agree with you. There are many things that can cause the discrepancies. The program itself is loosely written and seems to be generally unpredictable. In fact most of the time when people think that I am accusing them of 'something', 90% of the time I don't mean "cheating" (in the classical sense of the word i.e. changing code, using various enhancements)... most of the time I mean peoples inability to correctly operate a computer and the accompanying program BZFlag. Where as I, and many others, try hard to reduce our "interference" there are a great many players that seek to increase their interference via various means and methods. Some go to great lengths to manipulate OS input device controls, methods in which their network communicates, artificial enhancing lag, adjusting fps low to get super jumps and all kinds of things, using shoddy wireless, f5ing, downgrading their client version, etc, I can't even name them all I've see so many exotic attempts to make the tank act erratic. Most of the time when you ask the players they will tell you thieir intention is to do create these conditions. We need to start looking at erratic tank behavior as problematic behavior for league play. This way we don't have to know the cause -- we just see the effect.

So generally when I see a shot thru (front to back only -- I've given up on side to side thrus) it does raise lots of questions because clearly certain types of thrus should be very rare in occurance but are very frequent with some players. Sometimes I will tell players "I don't play them" and it's not because I think they cheat but further because I want them to know something is wrong with their computer or implementation of bzflag which may be intentional or not. Like you said this could occur from a rogue usb device polling too often on a bus, rogue program, or inproper OS / Video configurations, any number of things.

So in conclusion, I agree it is difficult to tell but mostly because people have figured out that if they have erratic connections it nearly impossible to verify it on replay. But perhaps in those cases where players get a large number of shot thrus maybe short bans or warnings should be given as a means to let those players know that they need to evaluate why their connection is bad or why thrus could be occuring. I know this is immensely difficult task and I don't have all the answers but I do know there is a general trend increasing in the GU populous to create the most erratic unpredicatble settings that will give the player an advantage, and in some cases this allows haxors with real code modifcations to hide their hax via constant erratic behavior. We need to fight these efforts to protect the integrity of the game.

I think the admins (as a whole) are doing a much better job (some already did a good job). Glad to see more activity on getting some players banned. I think once we finally get a couple more of these guys out of the league that deserve to go -- you'll have a really nice group of people playing in an honest and competitve league.

smoooth

ts
Dev Monkey
Dev Monkey
Posts: 970
Joined: Fri Jan 14, 2005 6:26 pm

Re: Lagstats in replays

Post by ts » Sat Feb 21, 2009 3:58 pm

smoooth wrote:ts - you're proving my point exactly.

How do you "suspect a handy cheat" when the replay doesn't represent what actually happened.

You can't say

1. Tank locations and shots in replays are always false.

and then say

2. If a tank walks thru all shots on replay then they are cheating.
I did not say that. Non moving players in replays are pretty accurate in most cases. Once the tanks begin to move, it's very hard to guess what the players in question saw.
smoooth wrote:Saturos/Foo -- I completely agree with you. There are many things that can cause the discrepancies. The program itself is loosely written and seems to be generally unpredictable.
It's what I wrote quite a while ago in this thread. :)
GU league: http://www.guleague.org/
An introduction to TCP: http://www.lafkon.net/tc/

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest