Lagstats in replays

Discussion for GU League Players
dexter
Private First Class
Private First Class
Posts: 212
Joined: Sun Apr 30, 2006 7:36 am
Location: Germany :o)

Lagstats in replays

Post by dexter »

Heya,

I find it increasingly irritating that you can not see the lag/jitter of players while watching a replay. It's hard to determine if the shot went through or missed if you don't know if the player had a lag/jitter-spike at that moment or not. Would it be possible to have the server run /lagstats during the match and then later paste the output in the replay? Or something like that, probably wouldn't work like that but maybe somebody else has a better idea. Just thought I'd put that out there in case anyone else felt the same. :)


~dex
User avatar
JeffM
Staff Sergeant
Staff Sergeant
Posts: 5196
Joined: Fri Dec 13, 2002 4:11 am

Re: Lagstats in replays

Post by JeffM »

the way replay works, makes this impossible.

when viewing a replay, your (yes you, the viewer) current lag with the server affects what you see, in addition to 1/2 the lag that happened during the recording (the lag from the various clients to the server). The lag from the server to the clients that played the game is in no way recorded, since all the recording happens on the server side.

replay should not be used as any sort of "proof" of anything, as it is in no way an accurate representation of any of the gamestates that any client saw.
ImageJeffM
User avatar
Mucho Maas
Private First Class
Private First Class
Posts: 515
Joined: Tue Sep 21, 2004 5:14 pm

Re: Lagstats in replays

Post by Mucho Maas »

JeffM wrote:when viewing a replay, your (yes you, the viewer) current lag with the server affects what you see
I am not quite understanding that point. I thought the replay contains all the events as seen by the server, with every event having a time signature corresponding to how the server received the packets. That information is broadcasted to the client viewing the replay with those same time signatures. So the lag of the client watching the replay should not effect the replay he sees. The lag of the client only matters when it is observing a game live. Is this not correct?

For the replay to contain information about the lag of each tank should technically be not to difficult I think. The server could query those lag values in preset intervals and store that information along with all other received packets in the recording. However this would still not give you exact values of lag and jitter for every moment. So short lag or jitter spikes would not be visible, unless those lag query intervals would be sufficiently small.
"meet the new fo0 , same as the old f0o ... no no no .. don't get fo0'ed again ... " - The Who
User avatar
JeffM
Staff Sergeant
Staff Sergeant
Posts: 5196
Joined: Fri Dec 13, 2002 4:11 am

Re: Lagstats in replays

Post by JeffM »

bzflag 2.0.x does not do any lag compensation, so while yes, the server does time stamp the packets when it gets them, that does not mean they get to you in the same time that it took for the original recipient to get them. This can subtly change the state between a live game and a recorded game, especially if jitter is high.

The recording system does not store lagstats, since all it does is store packets that are sent out. The recording system would have to be changed to "stamp" the lagstats of each player state with each event, and then maintain them for query by the replay viewer.

Of course anything is possible with coding changes. I feel these would be somewhat significant.

2.0.x does not use a unified timestamp on all it's movement and shot packets so getting per packet timings is also more difficult.
ImageJeffM
dexter
Private First Class
Private First Class
Posts: 212
Joined: Sun Apr 30, 2006 7:36 am
Location: Germany :o)

Re: Lagstats in replays

Post by dexter »

Okay, thanks for the fast replies and explanations.. guess it's up to a crazy coder for this to happen then. :)
User avatar
Mucho Maas
Private First Class
Private First Class
Posts: 515
Joined: Tue Sep 21, 2004 5:14 pm

Re: Lagstats in replays

Post by Mucho Maas »

JeffM wrote:This can subtly change the state between a live game and a recorded game, especially if jitter is high.
I think that is why dexter proposes to have lag and jitter values in the replay available, so one can make a better assumption to what the original recipient saw (not so much what he actually saw, but by how much it can differ to what is being seen in the replay).
But true, sounds like more effort than I thought initially. And it remains disputable what the gain would be if it was to be implemented.
"meet the new fo0 , same as the old f0o ... no no no .. don't get fo0'ed again ... " - The Who
dexter
Private First Class
Private First Class
Posts: 212
Joined: Sun Apr 30, 2006 7:36 am
Location: Germany :o)

Re: Lagstats in replays

Post by dexter »

Yeah, I mean, just because you have a clear through on a low-lag player doesn't mean they cheat or anything. If you're suspicious of a player and you see funny things frequently and you know they have low lag and jitter in those situations, it could help.
User avatar
JeffM
Staff Sergeant
Staff Sergeant
Posts: 5196
Joined: Fri Dec 13, 2002 4:11 am

Re: Lagstats in replays

Post by JeffM »

yes, I know what he means, and as I said, the system would have to be changed to store them. But showing them to you, would not necessarily mean that what you see will match up with what the user actually saw since the behavior of packets from the server to the recorded client is not part of the recreation. That aspect may not be an issue for you, but you should be aware of it.
ImageJeffM
User avatar
Mucho Maas
Private First Class
Private First Class
Posts: 515
Joined: Tue Sep 21, 2004 5:14 pm

Re: Lagstats in replays

Post by Mucho Maas »

This may differ from user to user using the replay. Of what I see, 80% of the benefit of the replay as we have it today is for players to verify after a match that what they think was a 100% shot through in the replay actually looks like a miss after all. Another 10% is to verify if the tank itself moved according to specification (tank speed, jump length, warping).
For this, the replay as it is implemented now works really great. In the future we could even have server side build-in verifications for all of those (or a hit-hint-indicator, there was a discussion about that in another thread). And for tank speed etc there already are basic server side implementations.
Only for the remaining 10% the replay leaves open questions. And it is these 10% where
JeffM wrote:the behavior of packets from the server to the recorded client is not part of the recreation
always will remain an unsolvable issue, I agree (discussing client installed checkers is a whole different subject).
JeffM wrote:The recording system would have to be changed to "stamp" the lagstats of each player state with each event, and then maintain them for query by the replay viewer.
Is this information already there in the game packets? If not, they would have to be explicitly queried in certain intervals, which then again may lead to "missing" short spikes. And this would again question the gain that such feature would bring, elas.
Foo wrote:And it remains disputable what the gain would be if it was to be implemented.
I am not sure I am understanding the code for that enough, but isn't lagvalue queries not communicated on a different communication channel as the one for sending and receiving game information packets? If that is true, the question would also be how reliable are lagstat values to describe client-server communication quality. But as I said, I might have interpreted the code wrongly in that regard.
"meet the new fo0 , same as the old f0o ... no no no .. don't get fo0'ed again ... " - The Who
User avatar
joevano
General
General
Posts: 1863
Joined: Sat Jun 18, 2005 1:08 pm
Location: South Bend, Indiana, USA

Re: Lagstats in replays

Post by joevano »

But based on what JeffM said, you cannot necessarily tell from a replay whether a 100% shot through was actually a miss or a hit. It is inconclusive because what you see is not what exactly what happened, as it includes any lag which you may have during the viewing.. plus any that they had *.5
JeffM wrote:replay should not be used as any sort of "proof" of anything, as it is in no way an accurate representation of any of the gamestates that any client saw.
There is nothing worse than aggressive stupidity. -- Johann Wolfgang von Goethe
"How many legs does a dog have if you call his tail a leg? Four. Calling a tail a leg doesn't make it a leg." -- Abraham Lincoln
User avatar
Mucho Maas
Private First Class
Private First Class
Posts: 515
Joined: Tue Sep 21, 2004 5:14 pm

Re: Lagstats in replays

Post by Mucho Maas »

joevano wrote:It is inconclusive because what you see is not what exactly what happened, as it includes any lag which you may have during the viewing
The lag the client has while viewing the replay should not matter, as explained above.
joevano wrote:plus any that they had *.5
That is why dexter is proposing to have the lag values included in the replay: to have at least a small indication of the latency involved in the viewed events.
Note: indication =/= proof

That nonetheless a replay is not a 100% proof was already acknowledged, and has also been considered in our interpretation of replays for the last 3 years.
However, with some shots one can assume that in order to have dodged those, the lag must have been sufficiently high. An example are close triple shots that leave no space between the bullets for a dodge, and thus requires driving around all three shots, which again takes time. And if the replay does not show such a dodge, one can make assumptions about the latency that must have been involved. This rarely happens though. But it are these cases where dexter's proposal could be helpful.
JeffM wrote:the behavior of packets from the server to the recorded client is not part of the recreation

I think the above point is closer to the problem. JeffM is right with the above when hes says that next to latency and jitter there may also be other reasons not capturable on replay.

@donny: If your post was a comment on my last post: When a shot that looked like a through shot during a match, appears to be a miss on the replay, one can safely assume that that shot was a miss. This one can clearly be attributed to the viewers latency during the game.
And from my experience using replays, this scenario is the most common. And for that purpose alone the time invested into it's implementation has been 100% worth it. The replay is a very useful feature. Thanks to those who implemented it.
"meet the new fo0 , same as the old f0o ... no no no .. don't get fo0'ed again ... " - The Who
User avatar
JeffM
Staff Sergeant
Staff Sergeant
Posts: 5196
Joined: Fri Dec 13, 2002 4:11 am

Re: Lagstats in replays

Post by JeffM »

yes, the lagstats system is different then the gameplay system, this is why it can't simply be recorded like the game packets.

To get lagstats you ask the server for the lag info for each player.

In a real game the server has the last set of lag info for each player that it pinged and it just gives you those values.

In a replay, there are no real players for the server to ping, and the recording doesn't keep the lag stat events since they are not events that are sent out to everyone, but received by each player. this means that when you ask for the lagstats during a replay, the server doesn't have a record of what the lag for each player was during that time in the recording.

The change would be to have the recorder save the ping response in the recording as events that are not packets, and update the player lagstats so that they can be use.
ImageJeffM
smoooth
Private First Class
Private First Class
Posts: 81
Joined: Sun Aug 05, 2007 6:13 pm

Re: Lagstats in replays

Post by smoooth »

I think the point here is not that we are looking for instantaneous data of lagstats at everypoint during the game. I think we'd like to see a lag snapshot at various points in the game as an attempt to get a more complete picture while watching the events in a replay. If we are able to see in a few snap shots that the jitter of an individual is high or fluctuating from high to low it will be hard to determine that any shot demonstrated in the replay is legit. HOWEVER, if we have several snapshots of lag which shows a player with consistent low lag and jitter -- then there is a greater probability that what is reflected in the replay is close to what actually occurred. In instances such as this we can determine that if those players are getting shot thurs then, within a realm of reasonable certainty, there maybe something underhanded going on.

smoooth
quantum dot
Private First Class
Private First Class
Posts: 1290
Joined: Sun May 16, 2004 10:19 pm
Location: Spain
Contact:

Re: Lagstats in replays

Post by quantum dot »

smoooth wrote:I think the point here is not that we are looking for instantaneous data of lagstats at everypoint during the game. I think we'd like to see a lag snapshot at various points in the game as an attempt to get a more complete picture while watching the events in a replay. If we are able to see in a few snap shots that the jitter of an individual is high or fluctuating from high to low it will be hard to determine that any shot demonstrated in the replay is legit. HOWEVER, if we have several snapshots of lag which shows a player with consistent low lag and jitter -- then there is a greater probability that what is reflected in the replay is close to what actually occurred. In instances such as this we can determine that if those players are getting shot thurs then, within a realm of reasonable certainty, there maybe something underhanded going on.

smoooth
I agree lagstats would provide valuable information to evaluate replays. One shot thru is nothing, but when you see the same player going thru bullets time after time and again with nearly no lag, it is a good time to talk to her/him. That's why one is always keen to see a '\lagstats' command while watching a replay about a fishy player.
Don't let school interfere with your education.
Mark Twain
User avatar
ducatiwannabe
Private First Class
Private First Class
Posts: 3258
Joined: Tue Aug 10, 2004 3:55 pm
Location: Planet Earth
Contact:

Re: Lagstats in replays

Post by ducatiwannabe »

Wouldn't it be possible to create a "bot" of sorts who would sit in observer mode during matches, rather like the logging bots people put up? It could at certain times perform /lagstats and then it would chat to the observer team with the lagstats received of the players involved. I'm not sure how possible this would be, plus it would be annoying.

But players could silence the bot if it got to them and also it would show the team chat on replay...correct?
User avatar
blast
General
General
Posts: 4931
Joined: Fri Mar 21, 2003 3:49 pm
Location: playing.cxx
Contact:

Re: Lagstats in replays

Post by blast »

I don't think team or private chat is recorded by replays.
"In addition to knowing the secrets of the Universe, I can assure you that I am also quite potty trained." -Koenma (Yu Yu Hakusho)

Image
smoooth
Private First Class
Private First Class
Posts: 81
Joined: Sun Aug 05, 2007 6:13 pm

Re: Lagstats in replays

Post by smoooth »

I don't understand why people say "one shot thru is nothing." Lots of people say that but I just don't get it. I've played for many many years and I've never had one shot thru on me -- period. A close shot happens more frequently -- but a true shot thru, dead center, front to back, I don't see how that can be easily overlooked. Provided that lag and jitter of both players is in check.

I think the common fallacy is that the replay doesn't reflect what actually happened. I hear that over and over again. But it does reflect, with some degree of accuracy, what actually took place. I think we are not looking for 100% accuracy. Too many people are looking for 100% proof -- there never is. However, often times you can make some small assumptions and give a conclusive result that is just as good. Wish some admins could take good data and solid information and draw their own conclusions.

I had one admin tell me that "what is on the replay can never be correlated to what happened during a match. " I asked in return, "so if you saw on a replay that I was walking thru every bullet, by that logic, you can't prove me cheating."He said, "well no then you would be cheating." You can't have it both ways -- either the replays reflects what happens or it doesn't. If it does than you can statistically tell who is cheating by correlating average lag/jitter stats with quantity of apparent shot thrus. What you will find is that it is very obvious who the cheaters are. If the replays doesn't reflect what actually occurred than no one can be banned for walking thru every single bullet.
User avatar
joevano
General
General
Posts: 1863
Joined: Sat Jun 18, 2005 1:08 pm
Location: South Bend, Indiana, USA

Re: Lagstats in replays

Post by joevano »

But you can have it both ways! One passed bullet is not statistically significant on a replay, but all passed bullets would be. And you would not know if you had passed bullet through you on a replay, as it would seem like a miss unless you watched every replay. I will point you to this excellent article...http://my.bzflag.org/w/Lag . Think of the server as the observer (C).
There is nothing worse than aggressive stupidity. -- Johann Wolfgang von Goethe
"How many legs does a dog have if you call his tail a leg? Four. Calling a tail a leg doesn't make it a leg." -- Abraham Lincoln
User avatar
Mucho Maas
Private First Class
Private First Class
Posts: 515
Joined: Tue Sep 21, 2004 5:14 pm

Re: Lagstats in replays

Post by Mucho Maas »

I think that article in http://my.bzflag.org/w/Lag has one flaw.
It describes the tank positions as seen from every point of view. What it doesn't show is how different the participating actors see the position of the bullets. The shot path of the bullet for all is ofcourse the same, as the information to calculate the shot path is transported together with the shot event. But in addition to tank positions, due to lag they would also see the bullets on a different position on that shot path. These adds to the difficulties of interpreting replays.

But I believe smoooth is quite aware of the effect the article is describing. It is very obvious with tanks moving angled towards incoming bullet. Smoooth I believe was referring to bullets that come right from the front or back of the tank. You need to have a considerable higher latency involved to have dodged those when the replay shows it as a shot through. This is because the movement to dodge it doesn't just involve driving forward, getting out of the shot line takes time in these circumstances. One could make an estimate of how many ms it takes for a tank to move out of the shot line when the bullet comes straight from the front.
Even with smoooth you will have shot throughs in a match when he dodges a bullet coming in side ways. But it aren't these he is referring to I believe.
However there is a difference in keyboard and mouse stearing. There is an acceleration when driving with keyboard, meaning that it takes a small fraction of time for the tank to gain maximum momentum. With mouse driving the speed change is performed instant. This is because the tank moves according the position of the mouse, not according the movement of the mouse. So the change of a tanks speed is not according a continuous function. So with mouse you are faster in getting out of a shot line from a still position. This may add to the perception that you have more shots going through a mouse driven tank than a keyboard driven tank.

I agree on the statistics. One or two frontal shot though in a match statistically is not very significant.
"meet the new fo0 , same as the old f0o ... no no no .. don't get fo0'ed again ... " - The Who
smoooth
Private First Class
Private First Class
Posts: 81
Joined: Sun Aug 05, 2007 6:13 pm

Re: Lagstats in replays

Post by smoooth »

Joevano -- I understand lag. But if you say its significant that every shot goes through on replay then that assumes that the replay is reflective of what actually happened. If that is the case -- then you can statistically gather that those with more thrus than others have a greater probability of having actually moved thru a bullet. Then we asked ourselves why. There maybe a good answer such as an exceptional dodger. BUT when you analyze exceptional dodgers and determine a reasonable level of shot thrus for even the best players -- and then there is someone who has a factor of 100 times more shot thrus than those... then you can gather that person is cheating.

Foo is correct in interpretting what I was saying. Talking about front to back thrus. In other words a player can not dodge a shot coming to the dead center middle of his tank while facing the bullet. This is the most obvious situation where an apparent thru should not occur. Also lag or jitter, in this scenario, is almost eliminated as a rationale by the physics of the game.

Foo -- Good point about your mouse and keyboard comparison. I have kinda known that for a while. It is a flaw in the game as changing input devices should not inhibit or enhance the physics of the tank movements. Maybe someday this will be fixed and us keyboard player can be brought into the same game as some of those super high DPI speedy mouse guys :) (not you foo but others who we are familar with :))

smoooth
User avatar
slime
Private First Class
Private First Class
Posts: 188
Joined: Fri Dec 30, 2005 3:08 am
Location: Omaha , Nebraska

Re: Lagstats in replays

Post by slime »

smoooth wrote:But if you say its significant that every shot goes through on replay then that assumes that the replay is reflective of what actually happened.
It's completely up to the GU League Admins to determine whether any evidence from the replay server is significant. You are looking for a concrete answer to whether or not a replay server can be used as evidence against a player, but there is no concrete answer to this. The admins run the league, so they are entitled to use whatever means they wish to gather evidence. Technically speaking, the replay servers aren't 100% reflective of what actually happened. However, if the admins wish to use it for evidence, there is no reason why they can't. A great trait our admins have is COMMON SENSE. Obviously, if someone drives through every single bullet on the replay server, it will be deduced as cheating. If a player has a few shot throughs in a match, the admins have enough common sense to deduce that it was most likely caused by lag.

I'm sort of confused as well on your thoughts of lag/jitter influencing a shot going through a player, dead center, from front to back. Most likely I am just not understanding what you are saying, seeing as it's late at night. Could you clear up these views for me please?
smoooth wrote:A close shot happens more frequently -- but a true shot thru, dead center, front to back, I don't see how that can be easily overlooked. Provided that lag and jitter of both players is in check.
smoooth wrote:In other words a player can not dodge a shot coming to the dead center middle of his tank while facing the bullet. This is the most obvious situation where an apparent thru should not occur. Also lag or jitter, in this scenario, is almost eliminated as a rationale by the physics of the game.
Also, you say that you have never had one shot through you (and I am assuming here you are talking about the dead center, front to back shot). From what I myself have observed throughout my years in the league, I'd say the majority (if not all) of those shot throughs occur with people using wireless internet. Thus, I'd assume you are NOT using wireless internet, which would explain why you have not had a shot go through you in this manner. This is all on my assumptions, though, so correct me if I'm wrong.
ts
Dev Monkey
Dev Monkey
Posts: 970
Joined: Fri Jan 14, 2005 6:26 pm

Re: Lagstats in replays

Post by ts »

slime wrote:A great trait our admins have is COMMON SENSE.
Honestly you have no chance to know. Most decisions are kept secret and when a ban is issued for example it may not be announced or the exact reasons are not revealed to the public. I have a very different opinion regarding the definition of a cheat. Anyway, banning based on suspicions alone is something I don't recommend, either.
slime wrote:Also, you say that you have never had one shot through you (and I am assuming here you are talking about the dead center, front to back shot). From what I myself have observed throughout my years in the league, I'd say the majority (if not all) of those shot throughs occur with people using wireless internet. Thus, I'd assume you are NOT using wireless internet, which would explain why you have not had a shot go through you in this manner. This is all on my assumptions, though, so correct me if I'm wrong.
Shot through tanks can be caused by numerous reasons. Even hardware problems could influence it. I'd not say the majority is caused by anything without exactly knowing what happens in all these cases. It is probably sane to assume that BZFlag's networking model may be one of the biggest reasons for shot throughs. A better networking model, involving the server keeping an accurate state of clients and decide the kills is supposed to help a lot. Sadly implementing such a thing is obviously not very easy.
GU league: http://www.guleague.org/
An introduction to TCP: http://www.lafkon.net/tc/
smoooth
Private First Class
Private First Class
Posts: 81
Joined: Sun Aug 05, 2007 6:13 pm

Re: Lagstats in replays

Post by smoooth »

What I mean by a dead center thru front to back is when the bullet is in a straight line towards a tank where the tank is facing the bullet. In other words the turret of the tank is directly lined up with an on coming bullet. In this scenario there i no way to "dodge" per se. As opposed to traditional dodging where the tank is perpendicular to the oncoming bullet and the tank can move to avoid the oncoming bullet.

What I am saying is that we all agree that the replays ARE reflective of what actually happened. My point in saying this is then there is a statistical number out there for what could be the average number of thrus for any given player. To gather that number you can coorelate it to lag and jitt or by perceived skill of the player to dodge a bullet. You come up with a "normalized" expected occurance of shot thrus. Then you analyze the players. What may come up is that 1 or 2 players have an exponentially higher number of shot thrus than the norm. Of course there is always going to be some acceptable level of deviation. Not all players are the same (different hardware, connections, skill level). However, when a couple players are off by a factor of, lets says, 50-100 times more than all other players.... you can be very certain that something is not correct with those two players.

TS I agree on the network model and what you are saying but agreed we have to deal with what we have not what we'd like it to be. We have to remember that sometimes hunches need to be made and also enforcement should be dealt with on best available information. If we wait until the "smoking gun" we'll be waiting forever. In situations where there is a front to back shot thru.. why not ban for 3 weeks. Hell, 1 day. May raise the standards of play and weed out the illegal players who are turning up all over the league these days.
ts
Dev Monkey
Dev Monkey
Posts: 970
Joined: Fri Jan 14, 2005 6:26 pm

Re: Lagstats in replays

Post by ts »

I was well aware of what you meant but still the state is not accurate enough to know what actually happened. It is likely it never can be in this networking model. The server does not know the exact client state, it can only resemble a rough idea of what the client „saw”. Please note the bullets in 2.0.x are not lag compensated, either. As consequence replays can not reconstruct the situations played, not even close. In 3.x the differences in replays will be even higher as the shots will be no more directly tied to a position and a vector but bullets will be also lag compensated. Last time I checked even ricochets didn't work in 3.x replays (I could not find what exactly broke it). My impression is that when even basic functionality does not work spending most time on additional features is not feasible. If anyone does, well they're free to do but I doubt it will get the game good enough (not much bugs, good enough to play using it) situation. At least not until there is a concentration of bug fixes.
GU league: http://www.guleague.org/
An introduction to TCP: http://www.lafkon.net/tc/
User avatar
FiringSquad
Sergeant
Sergeant
Posts: 849
Joined: Thu Jan 26, 2006 5:53 pm
Location: Ireland

Re: Lagstats in replays

Post by FiringSquad »

Why not just have client-side recording and activate it automatically for league matches?
There would have to be some transport mechanism developed so that the recording can be uploaded to the replay server upon request, but other than that it should not be too difficult to implement and would provide an accurate representation of what the client saw.
That way problems could be rectified by demonstrating how it looked in your client.
How big is a recording? Perhaps it is prohibitively large.
Post Reply