I'm disappointed that I missed out on this thread for months... real life was a real bitch for the last few months of 2012.

Anyway, I just read through the whole thread & I think I'm caught up on the various suggestions now. There's one simple idea I don't think was ever mentioned, though:

When you calculate an overall score (Best SSX 3 Across All Modes or Best Showoff Across All Games), you obviously need to assign weights to the different game modes or different titles. You've mentioned two options for that:
1. Percent of all scores on the scoreboard attributed to that game / mode
2. Percent of all tracks attributed to that game / mode

Have you considered this one?
3. Average number of scores submitted per track

If the average Survive scoreboard only has 20 scores, but the average Race scoreboard has 100 scores, then racing would count 5x as much as survive in calculating an overall rating. But if the average Survive scoreboard has 85 entries and the average Race scoreboard has 90, the two modes would be weighted almost equally. This initially treats each title / game mode equally (number of tracks does not matter), but then it rewards people for being the best at the titles / modes with the most competition.


Also, I think the fact that the tracks are shorter in 2012 than in previous games is actually what's artificially boosting the (last place) Racing scores in 2012, making it count too much compared to Showoff or Survive and compared to other games. (When you're calculating a mean, it doesn't actually matter how many values you have... The problem isn't that you're averaging 69 values instead of 10; the problem is that the mean of the 69 values is very different from the mean of the 10.)

In the Showoff and Survive scores, you're looking at the score as a percent of the best score. So it doesn't really matter how long the track is; if you get half as many points as the top dog, you'll always earn the same score on the Merq scoreboard. But in the Racing scores, you're just doing a flat subtraction. So being 30 seconds slower than first place on a 10-minute race is counted the same as being 30 seconds slower than first place on a 20-second race.

The tracks in 2012 are so short that I would expect the difference in time between first and last on the average 2012 track is much smaller than the difference in time between first and last on previous games' racing scoreboards. This is what's making the "no score" placeholder value earn so many points on 2012 racing.

... Sorry for the rambling post. Hopefully it makes at least some sense. My brain's a bit fried tonight.

If you want, I have over 50,000 Top Scores (over 90,000 total scores) to submit to the database to screw up your calculations. Just match scores.source to friends.personaid and scores.dropid to drops.dropid.

As trolly as that would be, I'll have to decline. The scoreboard is purely voluntary - I've had a lot of top players not submit their best scores for it and that's perfectly fine. Involuntarily adding people who signed up for YOUR service and then chucking them into the Grav standings would a) throw off everything, like you said and b) be sort of involuntary.

Mahkra, will respond to your reply once I've had time to process what you're proposing.

_________________

Oh, I wouldn't call it a proposal at this point, just a suggestion to spur some more discussion & maybe lead to some quality ideas. I haven't actually looked at any significant amount of data on the scoreboards, so I'm not sure how it would pan out with the actual real data.

... but if you did want to send me a dump of the scoreboard data, I'd be happy to play around with some different formulae and actually propose something specific.



Never mind that many of those 50k-90k scores are from people who never even signed up with ozmo, right? If I recall correctly, Winonanet pulls friends' data from the Ridernet feed, so many of the scores are probably from people who don't even know the site exists.

I will look into it if our current ideas just aren't cutting - but I think we're close. I had my programmer institute the changes I talked about above, and it does make quite a difference, but I'm not sure it's 100% effective. So I'm going to float some more ideas a little later this week and see what you think. And I'm counting on you to input your 2 cents since the amount of people who can follow this (or care to, anyway) is pretty small and I greatly appreciate feedback from those who enjoy it.

Actually, thinking about it, it might not be that bad of idea. The problem is how to send it without including any sort of privileged info (hashed passwords, etc.) Let me see what our programmer says about it. The scoreboard data isn't private by any stretch, and I would like to see what you come up with.


Yeah, it would be an enormous snowstorm of brand new information into the database, and let's say we created a new user for each user who isn't in our database (not to mention how meticulously we would have to match the incoming data with our existing database), we'd probably increase our userbase by what, at least 200%? You're a funny guy Ozmo.

_________________

As mahkra said, I pull friends of friends, which currently yields 3800+ people's scores. I have about 200 ACTUAL people who have signed in. So, yes, about 3500 people have no idea I have their scores, and no, I wouldn't put everyone in there.

I was being facetious, however...

50,000 scores is a scale out of any formula you might want to write. If it works for 500 scores, it should work with 50,000 scores. You might find that since people are only submitting their best scores on cherry-picked tracks, that a certain track might have an inflated average which throws off your weighting below a certain number of scores.

Okay, so I'm going to list the changes that we're going to make to the scoreboard rankings, and then I have some questions to ask you all in a post below that.

So, onto the gory details. I'm going to quote a PM that our programmer sent me and show my responses where they will inform the topic.



As a side note, I noticed, since I've been paying attention to the integers of last place people, that the last place score for SSX (original) users are 6649 - and that's normal because there are so few scores for that game. I think this might be a good indicator that I need to turn the integer to zero for people who don't have the miminum number of scores for a category. If someone has no scores on a certain category, they should get a zero. This is more of an injustice than the SSX 2012 standing. : I actually have thought of a more novel solution than the "minimum scores" solution, which I touch on below.


Yeah, looking at that and it seems right. RockLee dominates about 1/3 of the game, so #9 seems about right. Now, I want to do a little math here to see if I can wrap my head around how to explain the Best of SSX (2012) standings. I'll use Xedec as my example, since he dominates one category, but has no scores in the other categories.

Xedec
Racing: 9970.7
Showoff: 2392.91 (has no scores in showoff, so his integer is last place in that category - 1, correct? So 2393.91 - 1)
Survival: 1142.47 (has no scores in survival, so 1143.47 - 1)

Then we take these scores and multiply them by the percentage of tracks for each mode, correct?

9970.7 * (69/168 = 0.41) = 4087.99
2392.91 * (71/168 = 0.42) = 1005.02
1142.47 * (28/168 = 0.167) = 190.714

=5283.72. Actual total is 5296.81, which is close enough for my fuzzy math. Ok, very nice. Now if I change this to where Xedec has zero in those categories where he has no scores..., he would have 4087.99 without the placeholder scores bonus.

So let's say we institute our "minimum scores" threshhold and set it at 3 minimum scores in any category to then gain full benefit of last place - 1. The second wild card in this equation is Hero Complexity, who should logistically be in first place since he has 3 racing AND survival scores in addition to near top scores in EVERY showoff event, which has more tracks.

Hero Complexity
Racing (3 scores) : 1596.64 *.41 = 654.62
Showoff: 8603.39 = *.42 = 3613.42
Survival (3 scores) : 1471.72 = *.167 = 245.65

=4513.69

So in this case, Hero would bump Xedec out of first place (as he logically should since he has a wider variety of scores). He gets a bump of 900 points by doing 3 scores in each mode. So, while this is all good and well - it is a statement of fact to say that Hero has (inadvertently) done the bare minimum he can to get the most benefit. Part of me doesn't like creating a dynamic whereby someone can get maximum benefit for minimum effort. So here's what I'm proposing to rectify this, and let me know how difficult or not this would be:

So, here's my proposal to the board:

What if we tie the person's score in each mode to the percentage of tracks they've submitted for that mode? See below for my explanation:

So in the benefit above, since Hero has submitted 3 out of 69 scores in racing, he would get 4 percent (0.04) of his racing integer (654.62*.04 = ~ 26 points) He has submitted 3 scores on survival, so he would get 10.7 percent of his Survival Integer (245 .65 * .107= ~ 26 points). That would also even out a problem I spotted with the SSX (original) dichotomy where BlackDth is listed as top in SSX Overall, even though he has no scores in SSX (original) Racing. He's getting over 6000 points benefit just because of the low number of scores in SSX (original). Let's call this feature "Proportional Placeholder Scores".

This brings to mind that we still have an additional problem to fix - we haven't addressed the imbalance that SSX 2012 has thrown in the Best Overall Category with large imbalance of scores it's introduced. While I like Xedec/Hero/Etc, they shouldn't be featured at the top anymore than any one person who specialized in one game (like BlackDth did for SSX Original Showoff.)

What do you guys think?

_________________

I do not feel you should be getting placeholder scores. If you put no work into mastering a type, you get no credit. Beat it, freeloader. You can't be on the overall leaderboard if you have NO SCORES in the other 2/3rds of the game.

If you ABSOLUTELY MUST *groan* combine the scores, I do not like the weighting of Race = 41%, Trick = 42%, Survive = 16%. According to this, someone who busts their ass for DAYS and gets 10,000 on Survive (10,000 x 16% = 1,600 points) still does not numerically beat someone who pushed "go and forward" on every track to get a 5,000 in Racing (5,000 x 41% = 2,050 points). It is unfair to Survive, and even Racing (when compared to Trick).

You are looking at it wrong. Rather than dividing out 168 tracks into 68, 71, and 28 THEN multiplying it by it's percentage (1 of 68, 1 of 71, and 1 of 28), why not just take each score individually as 1 of 168?

This is going into AwesomeScore(tm) territory, but if you treat EACH score INDIVIDUALLY, you can then *groan* combine them. Calculate Xedec's ranking in JT2 Race / 168, + Xedec's ranking in Curnoe's Race / 168, etc.

If you combine GROUPS (race/trick/survive as a whole), they should equally be 33%. If you combine SCORES, they should equally be worth .005952381% (1/168th).

THIS.

_________________

Yes, tie it in to % of scores! Proportional placeholder scores? Wtf? You have not submitted a score you do not deserve a "proportional" score. Think cherry picked again, and someone could skip out runs like One Step Ahead. Yes, 2012 players should not benefit over players of the old games just for playing 2012.

_________________
When should I open my eyes?!

There's a lot of really weird kludges in the scoring...

Proportional Placeholders seems like an overly complicated way of doing almost the same thing as just eliminating Placeholder scores.

"Maximum Divisor Settings" is just a completely artificial scaling factor to try to shift the scores for one section of the scoreboard more in line with the rest of the board.

Even the 0.99^(time-best time) formula... why 0.99? Why not 0.94, or 0.98? It's an arbitrary number. And this formula does not scale well to tracks of different lengths.


I copied the data from all of the scoreboards last night and dumped it into an excel file, but I haven't had a chance to test any ideas yet for combining scores between games or between game modes. I'll try to post something later tonight or maybe tomorrow night.

I did have one thought for an alternate racing formula, though.
If showoff is catscore/cattopscore, why not simply make racing cattopscore/catscore? Then if you take double the time in a race, it's the exact same thing as earning half the points in a showoff.
And a preliminary glance at the data looks like this wouldn't be a huge change in the calculated scores.

Here's the distribution of scores (out of 10k) earned by every entry on every scoreboard.
(R/S/T is Race/Survive/Trick)

And here's how it would change with the simpler racing formula.

Interestingly, Racing is pretty top-heavy. A lot of scores are bunched rather close to the best score on each track. Survive is the exact opposite -- just a few people way ahead of the pack; most scores are bunched at the low end of the 10k spectrum. And Trick is somewhat flatter; there's a broad range of scores.

Placeholder scores must be included at the track level because of the way the Racing Formula works. All the racing times are lumped into one big sum and then plugged into the racing formula. You can't put a zero (0 seconds) down for a racing track because that's actually better as it leads to a lower penalty and a thus better ranking score. You can't favor users with no scores over those who actually submit. So there has to be placeholder scores of (last place - 1 second) for all racing tracks where a user doesn't have scores. Nor can you institute some generic penalty like 10 minutes because some track could come out where 10 minutes is a good time or what have you.

So it made sense at the time that we were creating the scoreboard rankings system that if we must have placeholder scores for racing, then we should include them for showoff, to be equal. Now, Showoff does NOT require placeholder scores, since it's a different formula, but again, to be fair we included them so as not to penalize showoff players more than racing players. And at that point, it made sense, and it worked for a long time, no one had many issues with it. It's wasn't until SSX 2012, with the huge influx of new scores and users and the issues that brings that I look at this and go "we could do this better."

Now because we MUST have placeholder scores for racing at the track level, this means that any "zeroing" of scores must be done once the scores for each mode are totalled up and coverted to ranking points (the integer out of 10,000).

Now I think it's pretty unanimous from everyone's feedback that if you have no scores in a mode, then you get a zero on your ranking points. I think this makes sense and I think we'll institute it no matter what. The question is what do we do when a user has only a few scores in a mode? There are two ways to do this that I can see:

1) We can use what we have on the main four "Best Overall Categories", which is a minimum number of scores (currently set at 3) for a user to gain the advantage of placeholder scores for each track they don't have scores. This is the "do barely enough to get by" option I mentioned above. A user could submit the bare minimum of scores, and then gain advantage of placeholder scores.

2) The second option is to make the percentage of their ranking integer directly scaled to some other statistic. I propose that we use the number of tracks they submit scores for. So if a user has submitted 1 score out of 8 tracks of a mode (racing, showoff, etc), then they get 1/8 of their placeholder score. If you guys can think of some other figure that represents a better mathematical representation of how competitive/good a user is for a mode, let's hear it. I'm open to just about anything.


Yes, of course. I mentioned previously in this topic that it is a formula to bring the maximum divisor more into line with the previous games, yes. Because players that specialize in SSX 2012 get a bonus to their last place ranking points due to simply specializing in SSX 2012. It's part and parcel of why RockLee is so high in the SSX Overall standings. He's getting like 6,000 points for modes he has no scores for. Obviously we're gonna fix that.


Ahh, mahkra, just do the math. If you work out the racing formula, you'll find out that ideally the number that is multiplied by 10,000 should be as close to 1.0 as possible. Lower numbers like 0.98 or 0.94 means that the penalty is much steeper for the same figure.

Example: 10,000 * 0.99^10 = 9043.82
10,000 * 0.98^10 = 8170.72 <-- a thousand point difference just .01 down.

The course on this continues the lower that number gets. So ideally it would want to be as close to 1.0 as possible, something like 0.9999999 or something. We just cut it off at 0.99 to prevent the computations from getting too long and overworking the server.


Yep, I mentioned this in the past. Part of the problem with survival compared with the other modes is that Survive has essentially no ceiling when it comes to scores, which means the ranges of scores are much, much broader. This is all in the process of being dealt with - what this is turning out to be will be a rather comprehensive rework of everything involved in calculating scores.

I look forward to the solutions you come up with. I think the more brains involved in this, the better.

_________________

Placeholder Scores:

Maximum Divisor:

.99^x:

Ok, so here are some actual suggestions from just a couple hours of cleaning up & playing around with the numbers:

• Calculate Ranking Points for each track instead of averaging everything first.
• No placeholders. No entry on a scoreboard means 0 Ranking Points for that track.
• Change the racing formula. I propose RankingPoints = 10,000 * (BestTime/Time)
• Average Ranking Points for all races in one game determines the best racer for that title.
  (Same for Trick/Survive)
• Average of those Race/Trick/Survive total Ranking Points determines best overall for the title.
• Average of the overall Ranking Points for each title determines the best overall across all titles.
• Average of the Race total Ranking Points for each title determines best overall racer across all titles.
  (Same for Trick/Survive)

And the general ideas behind the suggestions:

• No "kludge factor" numbers in any of the formulae. That's just an artificial bias on the scoreboard.
• No freeloading. No entry on a board means no points. If you want to be ranked highly overall, you need to play multiple modes & titles.
• When calculating best racer on one title, treat all tracks equally, regardless of length.
• When calculating best overall on one title, treat all modes equally, regardless of number of tracks.
• When calculating best across multiple titles, treat all titles equally, regardless of number of tracks or number of submissions to the scoreboard.
• When calculating best across mutliple modes and titles, treat all titles equally rather than all modes equally. (All modes of SSX 2012 is 1/6 of your score instead of Survive mode of all titles being 1/3 of your score.)

edit: accidentally typed the fraction inverted originally... fixed now

Oh, Assuming I didn't screw up copying the data & am not accidentally pulling results from the wrong model in my spreadsheets, here's how the scoreboards would look:

SSX - Race | Click to reveal hidden content

alpmaster 9709.71398
GiveMeIce 8932.969285
FirebrandX 8000.766302
HX YEH 6916.153791
Obi 6166.962833
gondee 6159.731051
happymachines 6069.687433
Tigerbalm 6001.280553
Elisefan#1 5030.703626
ssx4ever 5000

SSX Tricky - Race | Click to reveal hidden content

FirebrandX 9706.237423
the hot legend 9386.754258
Frost 8781.424068
ssx4ever 6960.631805
gondee 6698.75978
happymachines 5411.638439
HX YEH 5106.536199
MAD_JO3 4900.06605
SpicyStunner 4851.526072
Obi 4843.020509

SSX 3 - Race | Click to reveal hidden content

RE Virus 9530.424593
IceXtreme 8287.647387
Obi 7639.748802
archie 7533.954661
Lord Elevation 7530.749948
Legend_armlet 7387.595661
MAD_JO3 7253.917284
snowstorm 7253.486769
laidback0378 7153.279689
ChaosRacer2 6840.987884

SSX On Tour - Race | Click to reveal hidden content

IceXtreme 9864.171767
HX YEH 9792.628509
the hot legend 9693.82596
RE Virus 8474.770079
snowman 8116.619782
erictang05 4843.55665
ChaosRacer2 2863.79564
Aerodynamisch 2286.072896
archie 1969.110091
898ppp 1588.467981

SSX Blur - Race | Click to reveal hidden content

LLCoolDave 8927.219314
MrRuchie123 8616.648972
Yeti 5915.385391
Obi 4935.34102
XquietusX 4380.831542
Bombshell=Riggs 3465.065938
yako591 2762.810809
gondee 2699.738398
Deaddy 2664.830521
dfi 977.3016277

SSX 2012 - Race | Click to reveal hidden content

Xedec 9984.33928
KeKoFruiT 8617.547427
the hot legend 8593.385179
iTofu 7197.599553
minn 6701.302308
Madeo 6185.126913
RadicalPlayer 4503.07346
chromemaps 4440.921324
Th3 Apache Dog 3466.004324
deadshootme 2688.373819

SSX - Trick | Click to reveal hidden content

Alex 8489.550404
BlackDth 7392.712028
grandmort 5689.796016
Evensen 3863.35138
Lunt 3820.073148
MrChaos 3695.589603
alpmaster 3645.058625
Ca$flo 3383.579289
m0ir 3113.901165
Rainbow Rainbow 3070.133335

SSX Tricky - Trick | Click to reveal hidden content

9EDDY2 8270.594877
the hot legend 8156.265189
elgen 8000
SpicyStunner 6280.326941
MrChaos 6030.076734
naya82 5392.98921
Alex 4661.412829
PD~ 3608.794577
yako591 3542.547443
Goose Bay EK 3295.351459

SSX 3 - Trick | Click to reveal hidden content

Rastapo 9113.747385
RE Virus 8624.210208
Obi 7702.588164
Oniz 7651.906239
IceXtreme 7113.65144
JXR 6648.787543
Wanis 5936.171486
fliper 5857.349373
MrChaos 5782.377558
QuotidianPerfection 5555.60592

SSX On Tour - Trick | Click to reveal hidden content

royank 9516.381347
the hot legend 7947.063319
HX YEH 4076.041917
898ppp 3415.859193
MrChaos 2488.168919
Titan 2417.430576
snowman 2305.357419
RE Virus 1961.179928
eviltim 1803.450584
eastviewgolf38 1789.938031

SSX Blur - Trick | Click to reveal hidden content

yako591 9795.681251
XquietusX 7214.008967
MrRuchie123 4687.208911
Deaddy 1412.437461
mandlebaum 1379.297531
gondee 1342.778539
Dr. Spud 1122.389895
MdX MaxX 781.634131
StickNutzman 748.9206424
Bombshell=Riggs 681.9636104

SSX 2012 - Trick | Click to reveal hidden content

Hero Complexity 8147.256722
Pemdas82 7414.913879
yako591 5996.18865
the hot legend 4703.792398
eviltim 3943.675469
iTofu 2964.096286
FKuPAYme 2249.585527
Don 1925.352181
vix 1767.569447
fredde007 1732.956458

SSX 2012 - Survive | Click to reveal hidden content

Rocklee1113 6387.859061
chromemaps 5538.615098
MarcusAnnex 2834.136059
vix 2187.598774
Ahlyis 2162.207519
EnjeruKira17 2065.424386
KeKoFruiT 1741.045198
iTofu 1696.615681
the hot legend 1659.663515
XahGee 1482.121189

SSX - Overall | Click to reveal hidden content

alpmaster 6677.386302
GiveMeIce 5917.211322
grandmort 4962.685178
FirebrandX 4435.181777
Alex 4244.775202
HX YEH 3856.997998
BlackDth 3696.356014
Obi 3411.192623
gondee 3079.865526
happymachines 3034.843716

SSX Tricky - Overall | Click to reveal hidden content

the hot legend 8771.509723
9EDDY2 6045.163167
SpicyStunner 5565.926506
FirebrandX 5197.132546
naya82 4885.77634
Frost 4863.061091
elgen 4500
gondee 4499.526084
distancedave 3833.838471
ssx4ever 3820.84913

SSX 3 - Overall | Click to reveal hidden content

RE Virus 9077.3174
IceXtreme 7700.649413
Obi 7671.168483
Legend_armlet 6440.066524
JXR 5851.737501
ChaosRacer2 5288.237106
QuotidianPerfection 5280.44603
Wanis 5149.81951
laidback0378 4794.402997
archie 4724.014763

SSX On Tour - Overall | Click to reveal hidden content

the hot legend 8820.444639
HX YEH 6934.335213
IceXtreme 5279.856345
RE Virus 5217.975004
snowman 5210.988601
royank 4758.190674
erictang05 2845.90797
898ppp 2502.163587
archie 1633.440711
Aerodynamisch 1541.901761

SSX Blur - Overall | Click to reveal hidden content

MrRuchie123 6651.928942
yako591 6279.24603
XquietusX 5797.420255
LLCoolDave 4463.609657
Yeti 3248.168471
Obi 2467.67051
Bombshell=Riggs 2073.514774
Deaddy 2038.633991
gondee 2021.258468
dfi 743.4584948

SSX 2012 - Overall | Click to reveal hidden content

the hot legend 4985.613698
iTofu 3952.770507
KeKoFruiT 3635.707472
chromemaps 3501.38559
Xedec 3328.113093
yako591 3163.559444
Hero Complexity 3050.206304
Pemdas82 2471.63796
minn 2233.767436
Rocklee1113 2220.863006

All Titles - Race | Click to reveal hidden content

the hot legend 6889.358209
HX YEH 4315.268433
IceXtreme 3630.363831
Obi 3608.550206
RE Virus 3601.038934
MAD_JO3 2602.850758
crispncrunchy 2338.845711
Yeti 2207.053096
Xedec 1996.867856
FirebrandX 1941.247485

All Titles - Trick | Click to reveal hidden content

the hot legend 3568.067039
yako591 3222.402891
MrChaos 2999.368802
Alex 2354.545673
RE Virus 1764.231689
royank 1677.271781
Obi 1664.678916
BlackDth 1530.739534
Rastapo 1518.957898
898ppp 1457.244766

All Titles - Survive | Click to reveal hidden content

Rocklee1113 6387.859061
chromemaps 5538.615098
MarcusAnnex 2834.136059
vix 2187.598774
Ahlyis 2162.207519
EnjeruKira17 2065.424386
KeKoFruiT 1741.045198
iTofu 1696.615681
the hot legend 1659.663515
XahGee 1482.121189

G.O.A.T. | Click to reveal hidden content

the hot legend 4377.436925
HX YEH 3019.625324
Obi 2843.831864
RE Virus 2382.548734
IceXtreme 2163.417626
yako591 1869.013199
gondee 1831.096606
GiveMeIce 1757.973254
alpmaster 1683.144902
FirebrandX 1605.385721

G.O.A.T. (if we counted Race/Trick/Survive equally instead of each game equally - not recommended) | Click to reveal hidden content

the hot legend 4039.029588
chromemaps 2171.412029
Rocklee1113 2144.549129
HX YEH 1868.589918
yako591 1815.418928
RE Virus 1788.423541
Obi 1757.743041
IceXtreme 1643.965297
vix 1455.221899
Yeti 1245.62447

I'll look at all this tomorrow when my brains not fuzzy. Anyway...mahkra, where were you these last 5 years? I could have used someone like you to bounce stuff off of a long time ago. Instead, it was pretty much just me and RE Virus.

_________________

Well, gee, mahkra, why not just edit the forum database and put your name on my posts?

_________________
WinonaNET is a RiderNet experiment with a Cross-Platform Scoreboard: http://ssx.ozmonet.com

Would you believe I, absolutely, from the bottom of my heart, agree with you one-hundred thousand million percent? I don't like having to consistently "patch" our racing formula because I can't find anything better. It was good enough to achieve what I wanted, which was to have a way to objectively compare two fundamentally different modes and then use that to compare the various games in the series. Ideally, both formulas would not require any "kludging". They would scale flawlessly, and could be used to compare any past and even upcoming game in the series. If I didn't have to touch the inner workings of the scoreboard ever again, that would be just fine.


Agreed. I've been mulling over this idea once I realized that we could actually penalize users with no scores on a mode by removing their ranking points from that mode. Now, while this is all well and good to say - but as you'll discover, the devil is in the details, so I'm looking forward to see if you've come up with something that would elegantly do this.


I wouldn't ideally do this - each track should be weighted against itself, and not impact the others. That's why we had to introduce a divisor into the Peak Races on SSX 3 (another "kludge" as you call it.) Because it is SOOO long compared to the rest of the tracks in the game, having a really bad time on that track would seriously impact your overall score. The track divisor was used to lower the penalty that not having a good score in those few tracks were leading to.


It's not completely made-up - it's based on the average number of tracks across all the games. The average number of tracks between the six games is 20.1, so Thoul set it at 20. It needs to be relatively low compared to the other games in order to prevent SSX 2012 users from gaining that benefit I mentioned before.

With that said - do I hate doing that? Hell yeah. I would prefer that SSX 2012 stand on it's own without requiring even more kludging.


Ah, I see your point now. Yes, I agree, it is somewhat arbitrary. But it does a good, albeit rough, job of equating to the showoff scores. I've never seen a large dichotomy between racing and showoff as far as the top scores go, and racing is by far the most competitive event across all the games, so 10 seconds can mean a huge difference.



Ideally it shouldn't, but that's the inherent weakness of the racing formula.

Now we come to the part where I tackle less logistics and respond to your guiding principles behind the rankings system.


Agreed. How to do this?


Once we figure out how to do the first one, this one becomes automatic. In other words, of course.



Hmm...now comes the ugly part where gondee works on the math of this proposed formula to find out if it's any good.


Okay, good. This has potential.


Yep, agreed.


Here's where we run into the first major hang up. These two assume that each game is equal in terms of competition, and that is simply not true. No one can look at SSX Blur and say that it is as competitive than SSX 2012. Blur is a dead game, and hasn't had a score submitted for it in years. You can't give the same weight to the same games when the levels of competition are vastly different. No one can say that the one person who is #1 in Blur is the same level in skill as Xedec. We attempted to solve this by judging each game by the percentage of total scores it has in the database:


SSX 2012 blew all these games out of the water by simply having way more tracks, and thus far more scores. This is a problem for which we still haven't come up with a good solution. I am all ears for anything on this front.

On to the other principles:


Agreed, with all of these.


Disagree - because suppose in a future game, they make some very limited mode that only features 3 tracks. Should this be weighted the same as something like racing, which might have 20 or more tracks? Dominating a larger percentage of the game means you are better in skill, and should be treated as such. I think weighting each mode by the percentage of tracks it takes up in the game is a valid way of comparing one mode to the other. If you can think of some way of comparing them differently, by all means, list them.


No, as I mentioned above. There must be some way to quantify the level of skill for each title, or the scoreboard will ring hollow because it will not reflect reality. More popular games are more highly competed in, thus requiring a higher level of skill.


This is the purpose of "Best Racer/Showoffer/etc" in addition to Best Overall. Best Overall would obviously compile all modes.

So it looks like your racing formula might provide a valid way to solidify racing without requiring all the kludging we have to do at the moment. I welcome that 1000% - however it still has some weaknesses that must be addressed to reflect the state of the scoreboard a little better.

_________________

I considered a couple other ideas, too:
- count each track equally across multiple games
- count each overall game score according to how many scores are submitted for that game (current method)
- count each individual track score according to how many scores are submitted for that track

I shifted away from weighting tracks and games unequally, though, for a few reasons:
- Many scores are not really competing for first place... some tracks with 30 scores submitted may be much less competitive than other tracks with only 5 scores.
- You could submit bad scores under a new username just to artificially grow the scoreboards you're the best on & improve your ranking overall.
- Each game (or mode) represents mastery of one game engine / set of controls / set of rules. If a game has 20% more tracks, should it count 20% more? Does it actually take "more skill" to set the best times on 40 tracks than it does to set the best times on 30 tracks, especially when some of those 40 tracks are just shorter versions of others?
- A scoreboard might tend to get more activity if the initial scores posted are pretty bad & people can gradually take over first place from each other. Another scoreboard may get almost no activity because one guy sets the bar unfathomably high just 2 weeks after release and nobody can touch him. Should the first track count more than the second, just because there are more scores? Or maybe the second should count more than the first, because the score was so good that nobody could compete with it?


If you do want to count games unequally, though, the overall scoreboard probably *should* basically just be an SSX 2012 scoreboard. After all, it is almost as big as the previous 5 games combined...
(edit: see related afterthought at the end of this post)

They actually already made modes with limited numbers of tracks. In SSX3, there were separate modes for Big Air, Superpipe, etc. And Blur had Slalom. I didn't consider these to be different enough to warrant a new category, so I just lumped them in with Trick and Race. If a new mode is a completely trivial part of the game, we may be able to just consider it part of a pre-existing group. I think if enough dev resources are spent on a mode to make it actually legitimately different from what we already have, it's going to be featured prominently in the game.



This is a perfectly valid concept; it's just difficult to really determine how much skill something actually took.
Also, consider that the older scoreboards aren't closed. If Xedec wanted to be the Best SSX Player Ever, he could still go back and play those older games. And if Blur, with it's lesser numbers, really does not represent any skill on the scoreboards, he should score very well without too much trouble. But really, why should we give him credit when he hasn't actually done it yet?


I have not even once yet mentioned standard deviation!

Seriously though, yes, you did mention many of these ideas earlier in the thread. Sorry, not trying to claim them as my own original contributions; just trying to flesh out the ideas a bit more so gondee can consider them less abstractly.



Side note: If one wanted to consider most submissions per track, the overall scoreboard would become a Tricky/3 scoreboard. Of the 41 top tracks in terms of number of unique players submitting to the scoreboard, 15 are from Tricky (75% of the game) and 26 are from SSX3 (100% of the game).
(Blur's most popular track doesn't show up in that list until #150 or so...)

Actually, I think when I looked at weighting things by submissions per track, I was weighting each overall game score by average submissions per track. You could instead count every track individually, but weight the track score by the number of submissions to that track... Tricky and 3 would get a bump from having the most popular individual track scoreboards, and 2012 would get a bump from having so many different tracks... if I have time, I'll run the numbers that way tonight & see how it looks.

OK, here's proposal version 2. Some of this is repeated from the last one, but I'll list it all here anyway.

• No placeholder scores.
• Calculate Rank Points (RP) for each track.
• RP(race) = 10,000 * (best time / time) ; RP(trick) and RP(survive) = 10,000 * (score / best score)
• For ALL multi-track scoreboards (including overall best at one game and anything multi-game), Net Rank Points (NRP) are calculated as a weighted average of RP from each included track. Weighting factor for each track is the number of unique users (N) submitting scores to that track's scoreboard.
• Formula: NRP = sum [RP(track#) * N(track#)] for all included tracks/ sum [N(track#)] for all included tracks

This is intended to just be as simple as possible. If we're averaging overall game scores or overall category scores, it's very easy to introduce bias into the weighting formula or into the categorization of game modes. By simply counting every track in every game based on how competitive that scoreboard is, it eliminates much of that potential bias.

Also, in my earlier post I was concerned that people may not submit scores for a track if the top score is too hard to compete with, but I think the elimination of placeholder scores will alleviate that potential problem. With no placeholder scores, you really need to submit scores for lots of tracks to compete for overall titles.

I'll try to post a top ten preview for the composite scoreboards tomorrow. Was going to post that tonight before going to sleep, but my daughter just woke up & I need to get her back to bed.

Here's a preview of some scoreboards using the proposal in my post from last night:

SSX (all tracks) | Click to reveal hidden content

(1) 6,409 GiveMeIce
(2) 6,032 alpmaster
(3) 5,260 grandmort
(4) 4,536 HX YEH
(5) 4,534 FirebrandX
(6) 4,438 BlackDth
(7) 4,317 Obi
(8) 3,881 Alex
(9) 3,854 gondee
(10) 3,791 happymachines
(11) 3,746 Tigerbalm
(12) 3,110 Chris30
(13) 2,997 Elisefan#1
(14) 2,804 ssx4ever
(15) 2,418 Z
(16) 2,375 Evensen
(17) 2,339 Lunt
(18) 2,140 MrChaos
(19) 2,024 Ca$flo
(20) 2,010 Rainbow Rainbow

SSX Tricky (all tracks) | Click to reveal hidden content

(1) 8,769 the hot legend
(2) 6,480 9EDDY2
(3) 5,779 SpicyStunner
(4) 5,454 naya82
(5) 5,104 gondee
(6) 5,065 elgen
(7) 4,745 FirebrandX
(8) 4,495 Frost
(9) 4,230 distancedave
(10) 4,212 ssx4ever
(11) 4,003 happymachines
(12) 3,637 Obi
(13) 3,362 MrChaos
(14) 3,285 Rock AG
(15) 3,194 HX YEH
(16) 2,883 YoshiExcel
(17) 2,666 Alex
(18) 2,478 crispncrunchy
(19) 2,451 twinpeaks^^
(20) 2,446 Excalibur

SSX 3 (all tracks) | Click to reveal hidden content

(1) 9,079 RE Virus
(2) 7,806 Obi
(3) 7,778 IceXtreme
(4) 6,500 Legend_armlet
(5) 5,958 JXR
(6) 5,490 QuotidianPerfection
(7) 5,283 Wanis
(8) 5,150 ChaosRacer2
(9) 4,854 Rastapo
(10) 4,746 laidback0378
(11) 4,683 HX YEH
(12) 4,678 archie
(13) 4,642 Eurg
(14) 4,616 poach
(15) 4,593 JAMdrew
(16) 4,573 898ppp
(17) 4,277 Anticrobotic
(18) 4,144 Legologie
(19) 4,078 Oniz
(20) 3,958 vix

SSX On Tour (all tracks) | Click to reveal hidden content

(1) 8,578 the hot legend
(2) 6,874 HX YEH
(3) 5,484 RE Virus
(4) 5,170 IceXtreme
(5) 5,070 snowman
(6) 4,974 royank
(7) 2,976 erictang05
(8) 2,914 898ppp
(9) 2,066 archie
(10) 1,676 eviltim
(11) 1,545 Aerodynamisch
(12) 1,541 ChaosRacer2
(13) 1,444 Titan
(14) 1,436 MrChaos
(15) 1,225 eastviewgolf38
(16) 1,086 :.MonsterTricky.:
(17) 885 GiveMeIce
(18) 874 iglooman
(19) 852 Yeti
(20) 829 GriffUber

SSX Blur (all tracks) | Click to reveal hidden content

(1) 6,846 MrRuchie123
(2) 6,192 yako591
(3) 5,693 XquietusX
(4) 4,324 LLCoolDave
(5) 3,328 Yeti
(6) 2,785 Obi
(7) 2,637 gondee
(8) 2,496 Deaddy
(9) 2,187 Bombshell=Riggs
(10) 1,232 dfi
(11) 820 Rytyfe
(12) 820 Jesster
(13) 796 mandlebaum
(14) 684 Mwa!
(15) 640 Dr. Spud
(16) 477 StickNutzman
(17) 467 MdX MaxX
(18) 414 fyerblu80
(19) 301 Bildi
(20) 183 jckDsht

SSX 2012 (all tracks) | Click to reveal hidden content

(1) 6,039 the hot legend
(2) 4,876 iTofu
(3) 4,669 Xedec
(4) 4,580 KeKoFruiT
(5) 3,810 yako591
(6) 3,441 Hero Complexity
(7) 3,347 chromemaps
(8) 3,184 minn
(9) 3,108 Madeo
(10) 2,774 Pemdas82
(11) 2,611 RadicalPlayer
(12) 1,946 deadshootme
(13) 1,821 Th3 Apache Dog
(14) 1,786 vix
(15) 1,540 eviltim
(16) 1,519 Zockende_Socke
(17) 1,167 Rocklee1113
(18) 959 FTMMuckymouse
(19) 924 speed0fsound
(20) 913 FKuPAYme

Best Overall Racing | Click to reveal hidden content

(1) 7,742 the hot legend
(2) 4,889 Xedec
(3) 4,279 KeKoFruiT
(4) 3,556 HX YEH
(5) 3,531 iTofu
(6) 3,334 minn
(7) 3,116 Madeo
(8) 3,029 Obi
(9) 2,791 RE Virus
(10) 2,692 IceXtreme
(11) 2,450 RadicalPlayer
(12) 2,391 chromemaps
(13) 2,199 MAD_JO3
(14) 2,036 crispncrunchy
(15) 2,024 vix
(16) 1,879 Th3 Apache Dog
(17) 1,845 GiveMeIce
(18) 1,844 archie
(19) 1,665 ChaosRacer2
(20) 1,603 FirebrandX

Best Overall Showoff | Click to reveal hidden content

(1) 4,170 the hot legend
(2) 3,510 yako591
(3) 3,442 Hero Complexity
(4) 3,038 Pemdas82
(5) 2,811 MrChaos
(6) 2,296 RE Virus
(7) 2,152 Obi
(8) 2,104 Rastapo
(9) 1,978 eviltim
(10) 1,768 Oniz
(11) 1,763 IceXtreme
(12) 1,675 898ppp
(13) 1,550 PD~
(14) 1,546 JXR
(15) 1,491 Alex
(16) 1,367 Wanis
(17) 1,356 fliper
(18) 1,340 iTofu
(19) 1,317 vix
(20) 1,312 Legend_armlet

Best Overall Survival | Click to reveal hidden content

(1) 6,605 Rocklee1113
(2) 5,552 chromemaps
(3) 2,851 MarcusAnnex
(4) 2,310 Ahlyis
(5) 2,291 vix
(6) 2,111 EnjeruKira17
(7) 1,919 KeKoFruiT
(8) 1,785 iTofu
(9) 1,781 the hot legend
(10) 1,631 XahGee
(11) 1,582 Yeti
(12) 1,424 ticklemeozmo
(13) 1,320 EpicPandemic
(14) 1,295 yako591
(15) 766 elmorocks
(16) 685 deadshootme
(17) 650 Salariedcat7507
(18) 615 Hero Complexity
(19) 414 SweetWildFlowers
(20) 406 speed0fsound

Best Overall | Click to reveal hidden content

(1) 5,670 the hot legend (254 scores; average rank 6.57)
(2) 2,408 iTofu (131 scores; average rank 7.33)
(3) 2,399 Obi (51 scores; average rank 8.76)
(4) 2,352 RE Virus (51 scores; average rank 3.18)
(5) 2,340 yako591 (107 scores; average rank 4.54)
(6) 2,306 Xedec (69 scores; average rank 1.07)
(7) 2,261 KeKoFruiT (88 scores; average rank 3.68)
(8) 2,258 HX YEH (77 scores; average rank 8.39)
(9) 2,064 IceXtreme (50 scores; average rank 3.18)
(10) 1,726 vix (117 scores; average rank 12.87)
(11) 1,699 Hero Complexity (67 scores; average rank 1.85)
(12) 1,653 chromemaps (77 scores; average rank 5.94)
(13) 1,572 minn (62 scores; average rank 6.63)
(14) 1,569 Madeo (71 scores; average rank 11.01)
(15) 1,370 Pemdas82 (71 scores; average rank 2.61)
(16) 1,314 Legend_armlet (29 scores; average rank 15.1)
(17) 1,289 RadicalPlayer (43 scores; average rank 3.19)
(18) 1,268 MrChaos (33 scores; average rank 6.52)
(19) 1,201 898ppp (28 scores; average rank 7.82)
(20) 1,178 GiveMeIce (26 scores; average rank 6.31)

(edit: Just realized I'm triple-posting... sorry, bad form on my part.)

Just to break up the mah-"I-love-to-hear-myself-talk"-kra-fest, when can I get the WinonaNET API so I we can one-click submit to the scoreboard?

I just need the values you want me to submit, and how to submit them. Obviously, each track has a number, the game has a number, etc. And I'll use the link for the WinonaNET certificate for the verification image. They should still be validated, but at least it'll save some time and flood your scoreboard with real scores!

I don't really have any input in this thread but I do keep reading the latest posts now and again...



Oh, and I hope this doesn't turn into an argument over who came up with which idea for the scoreboard first either. It isn't a competition, it's a combined effort for the SSX community, however it ends up.

I just wanna reply to say I'm aware of the updates here and want to thank mahkra for all the work he's done on this. I need to work through it all to get a grasp on it all but at a glance it seems to be promising. Ideally, we'll make the process as simple as possible and get rid of ALL the kludging.


It is overcomplicated, yes. Needlessly so when it comes to "kludging" - ie. instituting stopgap measures on the racing formula. With that said, without the kludging, it's still more complicated than the average teenager will be willing to wade through. It must be that way to be sure it's fair to all sides. Can you think of a way off the top of your head to compare racing (where lower scores are better) and showoff (where higher scores are better)?



Yep, the thing is Clawz, it's always been this way on this ranking stuff. My goal is to make it as fair as possible and as representative as reality as we can. I hope we can clear it up too, and make it simpler.

_________________

I just thought of a couple new concepts you could add if the multi-track scoreboards were all calculated directly from the RP of every individual track. (i.e. no intermediate steps with subtotals for different games/modes.)

1. Completed Tracks Only
Select one merq member from a drop-down list. The custom scoreboard will now include all tracks that the selected member has submitted scores for & will show overall rankings based only on those tracks.

2. Head-to-Head
Select two merq members from drop-down lists. The custom scoreboard will now include all tracks that both members have submitted scores for & will compute overall rankings based only on those tracks.

3. Fully Custom Scoreboard
Select/deselect games, modes, or even individual tracks. Choose any subset of tracks to build your own custom overall scoreboard. Omit games that were old before you discovered SSX or individual mountain ranges in 2012 that you just don't like to play.

I was mainly just questioning whether people want a system that is more fancy than it is fair. Yes I can compare racing and tricking scores but it isn't going to be in the way you guys want to compare them, and I understand that so I will leave you to keep developing all the ideas and stuff in here. I will just answer your question though,