Apr 222011

I feel like RSVs (Relative Stat values) are the theoretician’s equivalent to Archeology. No matter how much time you spend doing it, the return you get for your time spent is far less than doing just about anything else.

In his recent post about Mew and RSVs, Tangedyn wrote “Increasing the number of iterations by 100 times to 1million will reduce the Error by 10 times, to ~0.015 requiring 0.0212 difference in RSV, which is far more acceptable.” That is a lot of iterations. In this post, Tangedyn elaborated on the time involved in calculating RSVs: “Takes about 2 hours to test a profile with 1M iterations :/.”

In case you have never done it, download Mew, grab a profile, make sure that the model selected is “Simulator”, and then do two runs, one with RSVs enabled and one without RSVs. You will quickly get an appreciation for how time consuming it is to calculate RSVs. While most people would agree that anything that is extremely time consuming should have a decent return on investment, I hesitate to say that about WoW players who will spend hours upon hours farming for a rare pet. However, RSV calculations do not reward us with a pet. Do they reward us with increased understanding that will help us gear better and conquer the content faster?

Let’s do a few tests and see. First of all, I am not going to do tests with 1 million iterations. I will use 40k iterations (which is still pretty slow). My goal is not to calculate a set of RSVs that can used for my gearing. Rather it is to try several tests to see how how RSVs change as I move my secondary stats around and to draw some conclusions as to whether or not the RSVs can help increase my dps. In my previous post about the homogenization of the secondary stats in 4.06, I already took the position that the difference in the secondary stats is not that much since the 4.06 changes. That would argue against spending time worrying about RSV values. Lets see if that holds true or not.

The setup I am using is starting with a balanced stats profile – Weapon DPS: 703.5, Strength: 142, Agility: 5118, Attack Power: 887, Haste Rating: 936, Hit rating: 604, Crit Rating: 954, Expertise Rating: 188, and Mastery rating: 1987 (this is an ilvl 362 setup). The testing is based on the 4.1 Berserk along with the Berserk changes I covered in my last post. I am not going to make an effort to reforge to different setups, rather I will simply subtract and add to my secondary stats in multiples of 100. For example, I might lower haste rating by 500 and increase crit rating by 500 if I want to test a profile with more crit. The first and most important thing you need to understand about RSVs (other than it can take 2 hours to get numbers that have a high confidence level) is that RSVs are calculated in a small range – about 100 stat points (I move around 91 stat points when I reforge my ilvl 369 chest). If you reforge more than a single piece of gear, it is likely you are changing your stats around more than the RSV code has tested. Enough discussion  – let’s see how different things affect my RSV.

Does the type of fight affect my RSVs:

Fight Description DPS Str Agi AP Haste Crit Mastery Hit Exp Error
5:00 Patchwork 24090.3 2.31 3.04 1.10 .81 1.02 1.19 .99 .98 .08
5:00 Mid-Combat Ravage 25133.7 2.40 3.16 1.14 .86 1.05 1.19 1.03 1.02 .08
5:00 Atramedes 19900.9 1.89 2.50 .90 .61 .84 .97 .89 .89 .07

Across all thee fights, taking into account the error (.08 x sqrt(2) = .11 and .07 x sqrt(2) = .10) we can safely say that haste is the worst stat for my profile on all three fights. It looks like mastery is the best on all three fights, but on the Atramedes fight we cannot say so with a 95% confidence without doing more iterations. It also looks like hit and expertise improve relative to the other stats on Atramedes, but again, we need to do more iterations to have a 95% confidence interval. So fight type does seem to have some effect on the ordering of the RSVs – but not a lot (obviously a fight like Atramedes has a statistically significant impact on all of the RSVs due to lost combat time).

How about the length of the fight?

Fight Description DPS Str Agi AP Haste Crit Mastery Hit Exp Error
5:00 Patchwork 24090.3 2.31 3.04 1.10 .81 1.02 1.19 .99 .98 .08
3:00 Patchwork 24673.0 2.34 3.10 1.11 1.21 1.05 1.19 1.10 1.09 .11
7:00 Patchwork 23977.9 2.31 3.05 1.10 1.03 1.03 1.18 .98 .98 .07

Look at the variation in haste. Even with only 40k iterations per sim run, that is significant. Haste is clearly sensitive to fight length – and in a way that is not obvious without doing more tests. It looks like hit and expertise are more valuable on a short fight but we cannot say that with a 95% confidence – so more iterations are needed to validate that.

What happens if I switch to Glyph of Berserk in place of Glyph of Tiger’s Fury? The previous tests were run with GoTF. Lets try the three fights with GoB and see what happens:

Fight Description DPS Str Agi AP Haste Crit Mastery Hit Exp Error
5:00 Patchwork 24000.7 2.29 3.04 1.09 .90 1.03 1.16 1.04 1.03 .08
5:00 Mid-Combat Ravage 24878.5 2.37 3.12 1.13 .85 1.04 1.16 1.04 1.02 .08
5:00 Atramedes 19948.2 1.89 2.51 .90 .59 .85 .97 .81 .81 .07

Comparing the GoB numbers to the GoTF above, it looks like the difference is not that great – at least for these three fight scenarios.

What if I move around stat points, taking from the lowest and putting them in the highest? I am going to focus on the 5:00 Patchwork fight where Crit is clearly better than Haste – so I am going to shift points from Haste to Crit and see what happens. Technically, Mastery is the better than Crit, but in my actual reforging I am already maxed on on Mastery.

Fight Description DPS Str Agi AP Haste Crit Mastery Hit Exp Error
5:00 Patchwork 24090.3 2.31 3.04 1.10 .81 1.02 1.19 .99 .98 .08
5:00 Patchwork, move 100 24097.7 2.31 3.06 1.10 .95 1.03 1.19 .96 .95 .08
5:00 Patchwork, move 200 24096.1 2.31 3.06 1.10 1.04 1.02 1.20 .97 .96 .08
5:00 Patchwork, move 500 24083.8 2.31 3.04 1.10 .97 1.01 1.21 .92 .91 .08

Once again Haste seems to be more volatile than the other stats, with some movement from Hit and Expertise, although more iterations will be needed to say that at a 95% confidence. The main thing to look at is the DPS line. I shifted 500 stat points from Haste to Crit (ending up with 436 Haste Rating and 1454 Crit Rating) and along the way, the range of the DPS varies only 14 dps from the high to the low. So even if, after a two hour run we can safely say that Crit rating has a higher RSV than Haste, does the potential 7 dps gain from moving 100 stat points from haste to crit justify the time spent?

TLDR: To get valid RSV values takes hours of simulation work. RSV values are only meaningful in a range of 100 stat points. Haste seems to be the most volatile stat (quoting Tangedyn from this post: “I’ve seen Haste’s RSV fluctuate enough that there is probably more than meet the eye, but at the moment there is no real proof pointing to anything as a cause of the the fluctuation or a Haste breakpoint.“). Even large swings (500 stat points) in how you allocate your secondary stats will have very little impact on your overall dps.  Quoting myself from my Homogenization post “The secondary stats are so close in value now that you you can pretty much choose any combination of crit/haste/exp/hit without significantly impacting your dps.” Bottom line – RSV calculations may be valuable to theoreticians, but do not use them as a guide for how to reforge your gear. As always, your time will be much better spent focusing on the fight mechanics and your rotation.

  13 Responses to “RSVs – Fact or Fiction?”

  1. I am still reading through the post and digesting the information, but something jumped out and bugged me. Why does the profile being used only have 142 Strength. I have no Strength gear, but I have the +50 Strength Glove Enchant (BiS) and came out with 162, so I’m just kinda curious, if the profile isn’t using BiS enchants, what is it using?

  2. Probably u have +20 stats enchant on chest either instead of +15

  3. Good post Leafkiller. And thanks for spending the time doing this, so we dont have to :)

    Btw, has Mew been optimized for speed? It does not seem unreasonable that it would take 2 hours to do 1 mio. iterations, but I’m just curious if there is anything to be gained in terms of faster running times. Has anyone tried running a profiler on it and see where the time is spent as the code is now?

    • Well, I last ran TPTP on it a few months ago, and optimized a few of the critical section stuff with caching. I’ll probably do a look at it again when I have time.

      The obvious thing to do for running 1M iterations would be to disable the highfreq which is rather costly.

  4. Good post.
    If anyone knows the answer to this I would love to hear the answer.
    I would argue that no one does a dps rotation perfectly. Which is going to give you the most bang for your buck getting your rotation back on track when you do screw up. My rotation is pretty solid if I do say so myself (I have a fair number of top 200s). But even on my best ones there are spots that I know I pooched.
    I guess my question is which stat after mastery is gonna get the rotation back to where it should be quicker.
    At first glance I would say hit but if you timed it wrong and your rip just fell off you are probably not at full energy. Is the energy regen from haste going to make up for the lost hit getting the rip back up quicker?

    • That is an interesting question – and not something I have seen discussed before.

      I would lean towards crit and hope for good RNG. If you need to get 5 combo points fast, haste is not going to get you there nearly as fast as a couple of crits.

      • This would actually be an interesting thing to test.

        Obviously crit would be essential to bring yourself to 5 combo points quickly, but crit is unreliable.

        Either way you’d end up needing an extreme amount of haste to make any real noticeable difference in such a short sample.

  5. I’d really like to see your results putting Haste at the “breakpoints” mentioned in the post you quoted. Try a RSV Mew run with Haste at say 803 then 853 (the 6.66% breakpoint), then at 903. Adjusting your value for crit accordingly. I am still interested if the values increase and decrease like mine do when I run my tests. I know I’ve harped on it in the WoWforums a bit but I still feel like there is a reason for the fluctuation in haste and that we can take advantage of it.

    I’d even be interested to see the RSV of haste at 852 and 854, but that is me being curious.

    • Almost forgot this. I’m also wondering what buffs you’re giving yourself in those Mew samples. Are you giving the 10% haste buff, or the 3% Dark Intent buff?

      The values I listed assume that you would only get the melee haste buff, which would -in raid- put you at 16.66%. To even further calculate the value of haste (at its true beneficial points which I keep calling “breakpoints”) you could then adjust haste to go up -in raid- to the next “break point” of 19.98 or up even to 20%.

      • No Dark Intent – but it does include the 10% melee haste buff.

        Take the profile in which you are testing your “haste breakpoints” and change the fight length. Assuming that haste allows you to an extra attacks at certain points which causes a step increase in dps, this would be subject to the number of OOC procs, the fight mechanics and also to the length of the fight (which I did test and saw haste jumping around in value). I don’t think you will find a set of “numbers” that you can hang your hat on as being points of optimization for haste rating.

  6. As you note, the main problem to me with the RSV are that they are only applicable in a very “local” part of the gearing.

    Something I would find interesting would be to do maybe a distributed computing effort to map out the DPS for a larger range of posisble stats. By putting this into an n-dimensional matrix it should be possible to interpolate between the points to get a fairly accurate dps estimate for a certain gearset.

    Of course this would only be relevant for a specific fight type, a certain spec, rotation etc. And it would also assume that there were no step responses in the stat values.

    Apart from this I suppose the issue would be how many points that needed to be calculated to get a reasonable dps estimate in any point.

  7. Are you using encounter duration jitter? If not it may account for the “large” variances in haste’s value.

    In addition to that with 40k iterations you’ll end up with a margin of error on the RSVs that means you’ll get inconclusive results (as you mentioned). If you come on IRC and tell me what you want tested I can run some 100k iterations (these result in 0.054~ error).

    Ultimately though as much as tangedyn and I have tried to work out the cause of the value of haste going up or down, there’s little conclusive evidence.

    The value shoots up when you use glyph of berserk, unheeded warning and have a lot of hit.

    With all things considered, the RSV section of mew is largely a tool for the advanced user and generally for most players it’d be easier for them to follow a best in slot list.

    There are no break points but the relationship between crit and haste specifically acts like a sine wave. You’ll end up with mew reporting one stat being better than another stat, then if you increase that, the value of the other will shoot up.

    You really should set NOTHING by this, honestly NOTHING, because the important number for most users is the end result (the output DPS).

    I’ve managed to get the best results with a slightly balanced profile (running 1m iteration tests, these only take me about 45m, Yawning would have the same calculation time as we both have the same CPU, if you want us to run stuff just ask), that is to say that more or less mastery + x items are left alone pretty much, I can show you this on irc if you wish.

    There is actually theoretical break points with haste related to glyph of berserk. You can reach the point where haste is worth equal to or more than mastery simply because the haste allows a guarenteed extra shred during the berserk. However, this occurs only when you’re using Unheeded Warning, presumably because you’re almost guarenteed a proc of the trinket during the berserk.

    Likewise there is funkyness going on with hit and expertise, it’s more than possible (and actually fairly common with certain trinket and glyph combinations) to have hit and expertise valued higher than crit and haste. I’ve not managed to get this to the point where I can draw any conclusions from it but it appears to be when you’re at a sufficiently high amount of crit/haste.

    Will there be a “break point” for haste that all right thinking cats go for?

    I can categorically say no. Why? Well as we all know the reason haste is valuable is because of the extra energy and auto attacks, nothing more, you won’t get an extra tick out of Rip, you won’t get a stepped value of haste at all (other than if you’re using UW and goberserk, see above) simply due to fight duration.

    Simulating a fight that lasts long enough (without duration jitter, which you should ALWAYS have on) that you get x.9 extra shreds will result in the value of haste increasing massively as if you were to get the extra haste you’d get a whole extra shred.

    As far as i’m concerned it’s always important to remember that theorycrafting and simulations are simply a tool. Sure, you can optimise your gear based on the simulation, but if you’re unlucky, or a fight has a special gimmick or lots of movement that the script doesn’t properly simulate, it’s down to you as an individual to gauge how to make the most out of your character.

Leave a Reply