A Brief Comparison of GotBuckets RAPM to ESPN RPM

11927423-mmmain

Today’s article is by guest contributor Ben Dowsett.  Follow him on twitter @Ben_Dowsett.

As will be the case from time to time, the world of advanced basketball metrics has been abuzz in the past week with ESPN’s release of RPM (Real Plus-Minus), their variation of a type of metric becoming more and more popular among smart NBA people, Adjusted Plus-Minus.  Derived from Jeremias Engelmann’s xRAPM (Expected Regularized Adjusted Plus-Minus), RPM sparked several types of conversations – how useful it could be as an overall player metric, whether its inclusion on ESPN (where it will be seen by many more casual fans) might paint the metric out of context to many, and even whether the name Real Plus-Minus sounds right.  And some people certainly just don’t care about the perfect catch-all number and would rather focus on the fabulous feats of athleticism occurring every night, unbothered by extremely complex statistical regressions.

But though not fully ignored, particularly among those who were already invested and knowledgeable on other APM-type metrics, a topic that didn’t receive quite enough attention was this: While RPM is now the most publicized Plus-Minus statistic, there are several others available.  So while comparisons to already-existing metrics like PER and Win Shares are interesting, to be sure, perhaps a way of evaluating RPM in better context would be to compare it with some of the APM metrics already in circulation.

And as it just so happens, GotBuckets is among those with a head start on this particular game.  What follows is a comparison of ESPN’s RPM with the @talkingpractice GotBuckets’ RAPM, using recent RPM and RAPM data.

Simple Numbers:

For starters, it’s important to note the different ways the two metrics are informed.  Without delving too deeply (more detailed description for RAPM here, for RPM here), the key difference is in box-score numbers – while RAPM is informed only by prior seasons’ RAPM and aging, RPM is additionally informed by box-score statistics and additional factors like player height, as well as prior seasons’ RPM.  This can create an improved, more complex metric in terms of number of variables, but can also cause certain biases to emerge – something we’ll get to in a bit.

I ran a few simple linear regressions between the two sets of data to see if any major predictive patterns emerged.  For these regressions, and for the purposes of this entire piece going forward, I have removed the bottom 25% for total minutes played from the data with the goal of reducing skew and any out-of-context outliers.

After doing so, the two sets show a 90% correlation (.825 r-squared value) with each other, not surprising given their many similarities.  Standard deviation for RPM (2.833) is somewhat higher than RAPM (2.118) – LeBron James, who leads both metrics, is just over three standard deviations above the mean for RPM but slightly over 3.5 above the mean for RAPM.  Offensive and defensive splits for each metric follow roughly the same overall trends with very similar relationships.  RPM shows a slightly more significant relationship with another overall player metric, Win Shares/48 Minutes, but this is to be expected given RPM’s box-score informed nature.

One element of this regression was of particular interest or surprise; when comparing the two metrics with certain others from basketball-reference.com, a large difference is seen in both metrics’ relationship with offensive versus defensive Win Shares.  RPM and RAPM offensive splits both show just shy of 70% positive correlation with Offensive Win Shares, but curiously, their corresponding defensive splits are only around 40% correlation with Defensive Win Shares.  This likely relates to Defensive Win Shares assigning defensive strength to every player on a good defensive team, and vice versa on bad defensive teams; Donald Sloan of the Indiana Pacers is considered 5 points per 100 possessions better on defense than Larry Sanders of the Milwaukee Bucks.

Player Trends:

Moving from the specifics of the raw numbers to the more general area of trends within and between the two sets of data yields some more interesting results.  In particular, the types of player that each metric values significantly more than the other gives us some insight into how the box-score stats weigh into RPM’s calculations.

RPM Favors:

First, I examined the players RPM values significantly more than RAPM, by the difference in their “rankings”.  Looking separately at the offensive and defensive splits – of the 30 players with the largest differential between their RPM defensive “ranking” and their RAPM defensive “ranking”, a whopping 27 were either centers or power forwards.  Defensive rebounding, in particular, appears to be a highly relevant category here; DeAndre Jordan, Andre Drummond, Pau Gasol, Gorgui Dieng and others reside in the top 25 league-wide for both “RPM defensive favoring” and defensive rebounding percentage.  We can also clearly see how RPM’s inclusion of height as a variable plays into the differences between the metrics, particularly defensively, as many of RPM’s favorite defenders are simply quite tall, including those above.

Offensively, the players RPM prefers over RAPM show a slight positional skew in the opposite direction, though not nearly as pronounced as the skew for defensive differential.  Eighteen of the top 30 RPM-favored offensive players by ranking differential are guards, and several of the highest differentials belong specifically to guards who share a couple characteristics: They play in elite offenses, but are complimentary players within these offenses that feature at least one All-Star offensively.  For example, Mario Chalmers and Norris Cole are third and fourth, respectively, for RPM-favored offensive ranking differential (Chalmers is in the bottom half according to RAPM, but in the top quarter RPM) – both get the pleasure of playing with LeBron and the rest of the big three in Miami.  Reggie Jackson and Thabo Sefalosha are also both within the top 10 for “offensive ranking differential”, and it’s likely not a coincidence that both play with Kevin Durant frequently.

Bigs with the highest RPM-favored offensive differential are, again, generally quite tall, and are some of the ones we’d traditionally consider the most offensively talented.  Marc Gasol and Marcin Gortat, both well-rounded offensive big men, make appearances here.  But the poster boy for this category, and for RPM-favored players in general, has to be Brook Lopez – in fact, Lopez leads all qualified players for both offensive and defensive RPM-favored ranking differential; combining both ends of the court, he sits in the top 20% for RPM, but in the bottom 40% for RAPM.  It’s easy to see why RPM likes him offensively, as he’s one of the most skilled and varied post players in the league, and his somewhat poor rebounding defensively is mitigated by his height and good shot-blocking numbers.  It’s quite possible that sample size plays a role here, given Lopez’s injury and the “regression to the mean” factored into the RAPM every off-season, but Lopez represents several of the types of trends seen here – height is a big factor for RPM versus RAPM, as are certain areas of the box-score, and this is where the largest differences appear to lie.

RAPM Favors:

When we look at the other side of the coin, players RAPM values significantly more than RPM, we see some different trends – to the keen eye, this comparison showcases some of the value inherent in a non-box-score-informed metric like RAPM.  Across the board, RAPM favors fewer “household names”, but many of the higher-regarded players compared with RPM are guys who showcase at least one particular skill that’s typically underrated or uncaptured altogether by a traditional box score.

Offensively, the top of the list is dominated mostly by big men.  A few of the top examples, like Chris Bosh and DeMarcus Cousins, are highly-regarded offensively for more traditional reasons (Cousins falls 22nd for centers on offense in the RPM list, but 5th for the RAPM).  Others, though, are strongly preferred by RAPM despite not producing in many of the same ways.  Bismack Biyombo, for example, ranks 59 spots higher for RAPM than RPM after my minutes adjustment (333rd versus 274th – not elite in either case, but a large differential), despite virtually no traditional offensive skill set whatsoever outside the ability to jump in the air reasonably well.  Why is he present, then?  Well, maybe because he’s one of the league’s elite screen-setters, a skill that’s basically impossible to quantify without optical tracking data; ESPN Insider Amin Elhassan recently did a piece on the best big screen-setters in the league using Vantage tracking, and Biyombo made an appearance.  Chuck Hayes (2nd-largest RAPM-favored offensive ranking differential) is another example – he’s undersized, which is a mark against him in RPM, but he does many little things well (screens, box-outs, etc.) and his teams are consistently better when he’s on the floor.

To those anticipating some groundbreaking findings herein, I certainly apologize – the results of my comparison line up almost exactly with what I and most statistically-capable folks would expect given the parameters of each metric.  RPM, with its additional height and box-score informed priors, leans in that direction – and to be sure, there’s great value in examining this added context.  RAPM, along with other more “purist”-style APM metrics, contains no such bias; while this may make certain players’ ratings more dependent on their team contexts, some people certainly prefer the approach of letting the scoreboard do most of the talking in a PM metric.

In the end, like any advanced metrics, proper context is paramount for evaluating the potential merits of either RPM or RAPM.  Try as we might, we’ll never invent a single “tell-all” metric – and who would want to?  This game is so much fun because of its incomplete information and inherent complexities.  Advanced Plus-Minus numbers provide clearer focus for another portion of the picture, and they’ll continue to gain popularity as more folks become comfortable with their correct applications.  We remain in a very exciting time for the game of basketball, and I can’t wait to see what’s coming next.

One Comment

on “A Brief Comparison of GotBuckets RAPM to ESPN RPM
One Comment on “A Brief Comparison of GotBuckets RAPM to ESPN RPM
  1. Pingback: 10 Surprises from the Release of ESPN’s “Real Plus Minus” Numbers - Quirks & Quiddities

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>

Current ye@r *