Does John Hollinger trust PER? Should we?

John Hollinger created Player Efficiency Rating (details about its calculation here). As John himself puts it,

The PER sums up all a player’s positive accomplishments, subtracts the negative accomplishments, and returns a per-minute rating of a player’s performance.

 

After a successful writing career for ESPN.com, the Memphis Grizzlies hired Hollinger as their VP of Basketball Operations. In his first season as VP of Basketball Operations, Hollinger’s most significant move was trading Rudy Gay for Tayshaun Prince, Austin Daye, and Ed Davis.

From a Wins Produced perspective, that’s a heck of a deal. Rudy Gay is unremarkable; Prince is solid; Daye seems to be productive when he’s played on the perimeter; and, Ed Davis is among the most underrated young big men in the league.

But what about from the perspective of PER?

Prior to the trade, Rudy Gay’s PER was 14.1, just below Hollinger’s average of 15. Prince’s? 12.7. Daye’s was also 12.7 in a weird coincidence, and like WP, PER says Davis is the best player of the bunch at 18.2.

Interesting, right?

When we look at minute distribution since the trade, it’s stranger still. For all intents and purposes, Tayshaun Prince is playing Rudy Gay’s minutes. And even though Daye and Davis have had some good moments, they’re not playing enough to matter.

Practically speaking, this trade was Rudy Gay for Tayshaun Prince.

From the perspective of PER, we’d expect the Grizzlies to be a little bit worse, at least so long as we look at it from a Prince-for-Gay standpoint, which is what it is in practice. Not a lot worse, but a little. But the Grizzlies are not worse. In fact, they’ve been quite a bit better since the trade (obligatory small sample size qualifier . That good play hasn’t been because TAyshaun Prince is playing like an All Star. He hasn’t been. But, Tayshaun Prince is quite a bit more productive all things considered, and that is certainly helping.

The question that then begs to be asked is, “Does Hollinger trust his own metric?”

There are a couple obvious counterpoints to my implicit point in that question. I’d like to quickly respond to some of them.

First, for those who followed the trade as it happened, it was obvious there was a financial reason to do this deal (and the one that immediately preceded it). Fair enough. But Hollinger flat out says that he believed Tayshaun would be able to replace Gay’s production, and that there were plenty of “basketball reasons” for the trade. So it wasn’t just a financial decision; it was also a basketball one. Reading between the lines, at worst it’s a lateral move that saves money. At least this year anyway, as Prince is already beginning his post-30 decent.

Second, Hollinger’s own trade machine predicted the Grizzlies would be better after the trade, but that prediction is heavily influenced Ed Davis’ PER, and as I mentioned, Ed Davis has hardly seen the floor. If this trade were about Ed Davis, this is an awfully strange way of showing it.

Third, you could also say that Davis is a big part of Hollinger’s plans, but from a long-term rather than short-term perspective. We’ll see how they cross that bridge when they get there, I guess, but color me skeptical. It’s hard to see them committing long-term, significant money to Davis after making all these money-saving cuts, and young big men like Davis tend to get MLE deals or better off rookie scale contracts. In a league where Darko makes millions, Ed Davis probably will too. That whole short supply of tall people and all.

To be fair to Hollinger, Rudy Gay isn’t a superstar by PER. He’s had some good, but not great, seasons. Even if Gay is as good as his best PER season, it’s reasonable for Hollinger not to want to pay him max money.

But my critical question is still worth asking, because it highlights the critical flaw in PER: players like Rudy Gay – who score lots of points by taking lots of shots – get significantly overrated by it, and players like Tayshaun Prince – who don’t score tons of points but are competent players overall – get underrated by it.

And maybe Hollinger knows this.

I am certainly anxious to see more of what he does so we can find out.

Advertisements

8 thoughts on “Does John Hollinger trust PER? Should we?

  1. I think you are commiting the mistake of imposing your way of seen metrics into Hollinger’s. If you go back to his columns in ESPN he has always used PER as more of a guide than as en end all number (I m pretty sure that he would rank KG ahead of Al Jefferrson to use an example two players that had exactly same PER difference as Prince/Gay), so that 1.4 diff might seem much smaller to him. Evenmore important Hollinger has always been a huge believer in usage. Prince usage in DET was 18, Gay usage in MEM was 25.5. I d say it’s not a stretch that even if you believe the1.4 drop in PER is a real downgrade, that 7.5% of possessions that suddenly gets moved towards the Gasol (19.4), Z Bo (18.1), Conley (17.0) trio would make for it. PER has always claimed that Gay was overrated and Hollinger in his columns has awalys been pretty cold towards ballhogs that have good but not great PERs..

    • I’m not quite sure what this means. Can you clarify?

      Evenmore important Hollinger has always been a huge believer in usage.

      To me, usage by itself doesn’t really say much. Usage relative to other box score stats matters, I think. Is that what you’re getting at?

  2. To expand on Filipe’s point, Hollinger wrote:

    “Bear in mind that PER is not the final, once-and-for-all evaluation of a player’s accomplishments during the season. … What PER can do, however, is summarize a player’s statistical accomplishments in a single number. That allows us to unify the disparate data on each player we try to track in our heads … so that we can move on to evaluating what might be missing from the stats.”

    http://espn.go.com/nba/columns/story?columnist=hollinger_john&id=2850240

    Essentially, Hollinger doesn’t believe PER effectively ranks players. He believes it effectively ranks players’ traditional box-score statistics. After setting that baseline, there is obviously a whole lot else that goes into ranking players, and I think Hollinger would admit as such.

    • Essentially, Hollinger doesn’t believe PER effectively ranks players. He believes it effectively ranks players’ traditional box-score statistics.

      So another way of saying this would be that Hollinger doesn’t trust PER to evaluate players.

      My language is admittedly more cynical and sharper, but at the end of the day, we’re saying the same essential thing:

      If you’re going to evaluate players, you’re not going to get that far with PER.

      • Sure, you can put it like that, and I don’t think Hollinger would disagree. You’re not going to get that far with any commonly used all-in-one metric, PER included. They’re just starting points, but helpful starting points.

    • Here’s the thing: when Hollinger says stuff like that, it seems an awful lot like a fudge factor that allows him to hide from criticism (such as the criticism that as long as you’re shooting > 35% you can raise your PER simply by shooting a lot).

      So he introduces other things that are outside the traditional statistics. In this article, he mentions defensive specialists (which we can actually capture with statistics and could to some extent in April, 2011 when that was published).

      But as far as I’m aware, he never actually explains what those “other” things are. By definition, if some action is impacting the outcome of a game, it has to be tangible and quantifiable. So what are those things?

      In other words, this language feels like an excuse made after his model has come under intense scrutiny. And it feels like an excuse because he’s never made an effort to capture any of those “other things” he says exists.

      • “By definition, if some action is impacting the outcome of a game, it has to be tangible and quantifiable.” Of course. But it’s not necessarily quantifiable by outsiders.

        Example: The Pistons can count what percentage of the time Jason Maxiell makes the correct defensive rotation. We can’t, because we don’t know precisely what the correct rotation is all the time.

        There are hundreds of things players do that can’t be capture in or derived from box score numbers. Teams, with perfect information about on-court responsibility, can tally that those stats. I don’t see how we can do it without speaking with scouts/coaches/etc. to develop a better understanding on a case-by-case basis of each team’s scheme, and even then, outside observation is still just an estimate.

  3. Pingback: Analytics Experience Wanted in Sports Execs Today

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s