[wintermute]

Over the last few weeks I've spent a lot of time browsing the SBR forum. A number of threads referenced a company called Accuscore - website is accuscore.com. They are in the business of predicting winners of professional sports contests using simulation techniques. I myself have written a baseball simulator with limited success so I started looking around to see if I could find out how well Accuscore has done.

To make a long story short, I sent the good folks at Accuscore about $30 to try out their system and more importantly to me - get access to their archived result files.

Here's what I found out.

Accuscore calculates the probability of each team winning a game by running 10000 or more simulations of the contest. If the calculated winning probability of the visiting team is greater than the probability assigned to that team by the sports book that you use, then you bet on that team. If not you bet on the opponent. You bet as close to line closing as possible.

7799 games were bet from April 3, 2006 to May 8, 2009 for a return of $26240.78 using $100.00 wagers. That's a ROI of 3.365 percent.

I've read comments in this forum to the affect that that's a pretty mediocre result - not so!!!

How do you determine how good a result that is? The way I do it when evaluating my own simulator is to use Monte Carlo techniques. I build a file containing one entry for each game I simulated - generally I run a simulation for the regular MLB season so I will have about 2430 games in the file. Each game entry has two numbers, the money you would have won on a $100 wager if you had picked the winning team and a $100 loss.

You then randomly pick one of the two numbers for each game entry in your sample and sum the numbers. That is one simulation. You do that a large number of times, keeping track of whether or not the sum exceeds the amount earned by your simulator.

For Accuscore the results for 100000 simulations were

11636 simulations had a sum > 0.00
1033 simulations had a sum > 10000.00
32 simulations had a sum > 20000.00
1 simulation had a sum > 26240.78

In other words, the probability of doing better than Accuscore is 1 in 100000 or .00001 That is very good. Incidentally I didn't use Ganchrow's Z technique since for a large number of games, the amount lost due to the vig becomes very large and is more and more difficult to overcome. In an 8000 game sample using an 8 cent line and $100 wagers you can expect to give the book about $12000. The Z method assumes no vig.

More to come...

[tweek]

From everything I've seen from Accuscore (though I haven't paid them any money), their record keeping is sketchy at best.

I wrote a script to go through all 2008 games and grab the accuscore probabilities (readily available from ESPN's preview page for each game), and ran it against my own personally logged lines from last year.

Needless to say, it was -EV...

What data do they provide to backup the $26k number?

[wintermute]

One of the problem's I had with the Accuscore result files was that the ROI was greater when the calculated probability was closer to the book probability than when it was further away.

What I mean by this is the following - assume vcalc is the calculated probability for the visitor and vprob is the book probability for the visitor. There is a corresponding hcalc and hprob for the home team.

When ( vcalc > vprob + 0.035 ) or ( hcalc > hprob + 0.035 ) the ROI is only 0.077 percent on 4644 games, barely break-even.

When ( vcalc > vprob and vcalc < vprob + 0.035 ) or ( hcalc > hprob and hcalc < hprob + 0.035 ) the ROI is 8.205 percent on 3155 games!!!

What gives? I find this fairly counter-intuitive. Surely if you calculate a probability well in excess of the book's probability, you expect to do better than if the calc is close to the prob. The only thing I can think of other than the possibility of cooked results is that a big difference in the probabilities means that the simulation is missing key data that is being accounted for by the book's line and should be ignored. Any other thoughts out there?

More to come...

[xyz]

When Accuscore computes its ROI, which book's odds do they use? There are substantial differences in using Matchbook, 5 cent lines, dime lines, and 20 cent lines.

[wintermute]

tweek and xyz:

They give you their calculated probs and the closing numbers they used to pick one team or the other. They don't say where the closing numbers come from but most seem to be very close to Pinnacle's - usually an 8 cent line. I calculated the $26K myself - it's not the same as their number but very close.

They say on their site that the ESPN numbers are preliminary and are not updated as we move closer to game start time. Lineup changes, weather changes can change calcs. When you pay you get all the changes as they happen.

[Jacey]

Originally Posted by wintermute

Lineup changes, weather changes can change calcs. When you pay you get all the changes as they happen.

So this is why you'd want to wait right up until game time, as opposed to a good number early. When a lineup is announced, or pitching change, etc, is it real time, or almost real time, updated on their website?

Using matchbook #s, especially now with the incredibly small juice, you could probably get a better ROI, I wonder how much

[Jacey]

site acting up

[Jacey]

weird

[Jacey]

f

[Jacey]

l

[Jacey]

gah

[twister]

Originally Posted by tweek

From everything I've seen from Accuscore (though I haven't paid them any money), their record keeping is sketchy at best.

I wrote a script to go through all 2008 games and grab the accuscore probabilities (readily available from ESPN's preview page for each game), and ran it against my own personally logged lines from last year.

Needless to say, it was -EV...

What data do they provide to backup the $26k number?

I don't know how you managed to turn Accuscores +78 units over the season edit: 2008 season (the way I tracked it) into -EV?

Are you sure you know how their system works? Firstly, the "Free Previews" from ESPN are usually anywhere from 8-12 hours out of date - Accuscore release their final simulation about 1 hour before game time.

Secondly, their system is based on LINE VALUE - not on playing the ML on whichever team is winning in the simulation.

The way their line value works, is that you take the closing ML from Pinnacle [in reality, an impossibility, thus I used the line about 3 mins before game time - which would be ample time to compute the "line value" and place the bet] and turn it into a "no-vig" probability.

i.e -104 -104 = 50%, 50%
-110 +102 = 51.4%, 48.6%

You compare this probability to the final accuscore simulation. The side that has the value is the side where the accuscore simulation is higher than the no-vig probability.

i.e for the second example, where the pinnacle no-vig probability is 51.4% for the road team, 48.6% for the home team, if the accuscore simulation went 50%-50%, you would take the home team on the ML. Alternatively, if the simulation was 52%-48%, you would take the road team on the ML.

So yeah, doing it like that ended the regular season with +78 units for me.

I no longer use accuscore, but if you have the will power to be able to bet every single game throughout the regular MLB season [it is futile doing it here and there as it goes through hot/cold streaks that are unpredictable], then you can pretty much guarentee yourself a winning season. They have been doing it since 2006 I think, I only jumped on the bandwagon last year.

Oh, the reason I no longer use them is because I wanted something I could control myself [I don't want to rely on Accuscore for my picks when next year they could close up shop], so I'm spending this season developing my own system. If you have no desire to "make your own picks", I would highly recommend Accuscore. Better your money served there than with some crook scamdicapper or Morrison-System type scam.

[tweek]

Originally Posted by twister

I don't know how you managed to turn Accuscores +78 units over the season edit: 2008 season (the way I tracked it) into -EV?

As I said, I did use ESPN preview numbers... From the stuff they post for free it didn't seem to me as though they were worthy of an investment...

[wintermute]

A potential problem with interpretation of Accuscore's results is the fact that Accuscore compares their calculations with a closing line. Well, which closing line? Lines at Pinnacle can change quite a bit over the last 5 minutes before the game starts and you can't wait forever to get your bet down. Accuscore can choose a line for result purposes that favors them and who's to know for sure whether they did it.

For example, vcalc calculated by Accuscore is 0.4323 with a closing line of 132, -140 with the visitor winning. That's a vprob of 0.4249 and $132 in the bank for a $100 wager. What if the actual line had been 131, -139. vprob would have been 0.4267 and Accuscore would have had a win but for only $131. 130, -138 for a vprob of 0.4285 would also have worked but for a win of $130.

For an even bigger gain, a line that gave probabilities very close to the calculated probabilities but for a loss could potentially be changed into a win by moving the line a cent or two in the right direction. This might explain why results are better when calculated probabilities are close to the book probabilities. They could be better because they have been adjusted to favor Accuscore.

To see what effect slight line changes have on Accuscore's results I reran the ROI calculations with the lines adjusted up and down by up to 5 cents.

In the following table, an adjustment of x means that x cents have been added to vclose and x cents have been subtracted from hclose.

Adjustment ROI
5 17986.19
4 17912.50
3 20849.54
2 23243.76
1 23673.82
0 26240.78
-1 23616.44
-2 20790.61
-3 18981.77
-4 17990.02
-5 16072.02

As you can see, results deteriorate as the line moves away from the recorded line. You still have a healthy profit but not quite as good as they claim.

I think at this stage of my examination of the data that Accuscore has a working system. It's open to conjecture however just how well it works.

You might be able to get a better result by shopping for better lines but very few books other than Matchbook ever beat Pinnacle's lines and some of the line advantage at Matchbook will go towards higher premiums.

[wintermute]

Answer to a PM:

ROI on dogs

$22070.89 on 3738 games, 5.904 percent.

[Bsims]

Originally Posted by wintermute

One of the problem's I had with the Accuscore result files was that the ROI was greater when the calculated probability was closer to the book probability than when it was further away.

What gives? I find this fairly counter-intuitive. Surely if you calculate a probability well in excess of the book's probability, you expect to do better than if the calc is close to the prob. The only thing I can think of other than the possibility of cooked results is that a big difference in the probabilities means that the simulation is missing key data that is being accounted for by the book's line and should be ignored. Any other thoughts out there?

More to come...

I've been running simulations since 2004 and have observed the same thing. I assign and expected return based on the win probabilities and the odds. I've divided the expected returns into groups by size. The results improve as the expected return increases, to a point. Generally the top group performs a bit less than the ones right below it.

Like you, I concluded that there is some information available that was not considered in the simulations, but somehow is reflected in the odds.

[Pokerjoe]

Originally Posted by wintermute

One of the problem's I had with the Accuscore result files was that the ROI was greater when the calculated probability was closer to the book probability than when it was further away.

What I mean by this is the following - assume vcalc is the calculated probability for the visitor and vprob is the book probability for the visitor. There is a corresponding hcalc and hprob for the home team.

When ( vcalc > vprob + 0.035 ) or ( hcalc > hprob + 0.035 ) the ROI is only 0.077 percent on 4644 games, barely break-even.

When ( vcalc > vprob and vcalc < vprob + 0.035 ) or ( hcalc > hprob and hcalc < hprob + 0.035 ) the ROI is 8.205 percent on 3155 games!!!

What gives? I find this fairly counter-intuitive. Surely if you calculate a probability well in excess of the book's probability, you expect to do better than if the calc is close to the prob. The only thing I can think of other than the possibility of cooked results is that a big difference in the probabilities means that the simulation is missing key data that is being accounted for by the book's line and should be ignored. Any other thoughts out there?

More to come...

I agree. In fact, I hold that as one of any system's tests: is it more effective as it varies from the line.

[Johnny 55]

Hope this info is useful to those using Accuscore:

I have been using Accuscore for the last 2 years to great success using a simple system where I play everything that has an EV of 16.0 or higher on professional sports and 25.0 or higher on college sports, I do not play hockey. For your basic spread lines that is Accuscore predicition of 60% and in college around 64% and in baseball most of the plays that have an EV of 16.0 are runlines and a few totals here and there, mostly unders. The baseball runlines have been extremely profitable and the NFL Totals were over 60%. There are definitely some problems with it, it is late updating injuries and can falter in late season situations when team's motivations come into question. Another nice thing about the daily pick predictions is you can get them through ESPN insider for $20/year instead of paying the Accuscore site $400/yr. for the same thing you get through ESPN.

[FrozenMAN]

yeah i've tracked the top 3 predicted winngin percent plays, ML/RL for each one, since April 9th. it started pretty bad given it was very early in the season and was mainly based on last year's stats and such and quickly got down 19.72 nits, that occured on April 16. Since then the predicted winners have been pretty good having 18/24 as winning days and for the season it's currently at +1.71 units going into today.

todays top plays are
rays 79% WINNER ML/RL +2.0 units
reds 66% LOSS ML/RL -2.38 units
red sox 66% WINNER ML/RL +2.0 units

for the day UP 1.62 UNITS

[Enforcer_23]

Originally Posted by FrozenMAN

y Since then the predicted winners have been pretty good having 18/24 as winning days and for the season it's currently at +1.71 units going into today.

todays top plays are
rays 79%
reds 66%
red sox 66%

[FrozenMAN]

yeah well see dude

[HUY]

So can someone run the picks agains their own lines and post the ROI?

[jaypavs]

I used accuscore for 8 months and bet every game where there was an edge and made money. I used every low juice sportsbook available and was kicked out of (or limits reduced ) everyone of them except matchbook. I got the best lines and the best vig available. I hit about 51.7%. I stopped because my results never matched theirs, they claimed to be hitting 54-56% which was not true. I figured if they were going to lie about their results how could I trust them going forward. Their closing pinnacle odds are right on in games where the line isn't an issue, yet when the final score is within 1/2 to 2 points of the closing lines that is when they deviate from the actual close to make their results better. I also found that the closer the simulation was to the point spread the better they claimed their results to be which makes no sense at all. Why would games where the simulation point spread is 1 point off perform at such a high rate (57%). If that were the case and the line had been 2 points the other way you would have lost over 50% of the time, this always bothered me and for good reason.

There is definitely some value in their sims but don't expect to hit what they claim, a 1/2 a point is the difference between winning and losing in this business and it doesn't take much to say a line was -6 rather than -6.5 to change a 50% losing player to a profitable 52% player.

[Pokerjoe]

Originally Posted by jaypavs

I used accuscore for 8 months and bet every game where there was an edge and made money. I used every low juice sportsbook available and was kicked out of (or limits reduced ) everyone of them except matchbook. I got the best lines and the best vig available. I hit about 51.7%. I stopped because my results never matched theirs, they claimed to be hitting 54-56% which was not true. I figured if they were going to lie about their results how could I trust them going forward. Their closing pinnacle odds are right on in games where the line isn't an issue, yet when the final score is within 1/2 to 2 points of the closing lines that is when they deviate from the actual close to make their results better. I also found that the closer the simulation was to the point spread the better they claimed their results to be which makes no sense at all. Why would games where the simulation point spread is 1 point off perform at such a high rate (57%). If that were the case and the line had been 2 points the other way you would have lost over 50% of the time, this always bothered me and for good reason.

There is definitely some value in their sims but don't expect to hit what they claim, a 1/2 a point is the difference between winning and losing in this business and it doesn't take much to say a line was -6 rather than -6.5 to change a 50% losing player to a profitable 52% player.

Well, if you're letting them pick the lines upon which they are measured, if they're fudging the lines to pick up some wins on close (to the spread) results, this is what it would look like.

[FrozenMAN]

playing the top 3 of the day ML/RL woulda got ya +1.62 units and to date you'd be at +3.33 units

[wintermute]

A final kick at the Accuscore cat.

If you bet all the games where ( vcalc > vprob - 0.01 and vcalc < vprob ) or ( hcalc > hprob - 0.01 and hcalc < hprob ) you have

a return of -13876.89 on 850 games ( -16.326% ROI )

If you bet all the games where ( vcalc > vprob and vcalc < vprob + 0.01 ) or ( hcalc > hprob and hcalc < hprob + 0.01 ) you have

a return of 10663.14 on 852 games ( 12.515% ROI )

Winning at a rate of 12 percent when the calculated line is 5 cents or less better than the book's line yet losing at a rate of 16 percent when it's 5 cents or less below the book's line sure look likes like cherry picking of closing lines to me.

[Wrecktangle]

I'm not surprised at this. Long ago I tried every model that could be had (bought) at the time, and every one crapped out, most in a very short time. One of the best was MicroBros WinPicks, which held up for a while due to the fact that you could search for winning formulas yourself. For a while, I could drop out data which I used to check my own form my modeling. This proved to be the best use of their product.

[Rich Boy]

Cant beat the books...