[Ganchrow]

I've recently starting using Pinnacle's XML line feed again (to Pinny's specifications) but am extremely disappointed with the speed of service -- especially during peak times when it becomes essentially unusable.

Is anyone having any luck with this? I think I may need to go back to scraping their HTML.

[jjgold]

Scrape their html and your ip wll be blocked if you put some auto reloader in it.

XML is ok but again your IP will be blocked if you grab the feed more than every 2 minutes or so.

I love you Ganch but your not playing smart today at all.

[ShamsWoof10]

what is the differance between their XML feed and that "Dynamic Line Dsiplay" ? I use DLD alot and it's pretty sweet.. what is XML like compared to this..?

THANKS

[Ganchrow]

Originally Posted by jjgold

Scrape their html and your ip wll be blocked if you put some auto reloader in it.

If you're going to scrape Pinnacle's HTML you just need to be careful ...

Originally Posted by jjgold

XML is ok but again your IP will be blocked if you grab the feed more than every 2 minutes or so.

They ask for 60 seconds between requests. That's pretty easy to do when each request takes 5 minutes to fulfill.

Originally Posted by jjgold

I love you Ganch but your not playing smart today at all.

You can tell me about it when we go out. At this point we'd even meet you in Newark.

[Ganchrow]

Originally Posted by ShamsWoof10

what is the differance between their XML feed and that "Dynamic Line Dsiplay" ? I use DLD alot and it's pretty sweet.. what is XML like compared to this..?

You'd use the XML line feed if you wanted raw data for analytical purposes. Attached is a sample, pared-down Pinnacle XML line file (change the extension from .txt to .xml in order to view in-browser).

[ShamsWoof10]

my browser couldn't get it to come up... can you cut and paste on your post a sample..?

Thanks

[Ganchrow]

Originally Posted by ShamsWoof10

my browser couldn't get it to come up... can you cut and paste on your post a sample..?

Thanks

You can also open it up with notepad (easier if you put back the .txt extension).

[Value]

Originally Posted by Ganchrow

I think I may need to go back to scraping their HTML.

what is scraping html?

[Ganchrow]

Originally Posted by Value

what is scraping html?

See http://en.wikipedia.org/wiki/HTML_scraping.

[ShamsWoof10]

What's the advantage to scraping and what's the advantage to xml feeds..? It looks like a buch of bionary code... Forgive me for being unaware but I can't make heads or tails of it... Just looks like bionary code..

[slash]

If you use spider programs, the xml feed is nice.

[ShamsWoof10]

Spider programs..? oh sh*t.. what the hell are those..? sounds interesting...

[Ganchrow]

Originally Posted by ShamsWoof10

What's the advantage to scraping and what's the advantage to xml feeds..? It looks like a buch of bionary code... Forgive me for being unaware but I can't make heads or tails of it... Just looks like bionary code..

Well first off, binary code will look something this:

00000000 00010001 00000100 11111111 00010011 00000111 00000100 00010010 00000100 11111111 00001111 00000100 00001110 00001111 00001011 00000100 11111111 00010001 00000100 00000000 00001011 00001011 00011000 11111111 00010011 00000111 00000000 00010011 11111111 00000101 00010100 00000010 00001010 00001000 00001101 00000110 11111111 00010010 00010011 00010100 00001111 00001000 00000011 11111111 11111111

While XML may not always be transparently human readable it's still much higher level and easier to visually parse than binary (not that a properly motivated person couldn't read binary with a modest effort ...)

The advantage to an XML feed over scraping is that it's provided by design and follows a guaranteed format. HTML pages may will frequently have their formatting modified and will so without warning or description . Furthermore, many companies look askance and those scraping their pages. XML should be the default choice.

As a general rule, there really is no advantage to scraping over XML. However, there may well be case-specific reasons why XML could be preferable.

For example:

• Pinnacle's XML feed does not provide access to the lines reflected in its drop-down box prices. If you wanted to automatically read this data you couldn't do so using their standard XML and instead would need to scrape the HTML.
• A sportsbook may not devote the same quality of technology to its XML provision server as it might to its HTML provision server (or servers). The reason for this is obvious. It's more profitable to invest resources to a website used by nearly all of its customers, than to a feed used by a relatively small number. One effect of this could be what I mentioned in the first post of this thread; namely that accessing the feed could become extremely slow process. Another effect could be an unreliable XML feed that often crashed.
• A company can effectively exert a lot more control over its XML feed than over its HTML.. The reason is b/c if a company sees you accessing its feed, it knows exactly what you're doing and can hence forbid you from accessing it too frequently. If you're scanning a web page, however, a book can't know for certain what you're doing and will typically allow you more latitude in terms of query frequency.
• Data providers that provide HTML data, sometimes charge money for dedicated XML feeds.

Anyway, this is by no means a complete list, but is just a few thoughts off the top of my head. These reasons (which may not even exist in any particular circumstance). may or may not outweigh the pronounced structural advantages of XML. Use XML unless you have a good reason not to.

[betplom]

I can't get it to work either.

[tacomax]

betplom nicky bumps!!!!!11111111111

[terrytateolb]

Originally Posted by Ganchrow

If you're going to scrape Pinnacle's HTML you just need to be careful ...

I've been writing a program that gets odds from Pinnacle by scraping their HTML, but earlier today my IP was banned by them. When you refer to being careful, what do you mean, i.e. what could I do differently that would prevent me from getting banned in the future?

[jjgold]

Terry you can only grab every 2 minutes if not they ban your IP

[fortunoman]

Originally Posted by Ganchrow

I've recently starting using Pinnacle's XML line feed again (to Pinny's specifications) but am extremely disappointed with the speed of service -- especially during peak times when it becomes essentially unusable.

Is anyone having any luck with this? I think I may need to go back to scraping their HTML.

Ganch-

tried to PM you to see if you wanted to exchange any feeds, but apparently your Inbox is full and the system wont deliver anymore PMs to you until you clear some space.

i have ~20 xml feeds for various books. i'd be willing to do a 1:1 exchange for feeds i dont have if there is any on my list that interests anybody reading this. anybody pls feel free to PM me if you are interested:

Feeds I have:
CentreBet
Gamebookers
Island Sports
NordicBet
Partybets
Sports Interaction
Stan James
WSEX
Bookmaker
Sportbet
Pinnacle
Linesmaker
Legends
Gamblers Palace
Betfred
BetJamaica
BlueSquare
BetClick
William Hill
BetAtHome
BetGun

[chantech]

Does anyone know what Pinny's limitations on screen scraping their site are? I.E. If I do it once every 3 minutes I should be ok?

[MonkeyF0cker]

Why would you scrape when they have the XML feed?

[jjgold]

Approx 2-3 minutes

Why scrape if you can just use sbrodds or another line service?

[MonkeyF0cker]

Umm. Some people want to populate line history databases in real time, jj.

[chantech]

Originally Posted by jjgold

Approx 2-3 minutes

Why scrape if you can just use sbrodds or another line service?

Correct me if I'm wrong, but I'm assuming sbrodds doesn't give both odds and line changes in real time?

We want to do our own analysis of how lines are changing from the time the line is posted to figure out the most optimal time to make offers and accept offers on a betting exchange and continuously optimize the prices we're getting.

[chantech]

Originally Posted by MonkeyF0cker

Why would you scrape when they have the XML feed?

I figured it would be faster since I didn't need the data for all sports and I didn't want to read in the whole XML feed. It's also easier to screen scrape than parse XML based on some of the tools we're using (I know that normally it's easier to parse XML).

[MonkeyF0cker]

Even if parsing the XML is less efficient with whatever language you are using (although I'm unaware of one where this is true), the process will still be more efficient as you'll likely need multiple scrapes for even the same sport (Full game, 1H, 2H, etc.). The full XML file is only processed once anyway. From there, you input a timestamp in your feed request and you only get line changes.

If you're set on scraping, however, I would simply call them and ask. Otherwise, you run the risk of having your IP blocked.

[jjgold]

your IP will get banned scrapping lines from Pinny

Even if you use xml feed you only better pull it every 2-3 minutes or your done.

[MrX]

Their own instructions state that once per minute is acceptable.

[MonkeyF0cker]

JJ, stick to videos please. As MrX said, the XML feed can be pulled every 60 seconds without recourse.

[dirk93]

Sorry to bring up and old topic but I was messing around with the XML feed and I think I set it to scrape too frequently and got blocked from Pinnacle. Anyone have any ideas on how to get unblocked? Also, I'm having trouble with the XML feed. I know the instructions are there I just can't seem to get it into a functional form. Any help?

[durito]

email them and ask them to unblock your ip. tell them you won't hit it more than once a minute again.

[squallsquall]

Anyone used the "sportType" parameter when calling the XML-feed? I'm only interested in MMA, but I can't very well set the parameter value to "Mixed Martial Arts" (which is the sport type name), as it includes spaces.

Any suggestions? Customer service won't help out with this.

[Aperjan]

Hi Everyone:
who can understand and parse pinny xml?
i can't understand what means lastGame and lastContest in xml document? what is it? for what is it? what is last?
how can i get only soccer data ? hopw can i get only one game data by gamenumber? is it possible?
what is contestantnum in participand node? what is rotnum in participant node?
what is period_status and period_updae? where i can read full manual for this xml file?

[creeky]

Originally Posted by Aperjan

Hi Everyone:
who can understand and parse pinny xml?
i can't understand what means lastGame and lastContest in xml document? what is it? for what is it? what is last?
how can i get only soccer data ? hopw can i get only one game data by gamenumber? is it possible?
what is contestantnum in participand node? what is rotnum in participant node?
what is period_status and period_updae? where i can read full manual for this xml file?

Any help? I'm also interested in the meanings for all of the elements

[katstale]

Pinny has indicated they might put buy/sell lines in the xml feed in the future. This might be worth a few folks sending them emails asking when this updated feature will be available.

[whitey]

Originally Posted by creeky

Any help? I'm also interested in the meanings for all of the elements

Pinnacle's feed is what we call "incremental" which means that when you request it you have the option of only receiving events/lines that have updated since your last call.

I suggest sticking with the parameter last parameter as its a little simpler to use. Basically when you receive the feed for the first time they give you a timestamp \\pinnacle_line_feed\PinnacleFeedTime you save this and send it as the value for the last parameter next time you request the feed. Doing so will cause only the lines and events that have changed since the last time you requested the feed to show up.

To request by only one league use the sportType parameter. For example, to request only soccer

http://xml.pinnaclesports.com/pinnac...ortType=Soccer