PHP Developers Network

A community of PHP developers offering assistance, advice, discussion, and friendship.
 
Loading
It is currently Wed Nov 26, 2014 8:28 pm

All times are UTC - 5 hours




Post new topic Reply to topic  [ 4 posts ] 
Author Message
 Post subject: curl doesnt quite cut it
PostPosted: Wed Aug 22, 2012 8:59 pm 
Offline
Forum Commoner

Joined: Fri Jan 01, 2010 1:56 am
Posts: 68
i am using curl to grab search results from web pages
i want to use it on this site
http://www.twitch.tv/search?query=dan+d ... dinh&stp=1
but the source code returned does not have the title, description etc in the source code
instead they have variables which look like {{title}}, {{description}} in its place
i have searched all of the java script files and all the other files in the source code for these variable values and cannot find them
but if the web browser can translate these variables into their values, there must be a way i can do it in php
does anyone know if there are other functions like curl that have the functionality i am looking for?


Top
 Profile  
 
PostPosted: Thu Aug 23, 2012 10:22 am 
Offline
Site Administrator
User avatar

Joined: Wed Aug 25, 2004 7:54 pm
Posts: 12696
Location: New York, NY, US
You will need to have a browser render the page after you fetch it. cURL just sends/gets requests -- it does not know how to render different data formats.

_________________
(#10850)


Top
 Profile  
 
PostPosted: Thu Aug 23, 2012 1:26 pm 
Offline
Forum Commoner

Joined: Fri Jan 01, 2010 1:56 am
Posts: 68
is there a way to simulate browser rendering?


Top
 Profile  
 
PostPosted: Fri Aug 24, 2012 1:54 pm 
Offline
Forum Contributor
User avatar

Joined: Thu May 11, 2006 8:58 pm
Posts: 305
Location: Utah, USA
chopficaro wrote:
is there a way to simulate browser rendering?

CasperJS is an awesome way to use a virtual browser to open a site, manipulate dom, take screenshots, click elements, submit forms, etc. On Ubuntu, you can install it with `sudo apt-get install casperjs` or OSX `brew install casperjs`. It uses a full V8 browser engine and executes JavaScript like a normal browser.


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 4 posts ] 

All times are UTC - 5 hours


Who is online

Users browsing this forum: PHPHelpNeeded, Yahoo [Bot] and 9 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Jump to:  
Powered by phpBB® Forum Software © phpBB Group