[TriLUG] perl: GET'ing large file with LWP
dragonstrider at gmail.com
Mon Sep 21 10:09:27 EDT 2009
I'd use two requests; HEAD for the headers, then GET to get it
afterwards after processing the headers. Yes, it's a bit redundant,
but otherwise you're writing the HTTP protocol yourself.
On Sun, Sep 20, 2009 at 1:45 PM, Joseph Mack NA3T <jmack at wm7d.net> wrote:
> I can GET an xml file with
> $ua = LWP::UserAgent->new;
> $url = 'http::/somewhere/';
> $response = $ua->get($url,
> ':content_file' =>
> The file takes about 10mins to download and I would like the user to know
> why their application appears to be doing nothing.
> Before downloading the file, I'd like to be able to
> o retrieve content-type. The content will not always be the file (it could
> be a short html error page saying that your cookie has expired)
> o retrieve length ($response->content). This will allow me to put up a
> progress bar.
> The only instructions on LWP I've found download the file first and then you
> look at the headers rather than the other way around.
> The download is html encoded eg "e; rather than ". I notice that
> lwp-request has an -o format option which I assume can change the output. Is
> there any way of removing the html encoding when downloading with LWP?
> Joseph Mack NA3T EME(B,D), FM05lw North Carolina
> jmack (at) wm7d (dot) net - azimuthal equidistant map
> generator at http://www.wm7d.net/azproj.shtml
> Homepage http://www.austintek.com/ It's GNU/Linux!
> TriLUG mailing list : http://www.trilug.org/mailman/listinfo/trilug
> TriLUG FAQ : http://www.trilug.org/wiki/Frequently_Asked_Questions
Personal e-mail: jtate AT dragonstrider DOT com
More information about the TriLUG