download http directory

Here’s a quick note on how to download the contents of an http directory via command line.

Using lftp to download an HTTP directory

Forget wget, lftp is what you covet!

What is lftp?

LFTP is a sophisticated ftp/http client, and a file transfer program supporting a number of network protocols. Like BASH, it has job control and uses the readline library for input. It has bookmarks, a built-in mirror command, and can transfer several files in parallel. It was designed with reliability in mind.

How do I use it to download the contents of an http directory?

sytax example #1:

$ lftp
lftp :~> mirror http: //example.org/somedirectory

sytax example #2:

$ lftp
lftp :~> mirror http://cans.cdm.depaul.edu/~badawi/courses/TDC368/codes/HW5/
Total: 1 directory, 6 files, 0 symlinks
New: 6 files, 0 symlinks
62538 bytes transferred
lftp :~> quit

Leave a Reply

To create code blocks or other preformatted text, indent by four spaces:

    This will be displayed in a monospaced font. The first four 
    spaces will be stripped off, but all other whitespace
    will be preserved.
    
    Markdown is turned off in code blocks:
     [This is not a link](http://example.com)

To create not a block, but an inline code span, use backticks:

Here is some inline `code`.

For more help see http://daringfireball.net/projects/markdown/syntax

You can use these HTML tags

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>