This link has been bookmarked by 509 people . It was first bookmarked on 15 Jul 2006, by Steve Wilson.
-
31 Dec 17
-
18 Jun 17
kevinoempty
-
24 Nov 15
-
22 Sep 15
-
21 Jun 15
-
27 Apr 15
-
14 Mar 15
-
04 Feb 15
Marek Gizela"curl is a command line tool and library for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMB, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, HTTP/2, cookies, user+password authentication (Basic, Plain, Digest, CRAM-MD5, NTLM, Negotiate and Kerberos), file transfer resume, proxy tunneling and more. "
-
20 Jan 15
-
27 Nov 14
-
23 Nov 14
-
14 Oct 14
-
07 Oct 14
-
01 Oct 14
-
12 Sep 14
-
28 Jul 14
-
28 Jun 14
-
SCP, SFTP
-
-
08 May 14
-
14 Apr 14
brent lawrence"curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks. "
curl tools web http programming opensource software linux command line url
-
13 Feb 14
James MobergCommand line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.
-
23 Jan 14
-
17 Jan 14
Miguel CamposProtocolo y metodologia de trabajo con el protocolo Curl que esta siendo recomendado por MELI
curl tools developers web programming opensource software meli documentación desarrollo
-
13 Dec 13
-
30 Sep 13
-
04 Aug 13
-
31 Jul 13
-
19 Jul 13
-
14 Jun 13
-
10 Jun 13
-
25 May 13
-
19 Mar 13
-
16 Mar 13
-
08 Mar 13
-
22 Jan 13
Margie BorschkeMessage: 9
Date: Tue, 22 Jan 2013 10:05:44 -0500
From: "Pete[r] Landwehr" <plandweh@cs.cmu.edu>
To: Casey Tesfaye <klt35@georgetown.edu>
Cc: "air-l@listserv.aoir.org" <air-l@listserv.aoir.org>
Subject: Re: [Air-L] Comment scraping
Message-ID:
<CACqYdy_N46oi0vDFi+1QP-0eR6_h2jsjCmhZ-bLkZwDFH=VTCA@mail.gmail.com>
Content-Type: text/plain; charset=ISO-8859-1
I will put in a plug for the painful-but-standard-and-entirely-free solution:
* scrape the comments using a free, command-line based program like
wget (http://www.gnu.org/software/wget/) or curl
(http://curl.haxx.se/)
* clean the text using the BeautifulSoup python package for parsing
HTML. (http://www.crummy.com/software/BeautifulSoup/)
...and I will put in a second, shameless plug for the text cleaning software put
out by the CASOS Center at CMU, AutoMap.
(http://www.casos.cs.cmu.edu/projects/automap/) While its main purpose
is for convert text data into network data, it incorporates a basic
HTML out-link scraper (probably not what you want), and a
remove-all-html-from-text-and-convert-symbols-to-english cleaner
(might be what you are looking for). This may not be what you need,
but if you are less familiar with coding solutions it should hopefully
help you get in the right direction.
Best,
pml
On Tue, Jan 22, 2013 at 8:54 AM, Casey Tesfaye <klt35@georgetown.edu> wrote:
> Jacob, Jasmine, etc,,
>
> That software looks great! But expensive! I wonder if there is a cheaper
> alternative? (I'm working with the same type of data)
>
> Otherwise, we the multipronged approach has been the best I've encountered:
> text file + screen shot + html file
>
> Thanks,
> Casey
>
>
> On Mon, Jan 21, 2013 at 10:57 PM, Jacob Groshek <jgroshek@gmail.com> wrote:
>
>> I highly recommend Discovertext. http://discovertext.com/
>>
>> Easy to use, good tech support if/when you need it. Built in coding
>> system. Also can export to spreadsheet (if necessary) with subscription.
>>
>> Best,
>>
>> Jacob
>>
>> --
>> Dr. Jacob Groshek
>> Assistant (Visiting) Professor
>> Digital Media and Research Methods
>> jgroshek.com <http://www.jgroshek.com/>
>>
>> Head, CTEC <http://aejmcctec.com/> / AEJMC <http://www.aejmc.org/>
>> Visiting Scholar, IAST <http://www.iast.fr/>
>> Full Member, NeSCoR <http://nescor.socsci.uva.nl/>
>>
>>
>>
>> On Tue, Jan 22, 2013 at 2:47 PM, Jasmine E McNealy <jemcneal@syr.edu>
>> wrote:
>>
>> > Hello All,
>> >
>> > I'm looking for ideas on the best software to use for comment scraping.
>> I
>> > plan on doing quantitative content and qualitative textual analysis on
>> the
>> > comments connected to an article on an online pubulication. The
>> > publication uses Disqus for comments, and ideally I'd like a program that
>> > would maintain the integrity of the comment relationships. Any and all
>> > ideas are appreciated.
>> >
>> > Thanks,
>> >
>> > JM
>> >
>> > Jasmine McNealy
>> > Assistant Professor
>> > S.I. Newhouse School of Public Communication
>> > Syracuse University
>> > 215 University Place
>> > Syracuse, NY 13210
>> > 315-443-115 -
02 Nov 12
-
22 Aug 12
-
curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.
-
Curl is free and open software that compiles and runs under a wide variety of operating systems. Curl exists thanks to efforts from many contributors.
The most recent stable version of curl is version 7.27.0, released on 27th of July 2012. Currently, 86 of the listed archives are of the latest version.
-
-
09 Aug 12
-
26 Jul 12
-
23 Jul 12
-
07 Jul 12
-
16 May 12
-
20 Apr 12
-
02 Mar 12
francesco arcaresecurl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, Gopher, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, Telnet and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.
-
22 Jan 12
Keith DavidsonCommand line utility for running various URL syntax commands including http, ftp, etc.
-
01 Dec 11
-
02 Nov 11
-
26 Oct 11
Free command line tool for interacting with web sites from the command prompt.
curl scripting api tools web programming http opensource software
-
20 Sep 11
-
08 Sep 11
-
05 Aug 11
-
08 Jul 11
-
25 Jun 11
-
09 Jun 11
-
23 May 11
-
12 May 11
cezarmcurl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP. curl supports SSL certificates, HTTP POST
tools web programming opensource software linux utilities http curl for:@twitter viaDelicious
-
28 Apr 11
-
25 Apr 11
-
07 Apr 11
-
05 Apr 11
-
curl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks
-
-
03 Mar 11
-
09 Jan 11
fmoyaeUtiliadad para hacer scrapping the http emulando un browser ademas de que la libraria php de facebook la utiliza.
-
27 Dec 10
-
13 Dec 10
-
18 Nov 10
-
17 Nov 10
aborrellcurl is a command line tool for transferring data with URL syntax, supporting DICT, FILE, FTP, FTPS, GOPHER, HTTP, HTTPS, IMAP, IMAPS, LDAP, LDAPS, POP3, POP3S, RTMP, RTSP, SCP, SFTP, SMTP, SMTPS, TELNET and TFTP. Curl supports HTTPS certificates, HTTP POST, HTTP PUT, FTP uploading, kerberos, HTTP form based upload, proxies, cookies, user+password authentication, file transfer resume, http proxy tunneling and a busload of other useful tricks
-
13 Nov 10
-
20 Oct 10
-
24 Sep 10
-
04 Sep 10
-
30 Aug 10
-
26 Aug 10
-
19 Aug 10
-
09 Aug 10
-
23 Jul 10
-
15 Jul 10
-
30 Jun 10
-
24 Jun 10
-
22 Jun 10
Jonathon RichtercURL: resumable download tool for those with slow bandwidth (e.g. rural areas)
-
30 May 10
Alfred Reichcurl is a command line tool for transferring data with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS, FILE, IMAP, SMTP, POP3 and RTSP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP for
-
28 May 10
-
18 May 10
-
15 May 10
-
13 May 10
-
12 May 10
almcurl is a command line tool for transferring data with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS, FILE, IMAP, SMTP, POP3 and RTSP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP form based upload, proxies, cookies, user+password authentication (Basic, Digest, NTLM, Negotiate, kerberos...), file transfer resume, proxy tunneling and a busload of other useful tricks.
-
07 May 10
-
02 May 10
mandarinecurl is a command line tool for transferring data with URL syntax, supporting FTP, FTPS, HTTP, HTTPS, SCP, SFTP, TFTP, TELNET, DICT, LDAP, LDAPS, FILE, IMAP, SMTP, POP3 and RTSP. curl supports SSL certificates, HTTP POST, HTTP PUT, FTP uploading, HTTP for
-
23 Apr 10
-
22 Apr 10
-
16 Apr 10
-
08 Apr 10
-
06 Apr 10
-
05 Apr 10
-
01 Apr 10
karlkappcurl is a command line tool for transferring data with URL syntax, supporting FTP, FTPS, HTTP...
-
23 Mar 10
-
18 Mar 10
-
15 Mar 10
-
12 Mar 10
-
18 Feb 10
Page Comments
Would you like to comment?
Join Diigo for a free account, or sign in if you are already a member.