WebSPHINX: A Personal, Customizable Web Crawler - The Diigo Meta page

• 05 Nov 09
  

  Vinod M

  wc crawler programming

• 18 Aug 09
  

  rameshchill

  Research-related info

• 09 Jul 09
  

  Swarna Srinivasan

  websphinx crawler

  • WebSPHINX is designed for advanced web users and Java programmers who want to
    crawl over a small part of the web (such as a single web site) automatically.
  • The WWW7 paper mentions a "CategoryClassifier", but I can't find it in
    the source code.  Where can I get it?
    

    The CategoryClassifier was part of an earlier web-crawling system, SPHINX,
    developed at Compaq SRC.  The original SPHINX code belongs to Compaq SRC
    and was never released.  WebSPHINX is an open-source reimplementation of
    the SPHINX interface.  CategoryClassifier was not part of this
    reimplementation because CategoryClassifier depended on some other software that
    belongs to SRC.
  • 2 more annotations...

  • • The crawler library is open source, licensed under an Apache-style license. The latest release is version
      0.5, released on July 8, 2002. See the change history to
      find out what's new.

      
      

      Download the source code here:

      
      

      websphinx.zip

    • • Acme.Spider is
        an elegant, single-threaded Java web crawler implemented as an Enumeration. Open
        source.
        
      • WebL is a scripting
        language for the Web, with primitive functions for getting web pages and posting
        forms, and a built-in structured pattern language for matching HTML and XML.
        Open source, implemented in Java.
        
      • LiveAgent Pro is a Java toolkit for
        developing web crawlers. Commercial, closed source.
        
      • Mapuccino (formerly known
        as WebCutter) is a Java web crawler designed specifically for web visualization.
        Closed source.
        
      • MacroBot is a web crawling
        environment using Basic. Runs only on Microsoft Windows. Commercial, closed
        source.
• 05 Feb 07
  

  Jorge Maestre

  crawler java tool web

• 22 Aug 06
  

  Di Ci

  webcrawler

• 11 Mar 06
  

  al bis

  eparkos

• 22 Feb 06
  

  vincent law

  webcrawler opensource

• 23 Jan 06
  

  Rachel Cavalier

  spider

• 13 Oct 05
  

  seabay becy123

Would you like to comment?

Join Diigo for a free account, or sign in if you are already a member.