Agents on, by and for the Web

Netscape world has a table of links to over 20 "retreival agents" which can be downloaded. They include:
Agentware (Autonomy) ; AnchorPage (Iconovex) ; AppControl (Seagate Software) ; EchoSearch (Iconovex) ; Freeloader (Individual) ; Headliner (Lanacom) ; Hotbot (Inktomi) ; MagnetSearch (CompassWare Development) ; Net Attache (Tympani Development) ; Netriever 2.0 (Metz Software) ; Personalized Commerce Agents (Personal) ; SEARCH'97 Information Server (Verity); SEARCH'97 Personal for Microsoft Exchange (Verity); SEARCH'97 Personal (Verity) ; SearchPDF for Web Servers (Verity) ; Secret Agent (Arial Communications); Smart Bookmarks 3.0 Beta 2 (FirstFloor) ; SmartSearch (Xilinx) ; Surfbot (SurfLogic) ; Web Retriever 2.1 (Folio) ; WebAnchor (Iconovex); WebCompass (Quarterdeck) ; Zooworks (Hitachi Software) ;
3/6/97
Using A Data Fusion Agent for Searching the WWW, Alan F. Smeaton and Francis Crimmins, Dublin City University, December 1996. This paper describes a meta search engine for searching the WWW. It is based upon a data fusion approach wherein a userÕs query is broadcast to 6 different WWW search engines (AltaVista, Excite, InfoSeek, Lycos, OpenText and WebCrawler) and the output from these searches is combined into a unified ranking of WWW pages. Unlike other meta search engines such as MetaCrawler, Savvy Search and ProFusion, our meta search engine is written in Java and launches a Java applet on the userÕs machine to handle the user interface, which is via a separate window from the userÕs normal WWW browser. The userÕs WWW browser is used to display individual WWW pages selected for viewing. This paper presents details of the system we have developed, concentrating on the client-server architecture for the fusion operation and outlines the extensions we plan for this work in order to deliver a WWW search agent using more sophisticated and effective information retrieval techniques than are currently available via todayÕs popular WWW search engines. 12/16/96
Firefly Network Inc. and Yahoo have launched a public beta of a website recommendation service using Automated Collaborative Filtering (ACF) technology. It allows users to find websites they are likely to find of interest and to find other people who share similar interests. To access the service, go to My Yahoo and then click [Firefly/My Agent] button at the top right of the page. They say
"Out of a constantly expanding universe of information, firefly will recommend sites you'll like and connect you to people who like what you like. First you give your opinion of a range of Weqb sites, then firefly finds other people who share those interests. You can interact with these neighbors, or have firefly recommend sites based on what they and you like. It's an automated, worldwide word-of-mouth system."
Automated Collaborative Filtering is a technique for providing recommendations (or dis-recommendations) based on statistical matches of peoples' evaluations of a set of objects in some domain. It has been developed and explored at the MIT Media Lab and used in a number of recommendation systems, such as HOMR, WEBHOUND and Firefly. Typical domains might be movies, books, web pages, or TV shows. An evaluation of an object is typically a simple scalar value, such as an integer from zero to ten or a real between 0.0 and 1.0, and might represent the degree to which the person "likes" the object. An ACF system maintains a database of users and keeps records of items they have evaluated. The set evaluations a person has made form a vector and represents that person's opinions. These evaluation vectors are compared to the vectors provided by other users and people with similar opinions can be discovered. From this, one can predict the evaluations a person might make for new objects in the domain. 12/11/96
AI Technology Watches What You Read, Sells Accordingly by James Glave, Wired News, 2 Dec 96. "Artificial Intelligence technology currently used to flag suspected credit-card fraud will now be brought to bear on the Net's ongoing commercial quest: targeted advertising. On Tuesday, Aptex, a San Diego company, will release SelectCast for Ad Servers, a "bolt-on brain" software package designed to work with advertising servers on high-traffic Web sites. SelectCast will monitor a user's surfing behavior across a particular Web site over time, build a user profile, then serve banners compatible with that profile. The software will "mine content" in that it will passively monitor what the user reads, then build a psychographic profile based on keywords and concepts occurring throughout that content. The confidential profiles reside on the specific Web server and are accessed and updated each time a user visits the site with help from their magic tracking cookie." MORE 12/2/96
Jigsaw is an HTTP server written in Java by the W3 consortium. It was designed to be portable, extensible and efficient. A third release of the alpha version was made available to non-W3 members early in October 1996. 11/17/96
John C. Mallery of the MIT AI lab has a full-featured, production-quality, HTTP-1.1-compliant web server wholly written in Common Lisp. It is freely available from http://www.ai.mit.edu/projects/iiip/doc/cl-http/home-page.html The server presently runs with full source-compatibility on Macintosh (MCL - Comes on the CD), UNIX (Allegro, LispWorks, Lucid), Windows NT (Allegro), and Lisp Machines (Symbolics 8.3, Open Genera 1.0). Additional ports are underway. This would be a good base for prototyping a server with agent-related features. 11/15/96
TechWeb has a page on off-line web browsing agents. Beyond Browsing--Offline Web Agents by Joel T. Patz. "Tired of waiting for web pages to download? Offline web agents are designed to log on, get the information you need, and log off while you are performing other tasks. They come in a variety of flavors, but the most popular are designed to log onto a web site, roam through a user-definable set of levels (that is, you control how many levels of links are followed) and return the pages to your hard disk, where you can read them at your leisure. ..." The page identifies seven systems -- Folio Web Retriever 2.0, The PointCast Network 1.1, Smart Bookmarks 2.0, WebEx 1.01, WebWhacker 2.0, Freeloader 2.0, METZ Netriever 1.1 -- and includes a head-to-head comparison and a feature chart. 10/15/96
Customized searching goes beyond the net. Gerd Meissner, who helped customize the German edition of Edupage, has developed a search service called BOB, The Human Search Engine, which combines searches of the Net with searches beyond the Net, to help you when you're looking for such things as: a special German saying? A bookstore in Bavaria? Or an old pal's address in Paderborn? From the requests made by private users, schools and non-profit organizations, the service chooses at least one every week to research and answer via e-mail -- for free. U.S. customers are welcome. The address is info@adline.de or http://www.adline.de. [from Edupage, 22 October 1996] 10/23/96
Do-I-Care: A Collaborative Web Agent, Brian Starr, Mark S. Ackerman, and Michael Pazzani, Proceedings of the ACM CHI'96 Conference, April, 1996. Abstract: Social filtering and collaborative resource discovery mechanisms often fail because of the extra burden, even tiny, placed on the user. This work proposes an innovative World Wide Web agent that uses a model of collaboration that leverages the natural incentives for individual users to easily provide for collaborative work. 10/13/96
Webcatcher is a free service which emails subscribers weekly lists of new URL's which are relevant to their interest chosen from a fixed set of about 100 possible topics. 9/15/96
Inference Corporation has developed iFind, an "Internet search tool that calls out in parallel all the best search engines on the internet, merges the results, removes redundancies, clusters the hits into neat understandable groupings, and returns it all to you faster than you can say "nothing but net"...". 8/31/96
A new and improved diffAgent server has been released which includes additional mediators. "A diffAgent watches information sources available via the web and e-mails you when it detects changes. In particular, it can:
- Watch your FedEx package for you and e-mail you when it sees the words "Package has been Delivered!" (make a package watcher agent)
- Monitor a list of query results at a search service like Altavista to see when new pages on your topic appear (make a web topic watcher agent)
- Keep track of news articles on a topic and mail you when it finds new ones (make a news topic watcher agent)
- Mail you when your name appears in a list of papers at an electronic archive (make a web page watcher agent)
- Tell you when the word "snow" appears on the Pittsburgh weather page (make a web page watcher agent)
8/15/96
Project Aristotle is a "clearinghouse for projects and research devoted to the automated categorization of Web resources." It focuses on projects and prototypes that have applied filtering systems, text extraction and/or categorization, or agents, robots or machine learning to the categorization of Web resources rather than general discussions of these approaches or technologies. 8/14/86
The Fusion system sends your web query to six different search engines in parallel: AltaVista, Excite, InfoSeek, Lycos, OpenText and WebCrawler. The results returned are then combined using the concepts of data fusion, thus improving the overall quality of the search. Fusion is written entirely in Java and was developed by Francis Crimmins (francis@compapp.dcu.ie) of Dublin City University. 8/12/96
Robots on the Web is a short overview of web robots. 7/31/96
Technology: Sun has released Tcl Plug-ins for Netscape Navigator, making it possible to create Web pages that include Tcl/Tk scripts. This provides an interesting alternative to the use of Java applets for Web-based agent programs. Sun's current version of the Tcl plug-in runs only with Netscape Navigator under Solaris, Macintosh, and Windows with support for other browsers and operating systems planned. 9/11/96
Thesis: Björn Hermans, "Intelligent Software Agents on the Internet: an inventory of currently offered functionality in the information society & a prediction of (near-)future developments" is now available on the World Wide Web.", Tilburg University, Tilburg, The Netherlands. Abstract: Software agents are a rapidly developing area of research. However, to many it is unclear what agents are and what they can (and maybe cannot) do. In the first part, this thesis will provide an overview of these, and many other agent-related theoretical and practical aspects. Besides that, a model is presented which will enhance and extend agents' abilities, but will also improve the way the Internet can be used to obtain or offer information and services on it. The second part is all about trends and developments. On the basis of past and present developments of the most important, relevant and involved parties and factors, future trends and developments are extrapolated and predicted. 9/9/96
The Similarities Engine is another web-based music recommending system developed by David Whiteis (WhiteisD@ari.net). Its database has been built up over the past year with input from over 18,000 informants. The SImilarites Engine was featured in a recent NYT article The Web's Magical Music Engine (ARTS@LARGE / By Matthew Mirapaul) 6/23/96
Forefront is a Houston company started by AI researcher Dr. G. Anthony Gorry (known for his work applying AI to medical problems) that is marketing several advanced web-based tools. WebWhacker, their latest and most interesting tool, is an "off line" web browser for Windows95 and NT. "By preselecting the Web sites that they want to monitor, users can set a schedule for Web pages, text, graphics and HTML links to be delivered to their desktop computers automatically and unattended. Powerful Wizards walk users through the automation process, requesting that they make only four simple decisions: what Web sites they want monitored, how the selected sites will be categorized, how much of a given site will be retrieved, and how often the content will be updated. " 6/23/96
Beyond Bookmarks: Schemes for Organizing the Web is a clearinghouse of sites that have applied or adopted standard classification schemes or controlled vocabularies to organize Web resources. 6/4/96
diffAgent is a system developed at CMU and Industry.Net Corporation that can monitor websites for specific changes. For example, you can ask it to track a specific package at Fedex and ask it to notify you when the package has been delivered. The system will continously monitor the status for you and send you an email when it detects the triggering event. 6/3/96
Agent-Based Engineering, the Web, and Intelligence,Charles J. Petrie Stanford Center for Design Research, 1996. Abstract: We describe the use of KQML-like Agents and their compatibility with the World-Wide Web. One distinguishing characteristic of such agents is the necessity for a peer-to-peer protocol vs. the client-server protocol of HTTP. This is indicative of a major conflict between the web and agent paradigms that must be resolved for integration of the two technologies, both of which are useful for design and engineering applications. We also note that "intelligence" is not a necessary property of useful agents and is not helpful in distinguishing agents from other kinds of software. agent introduction web 5/31/96
Ontology-Based Knowledge Discovery on the World-Wide Web. Sean Luke, Lee Spector, and David Rager. To appear in the AAAI96 Workshop on Internet-based Information Systems, 1996. Also available in Gzipped PostScript Format (.ps.gz). Abstract: This paper describes SHOE, a set of Simple HTML Ontology Extensions. SHOE allows World-Wide Web authors to annotate their pages with ontology-based knowledge about page contents. We present examples showing how the use of SHOE can support a new generation of knowledge-based search and knowledge discovery tools that operate on the World-Wide Web. paper web ontology 5/30/96
SHOE or Simple HTML Ontology Extensions is a project of the University of Maryland at College Park's Parallel Understanding Systems Group. SHOE is a proposed small extension to HTML which allows HTML authors to annotate their web documents with formal knowledge-representation semantics. SHOE is meant to make real intelligent agents on the web possible. [web ontology] 5/27/96.
Agent: Hitachi's ZooWorks automatically records and indexes each Web site and HTML document a user visits and captures this data in an easy to use personal index. 5/27/96
Agent: DocuMagix's HotPage is a Netscape Navigator plug-in which helps users capture and organize the information from the Web. They say, " With DocuMagix HotPage, you can view saved Web pages off-line within the Netscape Navigator viewer, link back to the original site(s) without needing to remember the exact URL, organize Web pages intuitively, merge them with other Windows documents, search your entire cabinet for a particular Web page that contains reference(s) to a particular topic, forward a Web page document by fax or e- mail (inside your company), mark annotations on a Web document, or even add URL links to any Windows documents." A white paper "The Next Step In Internet Information Management" is available. 5/27/96
Web: Robots in the Web: threat or treat?, Martijn Koster, NEXOR,ConneXions, Volume 9, No. 4, April 1995. Abstract: Robots have been operating in the World-Wide Web for over a year. In that time they have performed useful tasks, but also on occasion wreaked havoc on the networks. This paper investigates the advantages and disadvantages of robots, with an emphasis on robots used for resource discovery. New alternative resource discovery strategies are discussed and compared. It concludes that while current robots will be useful in the immediate future, they will become less effective and more problematic as the Web grows. 5/18/96
Web: One of the biggest list of pointers to web robots, spiders, wanderers, indexers, etc. by Martijn Koster, m.koster@webcrawler.com. 5/18/96
Technology: Autonomous Agent Programming using Java. The Applied Internet Technologies branch of SAIC has developed a framework in Java for the development of autonomous agents. We have provided links to the documentation of the Framework. 5/11/96
Technology: Java-To-Go - Itinerative Computing Using Java. Java-To-Go is an experimental infrastructure developed by William Li (wli@eecs.berkeley.edu) that assists in the development and experimentation of mobile agents and agent-based applications for itinerative computing (itinerative computing: the set of applications that requires site-to-site computations. Sites are usually traversed in sequence by a single mobile agent or in parallel by a group of agents). Agents are given the freedom to perform active computations (that is, computations are initiated by the agents at its volition) at one or more remote agent servers. In contrast, standard Java applets can only be invoked passively. 5/11/96
Technology: Ftp Software has released the CyberAgent Software Development Kit which provides numerous agent classes designed to expedite the development of Java-based mobile agents. The CyberAgent classes include templates to create an intelligent agent, start an agent, stop an agent, define a travel plan, allow access to OLE-enabled applications, and support secure agent communications. You can also use the agent classes with various third-party Java integrated development environments (IDEs). 5/11/96
Agent: ShopBot is an experimental prototype WWW shopping agent under development. It tries to shop for computer software or CD's. Fill in the name and manufacturer of the software product you're looking for, or the title and artist of the pop/rock CD, and let the ShopBot go to work. 5/11/96
Technology: Oracle Web Agent is a generic procedural gateway, which seamlessly invokes Oracle stored procedures, and provides an object-oriented, user-extendible framework for producing dynamic HTML pages using Oracle's PL/SQL scripting language. The Oracle Web Agent is implemented using CGI, enabling it to function with any Web Server. 5/11/96
Paper: Towards an Active Network Architecture, David L. Tennenhouse and David J. Wetherall, LCS, MIT. Abstract: Active networks allow users to inject customized programs into the nodes of the network. In this paper, we describe our vision of an active network architecture, outline our approach to its design, and survey the technologies that can be brought to bear on its implementation. In the course of this presentation we identify a number of research questions to be addressed and propose that the research community mount a joint effort to develop and deploy a wide area ActiveNet. 3/28/96
Paper: From Internet to ActiveNet, D.L. Tennenhouse, S.J. Garland, L. Shrira and M.F. Kaashoek, LCS, MIT. Abstract: ...Active Networks represent a new approach to network architecture that incorporates interposed computation. These networks are "active" in two ways: routers and switches within the network can act on, i.e., perform computations on, user data flowing through them; furthermore, users can "program" the network, by supplying their own programs to perform these computations. ... Our work is motivated by user "pull", as well as technology "push". The "pull" comes from the ad hoc collection of firewalls, Web proxies, multicast routers, mobile proxies, video gateways, etc. that perform user-driven computation at nodes "within" the network. These nodes are flourishing, suggesting user and management demand for their services. One goal of our work is to replace the present collection of ad hoc approaches with a generic capability that allows users to program their networks. The technology "push" is the emergence of "active technologies", supporting the encapsulation, transfer, interposition, and safe and efficient execution of program fragments. Today, active technologies are applied above the end-to-end network layer; for example, to allow clients and servers to exchange program fragments. Our innovation is to leverage and extend these technologies for use within the network - in ways that will fundamentally change today's model of what is "in" the network. 3/28/96
Technology: Microsoft "ActiveX" Technologies and their use for Agent Technologies. Microsoft's ActiveX Technologies facilitates the development of Internet applications and content and provides a modular method into which script engines (e.g., JavaScript and VB) can plug into web browsers and servers. A a newly-available flavor of VB, VB Script, will have "Safe" abilities, and could be considered for some glue for agent languages in itself. Microsoft has released a preliminary version of a ActiveX* Development Kit which also includes a preliminary version of Internet Explorer 3.0 for developers. 3/17/96
Project: Project PageSpace (TU Berlin and the University of Bologna) aims at supporting networked applications which require interaction between distributed software components and active processing. It is based on the Internet and the World Wide Web but introduces a notion of active Web-pages that are capable of executing code. 3/10/96
Software: General Magic announced the availability of a free pre-release version of its Telescript(TM) Active Web Tools, a tool kit for developing active, personalized services for the World Wide Web using agent technology. Until March 1, users can order a free pre-release CD-ROM version of Active Web Tools from General Magic's web site (http://www.genmagic.com/awt) or by calling 408-774-4043. 2/26/96
Workshop: The World Wide Web Consortium (W3C) and the Object Management Group (OMG) are sponsoring the Joint W3C/OMG Workshop ON Distributed Objects AND Mobile Code on June 24-25, 1996 in Boston MA. It will be aimed at extending web and object technologies to provide a richer global infrastructure for applications like electronic commerce, enterprise integration, digital libraries, concurrent engineering, and collaboration. One page position papers are required by March 11 for participation in the workshop. 2/7/96
Software: OSF WebWare 1.0 is a set of tools and services for building and deploying browsing associates and Web server-based applications that can be accessed through standard off-the-shelf browsers and servers. The services facilitate the location, management, and use of Web-based information and support group-related activities on the Web. In addition to the basic Strand toolkit (formerly known as OreO until Nabisco's lawyers noticed), a number of derived agents are available, such as: a Personal What's New Browsing Associate, a Personal What's Changed Proxy, a What's Changed Server Application, a Personal LinkTree Browsing Associate, a Group Full Text Indexer Proxy, and Group Annotation Service. The source is available free for academic, research and internal business purposes. 1/23/96
Workshop: The AAAI-96 Workshop on Internet-based Information Systems (August 4/5, 1996, Portland, Oregon) will examine the state of the art, and explore the future, of network-based systems for browsing, searching, and sharing information in text and other forms with a focus on interactivity and AI techniques. Electronic submissions are due by March 18, 1996. 1/19/96
Project: Human Computer Interaction---Intelligent Browsing Agents is an ARPA-sponsored project at OSF whose goals include making the WWW infrastructure agent-ready and agent-aware and providing an extensible set of agents that access info sources, network and user knowledge in the service of user-related objectives. 1/19/96
Book: Fah-Chun Cheong, Internet Agents - Spiders, Wanderers, Brokers and Bots, New Riders, 1995. UK29 US$32. ISBN 1-56205-463-5. CONTENTS: (1) Introduction: The World of Agents; The Internet - Past, Present, Future; WWW - Playground for Robots. (2) Web Robot Construction: Spiders for indexing the Web; Web Robots; HTTP: Protocol of Web Robots; WebWalker : Your Web maintenance Robot. (3) Agents and Money on the net: Web transaction security; ECash & Payment Services. (4) Bots in CyberSpace: Worms & Viruses; MUD Agents and Chatterbots. (5) Appendices.
PostModern Computing announced Black Widow , an object request broker which connects CORBA objects with web using Java. 1/4/96
Agents for searching the web
The IBM aqui page offers an interesting web-group-ware capability. They say:
12/7/95.
The OSF Wide-Area Intelligent Browsing Associates (WAIBA) project. 10/21/95
Silk - A New Internet Resource Discovery Client is based on the notion of a Uniform Resource Agent or URA. 8/29/95
Sun's HotJava is a dynamic, extensible WWW browser that showcases many of the capabilities of the Java(tm) language -- a new object-oriented programming language developed to solve a number of problems in modern programming practice.
For Java and HotJava announcements, send a "subscribe" message to java-announce-request@java.sun.com. For discussions, send a "subscribe" message to java-interest-request@java.sun.com or java-interest-digest-request@java.sun.com, and to hotjava-interest-request@java.sun.com or hotjava-interest-digest-request@java.sun.com. There is also a java-porting list and digest. The Java website is http://java.sun.com/ . For the Virtual Reality Markup Language (VRML), send a "subscribe www-vrml" message to info-rama@wired.com, or visit http://vrml.wired.com/ . [Gordon at The Otis Index , c.i.www.misc, 9/11/95.] (From TCC)9/21/95
Uniform Resource Agents (URA's) -- This paper proposes Uniform Resource Agents (URA's) as a means of specifying composite net-access tasks. Tasks are described as "composite" if they require the construction and instantiation of one or more Uniform Resource Locators (URL's) or Uniform Resource Names (URN's), and/or if they require transformation of information returned from instantiating URL's/URN's.

AgentWeb is maintained at the UMBC Lab for Advanced Information Technology by Tim Finin (finin@umbc.edu).