web crawler

noun, Digital Technology
1.
a computer program that automatically retrieves Web pages for use by search engines.
Compare bot3 .
Also, Web crawler, webcrawler.
Also called spider.
webcrawler in Technology
World-Wide Web
A free World-Wide Web search engine developed by Brian Pinkerton at the University of Washington and now moved to America Online, Inc. WebCrawler collects URLs by searching the Internet and allows users to perform keyword searches through a World-Wide Web browser.
(https://webcrawler.com/).
(1995-11-28)