# Class Web.Crawler.Crawler

Method create

  `Web.Crawler.Crawler Web.Crawler.Crawler(Queue _queue, function(:void) _page_cb, function(:void) _error_cb, function(:void) _done_cb, function(:void) _prepare_cb, string|array(string)|Standards.URI|array(Standards.URI) start_uri, mixed ... _args)`

Parameter `_page_cb`

  function called when a page is retreived. Arguments are: Standards.URI uri, mixed data, mapping headers, mixed ... args. should return an array containing additional links found within data that will be analyzed for insertion into the crawler queue (assuming they are allowed by the allow/deny rulesets.

Parameter `_error_cb`

  function called when an error is received from a server. Arguments are: Standards.URI real_uri, int status_code, mapping headers, mixed ... args. Returns void.

Parameter `_done_cb`

  function called when crawl is complete. Accepts mixed ... args and returns void.

Parameter `_prepare_cb`

  argument called before a uri is retrieved. may be used to alter the request. Argument is Standards.URI uri. Returns array with element 0 of Standards.URI uri, element 1 is a header mapping for the outgoing request.

Parameter `start_uri`

  location to start the crawl from.

Parameter `_args`

  optional arguments sent as the last argument to the callback functions.