Mail::SpamAssassin::Plugin::Redirectors(3pm)

NAME
Redirectors - Check for redirected URLs

SYNOPSIS
  loadplugin    Mail::SpamAssassin::Plugin::Redirectors

  url_redirector bing.com

  body HAS_REDIR_URL          eval:redir_url()
  describe HAS_REDIR_URL      Message has one or more redirected URLs

  body REDIR_URL_VALID        eval:redir_url_valid()
  describe REDIR_URL_VALID    Message has a redirector that returned a valid redirection

  body REDIR_URL_CHAINED      eval:redir_url_chained()
  describe REDIR_URL_CHAINED  Message has redirected URL chained to other redirectors

  body REDIR_URL_CHAINED_DOM      eval:redir_url_chained_domain()
  describe REDIR_URL_CHAINED_DOM  Message has redirected URL chained to another redirector on the same domain

  body REDIR_URL_MAXCHAIN     eval:redir_url_maxchain()
  describe REDIR_URL_MAXCHAIN Message has redirected URL that causes too many redirections

  body REDIR_URL_LOOP         eval:redir_url_loop()
  describe REDIR_URL_LOOP     Message has redirected URL that loops back to itself

  body REDIR_URL_404          eval:redir_url_code('404') # Can check any non-redirect HTTP code
  describe REDIR_URL_404      Message has redirected URL returning HTTP 404

DESCRIPTION
This plugin looks for URLs redirected by a list of URL redirector services. Upon finding a matching URL, plugin will send a HTTP request to the redirector service and retrieve the Location-header which points to the actual redirected URL. It then adds this URL to the list of URIs extracted by SpamAssassin which can then be accessed by uri rules and plugins such as URIDNSBL.

This plugin will follow chained redirections, where a redirected URL redirects to another redirector. Redirection depth limit can be set with "max_redir_url_redirections".

Maximum of "max_redir_urls" redirected URLs are checked in a message (10 by default). Setting it to 0 disables HTTP requests, allowing only redir_url() test to work and report found redirectors.

All supported rule types for checking redirector URLs and redirection status are documented in SYNOPSIS section.

NOTES
This plugin runs before priority 0 so that it may modify the parsed URI list prior to normal uri rules or the URIDNSBL plugin. It should run before DecodeShortURLs plugin so that redirected short uris are also checked.

USER SETTINGS

url_redirector  domain[/path] [domain[/path]...]     (default: none)
    Domains that should be considered as a URL redirector.

    Domain matching:

    •   A bare domain (e.g. "bing.com") matches that exact host and also "www.bing.com".

    •   A leading dot (e.g. ".sendgrid.com") matches any subdomain of the domain, to any depth. It does NOT match the bare domain itself; to match both, also list the bare domain ("sendgrid.com").

    An optional "/path" may follow the domain to restrict matching to URLs whose path begins with that string and ends at a path-segment boundary. "/tr/op" matches "/tr/op", "/tr/op/foo", and "/tr/op?x=1", but NOT "/tr/open" -- the configured prefix is treated as one or more whole path segments. Append a trailing slash ("/tr/op/") to require at least one more segment after it (matches "/tr/op/foo" but not bare "/tr/op"). Multiple entries for the same domain are additive (allowlist); a URL is followed if it matches any entry.

    A bare-domain entry without a path is equivalent to "domain/", which matches any path.

    Example:

     url_redirector bing.com
     url_redirector .sendgrid.com
     url_redirector .sendibt2.com/tr/cl/

    The last line follows "https://x.y.sendibt2.com/tr/cl/abc" but not "https://x.y.sendibt2.com/tr/op/abc".

url_redirector_use_selenium (default: 0)
    Enables the Selenium subsystem. When set to 1, hosts listed in "url_redirector_selenium" are fetched using Selenium (headless Chrome). When 0 (the default), those hosts are skipped entirely -- there is no fallback to LWP, because hosts only belong in "url_redirector_selenium" if their redirects cannot be discovered via LWP (e.g., JavaScript-driven redirects).

    This is a runtime kill switch: an operator can disable the Selenium subsystem without editing the host allowlist. While it is off, those hosts are simply not probed -- redir_url() still fires for them so detection rules continue to work.

    Due to how Selenium works, redir_url_code() and "redir_url_404" subs will not fire for hosts handled by Selenium; Selenium reports only "redirected" (301) or "not a redirect" (200), not real HTTP status codes.

url_redirector_selenium_host (default: 127.0.0.1)
    Set Selenium host to use.

url_redirector_selenium_port (default: 4444)
    Set Selenium port to use.

clear_url_redirector  [domain[/path]] [domain[/path]...]
    Clear configured url_redirector domains, for example to override default settings from an update channel. If no arguments are given, all entries are cleared. If domains are specified, only those are removed.

    When an entry includes a "/path", only that path is removed from the domain's allowlist; the domain entry itself is dropped only when its path list becomes empty. Use "domain/" to remove the "match any path" entry added by a bare-domain configuration.

url_redirector_get  domain[/path] [domain[/path]...]     (default: none)
    Domains that should be considered as an URL redirector, accessed using the HTTP GET method instead of HEAD. Syntax and matching rules are the same as "url_redirector": a bare domain matches that host and "www.", a leading dot matches the domain and any subdomain, and an optional "/path" prefix restricts the match.

url_redirector_selenium  domain[/path] [domain[/path]...]     (default: none)
    Domains that should be fetched using Selenium (headless Chrome) instead of LWP. Use this only for hosts whose redirects are NOT discoverable via LWP, such as JavaScript-driven redirects. Syntax and matching rules are the same as "url_redirector".

    Selenium executes JavaScript on the destination and always sends GET, which has weaker safety properties than the LWP path; reserve this list for hosts that genuinely require browser-driven redirect unwrapping.

    If the same host appears in both "url_redirector_selenium" and "url_redirector"/"url_redirector_get", the directive parsed last sets the method, and a warning is emitted.

    The Selenium subsystem is gated by "url_redirector_use_selenium". When that flag is off, hosts in this list are skipped entirely (there is no fallback to LWP, since these hosts wouldn't yield useful results via LWP).

url_skip_redirect_to  domain [domain...]     (default: none)
    Domains that, when found as a redirect destination, should cause the redirect chain to stop immediately. Useful to break loops caused by consent or gateway pages that redirect back into the chain.

    Example:

     url_skip_redirect_to consent.google.com

clear_url_skip_redirect_to  [domain] [domain...]
    Clear configured url_skip_redirect_to domains. If domains are specified, only those are removed from the list.

url_redirector_params regexp (default: (?:adurl|af_web_dp|cm_destination|continue|destination|destURL|goto|h|l|login|location|p1|pval|r|redir|redirect|redirectTo|return|returnUrl|referer|service|target|tid|u|url)=(.*))
    Regexp used to parse uri parameters in order to detect redirectors and to get redirected domains. The regexp must match only the redirected domain.

PRIVILEGED SETTINGS

url_redirector_cache_type     (default: none)
    The cache type that is being utilized. Currently only supported value is "dbi" that implies "url_redirector_cache_dsn" is a DBI connect string. DBI module is required.

    Example: url_redirector_cache_type dbi

url_redirector_cache_dsn		(default: none)
    The DBI dsn of the database to use.

    For SQLite, the database will be created automatically if it does not already exist, the supplied path and file must be read/writable by the user running spamassassin or spamd.

    For MySQL/MariaDB or PostgreSQL, see sql-directory for database table creation clauses.

    You will need to have the proper DBI module for your database. For example DBD::SQLite, DBD::mysql, DBD::MariaDB or DBD::Pg.

    Minimum required SQLite version is 3.24.0 (available from DBD::SQLite 1.59_01).

    Examples:

     url_redirector_cache_dsn dbi:SQLite:dbname=/var/lib/spamassassin/Redirectors.db

url_redirector_cache_username  (default: none)
    The username that should be used to connect to the database. Not used for SQLite.

url_redirector_cache_password  (default: none)
    The password that should be used to connect to the database. Not used for SQLite.

url_redirector_cache_ttl		(default: 86400)
    The length of time a cache entry will be valid for in seconds. Default is 86400 (1 day).

    See "url_redirector_cache_autoclean" for database cleaning.

ADMINISTRATOR SETTINGS

url_redirector_cache_autoclean	(default: 1000)
    Automatically purge old entries from database. Value describes a random run chance of 1/x. The default value of 1000 means that cleaning is run approximately once for every 1000 messages processed. Value of 1 would mean database is cleaned every time a message is processed.

    Set 0 to disable automatic cleaning and to do it manually.

url_redirector_loginfo           (default: 0 (off))
    If this option is enabled (set to 1), then redirected URLs and the decoded URLs will be logged with info priority.

url_redirector_timeout     (default: 5)
    Maximum time a redirection URL HTTP request can take, in seconds.

max_redir_urls                 (default: 10)
    Maximum amount of redirector URLs that will be looked up per message. Chained redirections are not counted, only initial redirection URLs found.

    Setting it to 0 disables HTTP requests, allowing only redir_url() test to work and report any found redirector URLs.

max_redir_url_redirections     (default: 10)
    Maximum depth of chained redirections that a redirector can generate.

url_redirector_user_agent       (default: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36)
    Set User-Agent header for HTTP requests. Some services require it to look like a common browser.