Mail::SpamAssassin::Plugin::HashBL(3pm)

NAME
HashBL - query hashed (and unhashed) DNS blocklists

SYNOPSIS
  loadplugin Mail::SpamAssassin::Plugin::HashBL

  # NON-WORKING usage examples below, replace xxx.example.invalid with real list
  # See documentation below for detailed usage

  header   HASHBL_EMAIL eval:check_hashbl_emails('ebl.example.invalid')
  describe HASHBL_EMAIL Message contains email address found on EBL
  tflags   HASHBL_EMAIL net

  # rewrite googlemail.com -> gmail.com, applied before acl/welcomelist
  hashbl_email_domain_alias gmail.com googlemail.com
  # only query gmail.com addresses
  hashbl_acl_freemail gmail.com
  header   HASHBL_OSENDR eval:check_hashbl_emails('rbl.example.invalid/A', 'md5/max=10/shuffle', 'X-Original-Sender', '^127\.', 'freemail')
  describe HASHBL_OSENDR Message contains email address found on HASHBL
  tflags   HASHBL_OSENDR net

  body     HASHBL_BTC eval:check_hashbl_bodyre('btcbl.example.invalid', 'sha1/max=10/shuffle', '\b([13][a-km-zA-HJ-NP-Z1-9]{25,34})\b')
  describe HASHBL_BTC Message contains BTC address found on BTCBL
  tflags   HASHBL_BTC net

  header   HASHBL_URI eval:check_hashbl_uris('rbl.example.invalid', 'sha1', '^127\.0\.0\.32$')
  describe HASHBL_URI Message contains uri found on rbl
  tflags   HASHBL_URI net

  body     HASHBL_ATTACHMENT eval:check_hashbl_attachments('attbl.example.invalid', 'sha256')
  describe HASHBL_ATTACHMENT Message contains attachment found on attbl
  tflags   HASHBL_ATTACHMENT net

  # Capture tag using SA 4.0 regex named capture feature
  header   __X_SOME_ID X-Some-ID =~ /^(?<XSOMEID>\d{10,20})$/
  # Query the tag value as is from a DNSBL
  header   HASHBL_TAG eval:check_hashbl_tag('idbl.example.invalid/A', 'raw', 'XSOMEID', '^127\.')

  # Fuzzy body hash lookup
  # Issues exactly 4 DNS TXT lookups.  Each TXT response carries the digest
  # of the indexed string; similarity is verified locally via zorder_compare().
  # sim_threshold sets the required similarity percentage.
  #
  body     HASHBL_FUZZY eval:check_hashbl_bodyfuzzy('fbl.example.invalid/TXT', 'sim_threshold=90')
  describe HASHBL_FUZZY Message body is similar to a known spam template
  tflags   HASHBL_FUZZY net

DESCRIPTION
This plugin supports multiple types of hashed or unhashed DNS blocklist queries.

Common OPTS that apply to all functions:
      raw          no hashing, query as is (can break if value is not valid DNS label)
      md5          hash query with MD5
      sha1         hash query with SHA1
      sha256       hash query with Base32 encoded SHA256
      case         keep case before hashing, default is to lowercase
      max=x        maximum number of queries (defaults to 10 if not specified)
      shuffle      if max exceeded, random shuffle queries before truncating to limit
      alldomains   do not ignore domains listed in uridnsbl_skip_domains

    Multiple options can be separated with slash.

    When rule OPTS is empty ('') or missing, default is used as documented by each query type. If any options are defined, then all needed options must be explicitly defined.

header RULE check_hashbl_emails('bl.example.invalid/A', 'OPTS', 'HEADERS', '^127\.')
    Check email addresses from DNS list. Note that "body" can be specified along with headers to search message body for emails. Rule type must always be "header".

    Optional DNS query type can be appended to list with /A (default) or /TXT.

    Default OPTS: sha1/notag/noquote/max=10/shuffle

    Additional supported OPTS:

      nodot    strip username dots from email
      notag    strip username tags from email
      nouri    ignore emails inside uris
      noquote  ignore emails inside < > or possible quotings
      user     query userpart of email only
      host     query hostpart of email only
      domain   query domain of email only (hostpart+trim_domain)

    Default HEADERS: ALLFROM/Reply-To/body

    HEADERS refers to slash separated list of Headers to process:

      ALL           all headers
      ALLFROM       all From headers as returned by $pms->all_from_addrs()
      EnvelopeFrom  message envelope from (Return-Path etc)
      <HeaderName>  any header as used with header rules or $pms->get()
      body          all emails found in message body

    If HEADERS is empty ('') or missing, default is used.

    Optional subtest regexp to match DNS answer (default: '^127\.').

    For existing public email blocklist, see: http://msbl.org/ebl.html

      # Working example, see https://msbl.org/ebl.html before usage
      header   HASHBL_EMAIL eval:check_hashbl_emails('ebl.msbl.org')
      describe HASHBL_EMAIL Message contains email address found on EBL
      tflags   HASHBL_EMAIL net

    Default regex for matching and capturing emails can be overridden with "hashbl_email_regex". Likewise, the default welcomelist can be changed with "hashbl_email_welcomelist". Only change if you know what you are doing, the default welcomelist includes abuse@, postmaster@, hostmaster@, domainmaster@ and few more; see plugin source code for more info about the defaults. "hashbl_email_regex" example: hashbl_email_regex \S+@\S+.com

header RULE check_hashbl_uris('bl.example.invalid/A', 'OPTS', '^127\.')
    Check all URIs parsed from message from DNS list.

    Optional DNS query type can be appended to list with /A (default) or /TXT.

    Default OPTS: sha1/max=10/shuffle

    Optional subtest regexp to match DNS answer (default: '^127\.').

[raw]body RULE check_hashbl_bodyre('bl.example.invalid/A', 'OPTS', '\b(match)\b', '^127\.')
    Search body for matching regexp and query the string captured. Regexp must have a single capture ( ) for the string ($1). Rule type must be "body" or "rawbody".

    Optional DNS query type can be appended to list with /A (default) or /TXT.

    Default OPTS: sha1/max=10/shuffle

    Additional supported OPTS:

      num      remove the chars from the match that are not numbers

      replace  if a regexp contains replace tags, replace the match of the regexp with the first option
               of the regexp, ex. +1 8O8.l23.4567 will be changed to +1 808.123.4567
               For the subsitution to work, replace tags must contain only single chars.

    Optional subtest regexp to match DNS answer (default: '^127\.').

header RULE check_hashbl_tag('bl.example.invalid/A', 'OPTS', 'TAGNAME', '^127\.')
    Query value of SpamAssassin tag _TAGNAME_ from DNS list.

    Optional DNS query type can be appended to list with /A (default) or /TXT.

    Default OPTS: sha1/max=10/shuffle

    Additional supported OPTS:

      ip        only query if value is valid IPv4/IPv6 address
      ipv4      only query if value is valid IPv4 address
      ipv6      only query if value is valid IPv6 address
      revip     reverse IP before query
      fqdn      only query if value is valid FQDN (is_fqdn_valid)
      tld       only query if value has valid TLD (is_domain_valid)
      trim      trim name from hostname to domain (trim_domain)

      If both ip/ipv4/ipv6 and fqdn/tld are enabled, only either of them is
      required to match.  Both fqdn and tld are needed for complete FQDN+TLD
      check.

    Optional subtest regexp to match DNS answer (default: '^127\.').

header RULE check_hashbl_attachments('bl.example.invalid/A', 'OPTS', '^127\.')
    Check all all message attachments (mimeparts) from DNS list.

    Optional DNS query type can be appended to list with /A (default) or /TXT.

    Default OPTS: sha1/max=10/shuffle

    Additional supported OPTS:

      minsize=x  skip any parts smaller than x bytes
      maxsize=x  skip any parts larger than x bytes

    Optional subtest regexp to match DNS answer (default: '^127\.').

    Specific attachment filenames can be skipped with "hashbl_ignore". For example "hashbl_ignore safe.pdf".

    Specific mime types can be skipped with "hashbl_ignore". For example "hashbl_ignore text/plain".

body RULE check_hashbl_bodyfuzzy('bl.example.invalid/TXT', 'OPTS')
    Computes a 32-bit MinHash signature of the full message body, splits it into 4 LSH bands of 8 bits each, and issues exactly 4 DNS TXT lookups of the form:

      fz2BVV.<list>  IN  TXT  "<8-char-digest>"

    where "fz2" is the algorithm+version prefix, "B" is the band index (0-3), and "VV" is the 8-bit band value in hex.

    Each TXT response carries the 8-char hex digest of the indexed string. The rule fires when "zorder_compare(query_digest, stored_digest)" meets the configured "sim_threshold".

    Supported OPTS:

      sim_threshold=N      integer 50-100, similarity % required to fire (default: 95)

    Example rules at different thresholds:

      # Near-identical bodies only
      body FUZZ_100 eval:check_hashbl_bodyfuzzy('fbl.example.invalid/TXT', 'sim_threshold=100')

      # Near-duplicates (~90%+ token overlap)
      body FUZZ_90  eval:check_hashbl_bodyfuzzy('fbl.example.invalid/TXT', 'sim_threshold=90')

hashbl_ignore value [value...]
    Skip any type of query, if either the hash or original value (email for example) matches. Multiple values can be defined, separated by whitespace. Matching is case-insensitive.

    Any host or its domain part matching uridnsbl_skip_domains is also ignored by default.

  check_hashbl_bodyfuzzy($list, $opts_str, $subtest)
Issues exactly 4 async DNS TXT lookups for the message body using the ZOrder MinHash+LSH scheme (4 bands x 8 bits, 32-bit digest).

Each TXT response carries the 8-char hex digest of the indexed string. Similarity is verified locally: "zorder_compare(query_digest, stored_digest)" must be >= "sim_threshold" for the rule to fire.