Mail::SpamAssassin::FuzzyHash::Util(3pm)

NAME
Mail::SpamAssassin::FuzzyHash::Util - Shared utilities for FuzzyHash algorithms

SYNOPSIS
  use Mail::SpamAssassin::FuzzyHash::Util qw(normalize_tokens hamming_distance);

  my @tokens = normalize_tokens($text);
  my $dist   = hamming_distance($hex_a, $hex_b);

FUNCTIONS

  normalize_tokens($text)
Normalises $text and returns a list of unique tokens suitable for fuzzy-hash computation.

Steps: ensure UTF-8 flag is set, apply NFKC normalisation (folds full-width ASCII spam chars and ligatures), lowercase, insert space boundaries around CJK characters, strip anything that is not a Unicode letter, digit, or space, collapse whitespace, split and deduplicate.

Returns "('__empty__')" for texts that produce no tokens after normalisation.

  hamming_distance($hex_a, $hex_b)
Returns the Hamming distance (number of differing bits) between two equal-length lowercase hex strings. The strings must have even length.