# Bounces

When a message to an email address bounces, Mailman’s bounce runner will register a bounce event. This registration is done through a utility.

    >>> from zope.component import getUtility
    >>> from zope.interface.verify import verifyObject
    >>> from mailman.interfaces.bounce import IBounceProcessor
    >>> processor = getUtility(IBounceProcessor)
    >>> verifyObject(IBounceProcessor, processor)
    True

## Registration

When a bounce occurs, it’s always within the context of a specific mailing list.

    >>> from mailman.app.lifecycle import create_list
    >>> mlist = create_list('test@example.com')
    >>> mlist.send_welcome_message = False

The bouncing email contains useful information that will be registered as well. In particular, the Message-ID is a key piece of data that needs to be recorded.

    >>> from mailman.testing.helpers import (specialized_message_from_string
    ...   as message_from_string)
    >>> msg = message_from_string("""\
    ... From: mail-daemon@example.org
    ... To: test-bounces@example.com
    ... Message-ID: <first>
    ...
    ... """)

There is a suite of bounce detectors that are used to heuristically extract the bouncing email addresses. Various techniques are employed including VERP, DSN, and magic. It is the bounce runner’s responsibility to extract the set of bouncing email addresses. These are passed one-by-one to the registration interface.

    >>> event = processor.register(mlist, 'anne@example.com', msg)
    >>> print(event.list_id)
    test.example.com
    >>> print(event.email)
    anne@example.com
    >>> print(event.message_id)
    <first>

Bounce events have a timestamp.

    >>> print(event.timestamp)
    2005-08-01 07:49:23

Bounce events have a flag indicating whether they’ve been processed or not.

    >>> event.processed
    False

When a bounce is registered, you can indicate the bounce context.

    >>> msg = message_from_string("""\
    ... From: mail-daemon@example.org
    ... To: test-bounces@example.com
    ... Message-ID: <second>
    ...
    ... """)

If no context is given, then a default one is used.

    >>> event = processor.register(mlist, 'bart@example.com', msg)
    >>> print(event.message_id)
    <second>
    >>> print(event.context)
    BounceContext.normal

A probe bounce carries more weight than just a normal bounce.

    >>> from mailman.interfaces.bounce import BounceContext
    >>> event = processor.register(
    ...     mlist, 'bart@example.com', msg, BounceContext.probe)
    >>> print(event.message_id)
    <second>
    >>> print(event.context)
    BounceContext.probe

## Processing

Bounce events are periodically processed via Bounce Runner to take actions for email addresses that bounce often. The first bounce in a day for an email address, in the context of a Mailinglist, increases the bounce score of their membership resource.

    >>> from mailman.interfaces.usermanager import IUserManager
    >>> user_manager = getUtility(IUserManager)
    >>> bart = user_manager.create_address('bart@example.com')
    >>> bart_member = mlist.subscribe(bart)

Initially, every member’s `bounce_score` is equal to 0.

    >>> print(bart_member.bounce_score)
    0

Once a `normal` bounce event is processed belonging to that member, the bounce score is increased by `1`:

    >>> event = processor.register(
    ...     mlist, 'bart@example.com', msg, BounceContext.normal)
    >>> print(event.message_id)
    <second>
    >>> processor.process_event(event)
    >>> print(event.processed)
    True
    >>> print(bart_member.bounce_score)
    1
    >>> print(bart_member.last_bounce_received)
    2005-08-01 07:49:23

However, `bounce_score` is bumped only once for a day, any other bounces for the same day have no effect on the score:

    >>> event = processor.register(
    ...     mlist, 'bart@example.com', msg, BounceContext.normal)
    >>> print(event.message_id)
    <second>
    >>> processor.process_event(event)
    >>> print(event.processed)
    True
    >>> print(bart_member.bounce_score)
    1

Bounce score that is older than Mailinglist’s configured `bounce_info_stale_after` number of days older is considered stale. It is reset to 1.0 if a bounce event is received after that many number of days.

We pretend last bounce was received 10 days ago, more than MailingList’s `bounce_info_stale_after` days

    >>> print(mlist.bounce_info_stale_after)
    7 days, 0:00:00
    >>> from mailman.utilities.datetime import now
    >>> from datetime import timedelta
    >>> bart_member.last_bounce_received = now() - timedelta(days=10)
    >>> bart_member.bounce_score = 5

Now, another event after 10 days will reset the score:

    >>> event = processor.register(
    ...     mlist, 'bart@example.com', msg, BounceContext.normal)
    >>> processor.process_event(event)
    >>> print(bart_member.bounce_score)
    1

## DeliveryStatus

If the `bounce_score` reaches the Mailinglist’s configured `bounce_score_threshold`, bouncing Member’s delivery is suspended:

    >>> print(mlist.bounce_score_threshold)
    5
    >>> bart_member.last_bounce_received = now() - timedelta(days=1)
    >>> bart_member.bounce_score = 4
    >>> event = processor.register(
    ...     mlist, 'bart@example.com', msg, BounceContext.normal)
    >>> processor.process_event(event)
    >>> print(bart_member.bounce_score)
    5
    >>> print(bart_member.preferences.delivery_status)
    DeliveryStatus.by_bounces

If Mailinglist is configured to do so, a notice is sent out the owners when a Member’s delivery is disabled:

    >>> print(mlist.bounce_notify_owner_on_disable)
    True
    >>> from mailman.testing.helpers import get_queue_messages
    >>> items = get_queue_messages('virgin', expected_count=1)
    >>> print(items[0].msg['Subject'])
    bart@example.com's subscription disabled on Test

## VERP Probes

Instead of immediately suspending the delivery of a Member, Mailman can be configured to send VERP probes to the sender after their bounce score has reached the Mailinglist’s threshold.

    >>> anne = user_manager.create_address('anne@example.com')
    >>> anne_member = mlist.subscribe(anne)
    >>> anne_member.bounce_score = 4
    >>> anne_member.last_bounce_received = now() - timedelta(days=1)

Next bounce event for anne should trigger a probe which resets bounce_score:

    >>> event = processor.register(
    ...    mlist, 'anne@example.com', msg, BounceContext.normal)
    >>> from mailman.testing.helpers import configuration
    >>> with configuration('mta', verp_probes='yes'):
    ...     processor.process_event(event)
    >>> print(anne_member.bounce_score)
    0
    >>> print(anne_member.preferences.delivery_status)
    None
    >>> items = get_queue_messages('virgin', expected_count=1)
    >>> msg = items[0].msg
    >>> print(msg.as_string())
    Subject: Test mailing list probe message
    From: test-bounces+...@example.com
    To: anne@example.com
    MIME-Version: 1.0
    Content-Type: multipart/mixed; boundary="..."
    Message-ID: ...
    Date: ...

    ...
    Content-Type: text/plain; charset="us-ascii"
    MIME-Version: 1.0
    Content-Transfer-Encoding: 7bit

    This is a probe message.  You can ignore this message.

    The test@example.com mailing list has received a number of bounces
    from you, indicating that there may be a problem delivering messages
    to anne@example.com.  A sample is attached below.  Please examine this
    message to make sure there are no problems with your email address.
    You may want to check with your mail administrator for more help.

    You don't need to do anything to remain an enabled member of the
    mailing list.

    If you have any questions or problems, you can contact the mailing
    list owner at

        test-owner@example.com

    ...
    Content-Type: message/rfc822
    MIME-Version: 1.0

    From: mail-daemon@example.org
    To: test-bounces@example.com
    Message-ID: <second>
    ...

When such a probe bounces, their delivery is then suspended immediately:

    >>> event = processor.register(
    ...     mlist, 'anne@example.com', msg, BounceContext.probe)
    >>> processor.process_event(event)
    >>> print(anne_member.preferences.delivery_status)
    DeliveryStatus.by_bounces

## Warnings and Unsubscription

When a Member’s delivery is disabled, they will received a configured number of warnings before they are removed as a subscriber of the mailing list.

    >>> print(mlist.bounce_you_are_disabled_warnings)
    3
    >>> # The warnings are sent after a configured interval.
    >>> print(mlist.bounce_you_are_disabled_warnings_interval)
    7 days, 0:00:00

For now, `anne` hasn’t received any warnings:

    >>> print(anne_member.total_warnings_sent)
    0

Bounce Runner invokes BounceProcessor to sends these warnings periodically and removes members when max number of warnings are sent.

    >>> processor.send_warnings_and_remove()
    >>> print(anne_member.total_warnings_sent)
    1
    >>> print(anne_member.last_warning_sent)
    2005-08-01 07:49:23
    >>> print(bart_member.total_warnings_sent)
    1
    >>> items = get_queue_messages('virgin', expected_count=2)
    >>> for item in sorted(items, key=lambda x: str(x.msg['to'])):
    ...     print('To: {}\nSubject: {}\n{}\n'.format(
    ...           item.msg['to'], item.msg['subject'], item.msg.get_payload()))
    To: anne@example.com
    Subject: Your subscription for Test mailing list has been disabled
    Your subscription has been disabled on the test@example.com mailing list
    because it has received a number of bounces indicating that there may
    be a problem delivering messages to anne@example.com.  You may want to
    check with your mail administrator for more help.

    If you have any questions or problems, you can contact the mailing
    list owner at

        test-owner@example.com

    To: bart@example.com
    Subject: Your subscription for Test mailing list has been disabled
    Your subscription has been disabled on the test@example.com mailing list
    because it has received a number of bounces indicating that there may
    be a problem delivering messages to bart@example.com.  You may want to
    check with your mail administrator for more help.

    If you have any questions or problems, you can contact the mailing
    list owner at

        test-owner@example.com

After Mailinglist’s configured `bounce_you_are_disabled_warnings` have been sent and another `bounce_you_are_disabled_warnings_interval` has elapsed:

    >>> print(mlist.bounce_you_are_disabled_warnings)
    3
    >>> anne_member.total_warnings_sent = 3
    >>> print(mlist.bounce_you_are_disabled_warnings_interval)
    7 days, 0:00:00
    >>> anne_member.last_warning_sent = (
    ...    now() - mlist.bounce_you_are_disabled_warnings_interval)

Now, the processor will unsubscribe `anne`:

    >>> processor.send_warnings_and_remove()
    >>> print(mlist.members.get_member('anne@example.com'))
    None

If Mailinglist’s `bounce_notify_owner_on_removal` is `True`, owners will receive a notification about the removal. `anne` will also be notified about about the un-subscription, depending on how the list’s `send_goodby_message` is configured to `True`:

    >>> print(mlist.bounce_notify_owner_on_removal)
    True
    >>> print(mlist.send_goodbye_message)
    True
    >>> items = get_queue_messages('virgin', expected_count=2)
    >>> for item in sorted(items, key=lambda x: str(x.msg['to'])):
    ...     print(item.msg['to'], item.msg['subject'])
    anne@example.com You have been unsubscribed from the Test mailing list
    test-owner@example.com anne@example.com unsubscribed from Test mailing list due to bounces