[Asrg] Mailing list signup handshakes

[Steve Atkins]

On Nov 29, 2008, at 10:14 AM, Michael Thomas wrote:

> Steve Atkins wrote:
>>
>> On Nov 29, 2008, at 9:20 AM, Michael Thomas wrote:
>>
>>>
>>> Even if you could do something clever here, would it make much if
>>> any operational difference? I got the impression from Mark Delany at
>>> Y! that mailing list traffic is a drop in a very large ocean. I'm  
>>> guessing
>>> that bulk mail is probably higher but still pretty much noise  
>>> level to
>>> large providers.
>>
>> Without explicit whitelisting in place, wanted bulk mail (mostly
>> broadcast mail) is one of the biggest components of false positives
>> in spam filters (due to both traffic patterns and content).
>>
>> Large providers expend quite a lot of effort to ensure delivery of
>> wanted bulk mail. If there were a magic wand they could wave
>> that would make all that free (or at least trivially automatable)  
>> then
>> they'd leap at it. That doesn't mean there's huge pent-up demand
>> for it, as the manual whitelisting approach mostly works, but there's
>> not total disinterest either.
>>
> Well, it seems with bulk auth (spf/dkim) getting to be pretty common
> for the whitelists, the work you allude to must mainly be in other  
> areas
> of the overall problem?

That sort of auth gives you the identity of the author, but that's just
the first aspect - necessary, but not sufficient. Reputation is another
aspect to that - how do you decide whether mail from that author
is wanted mail? And how do you differentiate between wanted
mail and unwanted mail from the same author?

There are some obvious approaches - manual vetting, wait for
mail to be delivered and track the reputation and so on - and they're
in use. But they're not cheap to do, and they don't work that well,
especially in the case where the same author sends mail that's
perceived in different ways be recipients. I know dozens
of FTEs who work on, basically, this stuff.

They also don't scale well, in terms of number of interacting parties,
which tends to leave small senders out in the cold.

> Or is it the case that the care and feeding of
> the ESP's whitelist is onerous too? If a large component is the  
> latter,
> you'd think that would be pretty easy to outsource to the enterprising
> minded.

I'm not sure what you mean by ESP's whitelist there.

Don't forget that one relevant identity here is the identity of
the author - Pfizer or Amazon - not the sender (which may
be the same as the author or a third party ESP).

Before we head off into the weeds, though, my only point here
is that there would be some interest from ISPs in a cheap way to
confirm that a particular email is solicited, but it's not a critical
problem.

The reason for that is that it is extremely difficult to distinguish
wanted bulk mail from unwanted bulk mail. Knowing whether a
particular message was solicited or not is a very useful
data point in making that decision.

Cheers,
   Steve