Archives for category: Email

Paid per email and why it won’t work

August 22, 09 //
2

A while back, people in IT kicked the tires on paying per email sent. Nothing ever came of it but the idea is coming back in the form of Yahoo sponsored CentMail. I initially heard of this thanks to a well written article by MacGregor Campbell over at the New Scientist. According to the article Yahoo is experimenting with Pay per email system, but with the cost of an email be only one US cent ($.01US) and that money going the sender’s charity of choice. The eliminates the original complaint of Pay per Email; different ISPs charging different rates. It also gets more money into charities which is always a good idea.

Problem is, it won’t work.

Simple Mail Transport Protocol, or SMTP, is the mechanism by which is email is sent. It’s simple and it works. There really isn’t a lot to it. You send a plain text stream that contains your message header and the email body to your domain’s SMTP server and it sends it out to the recipient’s mail server. I’m simplifying it a bit, but not by much. And due to its relatively straightforward design, SMTP is subject to abuse. We’ve all gotten the viagra and porn spam in our inbox. And there are methods to deal with spam like Domain Keys, SPF records, RBLs and server side spam filter software. And that all works, but it adds unnecessary levels of complexity to the email process. The CentMail idea will add just another layer to that.

I don’t work with that aspect of email much anymore, but my coworkers who work the phones do. Each of them is helping one of our customers get out of email purgatory. A customer will send some email that one of the ISP doesn’t like and it’ll get put on the ISPs blacklist.  The customer then calls us and one of the guys in Support has to fight with the ISP to get the customer’s Domain removed from the blacklist. The problems lay in that there is no standard on what classifies spam and how to handle it. Each email provider, be it an ISP like Charter or Comcast, or a webmail provider like Google and Yahoo have different requirements and have different levels of pain involved with getting a domain whitelisted.

And that’s not including the RBLs, or Remote Blacklists. These RBLs are another level of spam prevention and another level of complexity. They work by keeping a list of spam related IP address and Domains. Your mail client checks each incoming email against each RBL and drops mail that is on the list. Getting off the RBLs can be even more painful, ranging from paying a fee to clearing out an entire C-block of IPs.

The CentMail website, which is a horrible pun by the way, states “Anyone you email can automatically verify your donation and confirm you’re not a spammer. Since spammers send millions of emails every day, it is prohibitively expensive for them to donate even just 1¢ per email.” That won’t stop the spammers. I’m guessing that Yahoo is assuming that there will be checks on both ends of the email sending process. But unless they intentionally modifiy SMTP to be non standard compliant, all a spammer has to do is use an MTA that doesn’t check for the Cyberstamp to get their mail out.

From the process described on the New Scientist article, it sounds like there will be an addition to the email’s header that will contain the Cyberstamp signature, and a link will be embedded that will allow the sender to verify that your did donate that penny to the Humane Society. That’s great until the Spammers figure out a way to spoof the headers and make that link redirect the user to a phish site. There are enough people out there using unpatched versions of Windows XP that this would be presenting another vector for hackers to compromise new systems.

On that same vein, what is to prevent a receiver from filtering email that contains a cyberstamp to a charity that they find objectionable? Lets say I send you an email and I donate that penny to AIDS research or Planned Parenthood. Now lets say you work for a conservative company that finds both of objectionable. What is stopping them from just dropping those emails? This adds another layer of potential problems for CentMail.

And what’s keeping companies from creating their own systems or not even adopting it? This system would only work if everyone, and I mean everyone was on board with it. And the disturbing trend of ISPs hijacking DNS indicates that the ISPsa at least have no desire for adopting a standard that someone else created.

All in all, I don’t see Cyberstamps stopping the spam problem with the way email is currently setup. The RBLs and the spam filters work well. I’ve gotten maybe on 1 spam in the past 3 months on one of my email accounts, which is a lot less then what i get in my physical mailbox. In order for Centmail to work, you’d need a major upheval in the status quo and I don’t see it happening. Cyberstamps version 2 is doomed to fail even before its released.

Deleting a Crapton of Small Files from a Directory Using Find

July 14, 09 //
1

One of the fun situations you find yourself in when administrating people’s email (at least, I find myself in this situation frequently enough, and I am an uninteresting person, so this is what passes for a fun situation in my world) is the dreaded I-have-magically-hit-my-email-quota problem. The conversation usually runs something like this:

User: HALP I cannot send any email on this address!

Me: checks logs The mail logs are telling me that it can’t write to file due to quota issues. Looks like you’ve hit your $BIGNUM quota.

User: I don’t know why the system quota is saying I have $BIGNUM of mail when $MAILCLIENT informs me I have $MUCHSMALLERNUM of mail.

Me: cd’s into user’s maildir directory, starts running ll -ah like a boss

User: I mean, holy crap, I friggin’ hate email, why does the world have to hate me, I have $VERYIMPORTANTATTACHMENT to send to $BIGCLIENT and then this happens. Why don’t computers run on happiness and unicorn farts?

Me: Ah, it looks like this email address has $BIGNUM – $MUCHSMALLERNUM worth of mail in its .spam directory.

User: I have a dot spam directory? How come I can’t see it in $MAILCLIENT?

Me: You know, I could explain the concept of how hidden directories work in linux, and bemoan the fact that the mailserver uses them and most mail clients don’t, but in lieu of that, how about I delete all of that spam for you, thereby fixing your quota issue?

User: in a monotone Thank you sir may I have another

Here is the bash one-liner for removing the mails from this directory:

for file in $(find -type f -maxdepth 1); do rm -fv $file; done

“But sir! Sir!” you find yourself saying, “why on earth are you not using xargs? Surely piping find’s output to xargs is more memory-efficient than a naive for loop!”

While it is true that a for loop is less memory efficient than piping find to xargs, I always use it in this situation. The reason is simple; if a directory is $BIGNUM large, and most of the files in it are measured in tens of kilobytes, your find command will more often than not send too many arguments xarg’s way. Your time is too precious a commodity to see whether or not this is one of the situations when the stars align and the mail quota is small enough that the directory doesn’t have too many files for xargs. Just use the for loop. If the server is reasonably fast, it will handle it gracefully.

Every once in a while you’ll run into an enlightened user who realizes, just as you’re about to go and mow through the .spam directory, that hey, there was an important email last week that somehow never got through. They’d like to be able to poke at the last week’s .spam emails. Which means you can’t just delete all the mails willy-nilly. This is why God invented the -mtime flag.

for file in $(find -type f -maxdepth 1 -mtime +7); do rm -fv $file; done

That -mtime +7 causes find to snag only the files that haven’t been modified in the last seven days.

I really can’t get over how useful a little command like find can be. I mean, I’ve tried to get over it. Brought it into therapy a couple of times. Still can’t shake how useful it is. I usually refer to this find tutorial when I trip over the syntax.

One final tip: before you start deleting data with reckless abandon, I highly recommend substituting “do echo $file;” for "do rm -fv $file;" in the above one-liners. It never hurts to make sure that you’re actually grabbing the files you think you are before you delete them.