The Conference on Email and Anti-Spam (CEAS) invites participation in the CEAS 2008 Spam Challenge.
Registration
You can register for the spam challenge from the CEAS homepage. Follow the "Registration" link in the main navigation (left-hand side) and then select "Live Spam Challenge" as the ticket type.
Tasks
The challenge will consist of two tasks with partially overlapping data. Participants are welcome to take part in any or both of these tasks:
- Filtering a live email stream delivered using standard protocols ("live challenge") [new: Details].
- Filtering private email collections using the TREC spam filter evaluation toolkit [new: Details].
All types of anti-spam technologies are welcome to participate in the live challenge, including servers, appliances, proxies, personal filters and academic systems. Participants will be allowed to use any network resource including real-time blacklists, real-time white-lists, reverse-DNS queries, and privately owned and updated databases. SMTP will be used to deliver the live email stream to participants and to record spam filter responses.
Evaluation of filters on private email collections will be carried out by the organizers ("lab" evaluation). Participants who wish to take part in this task must implement the TREC spam-filter evaluation interface and submit their filters for evaluation. To preserve privacy, filters will not be allowed to use network resources for this task. All filters submitted for this task will also be evaluated on a copy of the live challenge e-mail stream. This task was conceived as a continuation of the TREC spam track effort, and we encourage TREC participants to submit existing systems for the evaluation.
Filter Submissions and Configuration new
Below are some pointers on how to set up and/or submit your filters for each competition task:
- Filters that implement the TREC spam-filter evaluation interface can be submitted here.
- Live challenge participants can hook up their filters to the live email stream at comp.ceas-challenge.cc (the "competition controller"). When you create your account on the competition controller, please use the email address that you provided when registering for the competition. Instructions on how to configure live filters are available here.
Evaluation Measures
In both the live filtering and "lab evaluation" tasks, filters will be required to provide a Ham/Spam classification for each message and an optional numeric score reflecting the filter's confidence in the prediction. Filters will be evaluated using two measures which combine the percentage of spam blocked and its false positive rate:
- Logistic average misclassification (LAM)
- Area under the ROC curve (AUC)
Filters which cannot provide a numeric score are welcome, however, such filters will be evaluated using only the LAM measure.
Training Feedback
A limited amount of feedback will be provided to train learning-based filters. Participants will be given the opportunity to actively request feedback for specific items in order to best exploit their feedback quota.
Participation Requirements
Participants will be required to provide a short (minimum 1 page) description of their system(s), to be included in the CEAS proceedings. Conference registration is not required for participation. Participants are however strongly encouraged to attend CEAS, which will be held in Mountain View, California, on August 21st and August 22nd, 2008. A Spam Challenge poster session will be held at the conference venue. Selected participants will also be given the opportunity for oral presentation of their systems.
Competition details, rules, scope and limitations will be announced at http://ceas.klika.eu, http://ceas.cc/challenge, and the competition mailing list. To join the competition mailing list, please fill out the mailing list registration form.
Key Dates
Key dates (tentative):
- Competition registration open May, 2008.
- Toolkit available June, 2008.
- Test stream beginning July 1st, 2008.
- Filter submissions for lab evaluation due: August 5th, 2008 at 14:00 GMT.
- Live competition starts: August 5th, 2008 at 14:00 GMT
- Live competition ends: August 8th, 2008 at 14:00 GMT.
Contact
The organizers can be reached at .