Search Stanford:

Strategic Vision

• Overview
• Technology Principles

Documents by Service Area

• Authentication
• Authorization
• Backups
• Calendar
• Desktop Management
• Directory
• Email
• Identity Management
• Integration
• Networking
• Storage
• Voice
• WWW Services
• Windows Infrastructure

Related Pages

• Planning Documents
• IT Services Projects
• Project Management Office

• IT Services
• Strategic Vision
• Email

Email Strategic Vision

Written by Russ Allbery, Huaqing Zheng and Ross Wilper, updated 1/29/07

This document tries to cover the strategic vision for email services at Stanford, focusing specifically on the centralized email services provided by and managed by IT Services. This includes the email gateway services, spam/virus filtering, POP/IMAP/webmail client services, and the mailing list service.

(In deference to Professor Donald Knuth, this document uses email in preference to e-mail.)

Principles

The goal of Stanford's email system is to provide a highly stable, centrally administered email service for all campus systems and users as well as for external systems and users who need to communicate with the campus population. In specific, we're aiming to provide the following:

• A centrally-managed mail service capable of handling the full email volume for the entire campus user population.
  
  
• Mail services that keep pace with both generally expected email features and user email access patterns (such as from mobile devices), with the understanding that email is extremely mission-critical and is used a primary communications path at all levels of the university.
  
  
• A mail system that balances strong security needs given the highly sensitive nature of email for most users and the widespread use of email as a vector for viruses and other forms of remote intrusion against the user need for simplicity and ease of access to their mail.
  
  

Email is an official university communications path, the primary communication method for many workgroups, and an expected and relied-upon part of daily life for the Stanford community. It is one of the highest-visibility services provided by IT Services. We have to take a measured and careful approach to any changes and maintain the highest level of stability and reliability that we can provide.

Current specific areas of focus are improving the reliability and scaling of the email system for the increased load caused by shifting more work to the server with technologies like IMAP, supporting ubiquitous access to one's mail from any system (including mobile devices), and ameliorating the serious security and usability challenges from the explosion of spam and email-borne viruses.

Technologies

Stable core technologies:

• IMAP and POP as network protocols for e-mail access, using both passwords sent with SSL encryption and checked by the server against Kerberos and native Kerberos authentication via GSSAPI.
  
  
• SMTP for mail transit, including SMTP over SSL for secure and authenticated mail transmission from client systems, including clients who are travelling to other sites.
  
  
• The Horde IMP webmail system (which has become a primary client for many of our users and at least a secondary client for nearly all of our users).
  
  
• Sophos PureMessage for virus filtering and spam tagging.
  
  
• LDAPv3 with GSSAPI authentication for mail forwarding information and as a store for vacation autoresponder information, using a locally-written vacation system and a client interface integrated with Stanford.You.
  
  
• Cyrus IMAP as the server software providing IMAP and POP access and server-side filtering.
  
  
• Eudora, Outlook, and Outlook Express for Windows clients, Eudora and OS X Mail, and Entourage for Mac clients, and mutt and Pine as Unix e-mail clients.
  
  
• Mail to news and news to mail bidirectional gatewaying using a locally-written system, used extensively for internal purposes and occasionally for classes and projects.
  
  

Technologies new to IT Services: (see Projects and Research below)

• Postfix for mail transit, offering a better security track record, less odd behavior, and better performance and reliability than sendmail. (Postfix is already in use for handling outgoing mail on many new servers.)
  
  
• Mailman for mailing list management and archiving.
  
  
• Thunderbird as a possible cross-platform email client with good IMAP handling.
  
  

Emerging technologies: (see Projects and Research below)

• Sieve as a specification language for server-side mail filtering. This is currently in use for automatic spam removal via a custom CGI application and users with Mulberry can edit their own Sieve rules, but there is no interface available for users who use other clients or pre-packaged rules other than spam filtering.
  
  
• Some form of encrypted email for HIPAA compliance. (The market has not yet standardized on a particular approach to this problem.)
  
  
• Murder or some other form of aggregator technology to provide easier access to group IMAP accounts and possibly easier migration of mailboxes between servers.
  
  
• A number of departments on campus utilize Exchange 2003 for combined email and calendaring functionality and to support mobile devices via DirectPush, Goodlink, or Blackberry connectors.
  
  

Deprecated technologies: (see Projects below)

• Majordomo. Our locally-modified Majordomo installation has now been replaced with Mailman.
  
  
• sendmail. Postfix now offers better performance and reliability with a stronger security track record and better Linux distribution support.
  
  
• KPOP. The KPOP protocol is Kerberos v4-only and therefore must be retired as we migrate to Kerberos v5, replaced with either GSSAPI authentication (preferred) or password authentication over SSL (more widely supported).
  
  
• The PC Leland mail proxy, which is KPOP-only.
  
  
• Zephyr for real-time email notification for Unix users. This must be replaced since Zephyr is tied to Kerberos v4, but no good replacement has yet been identified.
  
  

Other technologies in use: (These technologies are currently deployed and useful in specific circumstances, but either are not attractive for broader use or have a limited scope of applicability -- we are neither recommending expanding them nor recommending eliminating them at this time.)

• There are many other email clients in use on all platforms than the ones listed above. The ones listed under core technologies are the ones with a broad existing user base, sufficient to make it worthwhile to exert special effort towards documentation and client support. For all other clients, our position is to provide access to email through standardized protocols and any clients that speak those protocols properly should work correctly with our email system. We may not be able to provide as much specific help with those clients, but so far as possible our users should be able to use any email client they wish.
  
  
• qmail is in use as a transport agent for certain systems with specific email handling needs (generally complex rewriting or routing to programs), in particular the vacation responder and the mail to news gateway. We don't expect to use qmail in any additional areas in the future.
  
  

Projects

First:

• Investigate and test alternate backup mechanisms for the IMAP/POP SAN storage, since the current Tivoli backup is taking too long on the pobox servers. We are in the process of testing LVM snapshots and running backups from SAN volumes mounted on the spare server.
  
  
• Deploy a secure email system for HIPAA-related information. This would include some mechanism for encrypting e-mail on demand. The exact details of who would be affected and how the encryption would be handled still need to be determined. Policies and rules should be decided by an external governing body.
  
  
• Evaluate Exchange 2007 with Outlook Web Access, Goodlink, and Blackberry gateways for possible unified messaging solution.
  
  
• Upgrade the Horde/IMP Webmail to the Horde framework version 3. The upgrade will present a much more IMAPish interface for Webmail and will include a Sieve filter editor.
  
  
• Tie Mailman into the Workgroup system or some equivalent to tie mailing list membership with other institutional group management systems. This may also replace the current system for managing course mailing lists. While integrating mailing lists and workgroups, we may be able to use workgroups as mailing lists in Exchange as well.
  
  

Next:

• Implement automatic spam filtering for all users. By default, messages marked as spam will be filtered into a separate folder in IMAP and Webmail. Users will be provided with a tool to opt out of the spam filtering.
  
  
• Set up a departmental virtual domain service so that departments can have their own email name space but route an user's email to their departmental address to an user configurable forwarding address.
  
  

Later:

• Necessary upgrades and changes in technology driven by the Kerberos v4 to v5 migration (see the authentication vision). This in particular means dropping support for KPOP, dropping the PC Leland mail proxy, and replacing Zephyr for real-time Unix mail notification or dropping that service. Some of these changes may need to be done within the next one to two years, depending on the progress of the Kerberos upgrade project.
  
  

Research

• Evaluate Murder or some other aggregator system as a possible solution to the current problems with access to group IMAP accounts (such as clients that don't understand accessing different accounts from different servers) and possibly for easier migration of user mailboxes between servers.
  
  
• Several technical solutions of are being implemented to some portions of the spam, forgery, and virus problems with the current SMTP protocol. As yet, none of those solutions have emerged as a reasonable, stable, and deployable system, but this will probably change. When it changes, it's likely to change in a hurry, given the pent-up client demand for some systematic approach to the spam problem. We need to stay current on developments in this area and make sure that our infrastructure is in a position to adopt any solution that emerges from the pack (which also means moving to the most modern and best supported mail systems, such as Postfix instead of sendmail).
  
  
• The IMP webmail system works reasonably well, but isn't ideal and is missing many features found in the web-based email systems provided by other sites (particularly gmail). Other packages, projects, or products are likely to emerge in this area, and we should be evaluating those as they appear with an eye to possibly replacing our current webmail system.
  
  
• Evaluate other available desktop email clients and offer official support for an additional one if we can find one with better IMAP features and performance. Particularly on the PC side, none of our current officially-supported clients are satisfactory IMAP clients, which is a significant obstacle to expanding IMAP use. Thunderbird is a possibility as a cross-platform email client with an active development community that may be convinced to support Kerberos and our LDAP requirements..
  
  
• Microsoft Exchange will continue to be in demand by some clients, which means that we should continue to evaluate the feasibility of providing a campus-wide Exchange service as client needs change and keep abreast of any changes that may make a campus-wide service cost-effective.
  
  
• Explore deploying additional features of Exchange, particularly RPC over HTTPS to allow off-campus users to use Exchange without requiring a VPN, and Kerberos authentication between Outlook 2003 and Exchange 2003.
  
  

Finished Projects

• Move the IMAP/POP servers to Linux and set up infrastructure so that we can increase the default quota for users to 1GB to keep pace with current quota offerings from gmail, Hotmail, and our peer institutions (not to mention user demand) if we want to. We may take the more moderate course of 500MB to start with. The current Solaris systems are facing severe performance problems.
  
  
• Migrate from Majordomo to Mailman for mailing list management.