For starters you need to get peoples attention. Take the work done by Lorrie Faith Cranor's group at Carnegie Mellon. People can learn to avoid phishing scams but not by dragging them to a 4-hour session in a conference hall in the middle of the work day. Cranor's group use's "teachable moments" like after someone falls victim to phishing lure.

They have a prototype game called "Phishing Phil" (a commercial version is coming from Wombat Security Technologies) that helps people learn how to spot phishing lures. My initial reaction was "a game? are you kidding, how unprofessional" but then I realized those "professional" techniques don't work. In fact, I argued for more creative ways of teaching about data loss prevention using graphic workplace safety videos as an example to follow. More importantly, the researchers conducted a study and collected some data:

We evaluated the game through a user study: participants were tested on their ability to identify fraudulent web sites before and after spending 15 minutes engaged in one of three anti-phishing training activities (playing the game, reading an anti-phishing tutorial we created based on the game, or reading existing online training materials). We found that the participants who played the game were better able to identify fraudulent web sites compared to the participants in other conditions. We attribute these effects to both the content of the training messages presented in the game as well as the presentation of these materials in an interactive game format. Our results confirm that games can be an effective way of educating people about phishing and other security attacks.

I think we need to abandon our old, traditional, "professional" expectations about security training if we are going to make any progress in the effectiveness of security training.

See Can Phishing Be Foiled in the December 2008 issue of Scientific American for more on Cranor's work.