Showing posts with label spamassassin. Show all posts
Showing posts with label spamassassin. Show all posts

20100908

How to automate the SpamAssassin feeding

This post is my answer to the Sun Wikis' entry about Enabling Anti-Spam functionality in Convergence.

Technologies used:
Messaging Server
SpamAssassin
RBAC - execution profiles

Messaging Server use the two variables to hold the email accounts for feeding anti-spam system
with the positive and false positive spam messages:

  • service.feedback.spam

  • service.feedback.notspam


But there is no embedded mechanism to deal with them.
Here comes my solution.

Every time the Convergence user marks the spam (ot not spam), with the appropriate button from its interface,
the mail is being sent to the address provided within the configuration.
The email messages accumulate in the accounts and waiting for the action.

Scenario:

  • fetching messages from spam account

  • teaching spamassassin with spam

  • cleaing INBOX folder

  • fetching messages from notspam account

  • teaching SA with ham

  • cleaning account



Methods:

  • fetching messages from spam account


    1. imsexport


  • teaching spamassassin with spam


    1. sa-learn --spam


  • cleaing INBOX folder


    1. mboxutil -d

    2. mboxutil -c


  • fetching messages from notspam account


    1. imsexport


  • teaching SA with ham


    1. sa-learn --ham


  • cleaning account


    1. mboxutil -d

    2. mboxutil -c



Due to a fact the script will be invoked by root account, to get the access either to imsexport files or sa-learn with valid Bayes DB,
I have decided to use one of the Solaris RBAC mechanisms - profiles.

I would not describe the profile creation step-by-step, because the learning by example is much more valuable.

# /usr/sadm/bin/smexec add -H localhost -u root -- \
-n "SpamAssassin Administration" -t cmd -c /export/home/sa/bin/sa-learn -U 105 -G 102
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
There is no Solaris Management Console Server running on localhost.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\
# svcadm enable wbem

# /usr/sadm/bin/smexec add -H localhost -u root -- \
-n "SpamAssassin Administration" -t cmd -c /export/home/sa/bin/sa-learn -U 105 -G 102
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
Loading Tool: com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost
Login to localhost as user root was successful.
Download of com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost was successful.
You have entered a non-existent right SpamAssassin Administration.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\
# cd /usr/lib/help/profiles/locale/C
# cat RtSAAdmin.html
< HTML >
< HEAD >
< TITLE >< /TITLE >
< /HEAD >
< BODY >
SpamAssassin Administration right allows the user or role SA management.
< /BODY >
< /HTML >

^^^^^
Spaces has been added to HTML tags in due to blogspot problems with handling this kind of stuff within the post body.

# /usr/sadm/bin/smprofile add -H localhost -u root -- \
-n "SpamAssassin Administration" -d "Manage SpamAssassin" -m RtSAAdmin.html
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
Loading Tool: com.sun.admin.usermgr.cli.profile.UserMgrProfCli from localhost
Login to localhost as user root was successful.
Download of com.sun.admin.usermgr.cli.profile.UserMgrProfCli from localhost was successful.

# tail -1 /etc/security/prof_attr
SpamAssassin Administration:::Manage SpamAssassin:help=RtSAAdmin.html

# /usr/sadm/bin/smexec add -H localhost -u root -- \
-n "SpamAssassin Administration" -t cmd -c /export/home/sa/bin/sa-learn -U 105 -G 102
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
Loading Tool: com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost
Login to localhost as user root was successful.
Download of com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost was successful.

# tail -1 /etc/security/exec_attr
SpamAssassin Administration:solaris:cmd:::/export/home/sa/bin/sa-learn:uid=105;gid=102


So, running sa-learn by the user / role with "SpamAssassin Administration" profile assigned
allow access to bayes DB files with the proper rights.
Examples can be seen in the script code provided below.


#!/usr/bin/ksh
#------------------------------------
# sa-feed.ksh
# feed SpamAssassin with {NOT}SPAM
# from file in mbox format
#====================================
# author: Marcin Wisnios
# e-mail: wisnios at wisnios dot com
#------------------------------------
PATH=$PATH:/opt/sun/comms/messaging64/bin:/export/home/sa/bin

TDIR=$(mktemp -td) # temporary directory
FILE=$TDIR/INBOX # mbox file
MUSR=$(ps -o user -p $(pgrep -nf dispatcher) | tail -1) # messaging server runtime user
SUSR=$(ps -o user -p $(pgrep -nf spamd) | tail -1) # spamassassin runtime user
SPAM=$(configutil -o service.feedback.spam) # feedback account for spam
NOTSPAM=$(configutil -o service.feedback.notspam) # feedback account for not spam
BDBL=/export/home/sa/.spamassassin # Bayes DB location
SALEARN="pfexec sa-learn --dbpath $BDBL" # fixed part of sa-learn invocation

chown $MUSR $TDIR
chmod 0755 $TDIR

imsexport -s INBOX -d $TDIR -u $SPAM


[ -f $FILE ] && {
chown $SUSR $FILE

NSPAM_BEFORE=$($SALEARN --dump magic 2> /dev/null | grep nspam | awk '{print $3}')
$SALEARN --mbox --spam $FILE 2> /dev/null &&\
NSPAM_AFTER=$($SALEARN --dump magic 2> /dev/null | grep nspam | awk '{print $3}')

[ $NSPAM_BEFORE -lt $NSPAM_AFTER ] && {
mboxutil -d user/$SPAM/INBOX
mboxutil -c user/$SPAM/INBOX
}

rm $FILE
}

imsexport -s INBOX -d $TDIR -u $NOTSPAM

[ -f $FILE ] && {
chown $SUSR $FILE

NHAM_BEFORE=$($SALEARN --dump magic 2> /dev/null | grep nham | awk '{print $3}')
$SALEARN --mbox --ham $FILE 2> /dev/null &&\
NHAM_AFTER=$($SALEARN --dump magic 2> /dev/null | grep nham | awk '{print $3}')

[ $NHAM_BEFORE -lt $NHAM_AFTER ] && {
mboxutil -d user/$NOTSPAM/INBOX
mboxutil -c user/$NOTSPAM/INBOX
}
}

[ -d $TDIR ] && rm -rf $TDIR


Run the script in a way you like.
I put it directly to a crontab. Root's crontab.

Enjoy.

20100625

crle and missing libusb.so.1 library

Today I have installed SpamAssassin from a tar.gz archive on Solaris 10 10/09.
Though everything looked fine I faced the problem with invocation of sa-update - SpamAssassin rules updater.

$ sa-update
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
Killed
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
Killed
error: GPG validation failed!
The update downloaded successfully, but the GPG signature verification
failed.
channel: GPG validation failed, channel failed
$ gpg
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
Killed

(gnupg has been installed from a package from sunfreeware.com, but it was not a problem)

Quick truss session:

# truss gpg
[...]
stat64("/usr/local/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/local/ssl/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/openwin/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/X11R6/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/local/BerkeleyDB.4.7/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
[...]

find lookup:

# find /usr -name libusb.so.1
/usr/sfw/lib/libusb.so.1

...and I had to change the default library path.
The tool - crle - runtime linking environment configurator, was taken as the only fair solution.
Do not want to reproduce the manual pages, so, below is the syntax I had used.
Checking the current options:

# crle

Configuration file [version 4]: /var/ld/ld.config
Default Library Path (ELF): /lib:/usr/lib (system default)
Trusted Directories (ELF): /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib

Command line:
crle -c /var/ld/ld.config -s /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib


Complementation of default library path:

# crle -c /var/ld/ld.config -l /lib:/usr/lib:/usr/sfw/lib -s /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib

[CHECK]
# crle
Configuration file [version 4]: /var/ld/ld.config
Default Library Path (ELF): /lib:/usr/lib:/usr/sfw/lib
Trusted Directories (ELF): /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib

Command line:
crle -c /var/ld/ld.config -l /lib:/usr/lib:/usr/sfw/lib -s /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib
# exit
$ gpg
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: directory `/export/home/sa/.gnupg' created
gpg: new configuration file `/export/home/sa/.gnupg/gpg.conf' created
gpg: WARNING: options in `/export/home/sa/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/export/home/sa/.gnupg/secring.gpg' created
gpg: keyring `/export/home/sa/.gnupg/pubring.gpg' created
gpg: Go ahead and type your message ...
^C
gpg: signal 2 caught ... exiting
$ sa-update
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
$


Final notes
The syntax being used could be shortened to a form of:

# crle -u -l /usr/sfw/lib

where -u stands for existing configuration (file /var/ld/ld.config) update.
But be aware that by omitting the -u argument you turn your box into soldered fish tin, until you turn the other directories to the default path.

# crle -l /usr/sfw/lib
# init
ld.so.1: init: fatal: libpam.so.1: open failed: No such file or directory
Killed
# crle -u -l /lib -l /usr/lib -l /usr/sfw/lib
# init
Usage: init [0123456SsQqabc]
#