Showing posts with label solaris. Show all posts
Showing posts with label solaris. Show all posts

20100908

How to automate the SpamAssassin feeding

This post is my answer to the Sun Wikis' entry about Enabling Anti-Spam functionality in Convergence.

Technologies used:
Messaging Server
SpamAssassin
RBAC - execution profiles

Messaging Server use the two variables to hold the email accounts for feeding anti-spam system
with the positive and false positive spam messages:

  • service.feedback.spam

  • service.feedback.notspam


But there is no embedded mechanism to deal with them.
Here comes my solution.

Every time the Convergence user marks the spam (ot not spam), with the appropriate button from its interface,
the mail is being sent to the address provided within the configuration.
The email messages accumulate in the accounts and waiting for the action.

Scenario:

  • fetching messages from spam account

  • teaching spamassassin with spam

  • cleaing INBOX folder

  • fetching messages from notspam account

  • teaching SA with ham

  • cleaning account



Methods:

  • fetching messages from spam account


    1. imsexport


  • teaching spamassassin with spam


    1. sa-learn --spam


  • cleaing INBOX folder


    1. mboxutil -d

    2. mboxutil -c


  • fetching messages from notspam account


    1. imsexport


  • teaching SA with ham


    1. sa-learn --ham


  • cleaning account


    1. mboxutil -d

    2. mboxutil -c



Due to a fact the script will be invoked by root account, to get the access either to imsexport files or sa-learn with valid Bayes DB,
I have decided to use one of the Solaris RBAC mechanisms - profiles.

I would not describe the profile creation step-by-step, because the learning by example is much more valuable.

# /usr/sadm/bin/smexec add -H localhost -u root -- \
-n "SpamAssassin Administration" -t cmd -c /export/home/sa/bin/sa-learn -U 105 -G 102
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
There is no Solaris Management Console Server running on localhost.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\
# svcadm enable wbem

# /usr/sadm/bin/smexec add -H localhost -u root -- \
-n "SpamAssassin Administration" -t cmd -c /export/home/sa/bin/sa-learn -U 105 -G 102
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
Loading Tool: com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost
Login to localhost as user root was successful.
Download of com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost was successful.
You have entered a non-existent right SpamAssassin Administration.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
\
# cd /usr/lib/help/profiles/locale/C
# cat RtSAAdmin.html
< HTML >
< HEAD >
< TITLE >< /TITLE >
< /HEAD >
< BODY >
SpamAssassin Administration right allows the user or role SA management.
< /BODY >
< /HTML >

^^^^^
Spaces has been added to HTML tags in due to blogspot problems with handling this kind of stuff within the post body.

# /usr/sadm/bin/smprofile add -H localhost -u root -- \
-n "SpamAssassin Administration" -d "Manage SpamAssassin" -m RtSAAdmin.html
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
Loading Tool: com.sun.admin.usermgr.cli.profile.UserMgrProfCli from localhost
Login to localhost as user root was successful.
Download of com.sun.admin.usermgr.cli.profile.UserMgrProfCli from localhost was successful.

# tail -1 /etc/security/prof_attr
SpamAssassin Administration:::Manage SpamAssassin:help=RtSAAdmin.html

# /usr/sadm/bin/smexec add -H localhost -u root -- \
-n "SpamAssassin Administration" -t cmd -c /export/home/sa/bin/sa-learn -U 105 -G 102
Authenticating as user: root

Type /? for help, pressing accepts the default denoted by [ ]
Please enter a string value for: password ::
Loading Tool: com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost
Login to localhost as user root was successful.
Download of com.sun.admin.usermgr.cli.execs.UserMgrExecCli from localhost was successful.

# tail -1 /etc/security/exec_attr
SpamAssassin Administration:solaris:cmd:::/export/home/sa/bin/sa-learn:uid=105;gid=102


So, running sa-learn by the user / role with "SpamAssassin Administration" profile assigned
allow access to bayes DB files with the proper rights.
Examples can be seen in the script code provided below.


#!/usr/bin/ksh
#------------------------------------
# sa-feed.ksh
# feed SpamAssassin with {NOT}SPAM
# from file in mbox format
#====================================
# author: Marcin Wisnios
# e-mail: wisnios at wisnios dot com
#------------------------------------
PATH=$PATH:/opt/sun/comms/messaging64/bin:/export/home/sa/bin

TDIR=$(mktemp -td) # temporary directory
FILE=$TDIR/INBOX # mbox file
MUSR=$(ps -o user -p $(pgrep -nf dispatcher) | tail -1) # messaging server runtime user
SUSR=$(ps -o user -p $(pgrep -nf spamd) | tail -1) # spamassassin runtime user
SPAM=$(configutil -o service.feedback.spam) # feedback account for spam
NOTSPAM=$(configutil -o service.feedback.notspam) # feedback account for not spam
BDBL=/export/home/sa/.spamassassin # Bayes DB location
SALEARN="pfexec sa-learn --dbpath $BDBL" # fixed part of sa-learn invocation

chown $MUSR $TDIR
chmod 0755 $TDIR

imsexport -s INBOX -d $TDIR -u $SPAM


[ -f $FILE ] && {
chown $SUSR $FILE

NSPAM_BEFORE=$($SALEARN --dump magic 2> /dev/null | grep nspam | awk '{print $3}')
$SALEARN --mbox --spam $FILE 2> /dev/null &&\
NSPAM_AFTER=$($SALEARN --dump magic 2> /dev/null | grep nspam | awk '{print $3}')

[ $NSPAM_BEFORE -lt $NSPAM_AFTER ] && {
mboxutil -d user/$SPAM/INBOX
mboxutil -c user/$SPAM/INBOX
}

rm $FILE
}

imsexport -s INBOX -d $TDIR -u $NOTSPAM

[ -f $FILE ] && {
chown $SUSR $FILE

NHAM_BEFORE=$($SALEARN --dump magic 2> /dev/null | grep nham | awk '{print $3}')
$SALEARN --mbox --ham $FILE 2> /dev/null &&\
NHAM_AFTER=$($SALEARN --dump magic 2> /dev/null | grep nham | awk '{print $3}')

[ $NHAM_BEFORE -lt $NHAM_AFTER ] && {
mboxutil -d user/$NOTSPAM/INBOX
mboxutil -c user/$NOTSPAM/INBOX
}
}

[ -d $TDIR ] && rm -rf $TDIR


Run the script in a way you like.
I put it directly to a crontab. Root's crontab.

Enjoy.

20100625

crle and missing libusb.so.1 library

Today I have installed SpamAssassin from a tar.gz archive on Solaris 10 10/09.
Though everything looked fine I faced the problem with invocation of sa-update - SpamAssassin rules updater.

$ sa-update
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
Killed
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
Killed
error: GPG validation failed!
The update downloaded successfully, but the GPG signature verification
failed.
channel: GPG validation failed, channel failed
$ gpg
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
Killed

(gnupg has been installed from a package from sunfreeware.com, but it was not a problem)

Quick truss session:

# truss gpg
[...]
stat64("/usr/local/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/local/ssl/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/openwin/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/X11R6/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/local/BerkeleyDB.4.7/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
stat64("/usr/lib/libusb.so.1", 0x080473A0) Err#2 ENOENT
ld.so.1: gpg: fatal: libusb.so.1: open failed: No such file or directory
[...]

find lookup:

# find /usr -name libusb.so.1
/usr/sfw/lib/libusb.so.1

...and I had to change the default library path.
The tool - crle - runtime linking environment configurator, was taken as the only fair solution.
Do not want to reproduce the manual pages, so, below is the syntax I had used.
Checking the current options:

# crle

Configuration file [version 4]: /var/ld/ld.config
Default Library Path (ELF): /lib:/usr/lib (system default)
Trusted Directories (ELF): /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib

Command line:
crle -c /var/ld/ld.config -s /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib


Complementation of default library path:

# crle -c /var/ld/ld.config -l /lib:/usr/lib:/usr/sfw/lib -s /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib

[CHECK]
# crle
Configuration file [version 4]: /var/ld/ld.config
Default Library Path (ELF): /lib:/usr/lib:/usr/sfw/lib
Trusted Directories (ELF): /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib

Command line:
crle -c /var/ld/ld.config -l /lib:/usr/lib:/usr/sfw/lib -s /usr/lib/secure:/opt/sun/comms/calendar/SUNWics5/cal/lib
# exit
$ gpg
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
gpg: directory `/export/home/sa/.gnupg' created
gpg: new configuration file `/export/home/sa/.gnupg/gpg.conf' created
gpg: WARNING: options in `/export/home/sa/.gnupg/gpg.conf' are not yet active during this run
gpg: keyring `/export/home/sa/.gnupg/secring.gpg' created
gpg: keyring `/export/home/sa/.gnupg/pubring.gpg' created
gpg: Go ahead and type your message ...
^C
gpg: signal 2 caught ... exiting
$ sa-update
gpg: WARNING: using insecure memory!
gpg: please see http://www.gnupg.org/faq.html for more information
$


Final notes
The syntax being used could be shortened to a form of:

# crle -u -l /usr/sfw/lib

where -u stands for existing configuration (file /var/ld/ld.config) update.
But be aware that by omitting the -u argument you turn your box into soldered fish tin, until you turn the other directories to the default path.

# crle -l /usr/sfw/lib
# init
ld.so.1: init: fatal: libpam.so.1: open failed: No such file or directory
Killed
# crle -u -l /lib -l /usr/lib -l /usr/sfw/lib
# init
Usage: init [0123456SsQqabc]
#

20100607

svclog

Everytime I have to check the SMF log file quickly I suffer.
I suffer in due to lack of simple method which I would provide you today with.

C'n'P:

#!/usr/bin/ksh
#------------------------------------
# svclog
# ...run for your logs
#====================================
# author: Marcin Wisnios
# e-mail: wisnios at wisnios dot com
#------------------------------------

svcs $1 > /dev/null 2>&1
if [ $? -eq 1 ]; then
echo "No match."
exit
fi

if [ ! -z $1 ]; then
LOG=$(svcs -l $1 | sed 's/logfile[ ]*\(.*\)/\1/p;d')
echo "\n\t\t${LOG}\n"
else
echo "Usage: $0 FMRI [n]"
echo "n - number of lines to display"
exit
fi

if [ ! -z $2 ]; then
tail -$2 ${LOG}
else
tail -f ${LOG}
fi

Save as, for ex. /usr/local/bin/svclog and... run for your logs:

# svclog ds 3

/var/svc/log/application-sun-ds:default.log

Waiting for Directory Server instance '/instances/ds1' to start...
Directory Server instance '/instances/ds1' started: pid=434
[ Jun 7 16:53:00 Method "start" exited with status 0 ]
# svclog ms

/var/svc/log/application-sun-ms:default.log

Stopping dispatcher server 593 ... done
Stopping sched server 591 ... done
Stopping http server 590 ... done
Stopping pop server 589 ...... done
Stopping imap server 588 ... done
Stopping purge server 587 ... done
Stopping store server 584 .... done
Stopping watcher 583 ... done
[ Jun 7 23:40:12 Method "stop" exited with status 0 ]
[ Jun 7 23:40:12 Executing start method ("/opt/sun/comms/messaging64/bin/start-msg") ]
Connecting to watcher ...
Launching watcher ... 1463
Starting store server .... 1464
Checking store server status .....
- CUT -

Enjoy.

20100525

fc-fabric(ation) of server's dysfunction

Do you like minimization? I do.
But sometimes you can go one bridge too far.

Today I have switched off the set of useless services.
One after one...
Do I need... No!
Do I need... No!
Do I need Fibre Channel Fabric device support? No.

$ svcadm disable fc-fabric

Another one bites the dust.

After the couple of hours I decided to reboot the server to check its readiness to serve.
And... it failed.



Of course at the moment of failure I dit not realize it was caused by service unavailability of fc-fabric.
Unnecessary stress.

The solution has been revealed during the dependencies check.



Keep this in mind.

20100303

maximum number of instances of the package

Sometimes there is a need of existence of concurrent versions of the same package.
It is not a issue why, but how to achieve this behaviour.

During the normal installation procedure of the package, with the same name but different version then already installed one,
there is a chance to see a similar message:

# pkgadd -d somepkg*dstream
[...]
Current administration requires that a unique instance of the
<somepkg> package be created. However, the maximum number of
instances of the package which may be supported at one time on the
same system has already been met.

No changes were made to the system.

To resolve the conflict change the value of MAXINST variable from inside the pkginfo file
(during the package build process).

For example, to allow the coexistence of maximum number of two packages use the following:

MAXINST=2


# pkgadd -d somepkg*dstream
[...]

Installation of <somepkg.2> was successful.

20091124

Solaris ksh variations

Today i have read, and in the parallel - discovered, that Solaris has got three ksh variants.
Two of them are ksh88, and one ksh93.

$ grep -i ver /usr/bin/ksh /usr/xpg4/bin/sh /usr/dt/bin/dtksh
/usr/bin/ksh:@(#)Version M-11/16/88i
/usr/xpg4/bin/sh:@(#)Version M-11/16/88i
/usr/dt/bin/dtksh:@(#)Version M-12/28/93d
/usr/dt/bin/dtksh:@(#)Version M-12/28/93
/usr/dt/bin/dtksh:@(#)Version 12/28/93

The standard one - /usr/bin/ksh, and a POSIX-compliant veriant of ksh88 - /usr/xpg4/bin/sh.
Both of them are the components of SUNWcsu (Core Solaris (Usr)) package.
dtksh comes from SUNWdtbas.

$ ls -li /usr/bin/ksh /usr/xpg4/bin/sh /usr/dt/bin/dtksh
489 -r-xr-xr-x 3 root bin 171412 Aug 7 13:27 /usr/bin/ksh
26709 -r-xr-xr-x 1 root bin 620144 Jan 23 2005 /usr/dt/bin/dtksh
1536 -r-xr-xr-x 1 root bin 171412 Aug 7 13:27 /usr/xpg4/bin/sh
$ file /usr/bin/ksh /usr/xpg4/bin/sh /usr/dt/bin/dtksh
/usr/bin/ksh: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped
/usr/xpg4/bin/sh: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, stripped
/usr/dt/bin/dtksh: ELF 32-bit LSB executable 80386 Version 1, dynamically linked, not stripped, no debugging information available

As you can already saw (on the ls -li listing) there are also the three brothers-in-inode:

# ls -li /usr/bin/*ksh
489 -r-xr-xr-x 3 root bin 171412 Aug 7 13:27 /usr/bin/ksh
489 -r-xr-xr-x 3 root bin 171412 Aug 7 13:27 /usr/bin/pfksh
489 -r-xr-xr-x 3 root bin 171412 Aug 7 13:27 /usr/bin/rksh

It's the highest inode count from all of the Solaris shells (10u8, SUNWCall), the second place goes to csh with only two file names binded to its inode.

Fascinating.

20091122

Make yourself a package

In the world where the Good Security Practices becomes Science Fiction, there was an Admin who wants to train himself in Solaris Packaging.
He has written down the script, which has made his Lord the Legend ;-)
Now, let's put some light on it.

The script takes .ssh folder files from the specified user of template machine, and turns it into .ssh folder of root user on target host. There are also the configuration changes, inside of either /etc/default/login file, or /etc/ssh/sshd_config one.
Do not try this at (production) home!

# ./rootbox.sh
Generating package files
prototype
pkginfo
checkinstall
postinstall
postremove
login.sed
sshd_config.sed
Making package MMWrootbox.1.0.i386.pkg [/tmp]
success
Translating package format to a datastream
success
# cd /tmp
# pkgadd -d MMWrootbox.1.0.i386.pkg

The following packages are available:
1 MMWrootbox Root box
(i386) 1.0

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]:

Processing package instance from

Root box(i386) 1.0
Marcin Marian Wisnios
## Executing checkinstall script.
## Processing package information.
## Processing system information.
## Verifying disk space requirements.
## Checking for conflicts with packages already installed.
## Checking for setuid/setgid programs.

This package contains scripts which will be executed with super-user
permission during the process of installing this package.

Do you want to continue with the installation of [y,n,?] y

Installing Root box as

## Installing part 1 of 1.
/root/.ssh/authorized_keys
[ verifying class ]
Modifying /etc/default/login
Modifying /etc/ssh/sshd_config
[ verifying class ]
## Executing postinstall script.

Installation of was successful.
# date;svcs -x ssh| grep -i state
Sun Nov 22 08:50:57 CET 2009
State: online since Sun Nov 22 08:50:42 2009
# diff /etc/ssh/sshd_config /tmp/rootbox.bak/sshd_config
128c128
< PermitRootLogin without-password
---
> PermitRootLogin no
# diff /etc/default/login /tmp/rootbox.bak/login
18c18
< #CONSOLE=/dev/console
---
> CONSOLE=/dev/console
# pkginfo MMWrootbox
system MMWrootbox Root box
# pkginfo -l MMWrootbox
PKGINST: MMWrootbox
NAME: Root box
CATEGORY: system
ARCH: i386
VERSION: 1.0
BASEDIR: /
VENDOR: Marcin Marian Wisnios
DESC: Methods and keys to allow remote root user access
PSTAMP: 20091122085002
INSTDATE: Nov 22 2009 08:50
EMAIL: wisnios@gmail.com
STATUS: completely installed
FILES: 4 installed pathnames
2 shared pathnames
1 directories
3 blocks used (approx)

# pkgrm MMWrootbox

The following package is currently installed:
MMWrootbox Root box
(i386) 1.0

Do you want to remove this package? [y,n,?,q] y

## Removing installed package instance

This package contains scripts which will be executed with super-user
permission during the process of removing this package.

Do you want to continue with the removal of this package [y,n,?,q] y
## Verifying package dependencies in global zone
## Processing package information.
## Removing pathnames in class
Modifying /etc/ssh/sshd_config
Modifying /etc/default/login
## Removing pathnames in class
/root/.ssh/authorized_keys
/root/.ssh
## Executing postremove script.
## Updating system information.

Removal of was successful.
# svcs -x ssh|grep -i state; date
State: online since Sun Nov 22 08:52:33 2009
Sun Nov 22 08:52:50 CET 2009
# diff /etc/ssh/sshd_config /tmp/rootbox.bak/sshd_config
# diff /etc/default/login /tmp/rootbox.bak/login


http://cs-tools.googlecode.com/files/rootbox.sh

Enjoy.

20091028

Paranoid

So, have you ever seen the Solaris 10 boot process, line by line?
I do not talk about the kernel -m verbose mode, which only shows the particular SMF identifiers.
Have you ever wondered why the boot process has hanged? and what's the cause?

The Wisnios way is as follow.
I've decided to replace the master restarter binary with a shell script.

# mv /lib/svc/bin/svc.startd /lib/svc/bin/svc.startdd
# vi /lib/svc/bin/svc.startd


#!/bin/sh
truss -fa -t exec /lib/svc/bin/svc.startdd


# chmod 555 /lib/svc/bin/svc.startd
# chgrp sys /lib/svc/bin/svc.startd


You could also choose the more verbose mode by adding the -e switch to truss command.
Truss -f option follows all fork/vfork children, -a shows the argument strings within the exec() calls and -e shows the environment variables (for ex. SMF_FMRI).

Other traces are limited by imagination only.

20090703

Arming Messaging Server

Today I've got the three recipes to tighten the unwanted malicious user activity.
All of them are related to mappings file rules.

First i want to prohibit all the unauthenticated users from sending emails.
It could be achieved with a following line in the FROM_ACCESS table:

TCP|*|25|*|*|SMTP*|*|tcp_local|*@*|* $C$}$6,_canonical_name_{$N$ -$ Authentication$ required$ when$ sending$ with$ this$ envelope$ sender$ domain$E


The second step is to block attemtps of faking From header, after successful authentication
(for ex. using stolen password, or personal account in a negative manner).
I've prepared the suitable lines, also in FROM_ACCESS table:

*|SMTP*|*|tcp_auth|*@*|*@$4* $Y
*|SMTP*|*|tcp_auth|*@*|*@* $N$_Sender$ address$ rejected$ for$ $4

It reject the use of tcp_auth channel when domain of authenticated account is different from the one used within the From header.

The third method take advantage of check_metermaid.so library (included in Messaging Server installation).
MeterMaid could be used to throttle the agressive usage of mail server.
I've used two rules (to block them all ;-) ).
First one restrict the number of connections (15) in a unit of time (60 s.).
It's assigned under PORT_ACCESS table:

*|*|*|*|* $C$:A$[/opt/sun/comms/messaging64/lib/check_metermaid.so,throttle,ext_throttle,$3]$N421$ Connection$ declined$ at$ this$ time$E

Related thresholds are defined with a configutil command or by edition of msg.conf:

metermaid.config.secret = [your shared secret to authenticate incoming connections]
metermaid.config.serverhost = [host name or ip address of your metermaid server]
metermaid.table.ext_throttle.data_type = string
metermaid.table.ext_throttle.options = nocase
metermaid.table.ext_throttle.quota = 15
metermaid.table.ext_throttle.quota_time = 60

Ex.

# configutil -o metermaid.config.serverhost -v somehost.somedomain
# configutil -o metermaid.config.secret -v somesecret
and so on...

ext_throttle is defined by you throttling table name, and must be the same within the mappings and msg.conf files.

The second rule restrict number of total recipients sent to by a user (i've used the same limit values, but you could add the next throttle table with required thresholds).
It should be addes within the ORIG_SEND_ACCESS mapping tables:

tcp_auth|*|*|* $C$[/opt/sun/comms/messaging64/lib/check_metermaid.so,throttle,ext_throttle,$0]$NExcessive$ email$ sent$ -$ Please$ try$ again$ later$E

So, run imsimta cnbuild && imsimta restart and...

[...]
235 2.7.0 LOGIN authentication successful.
250 2.5.0 Address Ok.
250 2.1.5 marcin.wisnios@somedomain OK.
354 Enter mail, end with a single ".".
250 2.5.0 Ok.
250 2.5.0 Address Ok.
250 2.1.5 marcin.wisnios@somedomain OK.
354 Enter mail, end with a single ".".
250 2.5.0 Ok.
250 2.5.0 Address Ok.
250 2.1.5 marcin.wisnios@somedomain OK.
354 Enter mail, end with a single ".".
250 2.5.0 Ok.
[...]
250 2.5.0 Address Ok.
250 2.1.5 marcin.wisnios@somedomain OK.
354 Enter mail, end with a single ".".
250 2.5.0 Ok.
250 2.5.0 Address Ok.
550 5.7.1 Excessive email sent - Please try again later: marcin.wisnios@somedomain

Enjoy.

20090531

Solaris IP shortcuts

Today, I had to modify my routing table under Solaris 10u7 (5/09), and I made a typo.
Instead of writing 10.0.2.1, I typed 10.0.21... and it has worked.

sol10u7 [~]# route add default 10.0.21
add net default: gateway 10.0.21
sol10u7 [~]# netstat -rn

Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
default 10.0.0.21 UG 1 0
10.0.0.0 10.0.2.16 U 1 0 e1000g0
224.0.0.0 10.0.2.16 U 1 0 e1000g0
127.0.0.1 127.0.0.1 UH 1 54 lo0


I've also tried to use 10.1 and 10.1.1, and another success has been met.

sol10u7 [/etc/krb5]# route add default 10.1
add net default: gateway 10.1
sol10u7 [/etc/krb5]# netstat -rn

Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
default 10.0.2.2 UG 1 0
default 10.0.0.1 UG 1 0
10.0.0.0 10.0.2.16 U 1 24 e1000g0
224.0.0.0 10.0.2.16 U 1 0 e1000g0
127.0.0.1 127.0.0.1 UH 1 58 lo0
sol10u7 [/etc/krb5]# route add default 10.1.1
add net default: gateway 10.1.1
sol10u7 [/etc/krb5]# netstat -rn

Routing Table: IPv4
Destination Gateway Flags Ref Use Interface
-------------------- -------------------- ----- ----- ---------- ---------
default 10.0.2.2 UG 1 0
default 10.0.0.1 UG 1 0
default 10.1.0.1 UG 1 0
10.0.0.0 10.0.2.16 U 1 24 e1000g0
224.0.0.0 10.0.2.16 U 1 0 e1000g0
127.0.0.1 127.0.0.1 UH 1 58 lo0

The deletion works in the same manner.

sol10u7 [~]# route delete default 10.0.21
delete net default: gateway 10.0.21

Nice. Is it a bug, or is it a feature ;-) ?
I know there's a similar IPv6 behaviour - reducing the number of Zeros in address notation, but I didn't know it's also related to IPv4.

20090529

Create base64 jpegPhoto attribute

So, you wanna a picture in your user profile within the Directory Server entry?
Go for it.

Generate base-64 encoded value of the selected picture:

book [/tmp]# ldif -b jpegPhoto < marcin.jpg
jpegPhoto:: /9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAMgAA/+4ADkFkb2JlA
GTAAAAAAf/bAIQACAYGBgYGCAYGCAwIBwgMDgoICAoOEA0NDg0NEBEMDg0NDgwRDxITFBMSDxgYG
hoYGCMiIiIjJycnJycnJycnJwEJCAgJCgkLCQkLDgsNCw4RDg4ODhETDQ0ODQ0TGBEPDw8PERgWF
xQUFBcWGhoYGBoaISEgISEnJycnJycnJycn/8AAEQgBVAH0AwEiAAIRAQMRAf/EAK4AAAIDAQEBA
QAAAAAAAAAAAAMEAAIFAQYHCAEAAwEBAQAAAAAAAAAAAAAAAAECAwQFEAACAQMCBAMFBQUFBwMDB
[...]


Enter output to the ldif file or create the modification by hand:

book [~]$ ldapmodify -D cn=dirmgr
Enter bind password:
dn: uid=marcin,ou=People,o=wisnios.com,o=isp
changetype: modify
add: jpegPhoto
jpegPhoto:: /9j/4AAQSkZJRgABAgAAZABkAAD/7AARRHVja3kAAQAEAAAAMgAA/+4ADkFkb2JlA
GTAAAAAAf/bAIQACAYGBgYGCAYGCAwIBwgMDgoICAoOEA0NDg0NEBEMDg0NDgwRDxITFBMSDxgYG
hoYGCMiIiIjJycnJycnJycnJwEJCAgJCgkLCQkLDgsNCw4RDg4ODhETDQ0ODQ0TGBEPDw8PERgWF
xQUFBcWGhoYGBoaISEgISEnJycnJycnJycn/8AAEQgBVAH0AwEiAAIRAQMRAf/EAK4AAAIDAQEBA
QAAAAAAAAAAAAMEAAIFAQYHCAEAAwEBAQAAAAAAAAAAAAAAAAECAwQFEAACAQMCBAMFBQUFBwMDB
[...]
^D

20090514

zpool replace problem

Scenario.
Disk from pool1 failed. It was replaced with a new one, with appropriate cfgadm -c unconfigure and cfgadm -c configure commands. Process of resilvering has been completed, but there's still the unavailable device within a pool status. Any try to take a device offline causes the scrub process to start again.

thumper [~]# zpool status -v pool1
pool: pool1
state: DEGRADED
status: One or more devices could not be opened. Sufficient replicas exist for
the pool to continue functioning in a degraded state.
action: Attach the missing device and online it using 'zpool online'.
see: http://www.sun.com/msg/ZFS-8000-D3
scrub: resilver in progress, 2,32% done, 28h7m to go
config:

NAME STATE READ WRITE CKSUM
pool1 DEGRADED 0 0 0
raidz2 DEGRADED 0 0 0
c7t0d0p0 ONLINE 0 0 0
c6t0d0p0 ONLINE 0 0 0
c7t4d0p0 ONLINE 0 0 0
c6t4d0p0 ONLINE 0 0 0
c1t0d0p0 ONLINE 0 0 0
c0t0d0p0 ONLINE 0 0 0
c1t4d0p0 ONLINE 0 0 0
c0t4d0p0 ONLINE 0 0 0
c5t1d0p0 ONLINE 0 0 0
c4t1d0p0 ONLINE 0 0 0
c5t5d0p0 ONLINE 0 0 0
c4t5d0p0 ONLINE 0 0 0
c7t1d0p0 ONLINE 0 0 0
c6t1d0p0 ONLINE 0 0 0
spare DEGRADED 0 0 0
c7t5d0p0 UNAVAIL 0 0 0 cannot open
c1t3d0p0 ONLINE 0 0 0
c6t5d0p0 ONLINE 0 0 0
c1t1d0p0 ONLINE 0 0 0
c0t1d0p0 ONLINE 0 0 0
c1t5d0p0 ONLINE 0 0 0
c0t5d0p0 ONLINE 0 0 0
spares
c1t3d0p0 INUSE currently in use
c0t3d0p0 AVAIL

errors: No known data errors


Solution
Do exactly the same what has been done with a disk from the system point of view.
Replace it in place.

thumper [~]# zpool replace pool1 c7t5d0p0 c7t5d0p0


After resilver period of time:

thumper [~]# zpool status -v pool1
pool: pool1
state: ONLINE
scrub: resilver completed with 0 errors on Thu May 14 17:08:20 2009
config:

NAME STATE READ WRITE CKSUM
pool1 ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c7t0d0p0 ONLINE 0 0 0
c6t0d0p0 ONLINE 0 0 0
c7t4d0p0 ONLINE 0 0 0
c6t4d0p0 ONLINE 0 0 0
c1t0d0p0 ONLINE 0 0 0
c0t0d0p0 ONLINE 0 0 0
c1t4d0p0 ONLINE 0 0 0
c0t4d0p0 ONLINE 0 0 0
c5t1d0p0 ONLINE 0 0 0
c4t1d0p0 ONLINE 0 0 0
c5t5d0p0 ONLINE 0 0 0
c4t5d0p0 ONLINE 0 0 0
c7t1d0p0 ONLINE 0 0 0
c6t1d0p0 ONLINE 0 0 0
c7t5d0p0 ONLINE 0 0 0
c6t5d0p0 ONLINE 0 0 0
c1t1d0p0 ONLINE 0 0 0
c0t1d0p0 ONLINE 0 0 0
c1t5d0p0 ONLINE 0 0 0
c0t5d0p0 ONLINE 0 0 0
spares
c1t3d0p0 AVAIL
c0t3d0p0 AVAIL

errors: No known data errors

Voila!

20090507

Solaris IPSec with preshared keys

To implement transport mode ipsec configuration we have to complete the following steps:

  • assign ipsec policy (ipsecinit.conf)
  • configure Internet Key Exchange (IKE) daemon (in.iked) (config)
  • setup preshared key secret (ike.preshared)

The configuration should be made on both of the communicating nodes.
In below example i've used nodes named sol10u6 (10.0.2.15) and sol10u7 (10.0.2.16).
To generate the password ascii representation i've used a piece of code:


/*
IKE password converter
- Marcin Wisnios [marcin AT wisnios.com]
*/
#include <stdio.h>
#include <string.h>

#define PSIZE 512

int main(int argc, char* argv[])
{
int n;
size_t size;
char pass[PSIZE];

if (argc == 1)
{
fprintf(stderr, "Usage: %s password\n", argv[0]);
}
else
{
size=strlen(argv[1]);
if (size >= PSIZE)
{
fprintf(stderr, "Warning: possibility of buffer overflow on oversized password (>=%d). Exiting.\n", PSIZE);
return -1;
}

strncpy(pass, argv[1], size);

for(n=0; n<size; n++)
{
printf("%d", pass[n]);
}

printf("\n");
}

return 0;
}


I've used sample secret: example-preshared-key-secret-do-not-use-it.

sol10u6 [~]$ ./ikepconv example-preshared-key-secret-do-not-use-it
10112097109112108101451121141011151049711410110045107101121451151019911410111645100111451101111164511711510145105116

Let's start with ipsec policy settings.
Setup is as follows:

sol10u6 [~]# cat /etc/inet/ipsecinit.conf
[...]
{ laddr 10.0.2.15 raddr 10.0.2.16 } ipsec { encr_algs aes encr_auth_algs md5 sa shared }

sol10u7 [~]# cat /etc/inet/ipsecinit.conf
[...]
{ laddr 10.0.2.16 raddr 10.0.2.15 } ipsec { encr_algs aes encr_auth_algs md5 sa shared }

encr_algs aes - Use AES as Encapsulating Security Payload (ESP, IP Protocol: 50) encryption algorithm
encr_auth_algs md5 - Use MD5 as ESP authentication algorithm
sa shared - shared Security Association (SA) means the communication between the two nodes in one direction uses the same channel;
in opposite to "unique" which uses separate SA for each pair of source and destination ports

Apply IPSec policy to the system:

sol10u6 [~]# ipsecconf -a /etc/inet/ipsecinit.conf
sol10u6 [~]# ipsecconf
#INDEX 2
{ laddr 10.0.2.15 raddr 10.0.2.16 } ipsec { encr_algs aes encr_auth_algs md5 sa shared }

From now the communication is broken; IPSec has been instructed to encrypt and authenticate the traffic.
We need to provide the valid SA related to security policy.
To automate the process of key management I've used IKE daemon (in.iked).
It should be configured similary to below output:

sol10u6 [~]# cat /etc/inet/ike/config
[...]
### BEGINNING OF FILE

{
label "preshared"
local_id_type ip
local_addr 10.0.2.15
remote_addr 10.0.2.16

p1_xform
{ auth_method preshared oakley_group 5 encr_alg aes auth_alg md5 }
}

sol10u7 [~]# cat /etc/inet/ike/config
[...]
### BEGINNING OF FILE

{
label "preshared"
local_id_type ip
local_addr 10.0.2.16
remote_addr 10.0.2.15

p1_xform
{ auth_method preshared oakley_group 5 encr_alg aes auth_alg md5 }
}

oakley_group - The Oakley Diffie-Hellman group used for IKE SA key derivation. Acceptable values are currently 1 (768-bit), 2 (1024-bit), or 5 (1536-bit).

The last step is to configure the preshared secrets file.

sol10u6 [~]# cat /etc/inet/secret/ike.preshared
[...]
{
localidtype IP
localid 10.0.2.15
remoteidtype IP
remoteid 10.0.2.16
key 10112097109112108101451121141011151049711410110045107101121451151019911410111645100111451101111164511711510145105116
}

sol10u7 [~]# cat /etc/inet/secret/ike.preshared
[...]
{
localidtype IP
localid 10.0.2.16
remoteidtype IP
remoteid 10.0.2.15
key 10112097109112108101451121141011151049711410110045107101121451151019911410111645100111451101111164511711510145105116
}

key - continuous ASCII characters decimal representation of the passphrase "example-preshared-key-secret-do-not-use-it" (converted with ikepconv from the beginning of my post).
That's it: e(101)x(120)a(97)m(109)p(112)l(108)e(101)-(45)...

Start IKE daemon:

sol10u6 [~]# /usr/lib/inet/in.iked -d [debug mode]
sol10u6 [~]# /usr/lib/inet/in.iked

It's time for a simple test.

sol10u7 [~]# ping 10.0.2.15
10.0.2.15 is alive

sol10u6 [~]# snoop -c 1 src host 10.0.2.16
Using device /dev/e1000g0 (promiscuous mode)
sol10u7 -> sol10u6 ESP SPI=0xc072b272 Replay=6
1 packets captured


sol10u7 [~]# ping 10.0.2.15
10.0.2.15 is alive

sol10u6 [~]# snoop -c 1 -v src host 10.0.2.16
Using device /dev/e1000g0 (promiscuous mode)
ETHER: ----- Ether Header -----
ETHER:
ETHER: Packet 1 arrived at 17:46:25.51503
ETHER: Packet size = 150 bytes
ETHER: Destination = 8:0:27:ba:19:34, PCS Computer Systems GmbH
ETHER: Source = 8:0:27:30:9c:39, PCS Computer Systems GmbH
ETHER: Ethertype = 0800 (IP)
ETHER:
IP: ----- IP Header -----
IP:
IP: Version = 4
IP: Header length = 20 bytes
IP: Type of service = 0x00
IP: xxx. .... = 0 (precedence)
IP: ...0 .... = normal delay
IP: .... 0... = normal throughput
IP: .... .0.. = normal reliability
IP: .... ..0. = not ECN capable transport
IP: .... ...0 = no ECN congestion experienced
IP: Total length = 136 bytes
IP: Identification = 9596
IP: Flags = 0x0
IP: .0.. .... = may fragment
IP: ..0. .... = last fragment
IP: Fragment offset = 0 bytes
IP: Time to live = 255 seconds/hops
IP: Protocol = 50 (ESP)
IP: Header checksum = 7da9
IP: Source address = 10.0.2.16, sol10u7
IP: Destination address = 10.0.2.15, sol10u6
IP: No options
IP:
ESP: ----- Encapsulating Security Payload -----
ESP:
ESP: SPI = 0xc072b272
ESP: Replay = 10
ESP: ....ENCRYPTED DATA....

1 packets captured

INFO:
The IPSec definitions has been added to SMF in Solaris 10 update 7:
svc:/network/ipsec/manual-key:default
svc:/network/ipsec/ike:default
svc:/network/ipsec/ipsecalgs:default
svc:/network/ipsec/policy:default

Remember to enable ipsec/ike daemon service.

Enjoy.

20090225

port to pid / pid to port mapper

Finally, I've developed the mutual pid/port mapper.

Here it is. Feel free to use.

#!/bin/bash
#
# The mutual pid/port mapper
# - Marcin Wisnios [marcin AT wisnios.com]
#
case "$1" in
'')
echo "Usage: $0 PORT | -p PID | -n NAME | -a (ALL)"
exit 1
;;
'-p')
# pid search
P_ID=$2
;;
'-n')
# name search
P_ID=`pgrep $2`
;;
'-a')
# list all
P_ID=`ps -e -o pid,comm | sed '/PID/d; /ps/d; /sed/d; /bash/d; s/\([0-9]\{1,5\}\).*/\1/p;d'`
;;
*)
# port search
P_ID=`ps -e -o pid,comm | sed '/PID/d; /ps/d; /sed/d; /bash/d; s/\([0-9]\{1,5\}\).*/\1/p;d'`
PORT=$1
;;
esac

for P in $P_ID; do

N=`ps -o comm -p $P | sed '/COMMAND/d'`
N=`basename $N`

pfiles $P | sed 's/.*sockname:\ AF_INET[6]\{0,1\} \(.*\)\ \ port:\ \(.*\)/\1:\2/p;d' | while read L; do
if [ -z $PORT ]; then
echo "$L $P/$N"
else
echo "$L $P/$N" | sed "/:$PORT/p;d"
fi
done

done


Examples of use:

book [/home/wisnios]# ./ppm.sh 22
:::22 6963/sshd
::ffff:208.64.63.178:22 13491/sshd
::ffff:208.64.63.178:22 13507/sshd

book [/home/wisnios]# ./ppm.sh -n ns-slapd
:::389 7128/ns-slapd
:::636 7128/ns-slapd

book [/home/wisnios]# ./ppm.sh -p 20623
0.0.0.0:80 20623/cherokee-worker

20081127

Core system support - the journey begins

OK, I've installed 'core system support' software group. Solaris has started.
What now? There's for ex. no ssh service, no bash or zsh shell.
So I've decided to mount dvd image and install additional packages.

But... which dsk device is the right one?
No mounted device, no vfstab entry, no cdrom life marks on planet Solaris.

I've found it finally with:

for d in `ls /dev/dsk/*s2`; do prtvtoc $d; sleep 2; done

The right one is that with only one slice, with sector count of 4509440 (dvd image size):

* First Sector Last
* Partition Tag Flags Sector Count Sector Mount Directory
0 5 01 0 4509440 4509439
2 5 01 0 4509440 4509439

Mount:

mount -F hsfs -o ro /dev/dsk/c1t0d0s2 /cdrom

and install desired software:

pkgadd -R /a -d /cdrom/Solaris_10/Product
[...]

Select package(s) you wish to process (or 'all' to process all packages).
(default: all) [?,??,q]: SUNWsshcu,SUNWsshr,SUNWsshu,SUNWsshdr,SUNWsshdu,SUNWbash,SUNWzsh

I've used the -R switch because I've done the work inside the shell session invoked from the Installation menu.
The root system has been mounted at /a directory.

20081117

libcrypt_d.so.1: open failed: No such file or directory

During the rubygems installation process I've encountered the following problem:

thumper [/export/install/sources/rubygems-1.3.1]# ruby setup.rb
ld.so.1: ruby: fatal: libcrypt_d.so.1: open failed: No such file or directory
Killed


Because of a cryptographic technology export restrictions there's no build-in support for some of the algorithms.

To accomplish the task you should download and install 'Solaris 10 Encryption Kit for Sparc / x86'.
Enjoy the screener:


thumper [/export/install]# unzip sol-10-encrypt-GA-iso.zip
thumper [/export/install]# lofiadm -a /export/install/sol-10-encrypt-GA.iso
/dev/lofi/1
thumper [/export/install]# mount -F hsfs -o ro /dev/lofi/1 /mnt
thumper [/export/install]# mount -p | tail -1
/dev/lofi/1 - /mnt hsfs - no ro,noglobal,maplcase,rr,traildot
thumper [/export/install]# pkgadd -d /mnt/Encryption_10/i386/Packages

The following packages are available:
1 SUNWcrman Encryption Kit On-Line Manual Pages
(i386) 10.0,REV=52.0
2 SUNWcry Crypt Utilities
(i386) 11.10.0,REV=2005.01.21.16.34
3 SUNWcryr Solaris Root Crypto
(i386) 11.10.0,REV=2005.01.21.16.34

Select package(s) you wish to process (or 'all' to process
all packages). (default: all) [?,??,q]: all
[...]

thumper [/export/install]# umount /mnt
thumper [/export/install]# lofiadm -d /dev/lofi/1
thumper [/export/install]# find /usr/lib -name libcrypt_d*
/usr/lib/amd64/libcrypt_d.so
/usr/lib/amd64/libcrypt_d.so.1
/usr/lib/libcrypt_d.so
/usr/lib/libcrypt_d.so.1


To make your ruby/gem environment fully usable don't forget to install coreutils and make sunfreeware packages.
I've also linked the ginstall to install command:

thumper [/usr/local/bin]# ln -s install ginstall

It's a solution for the gem package builder error messages.

20080915

primary label corrupt

During the ZFS testing process I've used dd command to erase the disk fragment with /dev/urandom (testing the ability of raidz crash handling). After resilvering there was still information on my console and dmesg log:

Sep 15 07:42:00 thumper scsi: [ID 107833 kern.warning] WARNING: /pci@0,0/pci1022,7458@1/pci11ab,11ab@1/disk@0,0 (sd2):
Sep 15 07:42:00 thumper primary label corrupt; using backup


I've resolved this with a format command.
It could be done in the following way.

thumper [/]# format
Searching for disks...done


AVAILABLE DISK SELECTIONS:
0. c0t0d0
/pci@0,0/pci1022,7458@1/pci11ab,11ab@1/disk@0,0
[...]

Specify disk (enter its number): 0
selecting c0t0d0
[disk formatted]
Reading the primary EFI GPT label failed. Using backup label.
Use the 'backup' command to restore the primary label.

format> backup
Restoring primary label.
format> label
Ready to label disk, continue? y

format> quit

20080420

Disable Solaris GUI

It could be done by using one of the two methods.
The first one does its job immediate.

# svcadm disable cde-login

(You could also use the full FMRI, svc:/application/graphical-login/cde-login:default)

The second one takes place during the next startup.

# /usr/dt/bin/dtconfig -d

(Use -e switch to enable auto-start)

20070813

Solaris hostname

Setting the hostname - one of the most basic installation procedure. Unlike other operating systems a few places to visit. A few places to forget at least one of them.

Ex. setting the sunshine hostname on the machine with a bge0 interface and IP address 10.0.0.1

# echo -e "10.0.0.1\tsunshine" >> /etc/inet/hosts
# echo "sunshine" > /etc/hostname.bge0
# echo "sunshine" > /etc/nodename


There's one more file with the hostname and IP address information - /etc/inet/ipnodes. In Solaris 11 it's a symlink to /etc/inet/hosts file. So, you could leave it alone.

The change of the running system's name is as simple as:

# hostname sunshine

20070803

turn into mirror

In below example i've used single volume zfs pool to convert it into mirror zpool.
To show the possibilities of zfs in more extensive way it's based on file volumes instead of drives.

# mkfile -nv 1g mud
mud 1073741824 bytes
# zpool create pond /export/mud

I've created the pond pool based on a mud file.
Now, using the attach method of zpool command, I'll turn it into mirror with water (second file volume).

# mkfile -nv 1g water
water 1073741824 bytes
# zpool attach pond /export/mud /export/water
# zpool status pond
pool: pond
state: ONLINE
scrub: resilver completed with 0 errors on Wed Apr 11 09:43:22 2007
config:

NAME STATE READ WRITE CKSUM
pond ONLINE 0 0 0
mirror ONLINE 0 0 0
/export/mud ONLINE 0 0 0
/export/water ONLINE 0 0 0

errors: No known data errors