Tuesday, August 7, 2012

How to trace what’s being done by user’s on your Linux Server


Intrusions can take place from both authorized (insiders) and unauthorized (outsiders) users. My personal experience shows that unhappy user can damage the system, especially when they have a shell access. Some users are little smart and removes history file (such as ~/.bash_history) but you can monitor all user executed commands.

It is recommended that you log user activity using process accounting. Process accounting allows you to view every command executed by a user including CPU and memory time. With process accounting sys admin always find out which command executed at what time :)

The psacct package contains several utilities for monitoring process activities, including ac, lastcomm, accton and sa.
  • The ac command displays statistics about how long users have been logged on.
  • The lastcomm command displays information about previous executed commands.
  • The accton command turns process accounting on or off.
  • The sa command summarizes information about previously executed commmands.

Install psacct or acct package

Use up2date command if you are using RHEL ver 4.0 or less

# up2date psacct

Use yum command if you are using CentOS/Fedora Linux / RHEL 5:

# yum install psacct

Use apt-get command if you are using Ubuntu / Debian Linux:

$ sudo apt-get install acct OR # apt-get install acct

Start psacct/acct service

By default service is started on Ubuntu / Debian Linux by creating /var/account/pacct file. But under Red Hat /Fedora Core/Cent OS you need to start psacct service manually. Type the following two commands to create /var/account/pacct file and start services:

# chkconfig psacct on

# /etc/init.d/psacct start

Now let us see how to utilize these utilities to monitor user commands and time.

Display statistics about users' connect time

ac command prints out a report of connect time in hours based on the logins/logouts. A total is also printed out. If you type ac without any argument it will display total connect time:

[pankaj@Redhat ~]$ ac
        total       53.17

Display totals for each day rather than just one big total at the end:

[pankaj@Redhat ~]$ ac -d
Jul 23  total       19.65
Aug  5  total       12.10
Aug  6  total       20.67
Today   total        0.80

Display time totals for each user in addition to the usual everything-lumped-into-one value:

[pankaj@Redhat ~]$ ac -p
        pankaj                               0.09
        raj                                  0.97
        root                                47.07
        sohan                                5.11
        total       53.24

find out information about previously executed user commands

Use lastcomm command which print out information about previously executed commands. You can search command using usernames, tty names, or by command names itself.

Display command executed by pankaj user:

[pankaj@Redhat ~]$ lastcomm pankaj
lastcomm                pankaj   pts/2      0.00 secs Tue Aug  7 07:18
ac                      pankaj   pts/2      0.00 secs Tue Aug  7 07:17
ac                      pankaj   pts/2      0.00 secs Tue Aug  7 07:16
ac                      pankaj   pts/2      0.00 secs Tue Aug  7 07:16
grep                    pankaj   pts/2      0.00 secs Tue Aug  7 07:11
dircolors               pankaj   pts/2      0.00 secs Tue Aug  7 07:11
bash               F    pankaj   pts/2      0.00 secs Tue Aug  7 07:11
tput                    pankaj   pts/2      0.00 secs Tue Aug  7 07:11
tty                     pankaj   pts/2      0.00 secs Tue Aug  7 07:11

Search the accounting logs by command name:

[pankaj@Redhat ~]$ lastcomm useradd
useradd            S    root     pts/1      0.10 secs Tue Aug  7 07:11
useradd            F    root     pts/1      0.00 secs Tue Aug  7 07:11
useradd            F    root     pts/1      0.00 secs Tue Aug  7 07:11
useradd            F    root     pts/1      0.00 secs Tue Aug  7 07:11
useradd            F    root     pts/1      0.00 secs Tue Aug  7 07:11

Search the accounting logs by terminal name pts/1

[pankaj@Redhat ~]$ lastcomm pts/1

passwd             S    root     pts/1      0.04 secs Tue Aug  7 07:11
useradd            F    root     pts/1      0.00 secs Tue Aug  7 07:11
psacct                  root     pts/1      0.01 secs Tue Aug  7 07:11
touch                   root     pts/1      0.00 secs Tue Aug  7 07:11
accton             S    root     pts/1      0.00 secs Tue Aug  7 07:11

Summarizes accounting information

Use sa command to print summarizes information about previously executed commands. In addition, it condenses this data into a summary file named savacct which contains the number of times the command was called and the system resources used. The information can also be summarized on a per-user basis; sa will save this iinformation into a file named usracct.

[pankaj@Redhat ~]$ sa
      4         0.36re      0.12cp     31156k  up2date
     216       1.82re       0.02cp      647k
      19       1.64re       0.02cp      642k   ***other*
       2       0.01re       0.00cp     4123k   rhsmd
      48       0.01re       0.00cp      649k   find
      33       0.01re       0.00cp      504k   tmpwatch
       2       0.01re       0.00cp      662k   logrotate
      54       0.14re       0.00cp      687k   awk
      14       0.00re       0.00cp      498k   basename
      13       0.00re       0.00cp      495k   logger
      12       0.00re       0.00cp      743k   makewhatis*
       3       0.00re       0.00cp      505k   tr
       3       0.00re       0.00cp      496k   renice
       3       0.00re       0.00cp      507k   rm
       2       0.00re       0.00cp      596k   grep
       2       0.00re       0.00cp      742k   prelink*
       2       0.00re       0.00cp      742k   makewhatis.cron*
       2       0.00re       0.00cp      551k   sed
       2       0.00re       0.00cp      495k   ionice

Take example of first line:

4 0.36re 0.12cp 31156k up2date

Where,
  • 0.36re "real time" in wall clock minutes
  • 0.12cp sum of system and user time in cpu minutes
  • 31156k cpu-time averaged core usage, in 1k units
  • up2date command name
Display output per-user:

# sa -u
root       0.00 cpu      595k mem accton
root       0.00 cpu    12488k mem initlog
root       0.00 cpu    12488k mem initlog
root       0.00 cpu    12482k mem touch
root       0.00 cpu    13226k mem psacct
root       0.00 cpu      595k mem consoletype
root       0.00 cpu    13192k mem psacct           *
root       0.00 cpu    13226k mem psacct
root       0.00 cpu    12492k mem chkconfig
postfix    0.02 cpu    10696k mem smtpd
pankaj     0.00 cpu    19328k mem userhelper
pankaj     0.00 cpu    13018k mem id
pankaj     0.00 cpu    13460k mem bash             *
lighttpd   0.00 cpu    48240k mem php   
           *
Display the number of processes and number of CPU minutes on a per-user basis

[root@Redhat ~]# sa –m
root                                  566       3.44re       0.22cp      991k
postfix                                 2       0.21re       0.00cp     3138k
pankaj                                 

No comments:

Post a Comment