Keep walking

Whatever you need to grow, just do it faster…

Archive for the ‘UNIX-LINUX’ Category

UNIX: vi Editor

Posted by ZyK on 03/22/2012

General Introduction

The vi editor (short for visual editor) is a screen editor which is available on almost all Unix systems. Once you have learned vi, you will find that it is a fast and powerful editor. vi has no menus but instead uses combinations of keystrokes in order to accomplish commands. If you are just beginning to learn Unix, you might find the Pico editor easier to use (most command options are displayed at the bottom of the screen). If you use the Pine email application and have composed or replied to a message you have probably already used Pico as it is used for text entry. For more information please refer to the Pine/Pico page.

Starting vi

To start using vi, at the Unix prompt type vifollowed by a file name. If you wish to edit an existing file, type in its name; if you are creating a new file, type in the name you wish to give to the new file.

%vi filename

Then hit Return. You will see a screen similar to the one below which shows blank lines with tildes and the name and status of the file.~

~

“myfile” [New file]

vi’s Modes and Moods

vi has two modes: the command mode and the insert mode. It is essential that you know which mode you are in at any given point in time. When you are in command mode, letters of the keyboard will be interpreted as commands. When you are in insert mode the same letters of the keyboard will type or edit text. vi always starts out in command mode. When you wish to move between the two modes, keep these things in mind. You can type i to enter the insert mode. If you wish to leave insert mode and return to the command mode, hit the ESC key. If you’re not sure where you are, hit ESC a couple of times and that should put you back in command mode.

General Command Information

As mentioned previously, vi uses letters as commands. It is important to note that in general vi commands:

  • are case sensitive – lowercase and uppercase command letters do different things
  • are not displayed on the screen when you type them
  • generally do not require a Return after you type the command.

You will see some commands which start with a colon (:). These commands are ex commands which are used by the ex editor. ex is the true editor which lies underneath vi — in other words, vi is the interface for the ex editor.

Entering Text

To begin entering text in an empty file, you must first change from the command mode to the insert mode. To do this, type the letter i. When you start typing, anything you type will be entered into the file. Type a few short lines and hit Return at the end of each of line. Unlike word processors, vi does not use word wrap. It will break a line at the edge of the screen. If you make a mistake, you can use the Backspace key to remove your errors. If the Backspace key doesn’t work properly on your system, try using the Ctrl h key combination.

Cursor Movement

You must be in command mode if you wish to move the cursor to another position in your file. If you’ve just finished typing text, you’re still in insert mode and will need to press ESC to return to the command mode.

Moving One Character at a Time

Try using your direction keys to move up, down, left and right in your file. Sometimes, you may find that the direction keys don’t work. If that is the case, to move the cursor one character at the time, you may use the h, j, k, and lkeys. These keys move you in the following directions:

h	left one space		l	right one space j	down one space		k	up one space

If you move the cursor as far as you can in any direction, you may see a screen flash or hear a beep.

Moving among Words and Lines

While these four keys (or your direction keys) can move you just about anywhere you want to go in your file, there are some shortcut keys that you can use to move a little more quickly through a document. To move more quickly among words, you might use the following:

w	moves the cursor forward one word  b	moves the cursor backward one word (if in the middle of a 
 	word, b will move you to the beginning of the current word). e	moves to the end of a word.

To build on this further, you can precede these commands with a number for greater movement. For example, 5w would move you forward five words; 12b would move you backwards twelve words. [You can also use numbers with the commands mentioned earlier. For example, 5j would move you down 5 characters.]

Command Keys and Case

You will find when using vi that lower case and upper case command keys are interpreted differently. For example, when using the lower case w, b, and e commands, words will be defined by a space or a punctuation mark. On the other hand, W, B, and E commands may be used to move between words also, but these commands ignore punctuation.

Shortcuts

Two short cuts for moving quickly on a line include the $ and the 0 (zero) keys. The $ key will move you to the end of a line, while the 0 will move you quickly to the beginning of a line.

Screen Movement

To move the cursor to a line within your current screen use the following keys:

H	moves the cursor to the top line of the screen. M	moves the cursor to the middle line of the screen. L	moves the cursor to the last line of the screen.

To scroll through the file and see other screens use:

 ctrl-f	scrolls down one screen ctrl-b	scrolls up one screen ctrl-u	scrolls up a half a screen ctrl-d	scrolls down a half a screen

Two other useful commands for moving quickly from one end to the other of a document are G to move to the end of the file and 1G to move to the beginning of the file. If you precede G with a number, you can move to a specific line in the document (e.g. 15G would move you to line 15).

Moving by Searching

One method for moving quickly to a particular spot in your file is to search for specific text. When you are in command mode, type a / followed the text you wish to search for. When you press Return, the cursor will move to the first incidence of that string of text. You can repeat the search by typing n or search in a backwards direction by using N.

Basic Editing

To issue editing commands, you must be in command mode. As mentioned before, commands will be interpreted differently depending upon whether they are issued in lower or upper case. Also, many of the editing commands can be preceded by a number to indicate a repetition of the command.

Deleting (or Cutting) Characters, Words, and Lines

To delete a character, first place your cursor on that character. Then, you may use any of the following commands:

 x	deletes the character under the cursor. X	deletes the character to the left of your cursor. dw	deletes from the character selected to the end of the word. dd	deletes all the current line. D deletes from the current character to the end of the line.

Preceding the command with a number will delete multiple characters. For example, 10x will delete the character selected and the next 9 characters; 10X will delete the 10 characters to the left of the currently selected character. The command 5dw will delete 5 words, while 4dd deletes four lines.

Pasting Text using Put

Often, when you delete or cut text, you may wish to reinsert it in another location of the document. The Put command will paste in the last portion of text that was deleted since deleted text is stored in a buffer. To use this command, place the cursor where you wish the deleted text to appear. Then use p to reinsert the text. If you are inserting a line or paragraph use the lower case p to insert on the line below the cursor or upper case P to place in on the line above the cursor.

Copying Text with Yank

If you wish to make a duplicate copy of existing text, you may use the yank and put commands to accomplish this function. Yank copies the selected text into a buffer and holds it until another yank or deletion occurs. Yank is usually used in combination with a word or line object such as the ones shown below:

 yw	copies a word into a buffer (7yw copies 7 words) yy	copies a line into a buffer (3yy will copy 3 lines)

Once the desired text is yanked, place the cursor in the spot in which you wish to insert the text and then use the put command (p for line below or P for line above) to insert the contents of the buffer.

Replacing or Changing Characters, Words, and Lines

When you are using the following commands to replace text, you will be put temporarily into insert mode so that you can change a character, word, line, or paragraph of text.

 r	replaces the current character with the next character you enter/type.
   	Once you enter the character you are returned to command mode. R	puts you in overtype mode until you hit ESC which will then return
   	you to command mode. cw	changes and replaces the current word with text that you type.  A dollar
   	sign marks the end of the text you're changing.  Pressing ESC when you
   	finish will return you to command mode.

Inserting Text

If you wish to insert new text in a line, first position the cursor to the right of where you wish the inserted text to appear. Type i to get into insert mode and then type in the desired text (note that the text is inserted before the cursor). Press ESC to return to command mode.

Inserting a Blank Line

To insert a blank line below the line your cursor is currently located on, use the o key and then hit ESC to return to the command mode . Use O to insert a line above the line the cursor is located on.

Appending Text

You can use the append command to add text at any place in your file. Append (a) works very much like Insert (i) except that it insert text after the cursor rather than before it. Append is probably used most often for adding text to the end of a line. Simply place your cursor where you wish to append text and press a. Once you’ve finished appending, press ESC to go back to command mode.

Joining Lines

Since vi does not use automatic word wrap, it is not unusual in editing lines to end up with lines that are too short and that might be improved if joined together. To do this, place your cursor on the first line to be joined and type J. As with other commands, you can precede J with a number to join multiple lines (4J joins 4 lines).

Undoing

Be sure to remember this command. When you make a mistake you can undo it. DO NOTmove the cursor from the line where you made the change. Then try using one of the following two commands:

 u	undoes the last change you made anywhere in the file.  Using u again 
   	will "undo the undo". U	undoes all recent changes to the current line.  You can not have moved
	from the line to recover the original line.

Closing and Saving Files

When you edit a file in vi, you are actually editing a copy of the file rather than the original. The following sections describe methods you might use when closing a file, quitting vi, or both.

Quitting and Saving a File

The command ZZ(notice that it is in uppercase) will allow you to quit vi and save the edits made to a file. You will then return to a Unix prompt. Note that you can also use the following commands:

 :w	to save your file but not quit vi (this is good to do periodically in
	case of machine crash!). :q	to quit if you haven't made any edits. :wq	to quit and save edits (basically the same as ZZ).

Quitting without Saving Edits

Sometimes, when you create a mess (when you first start using vi this is easy to do!) you may wish to erase all edits made to the file and either start over or quit. To do this, you can choose from the following two commands:

 :e!	reads the original file back in so that you can start over. :q!	wipes out all edits and allows you to exit from vi.

More about Combining Commands, Objects, and Numbers

Now that you’ve learned some basic vi commands you might wish to expand your skills by trying some fancy combination steps. Some commands are generally used in combination with a text object. We’ve already seen some examples of this. For example, when you use the command dw to delete a word, that combines the delete (d) command with the word (w) text object. When you wish to delete multiple words, you might add a number to this combination. If you wished to delete 2 words you might use 2dw or d2w. Either of these combinations would work. So, as you can see, the general format for a command can be

(number) (command) (text object) or (command) (number) (text object)

You might wish to try out some of the following combinations of commands and objects:

Command Text Object
d (delete)	w (word to the left) y (yank/copy)	b (word to the right or backward) c (change)	e (end of word)
		H (top of the screen)
		L (bottom of the screen)
		M (middle of the screen)
		0 (zero - first character on a line)
		$ (end of a line)
		( (previous sentence)
		) (next sentence)
		[ (previous section)
		] (next section)

Repeating a Command

If you are doing repetitive editing, you may wish to use the same command over and over. vi will allow you to use the dot (.) to repeat the last basic command you issued. If for example, you wished to deleted several lines, you could use dd and then . (dot) in quick succession to delete a few lines.

A Quick Word about Customizing Your vi Environment

There are several options that you can set from within vi that can affect how you use vi. For example, one option allows you to set a right margin that will then force vi to automatically wrap your lines as you type. To do this, you would use a variation of the :set command. The :set command can be used to change various options in vi. In the example just described, you could, while still in vi, type :set wrapmargin=10 to specify that you wish to have a right margin of 10. Another useful option is :set number. This command causes vi to display line numbers in the file you are working on.

Other Options

To view a listing of other options, you could type :set all. To view only those options which are currently in effect, you can type set: by itself. Options that you set while in a vi session will apply during that session only. To make permanent changes to your vi environment, you could edit your .exrc file. However, you should not edit this file unless you know what you are doing!

Useful vi Commands

Cut/Paste Commands:

x 		delete one character (destructive backspace)
dw		delete the current word (Note: ndw deletes n numbered words)
dd 		delete the current line (Note: ndd deletes n numbered lines)
D		delete all content to the right of the cursor
d$		same as above
:u		undo last command
p,P		paste line starting one line below/above current cursor location
J		combine the contents of two lines 
"[a-z]nyy   	yank next n lines into named buffer [a-z] 
"[a-z]p/P	place the contents of selected buffer below/above the current line

Extensions to the Above Commands:

:3,18d		delete lines 3 through 18
16,25m30	move lines 16 through 25 to after line 30
23,29co62	copy specified lines and place after line 62

Cursor Relocation commands:

:[n]		goto line [n]
shift g		place cursor on last line of text
h/l/j/k		move cursor left, right, down and up 
^f/^b		move forward, backward in text, one page
^u/^d		move up, down one half page
$		move to end of line
0		move to beginning of line

Extensions to the Above:

b 		move backwards one word (Note: nb moves back n number of words)
e		move to end of current word
(		move to beginning of curent block
)		move to the end of current block

Searching and Substitution commands:

/ [string]	search forward for string
? [string]	search backwards for string
n		repeat last search
N		repeat search in opposite direction
cw		change the contents of the current word, (use ESC to stop
		replacement mode)
c$		Replace all content to the right of cursor (exit replacement
		mode with ESC)
c0		Replace all content to the left of cursor (exit with ESC)
:1,$s/s1/s2/g	(Yow!) global replacement of string1 with string2 
r 		replace current character with next character typed

Entering the Insert Mode:

i		Begin inserting text at current cursor location 
I		Begin inserting text at the beginning of the current line
a		Begin appending text, one character to the right of current
		cursor location
A		Begin appending text at the end of the current line
o/O		Begin entering text one line below\above current line
ESC		Exit insertion mode and return to command mode

Exiting and Entering VI

ZZ		save file and exit VI
:wq		same as above
:e!		return to last saved version of current file
:q		quit without save, (Note :q! is required if changes have been made)
:w		write without exit (:w! to force write)

Fancy Stuff:

:1,10w file   		write lines 1 through 10 to file newfile
:340,$w >> file		write lines 340 through the end of the file and append
			to file newfile
:sh			escape temporarily to a shell
^d			return from shell to VI
:![command]		execute UNIX command without leaving VI
:r![command]		read output of command into VI
:r[filename]		read filename into VI
:$r newfile		read in newfile and attach at the end of current document
:r !sort file		read in contents of file after it has been passed through
			the UNIX sort
:n			open next file (works with wildcard filenames,
			ex: vi file*)
:^g			list current line number
:set number		show line numbers
:set showinsert		show flag ("I") at bottom of screen when in insert mode
:set all		display current values of VI variables
:set ai			set autoindent; after this enter the insert mode and
			tab, from this point on VI will indent each line to
			this location.  Use ESC to stop the indentations.
^T 			set the autoindent tab one tab stop to the right 
^D 			set the autoindent tab one stop to the left
:set tabstop=n		sets default tab space to number n
>>			shift contents of line one tab stop to the right
<<			shift contents of line one tab stop to the left
___________________________________CCSF___________________________________

http://www.ccsf.edu/Pub/Fac/vi.html

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

20 Linux System Monitoring Tools Every SysAdmin Should Know

Posted by ZyK on 01/31/2012

Need to monitor Linux server performance? Try these built-in command and a few add-on tools. Most Linux distributions are equipped with tons of monitoring. These tools provide metrics which can be used to get information about system activities. You can use these tools to find the possible causes of a performance problem. The commands discussed below are some of the most basic commands when it comes to system analysis and debugging server issues such as:

  1. Finding out bottlenecks.
  2. Disk (storage) bottlenecks.
  3. CPU and memory bottlenecks.
  4. Network bottlenecks.

#1: top – Process Activity Command

The top program provides a dynamic real-time view of a running system i.e. actual process activity. By default, it displays the most CPU-intensive tasks running on the server and updates the list every five seconds.

Fig.01: Linux top commandFig.01: Linux top command

Commonly Used Hot Keys

The top command provides several useful hot keys:

Hot Key Usage
t Displays summary information off and on.
m Displays memory information off and on.
A Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system.
f Enters an interactive configuration screen for top. Helpful for setting up top for a specific task.
o Enables you to interactively select the ordering within top.
r Issues renice command.
k Issues kill command.
z Turn on or off color/mono

=> Related: How do I Find Out Linux CPU Utilization?

#2: vmstat – System Activity, Hardware and System Information

The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
# vmstat 3
Sample Outputs:

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 2540988 522188 5130400    0    0     2    32    4    2  4  1 96  0  0
 1  0      0 2540988 522188 5130400    0    0     0   720 1199  665  1  0 99  0  0
 0  0      0 2540956 522188 5130400    0    0     0     0 1151 1569  4  1 95  0  0
 0  0      0 2540956 522188 5130500    0    0     0     6 1117  439  1  0 99  0  0
 0  0      0 2540940 522188 5130512    0    0     0   536 1189  932  1  0 98  0  0
 0  0      0 2538444 522188 5130588    0    0     0     0 1187 1417  4  1 96  0  0
 0  0      0 2490060 522188 5130640    0    0     0    18 1253 1123  5  1 94  0  0

Display Memory Utilization Slabinfo

# vmstat -m

Get Information About Active / Inactive Memory Pages

# vmstat -a
=> Related: How do I find out Linux Resource utilization to detect system bottlenecks?

#3: w – Find Out Who Is Logged on And What They Are Doing

w command displays information about the users currently on the machine, and their processes.
# w username
# w vivek

Sample Outputs:

 17:58:47 up 5 days, 20:28,  2 users,  load average: 0.36, 0.26, 0.24
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    10.1.3.145       14:55    5.00s  0.04s  0.02s vim /etc/resolv.conf
root     pts/1    10.1.3.145       17:43    0.00s  0.03s  0.00s w

#4: uptime – Tell How Long The System Has Been Running

The uptime command can be used to see how long the server has been running. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.
# uptime
Output:

 18:02:41 up 41 days, 23:42,  1 user,  load average: 0.00, 0.00, 0.00

1 can be considered as optimal load value. The load can change from system to system. For a single CPU system 1 – 3 and SMP systems 6-10 load value might be acceptable.

#5: ps – Displays The Processes

ps command will report a snapshot of the current processes. To select all processes use the -A or -e option:
# ps -A
Sample Outputs:

  PID TTY          TIME CMD
    1 ?        00:00:02 init
    2 ?        00:00:02 migration/0
    3 ?        00:00:01 ksoftirqd/0
    4 ?        00:00:00 watchdog/0
    5 ?        00:00:00 migration/1
    6 ?        00:00:15 ksoftirqd/1
....
.....
 4881 ?        00:53:28 java
 4885 tty1     00:00:00 mingetty
 4886 tty2     00:00:00 mingetty
 4887 tty3     00:00:00 mingetty
 4888 tty4     00:00:00 mingetty
 4891 tty5     00:00:00 mingetty
 4892 tty6     00:00:00 mingetty
 4893 ttyS1    00:00:00 agetty
12853 ?        00:00:00 cifsoplockd
12854 ?        00:00:00 cifsdnotifyd
14231 ?        00:10:34 lighttpd
14232 ?        00:00:00 php-cgi
54981 pts/0    00:00:00 vim
55465 ?        00:00:00 php-cgi
55546 ?        00:00:00 bind9-snmp-stat
55704 pts/1    00:00:00 ps

ps is just like top but provides more information.

Show Long Format Output

# ps -Al
To turn on extra full mode (it will show command line arguments passed to process):
# ps -AlF

To See Threads ( LWP and NLWP)

# ps -AlFH

To See Threads After Processes

# ps -AlLm

Print All Process On The Server

# ps ax
# ps axu

Print A Process Tree

# ps -ejH
# ps axjf
# pstree

Print Security Information

# ps -eo euser,ruser,suser,fuser,f,comm,label
# ps axZ
# ps -eM

See Every Process Running As User Vivek

# ps -U vivek -u vivek u

Set Output In a User-Defined Format

# ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
# ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
# ps -eopid,tt,user,fname,tmout,f,wchan

Display Only The Process IDs of Lighttpd

# ps -C lighttpd -o pid=
OR
# pgrep lighttpd
OR
# pgrep -u vivek php-cgi

Display The Name of PID 55977

# ps -p 55977 -o comm=

Find Out The Top 10 Memory Consuming Process

# ps -auxf | sort -nr -k 4 | head -10

Find Out top 10 CPU Consuming Process

# ps -auxf | sort -nr -k 3 | head -10

#6: free – Memory Usage

The command free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
# free
Sample Output:

            total       used       free     shared    buffers     cached
Mem:      12302896    9739664    2563232          0     523124    5154740
-/+ buffers/cache:    4061800    8241096
Swap:      1052248          0    1052248

=> Related: :

  1. Linux Find Out Virtual Memory PAGESIZE
  2. Linux Limit CPU Usage Per Process
  3. How much RAM does my Ubuntu / Fedora Linux desktop PC have?

#7: iostat – Average CPU Load, Disk Activity

The command iostat report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).
# iostat
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 	06/26/2009
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.50    0.09    0.51    0.03    0.00   95.86
Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              22.04        31.88       512.03   16193351  260102868
sda1              0.00         0.00         0.00       2166        180
sda2             22.04        31.87       512.03   16189010  260102688
sda3              0.00         0.00         0.00       1615          0

=> Related: : Linux Track NFS Directory / Disk I/O Stats

#8: sar – Collect and Report System Activity

The sar command is used to collect, report, and save system activity information. To see network counter, enter:
# sar -n DEV | more
To display the network counters from the 24th:
# sar -n DEV -f /var/log/sa/sa24 | more
You can also display real time usage using sar:
# sar 4 5
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 		06/26/2009
06:45:12 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle
06:45:16 PM       all      2.00      0.00      0.22      0.00      0.00     97.78
06:45:20 PM       all      2.07      0.00      0.38      0.03      0.00     97.52
06:45:24 PM       all      0.94      0.00      0.28      0.00      0.00     98.78
06:45:28 PM       all      1.56      0.00      0.22      0.00      0.00     98.22
06:45:32 PM       all      3.53      0.00      0.25      0.03      0.00     96.19
Average:          all      2.02      0.00      0.27      0.01      0.00     97.70

=> Related: : How to collect Linux system utilization data into a file

#9: mpstat – Multiprocessor Usage

The mpstat command displays activities for each available processor, processor 0 being the first one. mpstat -P ALL to display average CPU utilization per processor:
# mpstat -P ALL
Sample Output:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in)	 	06/26/2009
06:48:11 PM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
06:48:11 PM  all    3.50    0.09    0.34    0.03    0.01    0.17    0.00   95.86   1218.04
06:48:11 PM    0    3.44    0.08    0.31    0.02    0.00    0.12    0.00   96.04   1000.31
06:48:11 PM    1    3.10    0.08    0.32    0.09    0.02    0.11    0.00   96.28     34.93
06:48:11 PM    2    4.16    0.11    0.36    0.02    0.00    0.11    0.00   95.25      0.00
06:48:11 PM    3    3.77    0.11    0.38    0.03    0.01    0.24    0.00   95.46     44.80
06:48:11 PM    4    2.96    0.07    0.29    0.04    0.02    0.10    0.00   96.52     25.91
06:48:11 PM    5    3.26    0.08    0.28    0.03    0.01    0.10    0.00   96.23     14.98
06:48:11 PM    6    4.00    0.10    0.34    0.01    0.00    0.13    0.00   95.42      3.75
06:48:11 PM    7    3.30    0.11    0.39    0.03    0.01    0.46    0.00   95.69     76.89

=> Related: : Linux display each multiple SMP CPU processors utilization individually.

#10: pmap – Process Memory Usage

The command pmap report memory map of a process. Use this command to find out causes of memory bottlenecks.
# pmap -d PID
To display process memory information for pid # 47394, enter:
# pmap -d 47394
Sample Outputs:

47394:   /usr/bin/php-cgi
Address           Kbytes Mode  Offset           Device    Mapping
0000000000400000    2584 r-x-- 0000000000000000 008:00002 php-cgi
0000000000886000     140 rw--- 0000000000286000 008:00002 php-cgi
00000000008a9000      52 rw--- 00000000008a9000 000:00000   [ anon ]
0000000000aa8000      76 rw--- 00000000002a8000 008:00002 php-cgi
000000000f678000    1980 rw--- 000000000f678000 000:00000   [ anon ]
000000314a600000     112 r-x-- 0000000000000000 008:00002 ld-2.5.so
000000314a81b000       4 r---- 000000000001b000 008:00002 ld-2.5.so
000000314a81c000       4 rw--- 000000000001c000 008:00002 ld-2.5.so
000000314aa00000    1328 r-x-- 0000000000000000 008:00002 libc-2.5.so
000000314ab4c000    2048 ----- 000000000014c000 008:00002 libc-2.5.so
.....
......
..
00002af8d48fd000       4 rw--- 0000000000006000 008:00002 xsl.so
00002af8d490c000      40 r-x-- 0000000000000000 008:00002 libnss_files-2.5.so
00002af8d4916000    2044 ----- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b15000       4 r---- 0000000000009000 008:00002 libnss_files-2.5.so
00002af8d4b16000       4 rw--- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b17000  768000 rw-s- 0000000000000000 000:00009 zero (deleted)
00007fffc95fe000      84 rw--- 00007ffffffea000 000:00000   [ stack ]
ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
mapped: 933712K    writeable/private: 4304K    shared: 768000K

The last line is very important:

  • mapped: 933712K total amount of memory mapped to files
  • writeable/private: 4304K the amount of private address space
  • shared: 768000K the amount of address space this process is sharing with others

=> Related: : Linux find the memory used by a program / process using pmap command

#11 and #12: netstat and ss – Network Statistics

The command netstat displays network connections, routing tables, interface statistics, masquerade connections, and multicast memberships. ss command is used to dump socket statistics. It allows showing information similar to netstat. See the following resources about ss and netstat commands:

#13: iptraf – Real-time Network Statistics

The iptraf command is interactive colorful IP LAN monitor. It is an ncurses-based IP LAN monitor that generates various network statistics including TCP info, UDP counts, ICMP and OSPF information, Ethernet load info, node stats, IP checksum errors, and others. It can provide the following info in easy to read format:

  • Network traffic statistics by TCP connection
  • IP traffic statistics by network interface
  • Network traffic statistics by protocol
  • Network traffic statistics by TCP/UDP port and by packet size
  • Network traffic statistics by Layer2 address
Fig.02: General interface statistics: IP traffic statistics by network interface Fig.02: General interface statistics: IP traffic statistics by network interface

Fig.03 Network traffic statistics by TCP connectionFig.03 Network traffic statistics by TCP connection

#14: tcpdump – Detailed Network Traffic Analysis

The tcpdump is simple command that dump traffic on a network. However, you need good understanding of TCP/IP protocol to utilize this tool. For.e.g to display traffic info about DNS, enter:
# tcpdump -i eth1 'udp port 53'
To display all IPv4 HTTP packets to and from port 80, i.e. print only packets that contain data, not, for example, SYN and FIN packets and ACK-only packets, enter:
# tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
To display all FTP session to 202.54.1.5, enter:
# tcpdump -i eth1 'dst 202.54.1.5 and (port 21 or 20'
To display all HTTP session to 192.168.1.5:
# tcpdump -ni eth0 'dst 192.168.1.5 and tcp and port http'
Use wireshark to view detailed information about files, enter:
# tcpdump -n -i eth1 -s 0 -w output.txt src or dst port 80

#15: strace – System Calls

Trace system calls and signals. This is useful for debugging webserver and other server problems. See how to use to trace the process and see What it is doing.

#16: /Proc file system – Various Kernel Statistics

/proc file system provides detailed information about various hardware devices and other Linux kernel information. See Linux kernel /proc documentations for further details. Common /proc examples:
# cat /proc/cpuinfo
# cat /proc/meminfo
# cat /proc/zoneinfo
# cat /proc/mounts

17#: Nagios – Server And Network Monitoring

Nagios is a popular open source computer system and network monitoring application software. You can easily monitor all your hosts, network equipment and services. It can send alert when things go wrong and again when they get better. FAN is “Fully Automated Nagios”. FAN goals are to provide a Nagios installation including most tools provided by the Nagios Community. FAN provides a CDRom image in the standard ISO format, making it easy to easilly install a Nagios server. Added to this, a wide bunch of tools are including to the distribution, in order to improve the user experience around Nagios.

18#: Cacti – Web-based Monitoring Tool

Cacti is a complete network graphing solution designed to harness the power of RRDTool’s data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices. It can provide data about network, CPU, memory, logged in users, Apache, DNS servers and much more. See how to install and configure Cacti network graphing tool under CentOS / RHEL.

#19: KDE System Guard – Real-time Systems Reporting and Graphing

KSysguard is a network enabled task and system monitor application for KDE desktop. This tool can be run over ssh session. It provides lots of features such as a client/server architecture that enables monitoring of local and remote hosts. The graphical front end uses so-called sensors to retrieve the information it displays. A sensor can return simple values or more complex information like tables. For each type of information, one or more displays are provided. Displays are organized in worksheets that can be saved and loaded independently from each other. So, KSysguard is not only a simple task manager but also a very powerful tool to control large server farms.

Fig.05 KDE System GuardFig.05 KDE System Guard {Image credit: Wikipedia}

See the KSysguard handbook for detailed usage.

#20: Gnome System Monitor – Real-time Systems Reporting and Graphing

The System Monitor application enables you to display basic system information and monitor system processes, usage of system resources, and file systems. You can also use System Monitor to modify the behavior of your system. Although not as powerful as the KDE System Guard, it provides the basic information which may be useful for new users:

  • Displays various basic information about the computer’s hardware and software.
  • Linux Kernel version
  • GNOME version
  • Hardware
  • Installed memory
  • Processors and speeds
  • System Status
  • Currently available disk space
  • Processes
  • Memory and swap space
  • Network usage
  • File Systems
  • Lists all mounted filesystems along with basic information about each.
Fig.06 The Gnome System Monitor applicationFig.06 The Gnome System Monitor application

Bonus: Additional Tools

A few more tools:

  • nmap – scan your server for open ports.
  • lsof – list open files, network connections and much more.
  • ntop web based tool – ntop is the best tool to see network usage in a way similar to what top command does for processes i.e. it is network traffic monitoring software. You can see network status, protocol wise distribution of traffic for UDP, TCP, DNS, HTTP and other protocols.
  • Conky – Another good monitoring tool for the X Window System. It is highly configurable and is able to monitor many system variables including the status of the CPU, memory, swap space, disk storage, temperatures, processes, network interfaces, battery power, system messages, e-mail inboxes etc.
  • GKrellM – It can be used to monitor the status of CPUs, main memory, hard disks, network interfaces, local and remote mailboxes, and many other things.
  • vnstat – vnStat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interface(s).
  • htop – htop is an enhanced version of top, the interactive process viewer, which can display the list of processes in a tree form.
  • mtr – mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.

Did I miss something? Please add your favorite system motoring tool in the comments.

Featured Articles:

(Src: http://www.cyberciti.biz)

Posted in Linux | Tagged: | 1 Comment »

Get Hardware Information On Linux Using dmidecode

Posted by ZyK on 01/31/2012

dmidecode command reads the system DMI table to display hardware and BIOS information of the server. Apart from getting current configuration of the system, you can also get information about maximum supported configuration of the system using dmidecode. For example, dmidecode gives both the current RAM on the system and the maximum RAM supported by the system.

This article provides an overview of the dmidecode and few practical examples on how to use dmidecode command.

1. Overview of dmidecode

Distributed Management Task Force maintains the DMI specification and SMBIOS specification. The output of the dmidecode contains several records from the DMI (Desktop Management interface) table.

Following is the record format of the dmidecode output of the DMI table.

Record Header: Handle {record id}, DMI type {dmi type id}, {record size} bytes
Record Value: {multi line record value}
  • record id: Unique identifier for every record in the DMI table.
  • dmi type id: Type of the record. i.e BIOS, Memory etc.,
  • record size: Size of the record in the DMI table.
  • multi line record values: Multi line record value for that specific DMI type.

Sample output of dmidecode command:

# dmidecode | head -15 
# dmidecode 2.9
SMBIOS 2.3 present.
56 structures occupying 1977 bytes.
Table at 0x000FB320.

Handle 0xDA00, DMI type 218, 11 bytes
OEM-specific Type
        Header and Data:
                DA 0B 00 DA B0 00 17 03 08 28 00

Handle 0x0000, DMI type 0, 20 bytes
BIOS Information
        Vendor: Dell Computer Corporation
        Version: A07
        Release Date: 01/13/2004

Get the total number of records in the DMI table as shown below:

# dmidecode | grep ^Handle | wc -l
56

(or)

# dmidecode | grep structures
56 structures occupying 1977 bytes.

2. DMI Types

DMI Type id will give information about a particular hardware component of your system. Following command with type id 4 will get the information about CPU of the system.

# dmidecode -t 4 # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0400, DMI type 4, 35 bytes
Processor Information
        Socket Designation: Processor 1
        Type: Central Processor
        Family: Xeon
        Manufacturer: Intel
        ID: 29 0F 00 00 FF FB EB BF
        Signature: Type 0, Family 15, Model 2, Stepping 9
        Flags:
                FPU (Floating-point unit on-chip)
                VME (Virtual mode extension)
                DE (Debugging extension)
                PSE (Page size extension)
                TSC (Time stamp counter)
                MSR (Model specific registers)

Following are the different DMI types available.

       Type   Information
       ----------------------------------------
          0   BIOS
          1   System
          2   Base Board
          3   Chassis
          4   Processor
          5   Memory Controller
          6   Memory Module
          7   Cache
          8   Port Connector
          9   System Slots
         10   On Board Devices
         11   OEM Strings
         12   System Configuration Options
         13   BIOS Language
         14   Group Associations
         15   System Event Log
         16   Physical Memory Array
         17   Memory Device
         18   32-bit Memory Error
         19   Memory Array Mapped Address
         20   Memory Device Mapped Address
         21   Built-in Pointing Device
         22   Portable Battery
         23   System Reset
         24   Hardware Security
         25   System Power Controls
         26   Voltage Probe
         27   Cooling Device
         28   Temperature Probe
         29   Electrical Current Probe
         30   Out-of-band Remote Access
         31   Boot Integrity Services
         32   System Boot
         33   64-bit Memory Error
         34   Management Device
         35   Management Device Component
         36   Management Device Threshold Data
         37   Memory Channel
         38   IPMI Device
         39   Power Supply

Instead of type_id, you can also pass the keyword to the -t option of the dmidecode command. Following are the available keywords.

       Keyword     Types
       ------------------------------
       bios        0, 13
       system      1, 12, 15, 23, 32
       baseboard   2, 10
       chassis     3
       processor   4
       memory      5, 6, 16, 17
       cache       7
       connector   8
       slot        9

For example, to get all the system baseboard related information execute the following command, which will display the type_id 2 and 10

# dmidecode -t baseboard # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0200, DMI type 2, 9 bytes
Base Board Information
        Manufacturer: Dell Computer Corporation
        Product Name: 123456
        Version: A05
        Serial Number: ..CN123456789098.

Handle 0x0A00, DMI type 10, 14 bytes
On Board Device 1 Information
        Type: SCSI Controller
        Status: Enabled
        Description: LSI Logic 53C1030 Ultra 320 SCSI
On Board Device 2 Information
        Type: SCSI Controller
        Status: Enabled
        Description: LSI Logic 53C1030 Ultra 320 SCSI
On Board Device 3 Information
        Type: Video
        Status: Enabled
        Description: ATI Rage XL PCI Video
On Board Device 4 Information
        Type: Ethernet
        Status: Enabled
        Description: Broadcom Gigabit Ethernet 1
On Board Device 5 Information
        Type: Ethernet
        Status: Enabled
        Description: Broadcom Gigabit Ethernet 2

3. Get Physical Memory (RAM) information using dmidecode

What is the maximum RAM supported by the system? In this example, this system can support maximum 8GB of RAM.

# dmidecode -t 16 # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x1000, DMI type 16, 15 bytes
Physical Memory Array
        Location: System Board Or Motherboard
        Use: System Memory
        Error Correction Type: Multi-bit ECC
 Maximum Capacity: 8 GB         Error Information Handle: Not Provided
        Number Of Devices: 4


How much memory can I expand to? From /proc/meminfo you can find out the total current memory of your system as shown below.

# grep MemTotal /proc/meminfo
MemTotal:      1034644 kB


In this example, the system has 1GB of RAM. Is this 1 x 1GB (or) 2 x 512MB (or) 4 x 256MB? This can be figured out by passing the type id 17 to the dmidecode command as shown below. Please note in the example below, if you have to expand upto 8GB of maximum RAM, you need to remove the existing 512MB from slot 1 and 2, and use 2GB RAM on all the 4 memory slots.

# dmidecode -t 17 # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x1100, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: 512 MB [Note: Slot1 has 512 MB RAM]
        Form Factor: DIMM
        Set: 1
        Locator: DIMM_1A
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

Handle 0x1101, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: 512 MB [Note: Slot2 has 512 MB RAM]
        Form Factor: DIMM
        Set: 1
        Locator: DIMM_1B
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

Handle 0x1102, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: No Module Installed [Note: Slot3 is empty]
        Form Factor: DIMM
        Set: 2
        Locator: DIMM_2A
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

Handle 0x1103, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000

        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: No Module Installed [Note: Slot4 is empty]
        Form Factor: DIMM
        Set: 2
        Locator: DIMM_2B
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

4. Get BIOS information using dmidecode

# dmidecode -t bios # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0000, DMI type 0, 20 bytes
BIOS Information
        Vendor: Dell Computer Corporation
        Version: A07
        Release Date: 01/13/2004
        Address: 0xF0000
        Runtime Size: 64 kB
        ROM Size: 4096 kB
        Characteristics:
                ISA is supported
                PCI is supported
                PNP is supported
                BIOS is upgradeable
                BIOS shadowing is allowed
                ESCD support is available
                Boot from CD is supported
                Selectable boot is supported
                EDD is supported
                Japanese floppy for Toshiba 1.2 MB is supported (int 13h)
                5.25"/360 KB floppy services are supported (int 13h)
                5.25"/1.2 MB floppy services are supported (int 13h)
                3.5"/720 KB floppy services are supported (int 13h)
                8042 keyboard services are supported (int 9h)
                Serial services are supported (int 14h)
                CGA/mono video services are supported (int 10h)
                ACPI is supported
                USB legacy is supported
                LS-120 boot is supported
                BIOS boot specification is supported
                Function key-initiated network boot is supported

Handle 0x0D00, DMI type 13, 22 bytes
BIOS Language Information
        Installable Languages: 1
                en|US|iso8859-1
        Currently Installed Language: en|US|iso8859-1

5. View Manufacturer, Model and Serial number of the equipment using dmidecode

You can get information about the make, model and serial number of the equipment as shown below:

# dmidecode -t system # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0100, DMI type 1, 25 bytes
System Information
        Manufacturer: Dell Computer Corporation
        Product Name: PowerEdge 1750
        Version: Not Specified
        Serial Number: 1234567
        UUID: 4123454C-4123-1123-8123-12345603431
        Wake-up Type: Power Switch

Handle 0x0C00, DMI type 12, 5 bytes
System Configuration Options
        Option 1: NVRAM_CLR:  Clear user settable NVRAM areas and set defaults
        Option 2: PASSWD:  Close to enable password

Handle 0x2000, DMI type 32, 11 bytes
System Boot Information
        Status: No errors detected

Posted in Linux, UNIX-LINUX | Tagged: | Leave a Comment »

IBM AIX: How to create a file system

Posted by ZyK on 11/30/2011

IBM AIX: Volume Management
IBM AIX Logical Volume Management
System Administrator Information

How to create a file system

  • Log into server and gain root access
  • Select a Volume Group to create the file system under: lsvg (shows what volume groups are on the server)
  • Check to see how much disk space is currently available on the desired volume group: lsvg VGNAME (Check for Free PPs:) If there is enough disk space proceed to the next step
  • From the results of the lsvg VGNAME Check for the PP size in Megabytes. This will be 16, 32, 64 or 128 MB. The PP size is VERY important to remember as this will determine the number of Physical Partitions that will be used. If the wrong number is used this will either give too much or not enough disk space to the file system but will not cause any failures. You can NOT shrink a file system after its created so this is the only chance to get it correct without deleting and starting over.
  • Open a calculator: Enter the number of Megabytes (MB) that you wish to have as the file system size in the calculator, divide that by the number of the PP Size (2048 “Size of file system” / 32 “PP Size” would equal 64 PP’s to be used in the file system creation)
  • Create the Logical Volume: Run the following command ‘smitty fs’
  • Select ‘Add a logical Volume’
  • Select ‘Volume Group Name’:
    • Enter the ‘Logical Volume NAME’: There is 15 character limit for this field.
    • Enter the number of Physical Partitions: Number of PPs from the above mathematical process
    • Select Logical volume TYPE: enter JFS or JFS2 based on requirements. Unless the customer requests JFS select JFS2.
    • Select Range of Physical Volumes:
    • Change from minimum to maximum
    • Press Enter to accept changes. System will run and return an “OK” in the upper left hand corner when the command is complete.
    • Exit Smitty lv and enter ‘smitty fs’
  • Select Add / Change / Show / Delete File Systems
  • Select either Journaled File Systems or Enhanced Journaled File Systems
  • Select Add an Enhanced Journaled File System on a Previously Defined Logical Volume
  • Select the Logical Volume Name that you created in the previous step
  • Enter the mount point:
    • Select yes for Mount AUTOMATICALLY at system restart
    • Press enter to create the file system
    • Exit smitty fs and run the following command: ‘mount /PATH
    • Confirm that the file system has been created (df –k |grep PATH)
    • Set the permissions and ownerships per customer request
    • Task is now complete

How to expand a file system

  • Log into server and gain root access
  • Run the command: ‘smitty fs’
  • Select Add / Change / Show / Delete File Systems
  • Select either Journaled File Systems or Enhanced Journaled File Systems
  • Select Change / Show Characteristics of an Enhanced Journaled File System and select the file system that you wish to expand
  • Convert the number of Megabytes / Gigabytes requested by the customer into 512K blocks:
  • Bring up http://www.mydasd.com/converison.htm in a web browser.
  • In the Convert what quantity? Field enter the amount of space requested by the customer. Make sure to click the proper options in the left box below the quantity field. The box on the right should always be set to Bytes. Always click the Convert button to make sure you have an updated number in the display box.
  • Open a calculator: Enter the number from the webpage and divide that number by 512 (10 GB = 10,737,418,240: 10737418240 / 512 = 20971520)
  • In the smitty fs window take the number that is currently listed in the SIZE of file system (in 512-byte blocks) field and add that to the number from the above line in your calculator (Current size + SUM from above line = new size)
  • Enter the finial number in your calculator into the SIZE of file system field
  • Press Enter to accept changes. System will run and return an “OK” in the upper left hand corner when the command is complete.
  • Task is now complete

How to create a volume group (Using Powerpath)

  • To bring LUNs in as disks type cfgmgr
  • Type lspv |grep hdiskpower (Look for disks that are not assigned, they will be listed as “None”
  • Type ‘smitty’ and press enter
  • Select ‘System Storage Management (Physical & Logical Storage)’
  • Select “Logical Volume Manager”
  • Select “Volume Groups”
  • Select “Add a Volume Group with PowerPath Devices”
  • Enter the Volume Group Name to be created
  • Select “Physical partition SIZE in megabytes” (choose 32 MB by default unless directed otherwise)
  • Select “PHYSICAL VOLUME names”
  • Select the disks to be added to the new Volume Group. Press enter
  • Make sure that the line that says Activate volume group AUTOMATICALLY is marked “Yes”
  • Press Enter start changes, you will be prompted “are you sure”.
  • If you are sure press enter. Make sure you have selected the right information as you can NOT back out the changes.
  • From the command line prompt: type ‘lspv |grep hdiskpower’ you should now see that the disks that were assigned as “None” are now assigned to your new Volume Group.
  • Exit smitty
  • Task is now complete

How to re-create a file system when there are sub file systems present

  • Create _new file system
  • Un-mount all /PATH/srvr####/ sub file systems
  • Copy data to fromthe /PATH/srvr#### file system to the _new one
  • Un-mount /PATH/srvr####
  • Re-name /PATH/srvr#### to /clocal/srvr####_old
  • Re-Mount _old file system
  • Change the mount order in /etc/filesystems so that /PATH/srvr####_new mounts before all of the sub file systems
  • Un-mount _new file system
  • Change mount point of _new and drop the _new part
  • Mount /PATH/srvr#### file system
  • Create sub file systems (cntr0001 dblog01) mount points (directories)
  • Run a mount on the rest of the file systems
Task AIX 5L Version 5
Storage structure A disk is composed of physical partitions.

A physical volume is a physical disk the same thing as a disk.

A volume group is composed of physical volumes.

A volume group is divided into logical volumes.

A filesystem is placed into a logical volume.

A logical volume is extensible and can reside on more than one volume.

Run multiple tasks in a GUI environment smit lvm
wsm
Move a logical volume to another logical volume migratepv
Create a logical volume mklv
Extend a logical volume extendlv
Remove a logical volume rmlv
Create a volume group mkvg
Remove a disk from a volume group reducevg
Add disks to a volume group extendvg
Change logical volume settings chlv
Display volume group information lsvg
Display performance statistics for storage lvmstat
Manage volumes chlv
mklv
rmlv
Add a copy to an existing volume mklvcopy

IBM Disk Utilities in a nutshell

IBM Volume Manager in a nutshell

Listing Volume Groups

lsvg

rootvg

HACMPvg

paritemvg

yatirimvg

pbackupvg

ybackupvg

Detailed information about a Volume Group

root@paritem:/ >lsvg paritemvg

VOLUME GROUP:paritemvgVG IDENTIFIER:00c076eb00004c000000010852f95257

VG STATE:activePP SIZE:32 megabyte(s)

VG PERMISSION:read/writeTOTAL PPs:3196 (102272 megabytes)

MAX LVs:256FREE PPs:5 (160 megabytes)

LVs:2USED PPs:3191 (102112 megabytes)

OPEN LVs:2QUORUM:3

TOTAL PVs:4VG DESCRIPTORS: 4

STALE PVs:0STALE PPs:0

ACTIVE PVs:4AUTO ON:no

MAX PPs per VG:32512

MAX PPs per PV:1016MAX PVs:32

LTG size (Dynamic): 1024 kilobyte(s)AUTO SYNC:no

HOT SPARE:noBB POLICY:relocatable

Listing logical volumes in a Volume Group

root@paritem:/ >lsvg -l paritemvg

paritemvg:

LV NAMETYPELPsPPsPVsLV STATEMOUNT POINT

parfslogjfs2log111open/syncdN/A

paritemlvjfs2319031904open/syncd/paritem

Detailed information about a Logival Volume

root@paritem:/ >lslv paritemlv

LOGICAL VOLUME:paritemlvVOLUME GROUP:paritemvg

LV IDENTIFIER:00c076eb00004c000000010852f95257.2 PERMISSION:read/write

VG STATE:active/completeLV STATE:opened/syncd

TYPE:jfs2WRITE VERIFY:off

MAX LPs:4096PP SIZE:32 megabyte(s)

COPIES:1SCHED POLICY:parallel

LPs:3190PPs:3190

STALE PPs:0BB POLICY:relocatable

INTER-POLICY:maximumRELOCATABLE:yes

INTRA-POLICY:middleUPPER BOUND:32

MOUNT POINT:/paritemLABEL:/paritem

MIRROR WRITE CONSISTENCY: on/ACTIVE

EACH LP COPY ON A SEPARATE PV ?: yes

Serialize IO ?:NO

Listing physical volumes in a Logical Volume

root@paritem:/ >lslv -l paritemlv

paritemlv:/paritem

PVCOPIESIN BANDDISTRIBUTION

hdisk5798:000:00020%160:160:159:160:159

hdisk6798:000:00020%160:160:159:160:159

hdisk7797:000:00020%160:160:159:160:158

hdisk4797:000:00019%160:159:159:160:159

Listing All Physical Volumes in the system

root@paritem:/ >lspv

hdisk000c076eb47dc9ccdrootvgactive

hdisk100c076eb48a321d1rootvgactive

hdisk200c076eb52dc8486HACMPvg

hdisk300c076eb52dc87cbHACMPvg

hdisk400c076eb52f945b8paritemvgactive

hdisk500c076eb52f948ecparitemvgactive

hdisk600c076eb52f94c26paritemvgactive

hdisk700c076eb52f94f61paritemvgactive

hdisk800c076eb52f9882eyatirimvg

hdisk900c076eb52f98b5cyatirimvg

hdisk1000c076eb52f98e8byatirimvg

hdisk1100c076eb52f99260yatirimvg

hdisk1200c076ebb148f11dpbackupvgactive

hdisk1300c076ebb148f470pbackupvgactive

hdisk1400c076ebb14935e2ybackupvg

hdisk1500c076ebb1493921ybackupvg

Detailed information about a Physical Volume

root@paritem:/ >lspv hdisk5

PHYSICAL VOLUME:hdisk5VOLUME GROUP:paritemvg

PV IDENTIFIER:00c076eb52f948ec VG IDENTIFIER00c076eb00004c000000010852f95257

PV STATE:active

STALE PARTITIONS:0ALLOCATABLE:yes

PP SIZE:32 megabyte(s)LOGICAL VOLUMES:1

TOTAL PPs:799 (25568 megabytes)VG DESCRIPTORS:1

FREE PPs:1 (32 megabytes)HOT SPARE:no

USED PPs:798 (25536 megabytes)MAX REQUEST:1 megabyte

FREE DISTRIBUTION:00..00..00..00..01

USED DISTRIBUTION:160..160..159..160..159

Listing logical volumes on a Pyhsical Volume

root@paritem:/ >lspv -l hdisk5

hdisk5:

LV NAMELPsPPsDISTRIBUTIONMOUNT POINT

paritemlv798798160..160..159..160..159 /paritem

Changing FileSystem Properties

# first check the avaliable free space

# previously explained

lsvg <volume group name>

# adding 40GB to a mount point

Chfs –a size=+20000000 /pbackup(20000000/512=39063MB)

# can also be made by smitty chfs

Smitty chfs > Change / Show Characteristics of an Enhanced Journaled File System > Select the file system

# changing a mount point

Chfs –d /pbackup /paritem_backup

# config file

/etc/filesystems

# further chfs options are in chfs manual pages

man chfs

# if there is hacmp managed system then

# changing filesystem should be made by “smitty hacmp”

Smitty hacmp > System Management > Logical Volume Management > Shared File Systems > Enhanced Journaled File Systems > Change / Show Characteristics of a Shared Enhanced Journaled File System > Select the File System

(http://www.softpanorama.org)

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

Logical Volume Manager (LVM) Commands for AIX

Posted by ZyK on 11/29/2011

Glossary

Term Definition
Journaled File System (JFS) File system that uses a journaled log for faster, more reliable data recovery
Logical Partition (LP) The LV is made up of LPs.  The LP corresponds to 1 or more (in the case of mirroring) PPs.
Logical Volume (LV) The VG is subdivided into logical volumes and each LV can have  a file system on it.
Physical Partition (PP) All physical volumes are subdivided into pps.  PPs are all the same size.
Physical Volume (PV) Disk that is being managed by LVM.
Rootvg Default volume group created during installation.  The vg holds the OS filesystems ( /,/usr, /home, /proc /opt,  /tmp,  /var and swap space )
Volume Group (VG) Area of storage that consists of one or more PVs

Command Summary

Command Definition
chfs -a size=<#512 byte blocks> <file system> Increases the size of a journaled file system to the total number of  512 byte blocks specified
chfs -a size=<+512 byte blocks> <mount point> Increases the size of a journaled file system by the addional number of 512 byte blocks specified.  For example “chfs -a size=+393216 /usr”
chlv -n <newname> <oldname> Change the name of a logical volume (it must be inactive)
crfs -v jfs -m <mount point> -g <volume group> -a size=<# of 512 byte blocks>

crfs -v jfs -m <mount point> -d <logical volume>

This command makes a logical volume, mount point with a journaled file system:

creates a jfs file system on a logical volume

df -k Shows the disk usage of logical volumes on the server.
exportvg <volume group> removes a volume group from a machine
extendvg <volume group> <physical volume> Adds a new physical volume to an existing volume group
importvg -y <volume group> <physical volume> add a volume group to another machine
lslv <logical volume> [-l, m] Lists information about the logical volumes.  The -l option lists the disks in the logical volume.
lspv <physical volume> [-l, M, p] Lists the disks on the server, including the physical volume will give details about that disk.  The -l option will list the details of how the filesystems are distributed on the disk.
lsvg <volume group> [-l] Lists the volume groups on the server, including the volume group name will give details about that vg. The -l option will list the logical volumes in the volume group.
lsvpcfg Lists each vpath and the hdisks that make up the vpath
mklv -y <new lv> <vg> Makes a logical volume in a volume group
mksysb -l -f <device> makes a bootable backup of rootvg
mkvg -y <volume group> <physical volume>  . . . <physical volume> Makes a volume group out of one or more physical volumes
mount <logical volume> <file system>   or
mount <filesystem>  if it is already in /etc/filesystems
Mounts the file system for use.
reducevg <volume group> <physical volume> Removes a physical volume from a volume group
rmfs <file system> removes a file system and it’s logical volume
rmlv <lv> Removes a logical volume (it must be inactive)
savevg -l -f <device> <volume group> makes a backup copy of another volume group
umount <file system>  dismount the file system Unmounts the filesystem.

Sample LVM Procedures:

Filesystem Procedures

Procedure to create a filesystem using JFS:

  • See below the procedure for creating a logical volume and a filesystem using JFS:

Procedure to extend the size of filesystem using JFS:

  1. “df” to see the filesystem, it’s current size, % utilization and the name of it’s logical volume
  2. “lslv <logical_volume>” to show information about the logical volume including it’s volume group name.
  3. “lsvg <volume_group>” to show information about the volume group, including number of free pp’s and the pp size
  4. If there are not enough free pp’s then see below for procedure to add a disk to a volume group.
  5. “chfs -a size= +4194304 <MOUNT_POINT>” to grow the filesystem by 2 GB (4194304=2*1024*1024*1024/512)
    • NOTE:  Growing the file system will automatically grow the logical volume
  6. df” shows the file system’s current size is 2 GB more than before.

Troubleshooting extending the size of a filesystem using JFS:

  • Error Message:  0516-787 extendlv: Maximum allocation for logical volume <LV_Name> is 512.
    • Maximum number of LPs for the logical volume has been exceeded – must increase the allocation
    • Calculate the number of LPs needed = LV Size in MB / LP size in MB
    • chlv -x <new_max_lps> <logical_volume>

Procedure to remove a file system

  1. Unmount the filesystem
  2. Remove the logical volume “rmlv <lv_name>”
  3. Remove the filesystem information from /etc/filesystems

Procedure to reduce the size of a file system – shareold is 8mb and needs to be reduced to 4mb

  1. Create the file system
    1. crfs -v jfs -m /usr/sharenew -g rootvg -a size=8192
    2. this makes a logical volume in the root volume group of 4MB that uses jfs
  2. Mount the volume
    1. mount /usr/sharenew
  3. Move the files from the old file system (/usr/shareold)
    1. cd /usr/shareold
    2. tar cf – | (cd /usr/sharenew; tar xvf -)
    3. cd
  4. Unmount the file systems
    1. umount /usr/sharenew
    2. umount /usr/shareold
  5. Remove the old file system and it’s logical volume
    1. rmfs /usr/shareold
    1. chfs -m /usr/shareold /usr/sharenew
  6. Mount the new filesystem
    1. mount /usr/shareold
  7. Delete the temporary mount point
    1. rmdir /usr/share

 

Logical Volume Procedures

Procedure to create a logical volume and filesystem in a volume group using JFS:

  1. lsvg to determine the size of the PP
  2. lslv in similar logical volumes to determine if mirroring is in effect
  3. Calculate the number of PPs needed for the logical volume
    1. bc
    2. scale=2
    3. <size of lv in MB>/<size of PP in MB>
    4. quit
  4. mklv -y  “<LV_NAME>” <VG_NAME> <# of LPS>  –> creates the logical volume
  5. crfs -v jfs -d <LV_NAME> -m /<MOUNTPOINT> -A yes   –> makes the filesystem, creates the mountpoint and puts it in /etc/filesystems
  6. mount /<MOUNTPOINT>  –> mounts the new fileystem
  7. df /<MOUNTPOINT>  –> verifies the mount and the size of the new filesystem
  8. Check the ownership and permissions of the new mount point
    • ls -ld <mountpoint>
    • chown owner:group <mountpoint>
    • chmod XXX <mountpoint>
  9. If mirroring is in effect, then mirror this logical volume to another disk (original and 1 mirror):
    • mklvcopy -s y <LV_NAME> 2

Check to see if  all of the logical volumes in a volume group are mirrored

  • lsvg -l

Mirror a logical volume after the fact

  •  mklvcopy -s y <LV_NAME> 2

Volume Group Procedures

Procedure to create a volume group:

  1. lsdev -C -c disk  -> lists available disks (and the hdisk#) on the server
  2. mkvg -y “<VG_NAME>” hdisk#  –> creates the volume group on the named hard disk
  3. varyonvg <VG_NAME>  –> activates the volume group

Procedure to add a disk to a volume group (extend the volume group)

  • extendvg <vg> <disk#>
    • Verify the disk has been successfully added to the vg
  • lsvg -p <vg>

Procedure to mirror the rootvg:

  1. lspv  –> determine the hdisk#
  2. extendvg rootvg hdisk<number>  –> add the hdisk to the volume group
  3. lspv  –>  verify that the hdisk has been successfully added to the volume group
  4. chvg -Q ‘n’ rootvg  –>  change the quorum so that the vg will stay active if one of the mirrors fail
  5. mirrorvg -S -c 2 rootvg  –> mirror all of the logical volumes in the volume group
  6. lsvg -l rootvg  –> verify successful mirroring (pps will appear “stale” until synchronization is complete).
  7. bosboot -a  –>  update the boot image information
  8. bootlist -m normal -o hdisk0 hdisk1  –> create a new bootlist
  9. bootlist -m normal -o  –> verify the bootlist is correct

Procedure to increase the number of LP’s available
Assume we receive an error that the maximum number of LP’s had been exceeded, and the maximum number of LP’s defined was 1100:

  1. “lsvg <volume_group>” to show the total PP’s available in the volume group =1250
  2. “lsvg -l <volume_group>” to show the total PP’s used in all logical volumes in that volume group (showed sys1log, the jfs log was using 2 PP’s)
  3. “chlv -x 1248 <logical_volume>” to change the maximum number of LP’s from 1100 to 1248 (1250 PP’s in the volume group – 2 PP’s used by the jfs log  = 1248 available)

Physical Disk Procedures

Procedure to find disks/vpaths that are unallocated

  • lsvpcfg
    • This will show disks/vpaths and the volume group they are allocated to
  • lspv|grep None
    • This will show pvs and whether they are asssociated with a volume group
    • Note:  For vpaths, the hdisks will show as none, but they may be allocated to a vpath – you must grep each hdisk with the lsvpcfg

Procedure to make a new lun available to AIX

  • Allocate the new lun on the SAN
  • Run “cfgmgr”
  • Verify the new vpatch/hdisk by running “lsvpcfg”
    • There should be a new vpath and it should be available with no volume group – if not, rerun cfgmgr

Procedure to list the PVs in a volume group:

  • lsvg -p <volume group>

(http://networktechnologist.com)

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

Configure Static Routes In Debian or Red Hat Enterprise Linux

Posted by ZyK on 11/29/2011

Static routes improves overall performance of your network (especially bandwidth saving). They are also useful in stub networks (i.e. there is only one link to the network). For example, each LAN (located at different offices) is connected to HQ IDC (Internet data center) using single T1/LL/Wan links.

For example under Red Hat/Fedora Linux you can add static router for eth0 network interface by editing /etc/sysconfig/network-scripts/route-eth0 file. Under Debian Linux add static route by editing /etc/network/interface file.

 

Task: Display Current Routing Table Using ip command

By using the ip command, you can setup and view static route. For example, to display current routing table you can type command:
# ip route show
Sample output:

192.168.2.0/24 dev eth1 proto kernel  scope link  src 192.168.2.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.2
default via 192.168.1.254 dev eth0

You can add static route using following command:
ip route add {NETWORK} via {IP} dev {DEVICE}
For example network 192.168.55.0/24 available via 192.168.1.254:
# ip route add 192.168.55.0/24 via 192.168.1.254 dev eth1
Alternatively, you can use old good route command:
# route add -net 192.168.55.0 netmask 255.255.255.0 gw 192.168.1.254 dev eth1

Linux Persistence Routes

The drawback of ‘ip’ or ‘route’ command is that, when Linux reboots it will forget static routes. So store them in configuration file. Static routing describes a system that does not implement adaptive routing. In these systems routes through a data network are described by fixed paths (statically). These routes are usually entered into the router by the system administrator

Red Hat (RHEL) / CentOS / Fedora Linux Persistence Static Routing

You need to open /etc/sysconfig/network-scripts/route-eth0 file to define static routes for eth0 interface:
# cat /etc/sysconfig/network-scripts/route-eth0
Sample Output:

GATEWAY0=192.168.1.254
NETMASK0=255.255.255.0
ADDRESS0=192.168.55.0
GATEWAY1=10.164.234.112
NETMASK1= 255.255.255.240
ADDRESS1=10.164.234.132

How do I define static routing for network 10.0.0.0/8 via 10.9.38.65 router?

Open /etc/sysconfig/network-scripts/route-eth0:
# vi /etc/sysconfig/network-scripts/route-eth0
Append following line:
10.0.0.0/8 via 10.9.38.65
Save and close the file. Restart networking:
# service network restart
Verify new routing table:
# route -n

Debian / Ubuntu Linux Persistence Static Routing

Open configuration file /etc/network/interfaces
# cat /etc/network/interfaces
Output:

auto eth0
iface eth0 inet static
address 192.168.1.2
netmask 255.255.255.0
gateway 192.168.1.254
up route add -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.2.1
down route del -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.2.1
Debian / Ubuntu Linux static routing for two interfaces:
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
        address 10.9.38.76
        netmask 255.255.255.240
        network 10.9.38.64
        broadcast 10.9.38.79
	### static routing ###
        post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.9.38.65
        pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 10.9.38.65
auto eth1
iface eth1 inet static
        address 204.186.149.140
        netmask 255.255.255.240
        network 204.186.149.128
        broadcast 204.186.149.143
        gateway 204.186.149.129
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 10.0.80.11 10.0.80.12
        dns-search nixcraft.in

Updated for accuracy.

(cyberciti.biz)

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

Unmirror rootvg in AIX

Posted by ZyK on 11/29/2011

Root Volume Group (rootvg) is a volume group containing the Base Operating System (BOS). Logical volume (lv) in rootvg may be doubled or more in copies with 2 or more physical volume (hard disk) for availability and reliability of the AIX system. The following steps are to unmirror a rootvg, if for whatever reason the rootvg needs to run on single logical volume (lv) on single physical volume (pv) only.

Check and Determine if rootvg is Mirrored

In mirror mode, each logical volume in rootvg such as filesystems “/”, “/usr”, “/var”, “/tmp”, “/home”, “/opt” and default boot, paging and jfslog LVs should be mirrored. In AIX, mirrorvg will create additional copy of image for all logical volumes in the volume group.

Use the following command to check if a rootvg is mirrored:

# lsvg -l rootvg

If the output shows that for each LP there are 2 PPs then its mirrored.

Check and Determine Which Disks the Mirrored rootvg is Located

For each logical volume (LV) name listed in output of “lsvg -l rootvg” command, run the following command:

lslv -m

The output date will tell you on which disk each copy of each logical partitions for each LV.

Unmirror rootvg

Important: The following instructions have the risk of making your AIX system unbootable or corrupting the data. So make you have advanced system administration experience before running the process of unmirroring.

To unmirror the root volume group (rootvg), follow the steps below (scenario: rootvg is contained on hdisk01 and mirrored onto hdisk11, and the steps will remove the mirror on hdisk11 (regardless of the disk from which you previously booted)):

  1. To unmirror the rootvg from hdisk11, enter the following command:unmirrorvg rootvg hdisk11This command turns quorum back on for rootvg. When unmirrorvg is executed, the default COPIES value for each logical volume becomes 1.
  2. To update the booted disk link, enter the following command:ln -f /dev/rhdisk01 /dev/ipldevice
  3. To reduce the disk out of rootvg, type the following command:reducevg rootvg hdisk11
  4. To initilize the boot record of the remaining disk again, enter the following command:bosboot -a -d /dev/hdisk01bosboot command is a must to initialize the boot record on the remaining disk hdisk01 again.
  5. To modify the boot list to remove the unmirrored disk, type the following command:bootlist -m normal hdisk01bootlist command is a must so that the system only boots to the disk remaining (hdisk01) in rootvg.
  6. Restart AIX machine, as unmirroring turns quorum back on for rootvg, a reboot is required for this to take effect.

Note: The reducevg command in step 3 will fail if there are non-mirrored logical volumes such as raw logical volumes and system dump devices on the disk.

————according to smit :

CODE

                                             Remove Copies from a Logical Volume

Type or select values in entry fields.
Press Enter AFTER making all desired changes.

[Entry Fields]
* LOGICAL VOLUME name                                 lvoracle
* NEW maximum number of logical partition             2                                                                     +
copies
PHYSICAL VOLUME name(s) to remove copies from      []
+

looks like you have to specified the on you want to removed-

 

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

20 Linux Server Hardening Security Tips (P2)

Posted by ZyK on 11/23/2011

#11: Configure Iptables and TCPWrappers

Iptables is a user space application program that allows you to configure the firewall (Netfilter) provided by the Linux kernel. Use firewall to filter out traffic and allow only necessary traffic. Also use the TCPWrappers a host-based networking ACL system to filter network access to Internet. You can prevent many denial of service attacks with the help of Iptables:

#12: Linux Kernel /etc/sysctl.conf Hardening

/etc/sysctl.conf file is used to configure kernel parameters at runtime. Linux reads and applies settings from /etc/sysctl.conf at boot time. Sample /etc/sysctl.conf:

# Turn on execshield
kernel.exec-shield=1
kernel.randomize_va_space=1
# Enable IP spoofing protection
net.ipv4.conf.all.rp_filter=1
# Disable IP source routing
net.ipv4.conf.all.accept_source_route=0
# Ignoring broadcasts request
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_messages=1
# Make sure spoofed packets get logged
net.ipv4.conf.all.log_martians = 1

#13: Separate Disk Partitions

Separation of the operating system files from user files may result into a better and secure system. Make sure the following filesystems are mounted on separate partitions:

  • /usr
  • /home
  • /var and /var/tmp
  • /tmp

Create septate partitions for Apache and FTP server roots. Edit /etc/fstab file and make sure you add the following configuration options:

  1. noexec – Do not set execution of any binaries on this partition (prevents execution of binaries but allows scripts).
  2. nodev – Do not allow character or special devices on this partition (prevents use of device files such as zero, sda etc).
  3. nosuid – Do not set SUID/SGID access on this partition (prevent the setuid bit).

Sample /etc/fstab entry to to limit user access on /dev/sda5 (ftp server root directory):

/dev/sda5  /ftpdata          ext3    defaults,nosuid,nodev,noexec 1 2

#13.1: Disk Quotas

Make sure disk quota is enabled for all users. To implement disk quotas, use the following steps:

  1. Enable quotas per file system by modifying the /etc/fstab file.
  2. Remount the file system(s).
  3. Create the quota database files and generate the disk usage table.
  4. Assign quota policies.
  5. See implementing disk quotas tutorial for further details.

#14: Turn Off IPv6

Internet Protocol version 6 (IPv6) provides a new Internet layer of the TCP/IP protocol suite that replaces Internet Protocol version 4 (IPv4) and provides many benefits. Currently there are no good tools out which are able to check a system over network for IPv6 security issues. Most Linux distro began enabling IPv6 protocol by default. Crackers can send bad traffic via IPv6 as most admins are not monitoring it. Unless network configuration requires it, disable IPv6 or configure Linux IPv6 firewall:

#15: Disable Unwanted SUID and SGID Binaries

All SUID/SGID bits enabled file can be misused when the SUID/SGID executable has a security problem or bug. All local or remote user can use such file. It is a good idea to find all such files. Use the find command as follows:
#See all set user id files:
find / -perm +4000
# See all group id files
find / -perm +2000
# Or combine both in a single command
find / \( -perm -4000 -o -perm -2000 \) -print
find / -path -prune -o -type f -perm +6000 -ls

You need to investigate each reported file. See reported file man page for further details.

#15.1: World-Writable Files

Anyone can modify world-writable file resulting into a security issue. Use the following command to find all world writable and sticky bits set files:
find /dir -xdev -type d \( -perm -0002 -a ! -perm -1000 \) -print
You need to investigate each reported file and either set correct user and group permission or remove it.

#15.2: Noowner Files

Files not owned by any user or group can pose a security problem. Just find them with the following command which do not belong to a valid user and a valid group
find /dir -xdev \( -nouser -o -nogroup \) -print
You need to investigate each reported file and either assign it to an appropriate user and group or remove it.

#16: Use A Centralized Authentication Service

Without a centralized authentication system, user auth data becomes inconsistent, which may lead into out-of-date credentials and forgotten accounts which should have been deleted in first place. A centralized authentication service allows you maintaining central control over Linux / UNIX account and authentication data. You can keep auth data synchronized between servers. Do not use the NIS service for centralized authentication. Use OpenLDAP for clients and servers.

#16.1: Kerberos

Kerberos performs authentication as a trusted third party authentication service by using cryptographic shared secret under the assumption that packets traveling along the insecure network can be read, modified, and inserted. Kerberos builds on symmetric-key cryptography and requires a key distribution center. You can make remote login, remote copy, secure inter-system file copying and other high-risk tasks safer and more controllable using Kerberos. So, when users authenticate to network services using Kerberos, unauthorized users attempting to gather passwords by monitoring network traffic are effectively thwarted. See how to setup and use Kerberos.

#17: Logging and Auditing

You need to configure logging and auditing to collect all hacking and cracking attempts. By default syslog stores data in /var/log/ directory. This is also useful to find out software misconfiguration which may open your system to various attacks. See the following logging related articles:

  1. Linux log file locations.
  2. How to send logs to a remote loghost.
  3. How do I rotate log files?.
  4. man pages syslogd, syslog.conf and logrotate.

#17.1: Monitor Suspicious Log Messages With Logwatch / Logcheck

Read your logs using logwatch or logcheck. These tools make your log reading life easier. You get detailed reporting on unusual items in syslog via email. A sample syslog report:

 ################### Logwatch 7.3 (03/24/06) ####################
        Processing Initiated: Fri Oct 30 04:02:03 2009
        Date Range Processed: yesterday
                              ( 2009-Oct-29 )
                              Period is day.
      Detail Level of Output: 0
              Type of Output: unformatted
           Logfiles for Host: www-52.nixcraft.net.in
  ##################################################################
 --------------------- Named Begin ------------------------
 **Unmatched Entries**
    general: info: zone XXXXXX.com/IN: Transfer started.: 3 Time(s)
    general: info: zone XXXXXX.com/IN: refresh: retry limit for master ttttttttttttttttttt#53 exceeded (source ::#0): 3 Time(s)
    general: info: zone XXXXXX.com/IN: Transfer started.: 4 Time(s)
    general: info: zone XXXXXX.com/IN: refresh: retry limit for master ttttttttttttttttttt#53 exceeded (source ::#0): 4 Time(s)
 ---------------------- Named End -------------------------
  --------------------- iptables firewall Begin ------------------------
 Logged 87 packets on interface eth0
   From 58.y.xxx.ww - 1 packet to tcp(8080)
   From 59.www.zzz.yyy - 1 packet to tcp(22)
   From 60.32.nnn.yyy - 2 packets to tcp(45633)
   From 222.xxx.ttt.zz - 5 packets to tcp(8000,8080,8800)
 ---------------------- iptables firewall End -------------------------
 --------------------- SSHD Begin ------------------------
 Users logging in through sshd:
    root:
       123.xxx.ttt.zzz: 6 times
 ---------------------- SSHD End -------------------------
 --------------------- Disk Space Begin ------------------------
 Filesystem            Size  Used Avail Use% Mounted on
 /dev/sda3             450G  185G  241G  44% /
 /dev/sda1              99M   35M   60M  37% /boot
 ---------------------- Disk Space End -------------------------
 ###################### Logwatch End #########################

(Note output is truncated)

#17.2: System Accounting with auditd

The auditd is provided for system auditing. It is responsible for writing audit records to the disk. During startup, the rules in /etc/audit.rules are read by this daemon. You can open /etc/audit.rules file and make changes such as setup audit file log location and other option. With auditd you can answers the following questions:

  1. System startup and shutdown events (reboot / halt).
  2. Date and time of the event.
  3. User respoisble for the event (such as trying to access /path/to/topsecret.dat file).
  4. Type of event (edit, access, delete, write, update file & commands).
  5. Success or failure of the event.
  6. Records events that Modify date and time.
  7. Find out who made changes to modify the system’s network settings.
  8. Record events that modify user/group information.
  9. See who made changes to a file etc.

See our quick tutorial which explains enabling and using the auditd service.

#18: Secure OpenSSH Server

The SSH protocol is recommended for remote login and remote file transfer. However, ssh is open to many attacks. See how to secure OpenSSH server:

#19: Install And Use Intrusion Detection System

A network intrusion detection system (NIDS) is an intrusion detection system that tries to detect malicious activity such as denial of service attacks, port scans or even attempts to crack into computers by monitoring network traffic.

It is a good practice to deploy any integrity checking software before system goes online in a production environment. If possible install AIDE software before the system is connected to any network. AIDE is a host-based intrusion detection system (HIDS) it can monitor and analyses the internals of a computing system.

Snort is a software for intrusion detection which is capable of performing packet logging and real-time traffic analysis on IP networks.

#20: Protecting Files, Directories and Email

Linux offers excellent protections against unauthorized data access. File permissions and MAC prevent unauthorized access from accessing data. However, permissions set by the Linux are irrelevant if an attacker has physical access to a computer and can simply move the computer’s hard drive to another system to copy and analyze the sensitive data. You can easily protect files, and partitons under Linux using the following tools:

#20.1: Securing Email Servers

You can use SSL certificates and gpg keys to secure email communication on both server and client computers:

Other Recommendation:

Recommend readings:

  1. Red Hat Enterprise Linux – Security Guide.
  2. Linux security cookbook- A good collections of security recipes for new Linux admin.
  3. Snort 2.1 Intrusion Detection, Second Edition – Good introduction to Snort and Intrusion detection under Linux.
  4. Hardening Linux – Hardening Linux identifies many of the risks of running Linux hosts and applications and provides practical examples and methods to minimize those risks.
  5. Linux Security HOWTO.

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

20 Linux Server Hardening Security Tips (P1)

Posted by ZyK on 11/23/2011

Securing your Linux server is important to protect your data, intellectual property, and time, from the hands of crackers (hackers). The system administrator is responsible for security Linux box. In this first part of a Linux server security series, I will provide 20 hardening tips for default installation of Linux system.

#1: Encrypt Data Communication

All data transmitted over a network is open to monitoring. Encrypt transmitted data whenever possible with password or using keys / certificates.

  1. Use scp, ssh, rsync, or sftp for file transfer. You can also mount remote server file system or your own home directory using special sshfs and fuse tools.
  2. GnuPG allows to encrypt and sign your data and communication, features a versatile key managment system as well as access modules for all kind of public key directories.
  3. Fugu is a graphical frontend to the commandline Secure File Transfer application (SFTP). SFTP is similar to FTP, but unlike FTP, the entire session is encrypted, meaning no passwords are sent in cleartext form, and is thus much less vulnerable to third-party interception. Another option is FileZilla – a cross-platform client that supports FTP, FTP over SSL/TLS (FTPS), and SSH File Transfer Protocol (SFTP).
  4. OpenVPN is a cost-effective, lightweight SSL VPN.
  5. Lighttpd SSL (Secure Server Layer) Https Configuration And Installation
  6. Apache SSL (Secure Server Layer) Https (mod_ssl) Configuration And Installation

#1.1: Avoid Using FTP, Telnet, And Rlogin / Rsh

Under most network configurations, user names, passwords, FTP / telnet / rsh commands and transferred files can be captured by anyone on the same network using a packet sniffer. The common solution to this problem is to use either OpenSSH , SFTP, or FTPS (FTP over SSL), which adds SSL or TLS encryption to FTP. Type the following command to delete NIS, rsh and other outdated service:
# yum erase inetd xinetd ypserv tftp-server telnet-server rsh-serve

#2: Minimize Software to Minimize Vulnerability

Do you really need all sort of web services installed? Avoid installing unnecessary software to avoid vulnerabilities in software. Use the RPM package manager such as yum or apt-get and/or dpkg to review all installed set of software packages on a system. Delete all unwanted packages.
# yum list installed
# yum list packageName
# yum remove packageName

OR
# dpkg --list
# dpkg --info packageName
# apt-get remove packageName

#3: One Network Service Per System or VM Instance

Run different network services on separate servers or VM instance. This limits the number of other services that can be compromised. For example, if an attacker able to successfully exploit a software such as Apache flow, he / she will get an access to entire server including other services such as MySQL, e-mail server and so on. See how to install Virtualization software:

#4: Keep Linux Kernel and Software Up to Date

Applying security patches is an important part of maintaining Linux server. Linux provides all necessary tools to keep your system updated, and also allows for easy upgrades between versions. All security update should be reviewed and applied as soon as possible. Again, use the RPM package manager such as yum and/or apt-get and/or dpkg to apply all security updates.
# yum update
OR
# apt-get update && apt-get upgrade
You can configure Red hat / CentOS / Fedora Linux to send yum package update notification via email. Another option is to apply all security updates via a cron job. Under Debian / Ubuntu Linux you can use apticron to send security notifications.

#5: Use Linux Security Extensions

Linux comes with various security patches which can be used to guard against misconfigured or compromised programs. If possible use SELinux and other Linux security extensions to enforce limitations on network and other programs. For example, SELinux provides a variety of security policies for Linux kernel.

#5.1: SELinux

I strongly recommend using SELinux which provides a flexible Mandatory Access Control (MAC). Under standard Linux Discretionary Access Control (DAC), an application or process running as a user (UID or SUID) has the user’s permissions to objects such as files, sockets, and other processes. Running a MAC kernel protects the system from malicious or flawed applications that can damage or destroy the system. See the official Redhat documentation which explains SELinux configuration.

#6: User Accounts and Strong Password Policy

Use the useradd / usermod commands to create and maintain user accounts. Make sure you have a good and strong password policy. For example, a good password includes at least 8 characters long and mixture of alphabets, number, special character, upper & lower alphabets etc. Most important pick a password you can remember. Use tools such as “John the ripper” to find out weak users passwords on your server. Configure pam_cracklib.so to enforce the password policy.

#6.1: Password Aging

The chage command changes the number of days between password changes and the date of the last password change. This information is used by the system to determine when a user must change his/her password. The /etc/login.defs file defines the site-specific configuration for the shadow password suite including password aging configuration. To disable password aging, enter:
chage -M 99999 userName
To get password expiration information, enter:
chage -l userName
Finally, you can also edit the /etc/shadow file in the following fields:

{userName}:{password}:{lastpasswdchanged}:{Minimum_days}:{Maximum_days}:{Warn}:{Inactive}:{Expire}:

Where,

  1. Minimum_days: The minimum number of days required between password changes i.e. the number of days left before the user is allowed to change his/her password.
  2. Maximum_days: The maximum number of days the password is valid (after that user is forced to change his/her password).
  3. Warn : The number of days before password is to expire that user is warned that his/her password must be changed.
  4. Expire : Days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login may no longer be used.

I recommend chage command instead of editing the /etc/shadow by hand:
# chage -M 60 -m 7 -W 7 userName
Recommend readings:

#6.2: Restricting Use of Previous Passwords

You can prevent all users from using or reuse same old passwords under Linux. The pam_unix module parameter remember can be used to configure the number of previous passwords that cannot be reused.

#6.3: Locking User Accounts After Login Failures

Under Linux you can use the faillog command to display faillog records or to set login failure limits. faillog formats the contents of the failure log from /var/log/faillog database / log file. It also can be used for maintains failure counters and limits.To see failed login attempts, enter:
faillog
To unlock an account after login failures, run:
faillog -r -u userName
Note you can use passwd command to lock and unlock accounts:
# lock account
passwd -l userName
# unlocak account
passwd -u userName

#6.4: How Do I Verify No Accounts Have Empty Passwords?

Type the following command
# awk -F: '($2 == "") {print}' /etc/shadow
Lock all empty password accounts:
# passwd -l accountName

#6.5: Make Sure No Non-Root Accounts Have UID Set To 0

Only root account have UID 0 with full permissions to access the system. Type the following command to display all accounts with UID set to 0:
# awk -F: '($3 == "0") {print}' /etc/passwd
You should only see one line as follows:

root:x:0:0:root:/root:/bin/bash

If you see other lines, delete them or make sure other accounts are authorized by you to use UID 0.

#7: Disable root Login

Never ever login as root user. You should use sudo to execute root level commands as and when required. sudo does greatly enhances the security of the system without sharing root password with other users and admins. sudo provides simple auditing and tracking features too.

#8: Physical Server Security

You must protect Linux servers physical console access. Configure the BIOS and disable the booting from external devices such as DVDs / CDs / USB pen. Set BIOS and grub boot loader password to protect these settings. All production boxes must be locked in IDCs (Internet Data Center) and all persons must pass some sort of security checks before accessing your server. See also:

#9: Disable Unwanted Services

Disable all unnecessary services and daemons (services that runs in the background). You need to remove all unwanted services from the system start-up. Type the following command to list all services which are started at boot time in run level # 3:
# chkconfig --list | grep '3:on'
To disable service, enter:
# service serviceName stop
# chkconfig serviceName off

#9.1: Find Listening Network Ports

Use the following command to list all open ports and associated programs:
netstat -tulpn
OR
nmap -sT -O localhost
nmap -sT -O server.example.com

Use iptables to close open ports or stop all unwanted network services using above service and chkconfig commands.

#9.2: See Also

#10: Delete X Windows

X Windows on server is not required. There is no reason to run X Windows on your dedicated mail and Apache web server. You can disable and remove X Windows to improve server security and performance. Edit /etc/inittab and set run level to 3. Finally, remove X Windows system, enter:
# yum groupremove "X Window System"

(cyberciti.biz)

Posted in UNIX-LINUX | Tagged: , | 1 Comment »

Unix, Linux, and variants

Posted by ZyK on 11/22/2011

Quick links

Unix ABCs
Linux ABCs
MS-DOS vs. Linux / Unix
Unix and Linux commands
Unix Top 10 commands
Unix Shortcuts
Linux variants
Unix variants
Unix and Linux history
Unix and Linux tips
Unix and Linux links
Linux and Unix news
Linux Q&A

Unix ABCs

Unix, which is not an acronym, was developed by some of the members of the Multics team at the bell labs starting in the late 1960‘s by many of the same people who helped create the C programming language. The Unix today, however, is not just the work of a couple of programmers. Many other organizations, institutes and various other individuals contributed significant additions to the system we now know today.

See additional Unix information and variants and information on the Unix variants page.

Linux ABCs

Developed by Linus Torvalds and further elaborated by a number of developers throughout the world, Linux (lee’nuhks/ or /li’nuks/,_not_/li:’nuhks) is a freely available multitasking and multiuser operating system. From the outset, Linux was placed under General Public License (GPL). The system can be distributed, used and expanded free of charge. In this way, developers have access to all the source codes, thus being able to integrate new functions easily or to find and eliminate programming bugs quickly. Thereby drivers for new adapters (SCSI controller, graphics cards, etc.) can be integrated very rapidly.

See additional Linux information and variants and information on the Linux variants page.

MS-DOS vs. Linux / Unix

If you are able to navigate using MS-DOS, you should be able to quickly pick up on the navigation of Linux and Unix. In the below chart is a listing of common MS-DOS commands with their Linux / Unix counterpart.

MS-DOS Linux / Unix
attrib chmod
backup tar
dir ls
cls clear
copy cp
del rm
deltree rm -R
rmdir
edit vi
pico
format fdformat / mount / umount
move / rename mv
type less <file>
cd cd
chdir
more < file more file
md mkdir
win startx

Linux / Unix Commands

  * See the Linux and Unix overview page for a brief description on all commands on one page.
A a2p | ac | alias | ar | arch | arp | as | at | awk
B basename | bash | bc | bdiff | bfs | bg | biff | break | bs | bye
C cal | calendar | cancel | cat | cc | cd | chdir | checkeq | checknr | chfn | chgrp | chkey | chmod | chown | chsh | cksum | clear | cls | cmp | col | comm | compress | continue | copy | cp | cpio | crontab | csh | csplit | ctags | cu | curl | cut
D date | dc | dd | df | deroff | dhclient | diff | dig | dircmp | dirname | dmesg | dos2unix | dpost | du
E echo | ed | edit | egrepelm | emacs | enable | env | eqn | ex | exit | expand | expr
F fc | fdisk | fg | fgrep | file | find | findsmb | finger | fmt | fold | for | foreach | fromdos | fsck | ftp
G getfacl | gprof | grep | groupadd | groupdel | groupmod | gunzip | gview | gvim | gzip
H halt | hash | hashstat | head | help | history | host | hostid | hostname
I id | ifconfig | ifdown | ifup | ip | init | isalist
J jobs | join
K keylogin | kill | ksh
L last | ld | ldd | less | lex | link | ln | lo | locate | login | logname | logout | lp | lpadmin | lpc | lpq | lpr | lprm | lpstat | ls
M mach | mail | mailcompat | mailx | make | man | merge | mesg | mii-tool | mkdir | mkfs | more | mount | mt | mv | myisamchk | mysql
N nc | neqn | netstat | newalias | newform | newgrp | nice | niscat | nischmod | nischown | nischttl | nisdefaults | nisgrep | nismatch | nispasswd | nistbladm | nmap | nohup | nroff | nslookup
O on | onintr | optisa
P pack | pagesize | passwd | paste | pax | pcat | perl | pg | pgrep | pico | pine | ping | pkill | poweroff | pr | priocntl | printf | ps | pvs | pwd
Q quit
R rcp | reboot | red | rehash | remsh | repeat | replace | rgview | rgvim | rlogin | rm | rmail | rmdir | rn | route | rpcinfo | rsh | rsync | rview | rvim
S s2p | sag | sar | scp | screen | script | sdiff | sed | sendmail | set | setenv | setfacl | settime | sftp | sh | shred | shutdown | sleep | slogin | smbclient | sort | spell | split | stat | stop | strip | stty | su | sudo | sysinfo | sysklogd
T tabs | tac | tail | talk | tar | tbl | tcopy | tcpdump | tee | telinit | telnet | test | time | timex | todos | top | touch | tput | tr | traceroute | tree | troff | tty
U ulumask | unalias | uname | uncompress | unhash | uniq | unmount | unpack | untar | until | unzip | uptime | useradd | userdel | usermod
V vacation | veditvgrind | vi | view | vim | vipw | vmstat
W w | wait | wc | wget | whereis | which  | while| who | whoami | whois | write
X X | xargs | xfd | xlsfonts | xset | xterm | xrdb
Y yacc | yes | yppasswd
Z zcat, zip, zipcloak, zipinfo, zipnote, zipsplit

* In addition to each of the above explanations, additional information about a specified command for your Unix or Linux variant can be found by using the man command.

Posted in UNIX-LINUX | Tagged: , | 1 Comment »

 
Follow

Get every new post delivered to your Inbox.