Keep walking

Whatever you need to grow, just do it faster…

Posts Tagged ‘LINUX’

20 Linux System Monitoring Tools Every SysAdmin Should Know

Posted by ZyK on 01/31/2012

Need to monitor Linux server performance? Try these built-in command and a few add-on tools. Most Linux distributions are equipped with tons of monitoring. These tools provide metrics which can be used to get information about system activities. You can use these tools to find the possible causes of a performance problem. The commands discussed below are some of the most basic commands when it comes to system analysis and debugging server issues such as:

  1. Finding out bottlenecks.
  2. Disk (storage) bottlenecks.
  3. CPU and memory bottlenecks.
  4. Network bottlenecks.

#1: top – Process Activity Command

The top program provides a dynamic real-time view of a running system i.e. actual process activity. By default, it displays the most CPU-intensive tasks running on the server and updates the list every five seconds.

Fig.01: Linux top commandFig.01: Linux top command

Commonly Used Hot Keys

The top command provides several useful hot keys:

Hot Key Usage
t Displays summary information off and on.
m Displays memory information off and on.
A Sorts the display by top consumers of various system resources. Useful for quick identification of performance-hungry tasks on a system.
f Enters an interactive configuration screen for top. Helpful for setting up top for a specific task.
o Enables you to interactively select the ordering within top.
r Issues renice command.
k Issues kill command.
z Turn on or off color/mono

=> Related: How do I Find Out Linux CPU Utilization?

#2: vmstat – System Activity, Hardware and System Information

The command vmstat reports information about processes, memory, paging, block IO, traps, and cpu activity.
# vmstat 3
Sample Outputs:

procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0      0 2540988 522188 5130400    0    0     2    32    4    2  4  1 96  0  0
 1  0      0 2540988 522188 5130400    0    0     0   720 1199  665  1  0 99  0  0
 0  0      0 2540956 522188 5130400    0    0     0     0 1151 1569  4  1 95  0  0
 0  0      0 2540956 522188 5130500    0    0     0     6 1117  439  1  0 99  0  0
 0  0      0 2540940 522188 5130512    0    0     0   536 1189  932  1  0 98  0  0
 0  0      0 2538444 522188 5130588    0    0     0     0 1187 1417  4  1 96  0  0
 0  0      0 2490060 522188 5130640    0    0     0    18 1253 1123  5  1 94  0  0

Display Memory Utilization Slabinfo

# vmstat -m

Get Information About Active / Inactive Memory Pages

# vmstat -a
=> Related: How do I find out Linux Resource utilization to detect system bottlenecks?

#3: w – Find Out Who Is Logged on And What They Are Doing

w command displays information about the users currently on the machine, and their processes.
# w username
# w vivek

Sample Outputs:

 17:58:47 up 5 days, 20:28,  2 users,  load average: 0.36, 0.26, 0.24
USER     TTY      FROM              LOGIN@   IDLE   JCPU   PCPU WHAT
root     pts/0    10.1.3.145       14:55    5.00s  0.04s  0.02s vim /etc/resolv.conf
root     pts/1    10.1.3.145       17:43    0.00s  0.03s  0.00s w

#4: uptime – Tell How Long The System Has Been Running

The uptime command can be used to see how long the server has been running. The current time, how long the system has been running, how many users are currently logged on, and the system load averages for the past 1, 5, and 15 minutes.
# uptime
Output:

 18:02:41 up 41 days, 23:42,  1 user,  load average: 0.00, 0.00, 0.00

1 can be considered as optimal load value. The load can change from system to system. For a single CPU system 1 – 3 and SMP systems 6-10 load value might be acceptable.

#5: ps – Displays The Processes

ps command will report a snapshot of the current processes. To select all processes use the -A or -e option:
# ps -A
Sample Outputs:

  PID TTY          TIME CMD
    1 ?        00:00:02 init
    2 ?        00:00:02 migration/0
    3 ?        00:00:01 ksoftirqd/0
    4 ?        00:00:00 watchdog/0
    5 ?        00:00:00 migration/1
    6 ?        00:00:15 ksoftirqd/1
....
.....
 4881 ?        00:53:28 java
 4885 tty1     00:00:00 mingetty
 4886 tty2     00:00:00 mingetty
 4887 tty3     00:00:00 mingetty
 4888 tty4     00:00:00 mingetty
 4891 tty5     00:00:00 mingetty
 4892 tty6     00:00:00 mingetty
 4893 ttyS1    00:00:00 agetty
12853 ?        00:00:00 cifsoplockd
12854 ?        00:00:00 cifsdnotifyd
14231 ?        00:10:34 lighttpd
14232 ?        00:00:00 php-cgi
54981 pts/0    00:00:00 vim
55465 ?        00:00:00 php-cgi
55546 ?        00:00:00 bind9-snmp-stat
55704 pts/1    00:00:00 ps

ps is just like top but provides more information.

Show Long Format Output

# ps -Al
To turn on extra full mode (it will show command line arguments passed to process):
# ps -AlF

To See Threads ( LWP and NLWP)

# ps -AlFH

To See Threads After Processes

# ps -AlLm

Print All Process On The Server

# ps ax
# ps axu

Print A Process Tree

# ps -ejH
# ps axjf
# pstree

Print Security Information

# ps -eo euser,ruser,suser,fuser,f,comm,label
# ps axZ
# ps -eM

See Every Process Running As User Vivek

# ps -U vivek -u vivek u

Set Output In a User-Defined Format

# ps -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
# ps axo stat,euid,ruid,tty,tpgid,sess,pgrp,ppid,pid,pcpu,comm
# ps -eopid,tt,user,fname,tmout,f,wchan

Display Only The Process IDs of Lighttpd

# ps -C lighttpd -o pid=
OR
# pgrep lighttpd
OR
# pgrep -u vivek php-cgi

Display The Name of PID 55977

# ps -p 55977 -o comm=

Find Out The Top 10 Memory Consuming Process

# ps -auxf | sort -nr -k 4 | head -10

Find Out top 10 CPU Consuming Process

# ps -auxf | sort -nr -k 3 | head -10

#6: free – Memory Usage

The command free displays the total amount of free and used physical and swap memory in the system, as well as the buffers used by the kernel.
# free
Sample Output:

            total       used       free     shared    buffers     cached
Mem:      12302896    9739664    2563232          0     523124    5154740
-/+ buffers/cache:    4061800    8241096
Swap:      1052248          0    1052248

=> Related: :

  1. Linux Find Out Virtual Memory PAGESIZE
  2. Linux Limit CPU Usage Per Process
  3. How much RAM does my Ubuntu / Fedora Linux desktop PC have?

#7: iostat – Average CPU Load, Disk Activity

The command iostat report Central Processing Unit (CPU) statistics and input/output statistics for devices, partitions and network filesystems (NFS).
# iostat
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 	06/26/2009
avg-cpu:  %user   %nice %system %iowait  %steal   %idle
           3.50    0.09    0.51    0.03    0.00   95.86
Device:            tps   Blk_read/s   Blk_wrtn/s   Blk_read   Blk_wrtn
sda              22.04        31.88       512.03   16193351  260102868
sda1              0.00         0.00         0.00       2166        180
sda2             22.04        31.87       512.03   16189010  260102688
sda3              0.00         0.00         0.00       1615          0

=> Related: : Linux Track NFS Directory / Disk I/O Stats

#8: sar – Collect and Report System Activity

The sar command is used to collect, report, and save system activity information. To see network counter, enter:
# sar -n DEV | more
To display the network counters from the 24th:
# sar -n DEV -f /var/log/sa/sa24 | more
You can also display real time usage using sar:
# sar 4 5
Sample Outputs:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in) 		06/26/2009
06:45:12 PM       CPU     %user     %nice   %system   %iowait    %steal     %idle
06:45:16 PM       all      2.00      0.00      0.22      0.00      0.00     97.78
06:45:20 PM       all      2.07      0.00      0.38      0.03      0.00     97.52
06:45:24 PM       all      0.94      0.00      0.28      0.00      0.00     98.78
06:45:28 PM       all      1.56      0.00      0.22      0.00      0.00     98.22
06:45:32 PM       all      3.53      0.00      0.25      0.03      0.00     96.19
Average:          all      2.02      0.00      0.27      0.01      0.00     97.70

=> Related: : How to collect Linux system utilization data into a file

#9: mpstat – Multiprocessor Usage

The mpstat command displays activities for each available processor, processor 0 being the first one. mpstat -P ALL to display average CPU utilization per processor:
# mpstat -P ALL
Sample Output:

Linux 2.6.18-128.1.14.el5 (www03.nixcraft.in)	 	06/26/2009
06:48:11 PM  CPU   %user   %nice    %sys %iowait    %irq   %soft  %steal   %idle    intr/s
06:48:11 PM  all    3.50    0.09    0.34    0.03    0.01    0.17    0.00   95.86   1218.04
06:48:11 PM    0    3.44    0.08    0.31    0.02    0.00    0.12    0.00   96.04   1000.31
06:48:11 PM    1    3.10    0.08    0.32    0.09    0.02    0.11    0.00   96.28     34.93
06:48:11 PM    2    4.16    0.11    0.36    0.02    0.00    0.11    0.00   95.25      0.00
06:48:11 PM    3    3.77    0.11    0.38    0.03    0.01    0.24    0.00   95.46     44.80
06:48:11 PM    4    2.96    0.07    0.29    0.04    0.02    0.10    0.00   96.52     25.91
06:48:11 PM    5    3.26    0.08    0.28    0.03    0.01    0.10    0.00   96.23     14.98
06:48:11 PM    6    4.00    0.10    0.34    0.01    0.00    0.13    0.00   95.42      3.75
06:48:11 PM    7    3.30    0.11    0.39    0.03    0.01    0.46    0.00   95.69     76.89

=> Related: : Linux display each multiple SMP CPU processors utilization individually.

#10: pmap – Process Memory Usage

The command pmap report memory map of a process. Use this command to find out causes of memory bottlenecks.
# pmap -d PID
To display process memory information for pid # 47394, enter:
# pmap -d 47394
Sample Outputs:

47394:   /usr/bin/php-cgi
Address           Kbytes Mode  Offset           Device    Mapping
0000000000400000    2584 r-x-- 0000000000000000 008:00002 php-cgi
0000000000886000     140 rw--- 0000000000286000 008:00002 php-cgi
00000000008a9000      52 rw--- 00000000008a9000 000:00000   [ anon ]
0000000000aa8000      76 rw--- 00000000002a8000 008:00002 php-cgi
000000000f678000    1980 rw--- 000000000f678000 000:00000   [ anon ]
000000314a600000     112 r-x-- 0000000000000000 008:00002 ld-2.5.so
000000314a81b000       4 r---- 000000000001b000 008:00002 ld-2.5.so
000000314a81c000       4 rw--- 000000000001c000 008:00002 ld-2.5.so
000000314aa00000    1328 r-x-- 0000000000000000 008:00002 libc-2.5.so
000000314ab4c000    2048 ----- 000000000014c000 008:00002 libc-2.5.so
.....
......
..
00002af8d48fd000       4 rw--- 0000000000006000 008:00002 xsl.so
00002af8d490c000      40 r-x-- 0000000000000000 008:00002 libnss_files-2.5.so
00002af8d4916000    2044 ----- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b15000       4 r---- 0000000000009000 008:00002 libnss_files-2.5.so
00002af8d4b16000       4 rw--- 000000000000a000 008:00002 libnss_files-2.5.so
00002af8d4b17000  768000 rw-s- 0000000000000000 000:00009 zero (deleted)
00007fffc95fe000      84 rw--- 00007ffffffea000 000:00000   [ stack ]
ffffffffff600000    8192 ----- 0000000000000000 000:00000   [ anon ]
mapped: 933712K    writeable/private: 4304K    shared: 768000K

The last line is very important:

  • mapped: 933712K total amount of memory mapped to files
  • writeable/private: 4304K the amount of private address space
  • shared: 768000K the amount of address space this process is sharing with others

=> Related: : Linux find the memory used by a program / process using pmap command

#11 and #12: netstat and ss – Network Statistics

The command netstat displays network connections, routing tables, interface statistics, masquerade connections, and multicast memberships. ss command is used to dump socket statistics. It allows showing information similar to netstat. See the following resources about ss and netstat commands:

#13: iptraf – Real-time Network Statistics

The iptraf command is interactive colorful IP LAN monitor. It is an ncurses-based IP LAN monitor that generates various network statistics including TCP info, UDP counts, ICMP and OSPF information, Ethernet load info, node stats, IP checksum errors, and others. It can provide the following info in easy to read format:

  • Network traffic statistics by TCP connection
  • IP traffic statistics by network interface
  • Network traffic statistics by protocol
  • Network traffic statistics by TCP/UDP port and by packet size
  • Network traffic statistics by Layer2 address
Fig.02: General interface statistics: IP traffic statistics by network interface Fig.02: General interface statistics: IP traffic statistics by network interface

Fig.03 Network traffic statistics by TCP connectionFig.03 Network traffic statistics by TCP connection

#14: tcpdump – Detailed Network Traffic Analysis

The tcpdump is simple command that dump traffic on a network. However, you need good understanding of TCP/IP protocol to utilize this tool. For.e.g to display traffic info about DNS, enter:
# tcpdump -i eth1 'udp port 53'
To display all IPv4 HTTP packets to and from port 80, i.e. print only packets that contain data, not, for example, SYN and FIN packets and ACK-only packets, enter:
# tcpdump 'tcp port 80 and (((ip[2:2] - ((ip[0]&0xf)<<2)) - ((tcp[12]&0xf0)>>2)) != 0)'
To display all FTP session to 202.54.1.5, enter:
# tcpdump -i eth1 'dst 202.54.1.5 and (port 21 or 20'
To display all HTTP session to 192.168.1.5:
# tcpdump -ni eth0 'dst 192.168.1.5 and tcp and port http'
Use wireshark to view detailed information about files, enter:
# tcpdump -n -i eth1 -s 0 -w output.txt src or dst port 80

#15: strace – System Calls

Trace system calls and signals. This is useful for debugging webserver and other server problems. See how to use to trace the process and see What it is doing.

#16: /Proc file system – Various Kernel Statistics

/proc file system provides detailed information about various hardware devices and other Linux kernel information. See Linux kernel /proc documentations for further details. Common /proc examples:
# cat /proc/cpuinfo
# cat /proc/meminfo
# cat /proc/zoneinfo
# cat /proc/mounts

17#: Nagios – Server And Network Monitoring

Nagios is a popular open source computer system and network monitoring application software. You can easily monitor all your hosts, network equipment and services. It can send alert when things go wrong and again when they get better. FAN is “Fully Automated Nagios”. FAN goals are to provide a Nagios installation including most tools provided by the Nagios Community. FAN provides a CDRom image in the standard ISO format, making it easy to easilly install a Nagios server. Added to this, a wide bunch of tools are including to the distribution, in order to improve the user experience around Nagios.

18#: Cacti – Web-based Monitoring Tool

Cacti is a complete network graphing solution designed to harness the power of RRDTool’s data storage and graphing functionality. Cacti provides a fast poller, advanced graph templating, multiple data acquisition methods, and user management features out of the box. All of this is wrapped in an intuitive, easy to use interface that makes sense for LAN-sized installations up to complex networks with hundreds of devices. It can provide data about network, CPU, memory, logged in users, Apache, DNS servers and much more. See how to install and configure Cacti network graphing tool under CentOS / RHEL.

#19: KDE System Guard – Real-time Systems Reporting and Graphing

KSysguard is a network enabled task and system monitor application for KDE desktop. This tool can be run over ssh session. It provides lots of features such as a client/server architecture that enables monitoring of local and remote hosts. The graphical front end uses so-called sensors to retrieve the information it displays. A sensor can return simple values or more complex information like tables. For each type of information, one or more displays are provided. Displays are organized in worksheets that can be saved and loaded independently from each other. So, KSysguard is not only a simple task manager but also a very powerful tool to control large server farms.

Fig.05 KDE System GuardFig.05 KDE System Guard {Image credit: Wikipedia}

See the KSysguard handbook for detailed usage.

#20: Gnome System Monitor – Real-time Systems Reporting and Graphing

The System Monitor application enables you to display basic system information and monitor system processes, usage of system resources, and file systems. You can also use System Monitor to modify the behavior of your system. Although not as powerful as the KDE System Guard, it provides the basic information which may be useful for new users:

  • Displays various basic information about the computer’s hardware and software.
  • Linux Kernel version
  • GNOME version
  • Hardware
  • Installed memory
  • Processors and speeds
  • System Status
  • Currently available disk space
  • Processes
  • Memory and swap space
  • Network usage
  • File Systems
  • Lists all mounted filesystems along with basic information about each.
Fig.06 The Gnome System Monitor applicationFig.06 The Gnome System Monitor application

Bonus: Additional Tools

A few more tools:

  • nmap – scan your server for open ports.
  • lsof – list open files, network connections and much more.
  • ntop web based tool – ntop is the best tool to see network usage in a way similar to what top command does for processes i.e. it is network traffic monitoring software. You can see network status, protocol wise distribution of traffic for UDP, TCP, DNS, HTTP and other protocols.
  • Conky – Another good monitoring tool for the X Window System. It is highly configurable and is able to monitor many system variables including the status of the CPU, memory, swap space, disk storage, temperatures, processes, network interfaces, battery power, system messages, e-mail inboxes etc.
  • GKrellM – It can be used to monitor the status of CPUs, main memory, hard disks, network interfaces, local and remote mailboxes, and many other things.
  • vnstat – vnStat is a console-based network traffic monitor. It keeps a log of hourly, daily and monthly network traffic for the selected interface(s).
  • htop – htop is an enhanced version of top, the interactive process viewer, which can display the list of processes in a tree form.
  • mtr – mtr combines the functionality of the traceroute and ping programs in a single network diagnostic tool.

Did I miss something? Please add your favorite system motoring tool in the comments.

Featured Articles:

(Src: http://www.cyberciti.biz)

Posted in Linux | Tagged: | 1 Comment »

Get Hardware Information On Linux Using dmidecode

Posted by ZyK on 01/31/2012

dmidecode command reads the system DMI table to display hardware and BIOS information of the server. Apart from getting current configuration of the system, you can also get information about maximum supported configuration of the system using dmidecode. For example, dmidecode gives both the current RAM on the system and the maximum RAM supported by the system.

This article provides an overview of the dmidecode and few practical examples on how to use dmidecode command.

1. Overview of dmidecode

Distributed Management Task Force maintains the DMI specification and SMBIOS specification. The output of the dmidecode contains several records from the DMI (Desktop Management interface) table.

Following is the record format of the dmidecode output of the DMI table.

Record Header: Handle {record id}, DMI type {dmi type id}, {record size} bytes
Record Value: {multi line record value}
  • record id: Unique identifier for every record in the DMI table.
  • dmi type id: Type of the record. i.e BIOS, Memory etc.,
  • record size: Size of the record in the DMI table.
  • multi line record values: Multi line record value for that specific DMI type.

Sample output of dmidecode command:

# dmidecode | head -15 
# dmidecode 2.9
SMBIOS 2.3 present.
56 structures occupying 1977 bytes.
Table at 0x000FB320.

Handle 0xDA00, DMI type 218, 11 bytes
OEM-specific Type
        Header and Data:
                DA 0B 00 DA B0 00 17 03 08 28 00

Handle 0x0000, DMI type 0, 20 bytes
BIOS Information
        Vendor: Dell Computer Corporation
        Version: A07
        Release Date: 01/13/2004

Get the total number of records in the DMI table as shown below:

# dmidecode | grep ^Handle | wc -l
56

(or)

# dmidecode | grep structures
56 structures occupying 1977 bytes.

2. DMI Types

DMI Type id will give information about a particular hardware component of your system. Following command with type id 4 will get the information about CPU of the system.

# dmidecode -t 4 # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0400, DMI type 4, 35 bytes
Processor Information
        Socket Designation: Processor 1
        Type: Central Processor
        Family: Xeon
        Manufacturer: Intel
        ID: 29 0F 00 00 FF FB EB BF
        Signature: Type 0, Family 15, Model 2, Stepping 9
        Flags:
                FPU (Floating-point unit on-chip)
                VME (Virtual mode extension)
                DE (Debugging extension)
                PSE (Page size extension)
                TSC (Time stamp counter)
                MSR (Model specific registers)

Following are the different DMI types available.

       Type   Information
       ----------------------------------------
          0   BIOS
          1   System
          2   Base Board
          3   Chassis
          4   Processor
          5   Memory Controller
          6   Memory Module
          7   Cache
          8   Port Connector
          9   System Slots
         10   On Board Devices
         11   OEM Strings
         12   System Configuration Options
         13   BIOS Language
         14   Group Associations
         15   System Event Log
         16   Physical Memory Array
         17   Memory Device
         18   32-bit Memory Error
         19   Memory Array Mapped Address
         20   Memory Device Mapped Address
         21   Built-in Pointing Device
         22   Portable Battery
         23   System Reset
         24   Hardware Security
         25   System Power Controls
         26   Voltage Probe
         27   Cooling Device
         28   Temperature Probe
         29   Electrical Current Probe
         30   Out-of-band Remote Access
         31   Boot Integrity Services
         32   System Boot
         33   64-bit Memory Error
         34   Management Device
         35   Management Device Component
         36   Management Device Threshold Data
         37   Memory Channel
         38   IPMI Device
         39   Power Supply

Instead of type_id, you can also pass the keyword to the -t option of the dmidecode command. Following are the available keywords.

       Keyword     Types
       ------------------------------
       bios        0, 13
       system      1, 12, 15, 23, 32
       baseboard   2, 10
       chassis     3
       processor   4
       memory      5, 6, 16, 17
       cache       7
       connector   8
       slot        9

For example, to get all the system baseboard related information execute the following command, which will display the type_id 2 and 10

# dmidecode -t baseboard # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0200, DMI type 2, 9 bytes
Base Board Information
        Manufacturer: Dell Computer Corporation
        Product Name: 123456
        Version: A05
        Serial Number: ..CN123456789098.

Handle 0x0A00, DMI type 10, 14 bytes
On Board Device 1 Information
        Type: SCSI Controller
        Status: Enabled
        Description: LSI Logic 53C1030 Ultra 320 SCSI
On Board Device 2 Information
        Type: SCSI Controller
        Status: Enabled
        Description: LSI Logic 53C1030 Ultra 320 SCSI
On Board Device 3 Information
        Type: Video
        Status: Enabled
        Description: ATI Rage XL PCI Video
On Board Device 4 Information
        Type: Ethernet
        Status: Enabled
        Description: Broadcom Gigabit Ethernet 1
On Board Device 5 Information
        Type: Ethernet
        Status: Enabled
        Description: Broadcom Gigabit Ethernet 2

3. Get Physical Memory (RAM) information using dmidecode

What is the maximum RAM supported by the system? In this example, this system can support maximum 8GB of RAM.

# dmidecode -t 16 # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x1000, DMI type 16, 15 bytes
Physical Memory Array
        Location: System Board Or Motherboard
        Use: System Memory
        Error Correction Type: Multi-bit ECC
 Maximum Capacity: 8 GB         Error Information Handle: Not Provided
        Number Of Devices: 4


How much memory can I expand to? From /proc/meminfo you can find out the total current memory of your system as shown below.

# grep MemTotal /proc/meminfo
MemTotal:      1034644 kB


In this example, the system has 1GB of RAM. Is this 1 x 1GB (or) 2 x 512MB (or) 4 x 256MB? This can be figured out by passing the type id 17 to the dmidecode command as shown below. Please note in the example below, if you have to expand upto 8GB of maximum RAM, you need to remove the existing 512MB from slot 1 and 2, and use 2GB RAM on all the 4 memory slots.

# dmidecode -t 17 # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x1100, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: 512 MB [Note: Slot1 has 512 MB RAM]
        Form Factor: DIMM
        Set: 1
        Locator: DIMM_1A
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

Handle 0x1101, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: 512 MB [Note: Slot2 has 512 MB RAM]
        Form Factor: DIMM
        Set: 1
        Locator: DIMM_1B
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

Handle 0x1102, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000
        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: No Module Installed [Note: Slot3 is empty]
        Form Factor: DIMM
        Set: 2
        Locator: DIMM_2A
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

Handle 0x1103, DMI type 17, 23 bytes
Memory Device
        Array Handle: 0x1000

        Error Information Handle: Not Provided
        Total Width: 72 bits
        Data Width: 64 bits
 Size: No Module Installed [Note: Slot4 is empty]
        Form Factor: DIMM
        Set: 2
        Locator: DIMM_2B
        Bank Locator: Not Specified
        Type: DDR
        Type Detail: Synchronous
        Speed: 266 MHz (3.8 ns)

4. Get BIOS information using dmidecode

# dmidecode -t bios # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0000, DMI type 0, 20 bytes
BIOS Information
        Vendor: Dell Computer Corporation
        Version: A07
        Release Date: 01/13/2004
        Address: 0xF0000
        Runtime Size: 64 kB
        ROM Size: 4096 kB
        Characteristics:
                ISA is supported
                PCI is supported
                PNP is supported
                BIOS is upgradeable
                BIOS shadowing is allowed
                ESCD support is available
                Boot from CD is supported
                Selectable boot is supported
                EDD is supported
                Japanese floppy for Toshiba 1.2 MB is supported (int 13h)
                5.25"/360 KB floppy services are supported (int 13h)
                5.25"/1.2 MB floppy services are supported (int 13h)
                3.5"/720 KB floppy services are supported (int 13h)
                8042 keyboard services are supported (int 9h)
                Serial services are supported (int 14h)
                CGA/mono video services are supported (int 10h)
                ACPI is supported
                USB legacy is supported
                LS-120 boot is supported
                BIOS boot specification is supported
                Function key-initiated network boot is supported

Handle 0x0D00, DMI type 13, 22 bytes
BIOS Language Information
        Installable Languages: 1
                en|US|iso8859-1
        Currently Installed Language: en|US|iso8859-1

5. View Manufacturer, Model and Serial number of the equipment using dmidecode

You can get information about the make, model and serial number of the equipment as shown below:

# dmidecode -t system # dmidecode 2.9
SMBIOS 2.3 present.

Handle 0x0100, DMI type 1, 25 bytes
System Information
        Manufacturer: Dell Computer Corporation
        Product Name: PowerEdge 1750
        Version: Not Specified
        Serial Number: 1234567
        UUID: 4123454C-4123-1123-8123-12345603431
        Wake-up Type: Power Switch

Handle 0x0C00, DMI type 12, 5 bytes
System Configuration Options
        Option 1: NVRAM_CLR:  Clear user settable NVRAM areas and set defaults
        Option 2: PASSWD:  Close to enable password

Handle 0x2000, DMI type 32, 11 bytes
System Boot Information
        Status: No errors detected

Posted in Linux, UNIX-LINUX | Tagged: | Leave a Comment »

Configure Static Routes In Debian or Red Hat Enterprise Linux

Posted by ZyK on 11/29/2011

Static routes improves overall performance of your network (especially bandwidth saving). They are also useful in stub networks (i.e. there is only one link to the network). For example, each LAN (located at different offices) is connected to HQ IDC (Internet data center) using single T1/LL/Wan links.

For example under Red Hat/Fedora Linux you can add static router for eth0 network interface by editing /etc/sysconfig/network-scripts/route-eth0 file. Under Debian Linux add static route by editing /etc/network/interface file.

 

Task: Display Current Routing Table Using ip command

By using the ip command, you can setup and view static route. For example, to display current routing table you can type command:
# ip route show
Sample output:

192.168.2.0/24 dev eth1 proto kernel  scope link  src 192.168.2.1
192.168.1.0/24 dev eth0  proto kernel  scope link  src 192.168.1.2
default via 192.168.1.254 dev eth0

You can add static route using following command:
ip route add {NETWORK} via {IP} dev {DEVICE}
For example network 192.168.55.0/24 available via 192.168.1.254:
# ip route add 192.168.55.0/24 via 192.168.1.254 dev eth1
Alternatively, you can use old good route command:
# route add -net 192.168.55.0 netmask 255.255.255.0 gw 192.168.1.254 dev eth1

Linux Persistence Routes

The drawback of ‘ip’ or ‘route’ command is that, when Linux reboots it will forget static routes. So store them in configuration file. Static routing describes a system that does not implement adaptive routing. In these systems routes through a data network are described by fixed paths (statically). These routes are usually entered into the router by the system administrator

Red Hat (RHEL) / CentOS / Fedora Linux Persistence Static Routing

You need to open /etc/sysconfig/network-scripts/route-eth0 file to define static routes for eth0 interface:
# cat /etc/sysconfig/network-scripts/route-eth0
Sample Output:

GATEWAY0=192.168.1.254
NETMASK0=255.255.255.0
ADDRESS0=192.168.55.0
GATEWAY1=10.164.234.112
NETMASK1= 255.255.255.240
ADDRESS1=10.164.234.132

How do I define static routing for network 10.0.0.0/8 via 10.9.38.65 router?

Open /etc/sysconfig/network-scripts/route-eth0:
# vi /etc/sysconfig/network-scripts/route-eth0
Append following line:
10.0.0.0/8 via 10.9.38.65
Save and close the file. Restart networking:
# service network restart
Verify new routing table:
# route -n

Debian / Ubuntu Linux Persistence Static Routing

Open configuration file /etc/network/interfaces
# cat /etc/network/interfaces
Output:

auto eth0
iface eth0 inet static
address 192.168.1.2
netmask 255.255.255.0
gateway 192.168.1.254
up route add -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.2.1
down route del -net 192.168.2.0 netmask 255.255.255.0 gw 192.168.2.1
Debian / Ubuntu Linux static routing for two interfaces:
auto lo
iface lo inet loopback
auto eth0
iface eth0 inet static
        address 10.9.38.76
        netmask 255.255.255.240
        network 10.9.38.64
        broadcast 10.9.38.79
	### static routing ###
        post-up route add -net 10.0.0.0 netmask 255.0.0.0 gw 10.9.38.65
        pre-down route del -net 10.0.0.0 netmask 255.0.0.0 gw 10.9.38.65
auto eth1
iface eth1 inet static
        address 204.186.149.140
        netmask 255.255.255.240
        network 204.186.149.128
        broadcast 204.186.149.143
        gateway 204.186.149.129
        # dns-* options are implemented by the resolvconf package, if installed
        dns-nameservers 10.0.80.11 10.0.80.12
        dns-search nixcraft.in

Updated for accuracy.

(cyberciti.biz)

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

20 Linux Server Hardening Security Tips (P2)

Posted by ZyK on 11/23/2011

#11: Configure Iptables and TCPWrappers

Iptables is a user space application program that allows you to configure the firewall (Netfilter) provided by the Linux kernel. Use firewall to filter out traffic and allow only necessary traffic. Also use the TCPWrappers a host-based networking ACL system to filter network access to Internet. You can prevent many denial of service attacks with the help of Iptables:

#12: Linux Kernel /etc/sysctl.conf Hardening

/etc/sysctl.conf file is used to configure kernel parameters at runtime. Linux reads and applies settings from /etc/sysctl.conf at boot time. Sample /etc/sysctl.conf:

# Turn on execshield
kernel.exec-shield=1
kernel.randomize_va_space=1
# Enable IP spoofing protection
net.ipv4.conf.all.rp_filter=1
# Disable IP source routing
net.ipv4.conf.all.accept_source_route=0
# Ignoring broadcasts request
net.ipv4.icmp_echo_ignore_broadcasts=1
net.ipv4.icmp_ignore_bogus_error_messages=1
# Make sure spoofed packets get logged
net.ipv4.conf.all.log_martians = 1

#13: Separate Disk Partitions

Separation of the operating system files from user files may result into a better and secure system. Make sure the following filesystems are mounted on separate partitions:

  • /usr
  • /home
  • /var and /var/tmp
  • /tmp

Create septate partitions for Apache and FTP server roots. Edit /etc/fstab file and make sure you add the following configuration options:

  1. noexec – Do not set execution of any binaries on this partition (prevents execution of binaries but allows scripts).
  2. nodev – Do not allow character or special devices on this partition (prevents use of device files such as zero, sda etc).
  3. nosuid – Do not set SUID/SGID access on this partition (prevent the setuid bit).

Sample /etc/fstab entry to to limit user access on /dev/sda5 (ftp server root directory):

/dev/sda5  /ftpdata          ext3    defaults,nosuid,nodev,noexec 1 2

#13.1: Disk Quotas

Make sure disk quota is enabled for all users. To implement disk quotas, use the following steps:

  1. Enable quotas per file system by modifying the /etc/fstab file.
  2. Remount the file system(s).
  3. Create the quota database files and generate the disk usage table.
  4. Assign quota policies.
  5. See implementing disk quotas tutorial for further details.

#14: Turn Off IPv6

Internet Protocol version 6 (IPv6) provides a new Internet layer of the TCP/IP protocol suite that replaces Internet Protocol version 4 (IPv4) and provides many benefits. Currently there are no good tools out which are able to check a system over network for IPv6 security issues. Most Linux distro began enabling IPv6 protocol by default. Crackers can send bad traffic via IPv6 as most admins are not monitoring it. Unless network configuration requires it, disable IPv6 or configure Linux IPv6 firewall:

#15: Disable Unwanted SUID and SGID Binaries

All SUID/SGID bits enabled file can be misused when the SUID/SGID executable has a security problem or bug. All local or remote user can use such file. It is a good idea to find all such files. Use the find command as follows:
#See all set user id files:
find / -perm +4000
# See all group id files
find / -perm +2000
# Or combine both in a single command
find / \( -perm -4000 -o -perm -2000 \) -print
find / -path -prune -o -type f -perm +6000 -ls

You need to investigate each reported file. See reported file man page for further details.

#15.1: World-Writable Files

Anyone can modify world-writable file resulting into a security issue. Use the following command to find all world writable and sticky bits set files:
find /dir -xdev -type d \( -perm -0002 -a ! -perm -1000 \) -print
You need to investigate each reported file and either set correct user and group permission or remove it.

#15.2: Noowner Files

Files not owned by any user or group can pose a security problem. Just find them with the following command which do not belong to a valid user and a valid group
find /dir -xdev \( -nouser -o -nogroup \) -print
You need to investigate each reported file and either assign it to an appropriate user and group or remove it.

#16: Use A Centralized Authentication Service

Without a centralized authentication system, user auth data becomes inconsistent, which may lead into out-of-date credentials and forgotten accounts which should have been deleted in first place. A centralized authentication service allows you maintaining central control over Linux / UNIX account and authentication data. You can keep auth data synchronized between servers. Do not use the NIS service for centralized authentication. Use OpenLDAP for clients and servers.

#16.1: Kerberos

Kerberos performs authentication as a trusted third party authentication service by using cryptographic shared secret under the assumption that packets traveling along the insecure network can be read, modified, and inserted. Kerberos builds on symmetric-key cryptography and requires a key distribution center. You can make remote login, remote copy, secure inter-system file copying and other high-risk tasks safer and more controllable using Kerberos. So, when users authenticate to network services using Kerberos, unauthorized users attempting to gather passwords by monitoring network traffic are effectively thwarted. See how to setup and use Kerberos.

#17: Logging and Auditing

You need to configure logging and auditing to collect all hacking and cracking attempts. By default syslog stores data in /var/log/ directory. This is also useful to find out software misconfiguration which may open your system to various attacks. See the following logging related articles:

  1. Linux log file locations.
  2. How to send logs to a remote loghost.
  3. How do I rotate log files?.
  4. man pages syslogd, syslog.conf and logrotate.

#17.1: Monitor Suspicious Log Messages With Logwatch / Logcheck

Read your logs using logwatch or logcheck. These tools make your log reading life easier. You get detailed reporting on unusual items in syslog via email. A sample syslog report:

 ################### Logwatch 7.3 (03/24/06) ####################
        Processing Initiated: Fri Oct 30 04:02:03 2009
        Date Range Processed: yesterday
                              ( 2009-Oct-29 )
                              Period is day.
      Detail Level of Output: 0
              Type of Output: unformatted
           Logfiles for Host: www-52.nixcraft.net.in
  ##################################################################
 --------------------- Named Begin ------------------------
 **Unmatched Entries**
    general: info: zone XXXXXX.com/IN: Transfer started.: 3 Time(s)
    general: info: zone XXXXXX.com/IN: refresh: retry limit for master ttttttttttttttttttt#53 exceeded (source ::#0): 3 Time(s)
    general: info: zone XXXXXX.com/IN: Transfer started.: 4 Time(s)
    general: info: zone XXXXXX.com/IN: refresh: retry limit for master ttttttttttttttttttt#53 exceeded (source ::#0): 4 Time(s)
 ---------------------- Named End -------------------------
  --------------------- iptables firewall Begin ------------------------
 Logged 87 packets on interface eth0
   From 58.y.xxx.ww - 1 packet to tcp(8080)
   From 59.www.zzz.yyy - 1 packet to tcp(22)
   From 60.32.nnn.yyy - 2 packets to tcp(45633)
   From 222.xxx.ttt.zz - 5 packets to tcp(8000,8080,8800)
 ---------------------- iptables firewall End -------------------------
 --------------------- SSHD Begin ------------------------
 Users logging in through sshd:
    root:
       123.xxx.ttt.zzz: 6 times
 ---------------------- SSHD End -------------------------
 --------------------- Disk Space Begin ------------------------
 Filesystem            Size  Used Avail Use% Mounted on
 /dev/sda3             450G  185G  241G  44% /
 /dev/sda1              99M   35M   60M  37% /boot
 ---------------------- Disk Space End -------------------------
 ###################### Logwatch End #########################

(Note output is truncated)

#17.2: System Accounting with auditd

The auditd is provided for system auditing. It is responsible for writing audit records to the disk. During startup, the rules in /etc/audit.rules are read by this daemon. You can open /etc/audit.rules file and make changes such as setup audit file log location and other option. With auditd you can answers the following questions:

  1. System startup and shutdown events (reboot / halt).
  2. Date and time of the event.
  3. User respoisble for the event (such as trying to access /path/to/topsecret.dat file).
  4. Type of event (edit, access, delete, write, update file & commands).
  5. Success or failure of the event.
  6. Records events that Modify date and time.
  7. Find out who made changes to modify the system’s network settings.
  8. Record events that modify user/group information.
  9. See who made changes to a file etc.

See our quick tutorial which explains enabling and using the auditd service.

#18: Secure OpenSSH Server

The SSH protocol is recommended for remote login and remote file transfer. However, ssh is open to many attacks. See how to secure OpenSSH server:

#19: Install And Use Intrusion Detection System

A network intrusion detection system (NIDS) is an intrusion detection system that tries to detect malicious activity such as denial of service attacks, port scans or even attempts to crack into computers by monitoring network traffic.

It is a good practice to deploy any integrity checking software before system goes online in a production environment. If possible install AIDE software before the system is connected to any network. AIDE is a host-based intrusion detection system (HIDS) it can monitor and analyses the internals of a computing system.

Snort is a software for intrusion detection which is capable of performing packet logging and real-time traffic analysis on IP networks.

#20: Protecting Files, Directories and Email

Linux offers excellent protections against unauthorized data access. File permissions and MAC prevent unauthorized access from accessing data. However, permissions set by the Linux are irrelevant if an attacker has physical access to a computer and can simply move the computer’s hard drive to another system to copy and analyze the sensitive data. You can easily protect files, and partitons under Linux using the following tools:

#20.1: Securing Email Servers

You can use SSL certificates and gpg keys to secure email communication on both server and client computers:

Other Recommendation:

Recommend readings:

  1. Red Hat Enterprise Linux – Security Guide.
  2. Linux security cookbook- A good collections of security recipes for new Linux admin.
  3. Snort 2.1 Intrusion Detection, Second Edition – Good introduction to Snort and Intrusion detection under Linux.
  4. Hardening Linux – Hardening Linux identifies many of the risks of running Linux hosts and applications and provides practical examples and methods to minimize those risks.
  5. Linux Security HOWTO.

Posted in UNIX-LINUX | Tagged: | Leave a Comment »

20 Linux Server Hardening Security Tips (P1)

Posted by ZyK on 11/23/2011

Securing your Linux server is important to protect your data, intellectual property, and time, from the hands of crackers (hackers). The system administrator is responsible for security Linux box. In this first part of a Linux server security series, I will provide 20 hardening tips for default installation of Linux system.

#1: Encrypt Data Communication

All data transmitted over a network is open to monitoring. Encrypt transmitted data whenever possible with password or using keys / certificates.

  1. Use scp, ssh, rsync, or sftp for file transfer. You can also mount remote server file system or your own home directory using special sshfs and fuse tools.
  2. GnuPG allows to encrypt and sign your data and communication, features a versatile key managment system as well as access modules for all kind of public key directories.
  3. Fugu is a graphical frontend to the commandline Secure File Transfer application (SFTP). SFTP is similar to FTP, but unlike FTP, the entire session is encrypted, meaning no passwords are sent in cleartext form, and is thus much less vulnerable to third-party interception. Another option is FileZilla – a cross-platform client that supports FTP, FTP over SSL/TLS (FTPS), and SSH File Transfer Protocol (SFTP).
  4. OpenVPN is a cost-effective, lightweight SSL VPN.
  5. Lighttpd SSL (Secure Server Layer) Https Configuration And Installation
  6. Apache SSL (Secure Server Layer) Https (mod_ssl) Configuration And Installation

#1.1: Avoid Using FTP, Telnet, And Rlogin / Rsh

Under most network configurations, user names, passwords, FTP / telnet / rsh commands and transferred files can be captured by anyone on the same network using a packet sniffer. The common solution to this problem is to use either OpenSSH , SFTP, or FTPS (FTP over SSL), which adds SSL or TLS encryption to FTP. Type the following command to delete NIS, rsh and other outdated service:
# yum erase inetd xinetd ypserv tftp-server telnet-server rsh-serve

#2: Minimize Software to Minimize Vulnerability

Do you really need all sort of web services installed? Avoid installing unnecessary software to avoid vulnerabilities in software. Use the RPM package manager such as yum or apt-get and/or dpkg to review all installed set of software packages on a system. Delete all unwanted packages.
# yum list installed
# yum list packageName
# yum remove packageName

OR
# dpkg --list
# dpkg --info packageName
# apt-get remove packageName

#3: One Network Service Per System or VM Instance

Run different network services on separate servers or VM instance. This limits the number of other services that can be compromised. For example, if an attacker able to successfully exploit a software such as Apache flow, he / she will get an access to entire server including other services such as MySQL, e-mail server and so on. See how to install Virtualization software:

#4: Keep Linux Kernel and Software Up to Date

Applying security patches is an important part of maintaining Linux server. Linux provides all necessary tools to keep your system updated, and also allows for easy upgrades between versions. All security update should be reviewed and applied as soon as possible. Again, use the RPM package manager such as yum and/or apt-get and/or dpkg to apply all security updates.
# yum update
OR
# apt-get update && apt-get upgrade
You can configure Red hat / CentOS / Fedora Linux to send yum package update notification via email. Another option is to apply all security updates via a cron job. Under Debian / Ubuntu Linux you can use apticron to send security notifications.

#5: Use Linux Security Extensions

Linux comes with various security patches which can be used to guard against misconfigured or compromised programs. If possible use SELinux and other Linux security extensions to enforce limitations on network and other programs. For example, SELinux provides a variety of security policies for Linux kernel.

#5.1: SELinux

I strongly recommend using SELinux which provides a flexible Mandatory Access Control (MAC). Under standard Linux Discretionary Access Control (DAC), an application or process running as a user (UID or SUID) has the user’s permissions to objects such as files, sockets, and other processes. Running a MAC kernel protects the system from malicious or flawed applications that can damage or destroy the system. See the official Redhat documentation which explains SELinux configuration.

#6: User Accounts and Strong Password Policy

Use the useradd / usermod commands to create and maintain user accounts. Make sure you have a good and strong password policy. For example, a good password includes at least 8 characters long and mixture of alphabets, number, special character, upper & lower alphabets etc. Most important pick a password you can remember. Use tools such as “John the ripper” to find out weak users passwords on your server. Configure pam_cracklib.so to enforce the password policy.

#6.1: Password Aging

The chage command changes the number of days between password changes and the date of the last password change. This information is used by the system to determine when a user must change his/her password. The /etc/login.defs file defines the site-specific configuration for the shadow password suite including password aging configuration. To disable password aging, enter:
chage -M 99999 userName
To get password expiration information, enter:
chage -l userName
Finally, you can also edit the /etc/shadow file in the following fields:

{userName}:{password}:{lastpasswdchanged}:{Minimum_days}:{Maximum_days}:{Warn}:{Inactive}:{Expire}:

Where,

  1. Minimum_days: The minimum number of days required between password changes i.e. the number of days left before the user is allowed to change his/her password.
  2. Maximum_days: The maximum number of days the password is valid (after that user is forced to change his/her password).
  3. Warn : The number of days before password is to expire that user is warned that his/her password must be changed.
  4. Expire : Days since Jan 1, 1970 that account is disabled i.e. an absolute date specifying when the login may no longer be used.

I recommend chage command instead of editing the /etc/shadow by hand:
# chage -M 60 -m 7 -W 7 userName
Recommend readings:

#6.2: Restricting Use of Previous Passwords

You can prevent all users from using or reuse same old passwords under Linux. The pam_unix module parameter remember can be used to configure the number of previous passwords that cannot be reused.

#6.3: Locking User Accounts After Login Failures

Under Linux you can use the faillog command to display faillog records or to set login failure limits. faillog formats the contents of the failure log from /var/log/faillog database / log file. It also can be used for maintains failure counters and limits.To see failed login attempts, enter:
faillog
To unlock an account after login failures, run:
faillog -r -u userName
Note you can use passwd command to lock and unlock accounts:
# lock account
passwd -l userName
# unlocak account
passwd -u userName

#6.4: How Do I Verify No Accounts Have Empty Passwords?

Type the following command
# awk -F: '($2 == "") {print}' /etc/shadow
Lock all empty password accounts:
# passwd -l accountName

#6.5: Make Sure No Non-Root Accounts Have UID Set To 0

Only root account have UID 0 with full permissions to access the system. Type the following command to display all accounts with UID set to 0:
# awk -F: '($3 == "0") {print}' /etc/passwd
You should only see one line as follows:

root:x:0:0:root:/root:/bin/bash

If you see other lines, delete them or make sure other accounts are authorized by you to use UID 0.

#7: Disable root Login

Never ever login as root user. You should use sudo to execute root level commands as and when required. sudo does greatly enhances the security of the system without sharing root password with other users and admins. sudo provides simple auditing and tracking features too.

#8: Physical Server Security

You must protect Linux servers physical console access. Configure the BIOS and disable the booting from external devices such as DVDs / CDs / USB pen. Set BIOS and grub boot loader password to protect these settings. All production boxes must be locked in IDCs (Internet Data Center) and all persons must pass some sort of security checks before accessing your server. See also:

#9: Disable Unwanted Services

Disable all unnecessary services and daemons (services that runs in the background). You need to remove all unwanted services from the system start-up. Type the following command to list all services which are started at boot time in run level # 3:
# chkconfig --list | grep '3:on'
To disable service, enter:
# service serviceName stop
# chkconfig serviceName off

#9.1: Find Listening Network Ports

Use the following command to list all open ports and associated programs:
netstat -tulpn
OR
nmap -sT -O localhost
nmap -sT -O server.example.com

Use iptables to close open ports or stop all unwanted network services using above service and chkconfig commands.

#9.2: See Also

#10: Delete X Windows

X Windows on server is not required. There is no reason to run X Windows on your dedicated mail and Apache web server. You can disable and remove X Windows to improve server security and performance. Edit /etc/inittab and set run level to 3. Finally, remove X Windows system, enter:
# yum groupremove "X Window System"

(cyberciti.biz)

Posted in UNIX-LINUX | Tagged: , | 1 Comment »

Unix, Linux, and variants

Posted by ZyK on 11/22/2011

Quick links

Unix ABCs
Linux ABCs
MS-DOS vs. Linux / Unix
Unix and Linux commands
Unix Top 10 commands
Unix Shortcuts
Linux variants
Unix variants
Unix and Linux history
Unix and Linux tips
Unix and Linux links
Linux and Unix news
Linux Q&A

Unix ABCs

Unix, which is not an acronym, was developed by some of the members of the Multics team at the bell labs starting in the late 1960‘s by many of the same people who helped create the C programming language. The Unix today, however, is not just the work of a couple of programmers. Many other organizations, institutes and various other individuals contributed significant additions to the system we now know today.

See additional Unix information and variants and information on the Unix variants page.

Linux ABCs

Developed by Linus Torvalds and further elaborated by a number of developers throughout the world, Linux (lee’nuhks/ or /li’nuks/,_not_/li:’nuhks) is a freely available multitasking and multiuser operating system. From the outset, Linux was placed under General Public License (GPL). The system can be distributed, used and expanded free of charge. In this way, developers have access to all the source codes, thus being able to integrate new functions easily or to find and eliminate programming bugs quickly. Thereby drivers for new adapters (SCSI controller, graphics cards, etc.) can be integrated very rapidly.

See additional Linux information and variants and information on the Linux variants page.

MS-DOS vs. Linux / Unix

If you are able to navigate using MS-DOS, you should be able to quickly pick up on the navigation of Linux and Unix. In the below chart is a listing of common MS-DOS commands with their Linux / Unix counterpart.

MS-DOS Linux / Unix
attrib chmod
backup tar
dir ls
cls clear
copy cp
del rm
deltree rm -R
rmdir
edit vi
pico
format fdformat / mount / umount
move / rename mv
type less <file>
cd cd
chdir
more < file more file
md mkdir
win startx

Linux / Unix Commands

  * See the Linux and Unix overview page for a brief description on all commands on one page.
A a2p | ac | alias | ar | arch | arp | as | at | awk
B basename | bash | bc | bdiff | bfs | bg | biff | break | bs | bye
C cal | calendar | cancel | cat | cc | cd | chdir | checkeq | checknr | chfn | chgrp | chkey | chmod | chown | chsh | cksum | clear | cls | cmp | col | comm | compress | continue | copy | cp | cpio | crontab | csh | csplit | ctags | cu | curl | cut
D date | dc | dd | df | deroff | dhclient | diff | dig | dircmp | dirname | dmesg | dos2unix | dpost | du
E echo | ed | edit | egrepelm | emacs | enable | env | eqn | ex | exit | expand | expr
F fc | fdisk | fg | fgrep | file | find | findsmb | finger | fmt | fold | for | foreach | fromdos | fsck | ftp
G getfacl | gprof | grep | groupadd | groupdel | groupmod | gunzip | gview | gvim | gzip
H halt | hash | hashstat | head | help | history | host | hostid | hostname
I id | ifconfig | ifdown | ifup | ip | init | isalist
J jobs | join
K keylogin | kill | ksh
L last | ld | ldd | less | lex | link | ln | lo | locate | login | logname | logout | lp | lpadmin | lpc | lpq | lpr | lprm | lpstat | ls
M mach | mail | mailcompat | mailx | make | man | merge | mesg | mii-tool | mkdir | mkfs | more | mount | mt | mv | myisamchk | mysql
N nc | neqn | netstat | newalias | newform | newgrp | nice | niscat | nischmod | nischown | nischttl | nisdefaults | nisgrep | nismatch | nispasswd | nistbladm | nmap | nohup | nroff | nslookup
O on | onintr | optisa
P pack | pagesize | passwd | paste | pax | pcat | perl | pg | pgrep | pico | pine | ping | pkill | poweroff | pr | priocntl | printf | ps | pvs | pwd
Q quit
R rcp | reboot | red | rehash | remsh | repeat | replace | rgview | rgvim | rlogin | rm | rmail | rmdir | rn | route | rpcinfo | rsh | rsync | rview | rvim
S s2p | sag | sar | scp | screen | script | sdiff | sed | sendmail | set | setenv | setfacl | settime | sftp | sh | shred | shutdown | sleep | slogin | smbclient | sort | spell | split | stat | stop | strip | stty | su | sudo | sysinfo | sysklogd
T tabs | tac | tail | talk | tar | tbl | tcopy | tcpdump | tee | telinit | telnet | test | time | timex | todos | top | touch | tput | tr | traceroute | tree | troff | tty
U ulumask | unalias | uname | uncompress | unhash | uniq | unmount | unpack | untar | until | unzip | uptime | useradd | userdel | usermod
V vacation | veditvgrind | vi | view | vim | vipw | vmstat
W w | wait | wc | wget | whereis | which  | while| who | whoami | whois | write
X X | xargs | xfd | xlsfonts | xset | xterm | xrdb
Y yacc | yes | yppasswd
Z zcat, zip, zipcloak, zipinfo, zipnote, zipsplit

* In addition to each of the above explanations, additional information about a specified command for your Unix or Linux variant can be found by using the man command.

Posted in UNIX-LINUX | Tagged: , | 1 Comment »

The Linux Cookbook: Tips and Techniques for Everyday Use

Posted by ZyK on 11/22/2011

  • Preface
    • Format of Recipes
    • Assumptions, Scope, and Exclusions
    • Typographical Conventions
    • Versions, Latest Edition, and Errata
    • Acknowledgments
  • PART ONE: Working with Linux
  • Introduction
    • Background and History
      • What’s Unix?
      • What’s Free Software?
      • What’s Open Source?
      • What’s Linux?
      • What’s Debian?
      • Unix and the Tools Philosophy
    • What to Try First
    • If You Need More Help
  • What Every Linux User Knows
    • Controlling Power to the System
      • Powering Up the System
      • Turning Off the System
    • Accounts and Privileges
      • Logging In to the System
      • Logging Out of the System
    • Console Basics
      • Switching between Consoles
      • Scrolling the Console Text
      • Keys for Console Manipulation
    • Running a Command
    • Changing Your Password
    • Listing User Activity
      • Listing Your Username
      • Listing Who Is on the System
      • Listing Who Is on and What They’re Doing
      • Listing the Last Times a User Logged In
    • Listing System Activity
      • Listing Your Current Processes
      • Listing All of a User’s Processes
      • Listing All Processes on the System
      • Listing Processes by Name or Number
    • Help Facilities
      • Finding the Right Tool for the Job
      • Listing a Description of a Program
      • Listing the Usage of a Tool
      • Reading a Page from the System Manual
      • Using the GNU Info System
      • Reading System Documentation and Help Files
  • The Shell
    • Keys for Command Line Editing
      • Passing Special Characters to Commands
      • Letting the Shell Complete What You Type
      • Repeating the Last Command You Typed
      • Running a List of Commands
    • Redirecting Input and Output
      • Redirecting Input to a File
      • Redirecting Output to a File
      • Redirecting Error Messages to a File
      • Redirecting Output to Another Command’s Input
    • Managing Jobs
      • Suspending a Job
      • Putting a Job in the Background
      • Putting a Job in the Foreground
      • Listing Your Jobs
      • Stopping a Job
    • Command History
      • Viewing Your Command History
      • Specifying a Command from Your History
    • Recording a Shell Session
    • Customizing Your Shell
      • Changing the Shell Prompt
      • Making a Command Alias
      • Adding to Your Path
      • Customizing Future Shells
  • The X Window System
    • Running X
      • Starting X
      • Stopping X
    • Running a Program in X
      • Specifying Window Size and Location
      • Specifying Window Colors
      • Specifying Window Font
      • Specifying Additional Window Attributes
    • Manipulating X Client Windows
      • Moving a Window
      • Resizing a Window
      • Destroying a Window
      • Minimizing a Window
      • Maximizing a Window
    • Moving around the Desktop
    • Running a Shell in X
    • Configuring X
      • Switching between Video Modes
      • Running X Clients Automatically
      • Changing the Root Window Parameters
      • Choosing a Window Manager
  • PART TWO: Files
  • Files and Directories
    • Naming Files and Directories
      • Making an Empty File
      • Making a Directory
      • Making a Directory Tree
    • Changing Directories
      • Changing to Your Home Directory
      • Changing to the Last Directory You Visited
      • Getting the Name of the Current Directory
    • Listing Directories
      • Listing File Attributes
      • Listing Directories Recursively
      • Listing Newest Files First
      • Listing Hidden Files
      • Listing Directories in Color
      • Listing Directory Tree Graphs
      • Additional Directory Listing Options
    • Copying Files and Directories
    • Moving Files and Directories
      • Changing File Names to Lowercase
      • Renaming Multiple Files with the Same Extension
    • Removing Files and Directories
      • Removing a File with a Strange Name
      • A Safe Way to Remove a File
    • Giving a File More than One Name
    • Specifying File Names with Patterns
    • Browsing Files
  • Sharing Files
    • Groups and How to Work in Them
      • Listing the Groups a User Belongs To
      • Listing the Members of a Group
    • File Ownership
      • Determining the Ownership of a File
      • Changing the Ownership of a File
    • Controlling Access to Files
      • Listing the Permissions of a File
      • Changing the Permissions of a File
      • Write-Protecting a File
      • Making a File Private
      • Making a File Public
      • Making a File Executable
  • Finding Files
    • Finding All Files That Match a Pattern
    • Finding Files in a Directory Tree
      • Finding Files in a Directory Tree by Name
      • Finding Files in a Directory Tree by Size
      • Finding Files in a Directory Tree by Modification Time
      • Finding Files in a Directory Tree by Owner
      • Running Commands on the Files You Find
      • Finding Files by Multiple Criteria
    • Finding Files in Directory Listings
      • Finding the Largest Files in a Directory
      • Finding the Smallest Files in a Directory
      • Finding the Smallest Directories
      • Finding the Largest Directories
      • Finding the Number of Files in a Listing
    • Finding Where a Command Is Located
  • Managing Files
    • Determining File Type and Format
    • Changing File Modification Time
    • Splitting a File into Smaller Ones
    • Comparing Files
      • Determining Whether Two Files Differ
      • Finding the Differences between Files
      • Patching a File with a Difference Report
    • Compressed Files
      • Compressing a File
      • Decompressing a File
    • File Archives
      • Creating a File Archive
      • Listing the Contents of an Archive
      • Extracting Files from an Archive
    • Tracking Revisions to a File
      • Checking In a File Revision
      • Checking Out a File Revision
      • Viewing a File’s Revision Log
  • PART THREE: Text
  • Viewing Text
    • Perusing Text
      • Perusing a Text File
      • Perusing Multiple Text Files
      • Commands Available While Perusing Text
    • Outputting Text
      • Showing Non-printing Characters
      • Outputting a Beginning Part of a Text
      • Outputting an Ending Part of a Text
      • Outputting a Middle Part of a Text
      • Outputting the Text between Strings
      • Outputting Text in a Dialect
    • Streaming Text
    • Viewing a Character Chart
  • Text Editing
    • Choosing the Perfect Text Editor
    • Emacs
      • Getting Acquainted with Emacs
      • Basic Emacs Editing Keys
      • Making Abbreviations in Emacs
      • Recording and Running Macros in Emacs
      • Inserting Special Characters in Emacs
    • Running a Vi Tutorial
    • Selecting Text
      • Cutting Text
      • Pasting Text
    • Editing Streams of Text
    • Concatenating Text
      • Writing Text to Files
      • Appending Text to a File
      • Inserting Text at the Beginning of a File
    • Including Text Files
  • Grammar and Reference
    • Spelling
      • Finding the Correct Spelling of a Word
      • Listing the Misspellings in a Text
      • Keeping a Spelling Word List
      • Interactive Spell Checking
      • Spell Checking in Emacs
    • Dictionaries
      • Listing Words that Match a Pattern
      • Listing the Definitions of a Word
      • Listing the Synonyms of a Word
      • Listing the Antonyms of a Word
      • Listing the Hypernyms of a Word
      • Online Dictionaries
    • Checking Grammar
      • Checking Text for Misused Phrases
      • Checking Text for Doubled Words
      • Checking Text for Readability
      • Checking Text for Difficult Sentences
      • Checking Text for Long Sentences
    • Word Lists and Reference Files
  • Analyzing Text
    • Counting Text
      • Counting the Characters in a Text
      • Counting the Words in a Text
      • Counting the Lines in a Text
      • Counting the Occurrences of Something
      • Counting Lines per Page in Emacs
    • Making a Concordance of a Text
    • Text Relevance
      • Sorting Text in Order of Relevance
      • Listing Relevant Files in Emacs
    • Finding Anagrams in Text
    • Finding Palindromes in Text
    • Text Cut-Ups
      • Making Simple Text Cut-Ups
      • Making Random Word Cut-Ups
      • Making Cut-Ups in Emacs
  • Formatting Text
    • Spacing Text
      • Eliminating Extra Spaces in Text
      • Single-Spacing Text
      • Double-Spacing Text
      • Triple-Spacing Text
      • Adding Line Breaks to Text
      • Adding Margins to Text
      • Swapping Tab and Space Characters
    • Paginating Text
      • Placing Headers on Each Page
      • Placing Text in Columns
      • Options Available When Paginating Text
    • Underlining Text
    • Sorting Text
    • Numbering Lines of Text
    • Reversing Text
  • Searching Text
    • Searching for a Word or Phrase
    • Regular Expressions — Matching Text Patterns
      • Matching Lines Beginning with Certain Text
      • Matching Lines Ending with Certain Text
      • Matching Lines of a Certain Length
      • Matching Lines That Contain Any of Some Regexps
      • Matching Lines That Contain All of Some Regexps
      • Matching Lines That Don’t Contain a Regexp
      • Matching Lines That Only Contain Certain Characters
      • Finding Phrases Regardless of Spacing
      • Finding Patterns in Certain Contexts
      • Using a List of Regexps to Match From
      • Regexps for Common Situations
    • Searching More than Plain Text Files
      • Matching Lines in Compressed Files
      • Matching Lines in Web Pages
    • Outputting the Context of a Search
    • Searching and Replacing Text
    • Searching Text in Emacs
      • Searching Incrementally in Emacs
      • Searching for a Phrase in Emacs
      • Searching for a Regexp in Emacs
      • Searching and Replacing in Emacs
    • Searching Text in Less
  • Typesetting and Word Processing
    • Choosing the Right Typesetting System for the Job
    • Converting Plain Text for Output
      • Outputting Text in a Font
      • Outputting Text as a Poster or Sign
      • Outputting Text with Language Highlighting
      • Outputting Text with Fancy Headers
      • Outputting Text in Landscape Orientation
      • Outputting Multiple Copies of Text
      • Selecting the Pages of Text to Output
      • Additional PostScript Output Options
    • LyX Document Processing
      • Features of LyX
      • Writing Documents with LyX
      • Learning More about LyX
    • Typesetting with TeX and Friends
      • Is It a TeX or LaTeX File?
      • Processing TeX Files
      • Processing LaTeX Files
      • Writing Documents with TeX and LaTeX
      • TeX and LaTeX Document Templates
    • Writing Documents with SGMLtools
      • Elements of an SGML Document
      • Checking SGML Document Syntax
      • Generating Output from SGML
    • Other Word Processors and Typesetting Systems
  • Fonts
    • X Fonts
      • Selecting an X Font Name
      • Listing Available X Fonts
      • Displaying the Characters in an X Font
      • Resizing the Xterm Font
    • Console Fonts
      • Setting the Console Font
      • Displaying the Characters in a Console Font
    • Text Fonts
      • Horizontal Text Fonts
      • Making a Text Banner
    • Other Font Tools
  • PART FOUR: Images
  • Viewing Images
    • Previewing Print Files
      • Previewing a DVI File
      • Previewing a PostScript File
      • Previewing a PDF File
    • Viewing an Image in X
      • Browsing Image Collections in X
      • Putting an Image in the Root Window
    • Browsing Images in a Console
    • Viewing an Image in a Web Browser
    • Browsing PhotoCD Archives
    • Additional Image Viewers
  • Editing Images
    • Transforming Images
      • Changing the Size of an Image
      • Rotating an Image
      • Adjusting the Colors of an Image
      • Annotating an Image
      • Adding Borders to an Image
      • Making an Image Montage
      • Combining Images
      • Morphing Two Images Together
    • Converting Images between Formats
    • Editing Images with the GIMP
    • Interactive Image Editors and Tools
  • Importing Images
    • Taking Screen Shots
      • Taking a Screen Shot in X
      • Taking a Screen Shot in a Console
    • Scanning Images
      • Listing Available Scanner Devices
      • Testing a Scanner
      • Scanning an Image
    • Extracting PhotoCD Images
      • Converting a PhotoCD Image
      • Removing PhotoCD Haze
  • PostScript
    • Manipulating PostScript Pages
      • Extracting DVI Pages to PostScript
      • Extracting Pages from a PostScript File
      • Combining PostScript Pages
      • Arranging PostScript Pages in Signatures
    • Manipulating PostScript Documents
      • Resizing a PostScript Document
      • Combining PostScript Documents
      • Arranging a PostScript Document in a Booklet
    • Converting PostScript
      • Converting PostScript to PDF
      • Converting PostScript to Plain Text
  • PART FIVE: Sound
  • Sound Files
    • Sound File Formats
    • Adjusting the Audio Controls
      • Listing the Current Audio Settings
      • Changing the Volume Level
      • Muting an Audio Device
      • Selecting an Audio Recording Source
    • Playing a Sound File
      • Playing an MP3 File
      • Playing a MIDI File
    • Recording a Sound File
    • Other Sound File Tools
  • Audio Compact Discs
    • Controlling CD Audio
      • Playing an Audio CD
      • Pausing an Audio CD
      • Stopping an Audio CD
      • Shuffling Audio CD Tracks
      • Displaying Information about an Audio CD
      • Ejecting an Audio CD
    • Sampling Sound from a CD
    • Writing an Audio CD-R
    • Other Audio CD Applications
  • Editing Sound Files
    • Working with Selections from Sound Files
      • Cutting Out Part of a Sound File
      • Pasting a Selection of Sound
      • Mixing Sound Files Together
    • Sound Effects
      • Changing the Amplitude of a Sound File
      • Changing the Sampling Rate of a Sound File
      • Adding Reverb to a Sound File
      • Adding Echo to a Sound File
      • Adding Flange to a Sound File
      • Adding Phase to a Sound File
      • Adding Chorus to a Sound File
      • Adding Vibro-Champ Effects to a Sound File
      • Reversing a Sound File
    • Converting Sound Files
      • Making an MP3 File
      • Converting MP3 to Another Format
    • Other Tools for Sound Editing
  • PART SIX: Productivity
  • Disk Storage
    • Listing a Disk’s Free Space
    • Listing a File’s Disk Usage
    • Floppy Disks
      • Formatting a Floppy Disk
      • Mounting a Floppy Disk
      • Unmounting a Floppy Disk
    • CD-ROMs
      • Mounting a CD-ROM
      • Unmounting a CD-ROM
  • Printing
    • Making and Managing Print Jobs
      • Sending a Print Job to the Printer
      • Printing Multiple Copies of a Job
      • Listing Your Print Jobs
      • Cancelling a Print Job
    • More Recipes for Printing
      • Printing in Emacs
      • Printing with Dvips
      • Printing the Contents of an Xterm Window
    • Preparing Files for Printing
      • Preparing a PostScript File for Printing
      • Preparing a DVI File for Printing
      • Preparing a PDF File for Printing
      • Preparing a Man Page for Printing
  • Cross-Platform Conversions
    • Using DOS and Windows Disks
      • Listing the Contents of a DOS Disk
      • Copying Files to and from a DOS Disk
      • Deleting Files on a DOS Disk
      • Formatting a DOS Disk
    • Using Macintosh Disks
      • Specifying the Macintosh Disk to Use
      • Listing the Contents of a Macintosh Disk
      • Copying Files to and from a Macintosh Disk
      • Deleting Files on a Macintosh Disk
      • Formatting a Macintosh Disk
    • Converting Text Files between DOS and Linux
    • Converting Microsoft Word Files
      • Converting Word to LaTeX
      • Converting Word to Plain Text
  • Reminders
    • Displaying the Date and Time
    • Playing an Audible Time Announcement
    • Calendars
      • Displaying a Calendar
      • Displaying a Calendar in Emacs
    • Managing Appointments
      • Making an Appointment File
      • Including Holidays in Your Reminders
      • Automatic Appointment Delivery
    • Contact Managers
      • Keeping a Free-Form Address List
      • Keeping a Contact Manager Database
    • Reminding Yourself of Things
      • Sending Yourself Email Reminders
      • Reminding Yourself When You Have to Leave
      • Running a Command on a Delay
  • Mathematics
    • Calculating Arithmetic
      • Making a Quick Arithmetic Calculation
      • Making Many Arithmetic Calculations
    • Outputting a Random Number
    • Listing a Sequence of Numbers
    • Finding Prime Factors
    • Converting Numbers
      • Converting an Amount between Units of Measurement
      • Converting an Arabic Numeral to English
    • Other Math Tools
  • PART SEVEN: Networking
  • Communications
    • Connecting to the Internet
      • Setting Up PPP
      • Controlling a PPP Connection
    • Faxing
      • Sending a Fax
      • Receiving a Fax
      • Receiving Faxes Automatically
      • Converting to and from Fax Format
    • Calling Out on a Modem
  • Email
    • Sending Mail
      • Mailing a User on the Same System
      • Mailing a File or the Output of a Command
      • Mailing the Contents of a URL
      • Special Mail Composition Keystrokes
    • Receiving Mail
      • Deleting Mail
      • Options Available while Reading Mail
    • Managing Mail
      • Viewing a Mail Folder
      • Setting Notification for New Mail
      • Counting How Many Messages You Have
      • Seeing Who Your Mail Is From
      • Verifying an Email Address
    • Mail Attachments
      • Reading a Mail Attachment
      • Sending a Mail Attachment
    • Making an Email Signature
    • Picking the Right Mail Application
  • The World Wide Web
    • Browsing the Web
      • Maintaining a List of Visited Web Sites
      • Opening a URL from a Script
      • Mozilla Browsing Tips
    • Viewing an Image from the Web
    • Reading Text from the Web
      • Perusing Text from the Web
      • Viewing a Site That Requires Authorization
      • Options Available while Browsing Text
    • Browsing the Web in Emacs
    • Getting Files from the Web
      • Saving a URL to a File
      • Archiving an Entire Web Site
      • Archiving Part of a Web Site
      • Reading the Headers of a Web Page
    • Writing HTML
      • Adding Parameters to Image Tags
      • Converting HTML to Another Format
      • Validating an HTML File
    • More Web Browsers and Tools
  • Other Internet Services
    • Connecting to Another System
      • Suspending a Connection with Another System
      • Connecting to Another System with Encryption
    • Transferring Files with Another System
      • Uploading a File
      • Downloading a File
    • Reading Usenet
      • Choosing a Newsreader
      • Finding Newsgroups for a Topic
    • Listing Online System and User Activity
      • Checking Whether a System Is Online
      • Checking Whether a User Is Online
      • Listing Who Is Logged In to a System
      • Finding the IP Address of a Host Name
      • Finding the Host Name of an IP Address
      • Listing the Owner of a Domain Name
    • Sending a Message to Another User’s Terminal
    • Chatting with Other Users
      • Chatting Directly with a User
      • Chatting with Users on IRC
      • Chatting with Users on ICQ
  • Administrative Issues
    • Linux and Hardware Compatibility
    • Shutting Down the System
      • Shutting Down Immediately
      • Shutting Down at a Certain Time
      • Cancelling a Shutdown
    • Installing Software
      • Getting and Installing Debian
      • Installing a Debian Package
      • Upgrading a Debian Package
      • Installing a Shell Script
    • Administrating Users
      • Making a User Account
      • Letting Users Access Hardware Peripherals
      • Letting Users Mount Drives
    • Displaying Information about the System
      • How Long Has the System Been Up?
      • What Version of Linux Am I Running?
      • What Version of Debian Am I Running?
  • Linux Resources on the Web
  • License
    • Design Science License
    • Applying Copyleft to Your Work
  • Program Index
  • Concept Index

(http://dsl.org/cookbook/cookbook_toc.html)

 

Posted in UNIX-LINUX | Tagged: , | Leave a Comment »

 
Follow

Get every new post delivered to your Inbox.