Friday, August 13, 2010

Linux Kernel panic reboot

By default after a kernel panic Linux just waits there for a sysadmin to hit the restart or powercycle button.  This is because of the value set on "kernel.panic" parameter.

[root@linux23 ~]# cat /proc/sys/kernel/panic
[root@linux23 ~]# sysctl -a | grep kernel.panic
kernel.panic = 0
[root@linux23 ~]#

To disable this and make the Linux OS reboot after a kernel panic, we have to set an integer N greater than zero to the paramter "kernel.panic", where "N" is the number of seconds to wait before a automatic reboot.  For example , if you set N = 10 , then the system waits for 10 seconds before automatic reboot. To make this permanent, edit /etc/sysctl.conf and set it.

[root@linux23 ~]# echo "10" > /proc/sys/kernel/panic
[root@linux23 ~]# grep kernel.panic /etc/sysctl.conf
kernel.panic = 10
[root@linux23 ~]#

Disable ipv6 on RHEL 4 and 5

Edit /etc/sysconfig/network and change
Edit /etc/modprobe.conf and add these lines (if they're not in it):
alias net-pf-10 off
alias ipv6 off

Stop the ipv6tables service by typing:
service ip6tables stop
Disable the ipv6tables service by typing:
chkconfig ip6tables off
After these changes, IPv6 will be disabled after the next reboot of your system.

Why is there no free RAM in Linux? or Why Memory usage is 100% in Linux?

Traditional Unix tools like 'top' often report a surprisingly small amount of free memory after a system has been running for a while. For instance, after about 3 hours of uptime, the machine I'm writing this on reports under 60 MB of free memory, even though I have 512 MB of RAM on the system. Where does it all go?

The biggest place it's being used is in the disk cache, which is currently over 290 MB. This is reported by top as "cached". Cached memory is essentially free, in that it can be replaced quickly if a running (or newly starting) program needs the memory.

The reason Linux uses so much memory for disk cache is because the RAM is wasted if it isn't used. Keeping the cache means that if something needs the same data again, there's a good chance it will still be in the cache in memory. Fetching the information from there is around 1,000 times quicker than getting it from the hard disk. If it's not found in the cache, the hard disk needs to be read anyway, but in that case nothing has been lost in time.

To see a better estimation of how much memory is really free for applications to use, run the command:
free m

The -m option stands for megabytes, and the output will look something like this:
             total       used       free     shared    buffers     cached
Mem:           503        451         52          0         14        293
-/+ buffers/cache:        143        360
Swap:         1027          0       1027

The -/+ buffers/cache line shows how much memory is used and free from the perspective of the applications. Generally speaking, if little swap is being used, memory usage isn't impacting performance at all.

Notice that I have 512 MB of memory in my machine, but only 503 is listed as available by free. This is mainly because the kernel can't be swapped out, so the memory it occupies could never be freed. There may also be regions of memory reserved for/by the hardware for other purposes as well, depending on the system architecture.

The difference among VIRT, RES, and SHR in top output
VIRT stands for the virtual size of a process, which is the sum of memory it is actually using, memory it has mapped into itself (for instance the video card's RAM for the X server), files on disk that have been mapped into it (most notably shared libraries), and memory shared with other processes. VIRT represents how much memory the program is able to access at the present moment.

RES stands for the resident size, which is an accurate representation of how much actual physical memory a process is consuming. (This also corresponds directly to the %MEM column.) This will virtually always be less than the VIRT size, since most programs depend on the C library.

SHR indicates how much of the VIRT size is actually sharable (memory or libraries). In the case of libraries, it does not necessarily mean that the entire library is resident. For example, if a program only uses a few functions in a library, the whole library is mapped and will be counted in VIRT and SHR, but only the parts of the library file containing the functions being used will actually be loaded in and be counted under RES.

The difference between buffers and cache
Buffers are associated with a specific block device, and cover caching of filesystem metadata as well as tracking in-flight pages. The cache only contains parked file data. That is, the buffers remember what's in directories, what file permissions are, and keep track of what memory is being written from or read to for a particular block device. The cache only contains the contents of the files themselves.

Corrections and additions to this section welcome; I've done a bit of guesswork based on tracing how /proc/meminfo is produced to arrive at these conclusions.

Swappiness (2.6 kernels)
Since 2.6, there has been a way to tune how much Linux favors swapping out to disk compared to shrinking the caches when memory gets full.

When an application needs memory and all the RAM is fully occupied, the kernel has two ways to free some memory at its disposal: it can either reduce the disk cache in the RAM by eliminating the oldest data or it may swap some less used portions (pages) of programs out to the swap partition on disk. It is not easy to predict which method would be more efficient.
The kernel makes a choice by roughly guessing the effectiveness of the two methods at a given instant, based on the recent history of activity.

Before the 2.6 kernels, the user had no possible means to influence the calculations and there could happen situations where the kernel often made the wrong choice, leading to thrashing and slow performance. The addition of swappiness in 2.6 changes this.

Swappiness takes a value between 0 and 100 to change the balance between swapping applications and freeing cache. At 100, the kernel will always prefer to find inactive pages and swap them out; in other cases, whether a swapout occurs depends on how much application memory is in use and how poorly the cache is doing at finding and releasing inactive items.

The default swappiness is 60. A value of 0 gives something close to the old behavior where applications that wanted memory could shrink the cache to a tiny fraction of RAM. For laptops which would prefer to let their disk spin down, a value of 20 or less is recommended.

As a sysctl, the swappiness can be set at runtime with either of the following commands:
# sysctl -w vm.swappiness=30
# echo 30 >/proc/sys/vm/swappiness

The default when Gentoo boots can also be set in /etc/sysctl.conf:
# Control how much the kernel should favor swapping out applications (0-100)
vm.swappiness = 30

Some patchsets allow the kernel to auto-tune the swappiness level as it sees fit; they may not keep a user-set value.

print commands in Linux

To see a list of available printers:

# lpstat -p -d

To print the file to printer "MyPrinter"

# lpr -P MyPrinter filename

To view (query) the print queue, use the lpq or lpstat command. When entered without arguments, it displays the contents of the default print queue.

# lpq
# lpstat

To list the default printer

# lpstat -d

To know the status of default printer

# lpstat -p

If the printer status is disabled, you need to enable the printer using "enable" command as follows.

# /usr/bin/enable <PrinterName>

To disable the printer

# /usr/bin/disable -c <PrinterName>

To reset the print queue, you have to disable and reenable as follows

# /usr/bin/disable -c <PrinterName>
# /usr/bin/enable <PrinterName>

To remove the print job

# lprm <job Number>

How can I unlock a SAN disk in AIX?

I got my LUN mapped to my system, but when I try to create my Volume Group with   mkvg -f vpath100   all I get is an I/O error. What can I do?
Probably there is still a SAN lock on the disk. Try to unlock it with:
 # lquerypv -ch /dev/vpath100
and retry to create your Volume Group.

How can I directly read out the VGDA of a PV (hdisk)?

Information about VGx, LVx, filesystems, etc. are stored in the ODM. But these information are also written to the VGDA of the disks itself. You can read the information directly from the disk's VGDA with a command like this:
 # lqueryvg -Atp hdisk100
You can use
 # redefinevg -d hdisk100 myvg
to synchronize the ODM with the information of the VGDA. You can also synchronize the VGDA with the information stored in the ODM:
 # synclvodm myvg

How to set a static routes in AIX

You can use the route command to set a static route. But this way you don't get it back after reboot.
To make a route persistent you need to change inet0. First check which routes are already set:
 # lsattr -El inet0 -a route
 route net,-hopcount,0,,0, Route True
 route net,-hopcount,,,,,,, Route True

These routes would be set with:
 # chdev -l inet0 -a route=net,-hopcount,0,,0,
 # chdev -l inet0 -a route=net,-hopcount,,,,,,,

To remove these specific static routes:
 # chdev -l inet0 -a delroute=net,-hopcount,0,,0,
 # chdev -l inet0 -a delroute=net,-hopcount,,,,,,,

In this route string is the netmask, the destination net, and the gateway.
For hostroutes the keyword net has to be replaced with host.

Difference between Ethernet Interface(en0) and Ethernet adapter (ent0)

AIX differentiates between hardware adapters, their interfaces and protocols standards associated.

To recognize interface card AIX uses three notations:
ent, en and et.
All are different and are described below and the sake of completeness I am using 0 at the end:
The notation ent0 is used to specify the hardware adapter. It has nothing to do with the TCP/IP address. The parameters associated with ent0 can be seen as below:
# lsattr –El ent0
It will show parameters related to card.
It shows adapter_names, alt_addr, auto_recovery, backup_adapter, hash_mode, mode, netaddr, noloss_failover, num_retries, retry_time, use_alt_addr, use_jumbo_frame.
en0 represents the interface associated with hardware adapter ent0. The notation en0 is used for Standard Ethernet(inet). The TCP/IP address is associated with this interface.
The parameters associated with en0 can be seen as below:
#lsattr –El en0
It'll shows all the parameters related with the interface en0 of the adapter ent0.
It shows alias4, alias6, arp, authority, broadcast=1500, mtu, netaddr, netaddr6, netmask, prefixlen, remmtu, rfc1323, security, state, tcp_mssdflt, tcp_nodelay, tcp_recvspace, tcp_sendspace.
Rest everything is same except mtu(Maximum Transfer Unit) value. Which is 1500 as per the standard ethernet protocol.
et0 represents the interface associated with hardware adapter ent0. The notation et0 is used for IEEE 802.3 Ethernet(inet). If you are using standard ethernet protocol then it will not have TCP/IP address.
The parameters associated with et0 can be seen as below:
#lsattr –El et0
It'll shows all the parameters related with the interface et0 of the adapter ent0.
It shows alias4, alias6, arp, authority, broadcast, mtu=1492, netaddr, netaddr6, netmask, prefixlen, remmtu, rfc1323, security, state, tcp_mssdflt, tcp_nodelay, tcp_recvspace, tcp_sendspace.
Note here as well that the MTU shown will be 1492 as per IEEE 802.3 standard. Rest all parameters will be same. Also, netaddr, netmask fields will be empty fr et0.

Are these terms interchangable or is there a difference between them? I always get confused with these terms. What does en0 and ent0 mean and the difference between these?

AIX differentiates between a network adapter and network interface:

Network adapter Represents the layer-2 device, for example, the Ethernet adapter ent0 has a MAC address, such as 06:56:C0:00:20:03

Network interface Represents the layer-3 device, for example the Ethernet interface en0 has an IP address, such as

Typically, a network interface is attached to a network adapter, for example, an Ethernet interface en0 is attached to an Ethernet adapter ent0.

There are also some network interfaces in AIX that are not attached to a network adapter, for example, the loopback interface lo0 or a Virtual IP Address (VIPA) interface, such as vi0, if defined.

Hhow to disable first time password change in AIX

Clear the ADMCHG flag of the user account with "pwdadm -c <user>".


# pwdadm –c username

Disable the Ctrl-Alt-Delete shutdown keys in Linux

On a production system it is recommended that you disable the [Ctrl]-[Alt]-[Delete] shutdown. It is configured using /etc/inittab (used by sysv-compatible init process) file. The inittab file describes which processes are started at bootup and during normal operation. You need to open this file and remove (or comment it) ctrlaltdel entry.
Ctrlaltdel specifies the process that will be executed when init receives the SIGINT signal. SIGINT is the symbolic name for the signal thrown by computer programs when a user wishes to interrupt the process, for example reboot/shutdown system using [Ctrl]-[Alt]-[Del].). This means that someone on the system console has pressed the CTRL-ALT-DEL key combination. Typically one wants to execute some sort of shutdown either to get into single-user level or to reboot the machine.

Disable CTRL+ALT+Del keys

Open /etc/inittab file, enter:
# vi /etc/inittab
Search for line that read as follows:
ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
And remove the line or comment out the above line by putting a hash mark (#) in front of it:
# ca:12345:ctrlaltdel:/sbin/shutdown -t1 -a -r now
Save the file and exit to shell promot. Reboot system to take effect or type command:
# init q

How to change Max length of username in aix

On AIX 5.3 to change default length of username:

# chdev -l sys0 -a max_logname=9
sys0 changed

To check the current length of username:

# lsattr -El sys0 -a max_logname
max_logname 9 Maximum login name length at boot time True

# getconf LOGIN_NAME_MAX 21

HMC Commands for Documenting LPAR Configuration

For power systems p servers that are HMC managed, there are some tools you can use to document your LPAR profile configurations that in conjunction with other traditional data such as AIX snap files or prtconf output can be useful should a server need to be manually rebuilt. The tools that can make the tasks of documenting the LPARs' configuration include system plan and some queries of the profile data. Queries can be run from the HMC command line and the text output does provide a compact way of describing profile definitions which can also be used to recreate profiles from scripts if desired. The system plan output file can be used to restore profiles automatically should the need ever arise.
Following are some simple queries you can run to document the LPAR configurations.

If you are running HMC v7r3.4 or higher then the following command should allow you to create a system plan that details the profile definitions.

mksysplan -f <filename.sysplan> -m <server name> --novios

The plan can be exported using the HMC GUI and used in case you have to manually rebuild your profiles or the plan might be used to automatically recreate the profiles should restoring profile data from other backup methods fail.

Another command you can use to gather your profile data in case there are issues with system plan follows.

lssyscfg -r prof -m <server name>

Some additional queries you can do if you have VIO servers that are active include following.

lshwres -m <server name> -r virtualio --rsubtype scsi

To get the name of your server to use in the above commands you can run

lssyscfg -r sys -F name

The data you get back from querying the profile data with the lssyscfg command and querying the virtual scsi data from lshwres can be saved along with any other server configuration data you might and would provide good documentation of your LPAR configurations should you ever need to manually rebuild a server the servers.

For more information regarding system plan or other HMC commands you can used the related topics links posted at the end of this tech-note.

Sunday, August 8, 2010

How to mount an ISO file in Linux

To mount an ISO file on Linux machine, you should mount it as "loop" back filesystem. The option to be used in mount command is "-o loop".

Check out the below example.

# mount -o loop boot.iso /mnt
# df /mnt
Filesystem           1K-blocks      Used Available Use% Mounted on
                        131708    131708         0 100% /mnt
# ls /mnt
images  isolinux

# mount | grep /mnt
/media/ boot.iso on /mnt type iso9660 (ro,loop=/dev/loop0)

Saturday, August 7, 2010

How to Disable ping to Linux server?

How to Disable ping to Linux server?



To disable ping

echo 1 > /proc/sys/net/ipv4/icmp_echo_ignore_all
To enable ping
echo 0 > /proc/sys/net/ipv4/icmp_echo_ignore_all

Enable remote desktop remotely

1. Run Regedit

2. Select File | Connect Network Registry

3. Enter the name of the remote computer and select Check Names

4. Go to hklm\system\currentcontrolset\control\terminal server\FdenyTSConnection=1

5. Change the FdenyTSConnection to 0

Now you should be able to connect to the remote computer using Remote Desktop.

6. Reboot the remote machine by issuing the following command in Command Prompt:

shutdown -m \\hostname –r

7. Remote Desktop for the remote computer has been enabled and listening on default Remote Desktop port for any incoming Remote Desktop Connection. For security reason, you may want to consider changing the Remote Desktop listening port. There is also plenty of freeware utility that allows user to remotely enable Remote Desktop without modifying registry.




psexec \\computername reg add "hklm\system\currentcontrolset\control\terminal server" /f /v fDenyTSConnections /t REG_DWORD /d 0

You can also look at the following URL:


Memory utilization on AIX systems typically runs around 100%. This is often a source of concern. However, high memory utilization in AIX does not imply the system is out of memory. By design, AIX leaves files it has accessed in memory. This significantly improves performance when AIX reaccesses these files because they can be reread directly from memory, not disk*. When AIX needs memory, it discards files using a "least used" algorithm. This generates no I/O and has almost no performance impact under normal circumstances.

Sustained paging activity is the best indication of low memory. Paging activity can be monitored using the "vmstat" command. If the "page-in" (PI) and "page-out" (PO) columns show non-zero values over "long" periods of time, then the system is short on memory. (All systems will show occasional paging, which is not a concern.)

Memory requirements for applications can be empirically determined using the AIX "rmss"command. The "rmss" command is a test tool that dynamically reduces usable memory. The onset of paging indicates an application's minimum memory requirement.

Finally, the "svmon" command can be used to list how much memory is used each process. The interpretation of the svmon output requires some expertise. See the AIX documentation for details. The vmo parameters also needs to be tuned.

How to stop crontab writing to var/spool/mail/root file

My crontab is writing to /var/spool/mail/root file... Is there anyway I can stop this? The crontab is running every minute and it would fill the mail pretty quickly.




Can i see the crontab rule ?? you must of put to sent email to root
If use the crontab rule like in below example, it will not send any email

*/5 17,18 * * 1-5 / >/dev/null 2>&1


At the top of your crontab file just add:
This will disable mail sending by crond.


Recommended Solution:

Try to set that in the script itself rather than the whole crontab file

Bandwidth vs Latency

Latency versus Bandwidth - What is it?

One of the most commonly misunderstood concepts in networking is speed and capacity. Most people believe that capacity and speed are the same thing. For example, it's common to hear "How fast is your connection?" Invariably, the answer will be "640K", "1.5M" or something similar. These answers are actually referring to the bandwidth or capacity of the service, not speed.

Speed and bandwidth are interdependent. The combination of latency and bandwidth gives users the perception of how quickly a webpage loads or a file is transferred. It doesn't help that broadband providers keep saying "get high speed access" when they probably should be saying "get high capacity access". Notice the term "Broadband" - it refers to how wide the pipe is, not how fast.


Latency is delay.

For our purposes, it is the amount of time it takes a packet to travel from source to destination. Together, latency and bandwidth define the speed and capacity of a network.

Latency is normally expressed in milliseconds. One of the most common methods to measure latency is the utility ping. A small packet of data, typically 32 bytes, is sent to a host and the RTT (round-trip time, time it takes for the packet to leave the source host, travel to the destination host and return back to the source host) is measured.

The following are typical latencies as reported by others of popular circuits type to the first hop. Please remember however that latency on the Internet is also affected by routing that an ISP may perform (ie, if your data packet has to travel further, latencies increase).

Ethernet .3ms
Analog Modem 100-200ms
ISDN 15-30ms
DSL/Cable 10-20ms
Stationary Satellite >500ms, mostly due to high orbital elevation

DS1/T1 2-5ms


Bandwidth is normally expressed in bits per second. It's the amount of data that can be transferred during a second.

Solving bandwidth is easier than solving latency. To solve bandwidth, more pipes are added. For example, in early analog modems it was possible to increase bandwidth by bonding two or more modems. In fact, ISDN achieves 128K of bandwidth by bonding two 64K channels using a datalink protocol called multilink-ppp.

Bandwidth and latency are connected. If the bandwidth is saturated then congestion occurs and latency is increased. However, if the bandwidth of a circuit is not at peak, the latency will not decrease. Bandwidth can always be increased but latency cannot be decreased. Latency is the function of the electrical characteristics of the circuit.

Bandwidth vs. Latency

There are two factors in online gaming when it comes to your internet connection: bandwidth and latency.  Let's review on what each of them means:

    * Bandwidth – The average rate of successful data transfer through a communications path (usually measured in KB/s or Mb/s – that's megabits, not megabytes!  See below) 

    * Latency – The time it takes for data to be passed from the source to a server and back again (usually measured in milliseconds)

It's the Latency, Stupid

Stuart Cheshire, May 1996.

(Revised periodically)

Copyright © Stuart Cheshire 1996-2001

Years ago David Cheriton at Stanford taught me something that seemed very obvious at the time -- that if you have a network link with low bandwidth then it's an easy matter of putting several in parallel to make a combined link with higher bandwidth, but if you have a network link with bad latency then no amount of money can turn any number of them into a link with good latency.

It's now many years later, and this obvious fact seems lost on the most companies making networking hardware and software for the home. I think it's time it was explained again in writing.

Fact One: Making more bandwidth is easy.

Imagine you live in a world where the only network connection you can get to your house is a 33kbit/sec modem running over a telephone line. Imagine that this is not enough for your needs. You have a problem.

The solution is easy. You can get two telephone lines, and use them together in parallel, giving you a total of 66kbit/sec. If you need even more you can get ten telephone lines, giving you 330kbit/sec. Sure, it's expensive, and having ten modems in a pile is inconvenient, and you may have to write your own networking software to share the data evenly between the ten lines, but if it was important enough to you, you could get it done.

It may not be cheap, but at least it's possible.

People with ISDN lines can already do this. It's called "bonding" and it uses two 56 (or 64) kbit/sec ISDN channels in parallel to give you a combined throughput of 112 (or 128) kbit/sec.

Fact Two: Once you have bad latency you're stuck with it.

If you want to transfer a large file over your modem it might take several seconds, or even minutes. The less data you send, the less time it takes, but there's a limit. No matter how small the amount of data, for any particular network device there's always a minimum time that you can never beat. That's called the latency of the device. For a typical Ethernet connection the latency is usually about 0.3ms (milliseconds -- thousandths of a second). For a typical modem link the latency is usually about 100ms, about 300 times worse than Ethernet.

If you wanted to send ten characters over your 33kbit/sec modem link you might think the total transmission time would be:

80 bits / 33000 bits per second = 2.4ms.

but it doesn't. It takes 102.4ms because of the 100ms latency introduced by the modems at each end of the link.

If you want to send a large amount of data, say 100K, then that takes 25 seconds, and the 100ms latency isn't very noticable, but if you want send a smaller amount of data, say 100bytes, then the latency is more than the transmission time.

Why would you care about this? Why do small pieces of data matter? For most end-users it's the time it takes to transfer big files that annoys them, not small files, so they don't even think about latency when buying products. In fact if you look at the boxes modems come in, they proudly proclaim "14.4 kbps", "28.8 kbps" and "33.6 kbps", but they don't mention the latency anywhere. What most end-users don't know is that in the process of transferring those big files their computers have to send back and forth hundreds of little control messages, so the performance of small data packets directly affects the performance of everything else they do on the network.

Now, imagine the same scenario as before. You live in a world where the only network connection you can get to your house is a modem running over a telephone line. Your modem has a latency of 100ms, but you're doing something that needs lower latency. Maybe you're trying to do computer audio over the net. 100ms may not sound like very much, but it's enough to cause a noticable delay and echo in voice communications, which makes conversation difficult. Maybe you're trying to play an interactive game over the net. The game only sends tiny amounts of data, but that 100ms delay is making the interactivity of the game decidedly sluggish.

What can you do about this?


You can compress the data, but it doesn't help. It was already small to start with, and that 100ms latency is still there.

You can get 80 phone lines in parallel, and send one single bit over each phone line, but that 100ms latency is still there.

Once you've got yourself a device with bad latency there's absolutely nothing you can do about it (except throw out the device and get something else).

Fact Three: Current consumer devices have appallingly bad latency.

A typical Ethernet card has a latency less than 1ms. The Internet backbone as a whole also has very good latency. Here's a real-world example:

  • The distance from Stanford to Boston is 4320km.
  • The speed of light in vacuum is 300 x 10^6 m/s.
  • The speed of light in fibre is roughly 66% of the speed of light in vacuum.
  • The speed of light in fibre is 300 x 10^6 m/s * 0.66 = 200 x 10^6 m/s.
  • The one-way delay to Boston is 4320 km / 200 x 10^6 m/s = 21.6ms.
  • The round-trip time to Boston and back is 43.2ms.
  • The current ping time from Stanford to Boston over today's Internet is about 85ms:
    [cheshire@nitro]$ ping -c 1
    PING ( 56 data bytes
    64 bytes from icmp_seq=0 ttl=238 time=84.5 ms
  • So: the hardware of the Internet can currently achieve within a factor of two of the speed of light.

So the Internet is doing pretty well. It may get better with time, but we know it can never beat the speed of light. In other words, that 85ms round-trip time to Boston might reduce a bit, but it's never going to beat 43ms. The speed's going to get a bit better, but it's not going to double. We're already within a factor of two of the theoretical optimum. I think that's pretty good. Not many technologies can make that claim.

Compare this with a modem. Suppose you're 18km from your ISP (Internet Service Provider). At the speed of light in fibre (or the speed of electricity in copper, which is about the same) the latency should be:

18000 / (180 x 10^6 m/s) = 0.1ms

The latency over your modem is actually over 100ms. Modems are currently operating at level that's 1000 times worse than the speed of light. They have a long way to go before they get close to what the rest of the Internet is achieving.

Of course no modem link is ever going to have a latency of 0.1ms. I'm not expecting that. The important issue is the total end-to-end transmission delay for a packet -- the time from the moment the software makes the call to send the packet to the moment the last bit of the packet arrives the destination and the packet delivered to the software at the receiving end. The total end-to-end transmission delay is made up of fixed latency (including the speed-of-light propagation delay), plus the transmission time. For a 36 byte packet the transmission time is 10ms (the time it takes to send 288 bits at a rate of 28800 bits per second). When the actual transmission time is about 10ms, working to make the latency 0.1ms would be silly. All that's needed is that the latency should not be so huge that it completely overshadows the transmission time. For a modem that has a transmission rate of 28.8kb/s, a sensible latency target to aim for is about 5ms.

Fact Four: Making limited bandwidth go further is easy.

If you know you have limited bandwidth, there are many techniques you can use to reduce the problem.


If you know you have limited bandwidth, compression is one easy solution.

You can apply general purpose compression, such as gzip, to the data.

Even better, you can apply data-specific compression, because that gets much higher compression ratios. For example, still pictures can be compressed with JPEG, Wavelet compression, or GIF. Moving pictures can be compressed with MPEG, Motion JPEG, Cinepak, or one of the other QuickTime codecs. Audio can be compressed with uLaw, and English text files can be compressed with dictionary-based compression algorithms.

All of these compression techniques trade off use of CPU power in exchange for lower bandwidth requirements. There's no equivalent way to trade off use of extra CPU power to make up for poor latency.

All modern modems have compression algorithms built-in. Unfortunately, having your modem do compression is nowhere near as good as having your computer do it. Your computer has a powerful, expensive, fast processor in it. Your modem has a feeble, cheap, slow processor in it. There's no way your modem can compress data as well or as quickly as your computer can. In addition, in order to compress data, your modem has to hold on to the data until it has a block that's big enough to compress effectively. That adds latency, and once added, there's no way for you to get rid of latency. Also, the modem doesn't know what kind of data you are sending, so it can't use the superior data-specific compression algorithms. In fact, since most images and sounds on Web pages are compressed already, the modem's attempts to compress the data a second time is futile, and just adds more latency without giving any benefit.

This is not to say that having a modem do compression never helps. In the case where the host software at the endpoints is not very smart, and doesn't compress its data appropriately, then the modem's own compression can compensate somewhat for that deficiency and improve throughput. The point is that modem compression only helps dumb software, and it actually hurts smart software by adding extra delay. For someone planning to write dumb software this is no problem. For anyone planning to write smart software this should be a big cause for concern.

Bandwidth-conscious code

Another way to cope with limited bandwidth is to write programs that take care not to waste bandwidth.

For example, to reduce packet size, wherever possible Bolo uses bytes instead of 16-bit or 32-bit words.

For many kinds of interactive software like games, it's not important to carry a lot of data. What's important is that when the little bits of data are delivered, they are delivered quickly. Bolo was originally developed running over serial ports at 4800 bps and could support 8 players that way. Over 28.8 modems it can barely support 2 players with acceptable response time. Why? A direct-connect serial port at 4800 bps has a transmission delay of 2ms per byte, and a latency that is also 2ms. To deliver a typical ten byte Bolo packet takes 22ms. A 28800 bps modem has transmission delay of 0.28ms per byte, but a latency of 100ms, 50 times worse than the 4800 bps serial connection. Over the 28.8 modem, it takes 103ms to deliver a ten byte packet.

Send less data

A third way to cope with limited bandwidth is simply to send less data.

If you don't have enough bandwidth to send high resolution pictures, you can use lower resolution.

If you don't have enough bandwidth to send colour images, you can send black and white images, or send images with dramatically reduced colour detail (which is actually what NTSC television does).

If you don't have enough bandwidth to send 30 frames per second, you can send 15fps, or 5fps, or fewer.

Of course these tradeoffs are not pleasant, but they are possible. You can either choose to pay more money to run multiple circuits in parallel for more bandwidth, or you can choose to send less data to stay within the limited bandwidth you have available.

If the latency is not good enough to meet your needs you don't have the same option. Running multiple circuits in parallel won't make your latency any better, and sending less data won't improve it either.


One of the most effective techniques throughout all areas of computer science is caching, and that is just as true in networking.

If visit a web site, your Web browser can keep a copy of the text and images on your computer's hard disk. If you visit the same site again, all your Web browser has to do check that the copies it has stored are up to date -- i.e. check that the copies on the Web server haven't been changed since the date and time the previous copies were downloaded and cached on the local disk.

Checking the date and time a file was last modified is a tiny request to send across the network. This kind of request is so small that the throughput of your modem makes no difference -- latency is all that matters.

Recently companies have started providing CDROMs of entire Web sites to speed Web browsing. When browsing these Web sites, all the Web browser has to do is check the modification date of each file it accesses to make sure that the copy on the CDROM is up to date. It only has to download files that have changed since the CDROM was made. Since most of the large files on a Web site are images, and since images on a Web site change far less frequently than the HTML text files, in most cases very little data has to be transferred.

Since for the most part the Web browser is only doing small modification date queries to the Web server, the performance the user experiences is entirely dominated by the latency of the connection, and the throughput is virtually irrelevant.

Another analogy

Even smart people have trouble fully grasping the implications of these latency issues. It's subtle stuff.

The Cable TV industry is hyping "cable modems" right now, claiming that they're "1000 times 'faster' than a telephone modem." Given the lack of public awareness of the importance of latency, I wouldn't be in the least surprised if many of them have latency that is just as bad, or maybe even worse, than telephone modems. (The results from some early prototype cable modems, however, look quite promising. Lets hope the production ones are as good.)

Another device in a similar position is the DirecPC satellite dish, which is supposed to be "14 times faster than a 28.8 modem". Is it really? Here are some excerpts of what Lawrence Magid had to say about it in his article in the San Jose Mercury News (2nd February 1997):

The system is expensive, requires a relatively elaborate installation and configuration and, in the end, doesn't necessarily speed up your access to the World Wide Web.

I set up two nearly identical PCs side by side. One was connected to the Net at 28.8kbps and the other with DirecPC. In most cases the satellite system displayed Web pages a bit faster than the one with a modem, but not by much.

In some cases, the modem-equipped PC was faster, especially with sites that don't have a great deal of graphics.

Alluring as its promise may be, DirecPC for now doesn't offer spectacular advantages for normal Web surfing, even though it does carry a spectacular price.

Do we see a pattern starting to emerge yet?

Part of the problem here is misleading use of the word "faster".

Would you say that a Boeing 747 is three times "faster" than a Boeing 737? Of course not. They both cruise at around 500 miles per hour. The difference is that the 747 carries 500 passengers where as the 737 only carries 150. The Boeing 747 is three times bigger than the Boeing 737, not faster.

Now, if you wanted to go from New York to London, the Boeing 747 is not going to get you there three times faster. It will take just as long as the 737.

In fact, if you were really in a hurry to get to London quickly, you'd take Concorde, which cruises around 1350 miles per hour. It only seats 100 passengers though, so it's actually the smallest of the three. Size and speed are not the same thing.

On the other hand, If you had to transport 1500 people and you only had one aeroplane to do it, the 747 could do it in three trips where the 737 would take ten, so you might say the Boeing 747 can transport large numbers of people three times faster than a Boeing 737, but you would never say that a Boeing 747 is three times faster than a Boeing 737.

That's the problem with communications devices today. Manufacturers say "speed" when they mean "capacity". The other problem is that as far as the end-user is concerned, the thing they want to do is transfer large files quicker. It may seem to make sense that a high-capacity slow link might be the best thing for the job. What the end-user doesn't see is that in order to manage that file transfer, their computer is sending dozens of little control messages back and forth. The thing that makes computer communication different from television is interactivity, and interactivity depends on all those little back-and-forth messages.

The phrase "high-capacity slow link" that I used above probably looked very odd to you. Even to me it looks odd. We've been used to wrong thinking for so long that correct thinking looks odd now. How can a high-capacity link be a slow link? High-capacity means fast, right? It's odd how that's not true in other areas. If someone talks about a "high-capacity" oil tanker, do you immediately assume it's a very fast ship? I doubt it. If someone talks about a "large-capacity" truck, do you immediately assume it's faster than a small sports car?

We have to start making that distinction again in communications. When someone tells us that a modem has a speed of 28.8 kbit/sec we have to remember that 28.8 kbit/sec is its capacity, not its speed. Speed is a measure of distance divided by time, and 'bits' is not a measure of distance.

I don't know how communications came to be this way. Everyone knows that when you buy a hard disk you should check what its seek time is. The maximum transfer rate is something you might also be concerned with, but the seek time is definitely more important. Why does no one think to ask what a modem's 'seek time' is? The latency is exactly the same thing. It's the minimum time between asking for a piece of data and getting it, just like the seek time of a disk, and it's just as important.

Lessons to learn

ISDN has a latency of about 10ms. Its throughput may be twice that of a modem, but its latency is ten times better, and that's the key reason why browsing the web over an ISDN link feels so much better than over a modem. If you have the option of ISDN, and a good ISP that supports it, and it is not too expensive in your area, then get it.

One of the reasons that telephone modems have such poor latency is that they don't know what you're doing with your computer. An external modem is usually connected through a serial port. It has no idea what you are doing, or why. All it sees is an unstructured stream of bytes coming down the serial port.

Ironically, the Apple Geoport telecom adapter, which has suffered so much criticism, may offer an answer to this problem. The Apple Geoport telecom adapter connects your computer to a telephone line, but it's not a modem. All of the functions of a modem are performed by software running on the Mac. The main reason for all the criticism is that running this extra software takes up memory slows down the Mac, but it could also offer an advantage that no external modem could ever match. Because when you use the Geoport adapter the modem software is running on the same CPU as your TCP/IP software and your Web browser, it could know exactly what you are doing. When your Web browser sends a TCP packet, there's no need for the Geoport modem software to mimic the behaviour of current modems. It could take that packet, encode it, and start sending it over the telephone line immediately, with almost zero latency.

Sending 36 bytes of data, a typical game-sized packet, over an Apple Geoport telecom adapter running at 28.8kb/s could take as little as 10ms, making it as fast as ISDN, and ten times faster than the current best modem you can buy. For less than the price of a typical modem the Geoport telecom adapter would give you Web browsing performance close to that of ISDN. Even better, all the people who already own Apple Geoport telecom adapters wouldn't need to buy anything at all -- they'd just need a software upgrade. Even better, Microsoft wouldn't be able to just copy it for Windows like they do with everything else they see on the Mac, because Wintel clones don't have anything like a Geoport for Microsoft to use. What a PR triumph for Apple that would be! It really would show that Apple is the company that understands the Internet. I'm know that in practice there would be other factors that prevent us from getting the delay all the way down to 10ms, but I'm confident that we could get a long way towards that goal.

So far Apple has shown no interest in making use of this opportunity.

Bandwidth Still Matters

Having said all this, you should not conclude that I believe that bandwidth is unimportant. It is very important, but in a way that most people do not think of. Bandwidth is important not only for it's own sake, but also for it's effect on overall latency. As I said above, the important issue is the total end-to-end transmission delay for a packet.

Many people believe that having a private 64kb/sec ISDN connection is just as good, or even better than having a 1/150 share of a 10MB/sec Ethernet. Telephone companies argue that ISDN is just as good as new technologies like cable modems, because while cable modems have much higher bandwidth, that bandwidth is shared between lots of users, so the average works out the same. This idea, that you can average packets as if they were a fluid in a pipe, is flawed, as the following example will show:

Say we have a game where the state of the virtual world amounts to 40K of data. We have a game server, and in this simple example, the game server transmits the entire current game state to the player once every 10 seconds. That's 40K every 10 seconds, or an average of 4K/sec, or 32kb/sec. That's only half the capacity of a 64kb/sec ISDN line, and 150 users doing this on an Ethernet is only half the capacity of the Ethernet. So far so good. Both links are running at only 50% capacity, so the performance should be the same, right? Wrong. On the Ethernet, when the server sends the 40K to a player, the player can receive that data as little as 32ms later (320kb / 10Mb/sec). If the server is not the only machine sending packets on the Ethernet, then there could be contention for the shared medium, but even in that case the average delay before the player receives the data is only 64ms. On the ISDN line, when the server sends the 40K to a player, the player receives that data 5 seconds later (320kb / 64kb/sec). In both cases the users have the same average bandwidth, but the actual performance is very different. In the Ethernet case the player receives the data almost instantly, but in the ISDN case, by the time the player gets the game information it is already 5 seconds out of date.

The standard mistake is to assume that a 40K chunk every ten seconds and a uniform rate of 4K/second are the same thing. They're not. If they were then ISDN, ATM, and all the other telephone company schemes would be good ideas. The telephone companies assume that all communications are like the flow of fluid in a pipe. You just tell them the rate of flow you need, and they tell you how big the pipe has to be. Audio streams, like voice, are like the flow of fluid in a pipe, but computer data is not. Computer data comes in lumps. The standard mistake is to say that if I want to send 60K of data once per minute, that's exactly the same as sending 1K per second. It's not. A 1K per second connection may be sufficient *capacity* to carry the amound of data you're sending, but that doesn't mean it will deliver the 60K lump of data in a timely fashion. It won't. By the time the lump finishes arriving, it will be one minute old. Just because you don't send data very often doesn't mean you want it delivered late. You may only write to your aunt once a year, but that doesn't mean that on the occasions when you do write her a letter you'd like it to take a year to be delivered.

The conclusion here is obvious. If you're given the choice between a low bandwidth private connection, or a small share of a larger bandwidth connection, take the small share.

Again, this is painfully obvious outside the computer world. If a politician said they would build either a large shared freeway, or a million tiny separate private footpaths, one reserved for each citizen, which would you vote for?


A lot of people have sent me e-mail disputing what I've said here. A lot of people have sent me e-mail simply asserting that their modem isn't slow at all, and the slow performance they see is due to the rest of the Internet being slow, not their modem link.

To try to get to the truth of the matter, I'm going to do a small-scale survey. If you think your modem has low latency, please try an experiment for me. Run a "traceroute" to some destination a little way away. On the West coast of the US might be a good host to trace to. From the East coast of the US you can trace to In other places, pick a host of your choice (or use one of those two if you like).

On Unix, you can run a trace by typing "traceroute " (if you have traceroute installed). On the Mac, get Peter Lewis's Mac TCP Watcher and click the "Trace" button. On Windows '95, you have to open a DOS window and type a command like in Unix, except on Windows '95 the "traceroute" command is called "TRACERT". Jack Richard wrote a good article about traceroute for Boardwatch Magazine.

When you get your trace, send it to me, along with any other relevant information, like what brand of modem you're using, what capacity of modem (14.4/28.8/33k/64k ISDN, etc.), whether it is internal or external, what speed serial port (if applicable), who your Internet Service Provider is, etc.

I'll collect results and see if any interesting patterns emerge. If any particular brands of modems and/or ISPs turn out to have good latency, I'll report that.

To start things off, here's my trace:

Name:  Stuart Cheshire
Modem: No modem (Quadra 700 built-in Ethernet)
ISP: BBN (Bolt, Beranek and Newman)

Hop Min Avg Max IP Name
1 3/3 0.003 0.003 0.004
2 3/3 0.003 0.006 0.013
3 3/3 0.004 0.004 0.004
4 3/3 0.003 0.003 0.004
5 3/3 0.004 0.004 0.005
6 2/3 0.006 0.006 0.007
7 3/3 0.036 0.036 0.037
8 3/3 0.036 0.160 0.406
9 3/3 0.056 0.058 0.059
10 3/3 0.056 0.058 0.059
11 3/3 0.076 0.077 0.078
12 3/3 0.076 0.076 0.076
13 3/3 0.077 0.077 0.078
14 3/3 0.080 0.081 0.083
15 3/3 0.080 0.145 0.212
16 3/3 0.079 0.081 0.084
17 3/3 0.083 0.096 0.104
18 3/3 0.082 0.082 0.084
19 3/3 0.082 0.085 0.089

You can see it took my Mac (Quadra 700 running Open Transport) 3ms to get to jenkins-gateway. This is not particularly fast. With a good Ethernet interface it would be less than 1ms. From there, it took 1ms to get to paloalto-br1 (near to Stanford) and another 2ms to get to oakland-br1 (across the bay from San Francisco).

From oakland-br1 to denver-br1 took 30ms, from denver-br1 to chicago1-br1 took 20ms, and from chicago1-br1 to boston1-br1 took another 20ms.

The last stretch from boston1-br1 to took another 6ms.

So to summarise where the time's going, there's 6ms spent at each end, and 70ms spent on the long-haul getting across the country. Remember those are round-trip times -- the one-way times are half as much.

Now, let's find out what the breakdown looks like when we try the same experiment with a modem. Send in your results! Hopefully we'll find at least one brand of modem that has good latency.

Note: October 1997. Now that I've got a decent collection of results, please only send me your results if they're a lot faster (or slower) than what's already on the list. Also, please send me results only for consumer technologies. If you're company has a T-1 Internet connection, or if you are a student in University houseing with a connection even faster than that, then it's not a great suprise to find that your connection has good latency. My goal here is to find what consumer technologies are available that offer good latency.

Are we there yet?

The good news is that since I first wrote this rant I've started to see a shift in awareness in the industry. Here are a couple of examples:

From Red Herring, June 1997, page 83, Luc Hatlestad wrote:

Matthew George is the vice president of techhnology at Engage... To Mr George, latency issues are more about modems than about network bandwidth. "Network latency in and of itself is not material to game playing; at least 70 to 90 percent of latency problems we see are due to the end points: the modems," he says.

From MacWeek, 12th May 1997, page 31, Larry Stevens wrote about the new 56k modems:

Greg Palen, director of multimedia at Galzay Marketing Group, a digital communications, prepress and marketing company in Kansas City, Kan., is one of those taking a wait-and-see attitude. "We can always use more bandwidth, but modem speed is not the primary issue at this point. The main issue is latency.

Some modem makers are finally starting to care about latency. One major modem manufacturer has contacted me, and we've been investigating where the time goes. It seems that there is room for improvement, but unfortunately modems will never be able to match ISDN. The problem is that over a telephone line, electrical signals get "blurred" out. In order to decode just one single bit, a 33.6kb/s modem needs to take not just a single reading of the voltage on the phone line at that instant, but that single reading plus another 79 like it, spaced 1/6000 of a second apart. A mathematical function of those 80 readings gives the actual result. This process is called "line equalization". Better line equalization allows higher data rates, but the more "taps" the equalizer has the more delay it adds. The V.34 standard also specifies particular scrambling and descrambling of the data, which also take time. According to this company, the theoretical best round-trip delay for a 14.4kb/s modem (no compression or error recovery) should be 40ms, and for a 33.6kb/s modem 64ms. The irony here is that as the capacity goes up, the best-case latency gets worse instead of better. For a small packet, it would be faster for your modem to send it at 9.6kb/s than at 33.6kb/s!

I don't know what the theoretical optimum for a 56kb/s modem is. The sample rate with these is 16000 times per second (62.5us between samples) but I don't know how many taps the equalizer has.

Linux Tuning Parameters

To successfully run enterprise applications, such as a database server, on your Linux distribution, you may be required to update some of the default kernel parameter settings. For example, the 2.4.x series kernel message queue parameter msgmni has a default value (for example, shared memory, or shmmax is only 33,554,432 bytes on Red Hat Linux by default) that allows only a limited number of simultaneous connections to a database. Here are some recommended values (by the IBM DB2 Support Web site) for database servers to run optimally:
- kernel.shmmax=268435456 for 32-bit
- kernel.shmmax=1073741824 for 64-bit
- kernel.msgmni=1024
- fs.file-max=8192
- kernel.sem="250 32000 32 1024"
Shared Memory
To view current settings, run command:
# more /proc/sys/kernel/shmmax
To set it to a new value for this running session, which takes effect immediately, run command:
# echo 268435456 > /proc/sys/kernel/shmmax
To set it to a new value permanently (so it survives reboots), modify the sysctl.conf file:
kernel.shmmax = 268435456
To view current settings, run command:
# more /proc/sys/kernel/sem
250 32000 32 1024
To set it to a new value for this running session, which takes effect immediately, run command:
# echo 500 512000 64 2048 > /proc/sys/kernel/sem
Parameters meaning:
SEMMSL - semaphores per ID
SEMMNS - (SEMMNI*SEMMSL) max semaphores in system
SEMOPM - max operations per semop call
SEMMNI - max semaphore identifiers
To view current settings, run command:
# ulimit -a
To set it to a new value for this running session, which takes effect immediately, run command:
# ulimit -n 8800
# ulimit -n -1 // for unlimited; recommended if server isn't shared

Alternatively, if you want the changes to survive reboot, do the following:

- Exit all shell sessions for the user you want to change limits on.
- As root, edit the file /etc/security/limits.conf and add these two lines toward the end:
        user1        soft    nofile          16000
        user1        hard    nofile          20000
  ** the two lines above changes the max number of file handles - nofile - to new settings.
- Save the file.
- Login as the user1 again. The new changes will be in effect.
Message queues
To view current settings, run command:
# more /proc/sys/kernel/msgmni
# more /proc/sys/kernel/msgmax
To set it to a new value for this running session, which takes effect immediately, run command:
# echo 2048 > /proc/sys/kernel/msgmni
# echo 64000 > /proc/sys/kernel/msgmax

Gigabit-based network interfaces have many performance-related parameters inside of their device driver such as CPU affinity. Also, the TCP protocol can be tuned to increase network throughput for connection-hungry applications.

Tune TCP
To view current TCP settings, run command:
# sysctl net.ipv4.tcp_keepalive_time
net.ipv4.tcp_keepalive_time = 7200 // 2 hours
where net.ipv4.tcp_keepalive_time is a TCP tuning parameter.
To set a TCP parameter to a value, run command:
# sysctl -w net.ipv4.tcp_keepalive_time=1800
A list of recommended TCP parameters, values, and their meanings:
Tuning Parameter        Tuning Value                    Description of impact
net.ipv4.tcp_tw_recycle         1                       Reuse sockets in the time-wait state
net.core.wmem_max               8388608                 Increase the maximum write buffer queue size
net.core.rmem_max               8388608                 Increase the maximum read buffer queue size
net.ipv4.tcp_rmem               4096 87380 8388608      Set the minimum, initial, and maximum sizes for the read buffer. Note that this maximum should be less than or equal to the value set in net.core.rmem_max.
net.ipv4.tcp_wmem               4096 87380 8388608      Set the minimum, initial, and maximum sizes for the write buffer. Note that this maximum should be less than or equal to the value set in net.core.wmem_max.
timeout_timewait                echo 30 > /proc/sys/net/ipv4/tcp_fin_timeout 

Determines the time that must elapse before TCP/IP can release a closed connection and reuse its resources. This interval between closure and release is known as the TIME_WAIT state or twice the maximum segment lifetime (2MSL) state. During this time, reopening the connection to the client and server cost less than establishing a new connection. By reducing the value of this entry, TCP/IP can release closed connections faster, providing more resources for new connections. Adjust this parameter if the running application requires rapid release, the creation of new connections, and a low throughput due to many connections sitting in the TIME_WAIT state.
Disk I/O
Choose the Right File System
Use 'ext3' file system in Linux.
- It is enhanced version of ext2
- With journaling capability - high level of data integrity (in event of unclean shutdown)
- It does not need to check disks on unclean shutdown and reboot (time consuming)
- Faster write - ext3 journaling optimizes hard drive head motion
# mke2fs -j -b 2048 -i 4096 /dev/sda
mke2fs 1.32 (09-Nov-2002)
/dev/sda is entire device, not just one partition!
Proceed anyway? (y,n) y
Filesystem label=
OS type: Linux
Block size=2048 (log=1)
Fragment size=2048 (log=1)
13107200 inodes, 26214400 blocks
1310720 blocks (5.00%) reserved for the super user
First data block=0
1600 block groups
16384 blocks per group, 16384 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
        16384, 49152, 81920, 114688, 147456, 409600, 442368, 802816, 1327104,
        2048000, 3981312, 5619712, 10240000, 11943936

Writing inode tables: done
Writing superblocks and filesystem accounting information: done

This filesystem will be automatically checked every 28 mounts or
180 days, whichever comes first.  Use tune2fs -c or -i to override.
Use 'noatime' File System Mount Option
Use 'noatime' option in the file system boot-up configuration file 'fstab'. Edit the fstab file under /etc. This option works the best if external storage is used, for example, SAN:
# more /etc/fstab
LABEL=/                 /                       ext3    defaults        1 1
none                    /dev/pts                devpts  gid=5,mode=620  0 0
none                    /proc                   proc    defaults        0 0
none                    /dev/shm                tmpfs   defaults        0 0
/dev/sdc2               swap                    swap    defaults        0 0
/dev/cdrom              /mnt/cdrom              udf,iso9660 noauto,owner,kudzu,ro 0 0
/dev/fd0                /mnt/floppy             auto    noauto,owner,kudzu 0 0
/dev/sda                /database               ext3    defaults,noatime 1 2
/dev/sdb                /logs                   ext3    defaults,noatime 1 2
/dev/sdc                /multimediafiles        ext3    defaults,noatime 1 2
Tune the Elevator Algorithm in Linux Kernel for Disk I/O
After choosing the file system, there are several kernel and mounting options that can affect it. One such kernel setting is the elevator algorithm. Tuning the elevator algorithm helps the system balance the need for low latency with the need to collect enough data to efficiently organize batches of read and write requests to the disk. The elevator algorithm can be adjusted with the following command:
# elvtune -r 1024 -w 2048 /dev/sda
/dev/sda elevator ID 2
read_latency: 1024
write_latency: 2048
max_bomb_segments: 6
The parameters are: read latency (-r), write latency (-w) and the device affected.
Red Hat recommends using a read latency half the size of the write latency (as shown).
As usual, to make this setting permanent, add the 'elvtune' command to the
/etc/rc.d/rc.local script.

Disable Unnecessary Daemons (They Take up Memory and CPU)
There are daemons (background services) running on every server that are probably not needed. Disabling these daemons frees memory, decreases startup time, and decreases the number of processes that the CPU has to handle. A side benefit to this is increased security of the server because fewer daemons mean fewer exploitable processes.

Some example Linux daemons running by default (and should be disabled).  Use command:
#/sbin/chkconfig --levels 2345 sendmail off
#/sbin/chkconfig sendmail off
Advanced power management daemon
Automatically mounts file systems on demand (i.e.: mounts a CD-ROM automatically)
Common UNIX Printing System
HP OfficeJet support
ISDN modem support
Used in support of exporting NFS shares
Used for file locking with NFS
PCMCIA support on a server
Red Hat Network update service for checking for updates and security errata
Mail Transport Agent
Font server for X Windows
Shutdown GUI
Normally, there is no need for a GUI on a Linux server. All administration tasks can be achieved by the command line, redirecting the X display or through a Web browser interface. Modify the 'inittab' file to set boot level as 3:
To set the initial runlevel (3 instead of 5) of a machine at boot,
modify the /etc/inittab file