How to Use Google to Search With in a Single Web Site?

Ever want to use Google to search a single Web site?

You can use Google's site: syntax to restrict your search to a single Web site. Make sure there's no space between site: and your Web site. Follow with a space and then your search terms. You don't need to use the "http://" portion of your URL.

site:google.about.com power search tricks

You could also widen the search to include all the About.com Guide sites:

site:about.com google

This same search can be widened to include all the Web sites within a domain.

site:edu text buyback dates

Google's site: syntax can be mixed with other syntax, such as AND and OR searches.

rsync examples from command line.

To copy all the files in /home/lokams/* of remote host to local directory /backup/home/lokams/
# rsync -e ssh -avz --delete --exclude dir/* --delete-excluded --stats user@remotehost:/home/lokams/ /backup/home/lokams/

To copy the directory /home/lokams of remote host to local directory /backup/home. Directory "lokams" will get created in /backup/home.
# rsync -e ssh -avz --delete --exclude-from=~/.rsync/excludeFile --delete-excluded --stats user@remotehost:/home/lokams /backup/home/

To ignore the permissions errors while copying to remote windows/mac systems, use "-rlt" option.
# rsync -e ssh -avz --delete --exclude dir/* --delete-excluded --stats -rlt user@remotehost:/home/lokams/ /backup/home/lokams/

rsync behaves differently if the source directory has a trailing slash. Study and learn the difference between the following two commands before moving on. Use the -n option to rsync when testing to preview what would happen.

$ rsync -n -av /tmp .
$ rsync -n -av /tmp/ .

More rsync examples:
--------------------------

# rsync -crogpvltz -e ssh --exclude "bin" --exclude "ifany" --delete --delete-after --bwlimit 20 root@10.144.17.1:/apps/uae/ /apps/uae > /apps/uae/logs/sync.log

# rsync -avz -e ssh root@remotehost:/backup/reptest /backtmp/reptest/

# rsync -avz -e ssh /logstage/archive/DXBSEC/NODE0000/ db2inst1@10.20.202.236:/db2/archivelog/security/logstage/retrieve/DXBSEC/NODE0000/

# rsync -e ssh -avz --timeout=999 --delete --exclude dir-or-file-to-exclude --delete-excluded --stats -rlt user@remotehost:/home/lokams/

rsync useful options:
-------------------------

-a, --archive archive mode, equivalent to -rlptgoD
-n, --dry-run show what would have been transferred ( Preview mode )

-c - always checksum
-r - recursive into directories
-o - preserve owner
-g - preserve group
-p - preserve permissions
-t - preserve times
-v - verbose
-l - copy symlinks as symlinks
-z - compress file data
-P - show progress during transfer
-q - quite (decrease verbosity)
-e - specify the remote shell
-b - make backup
-R - Use relative path names
-u - skip files that are newer on the receiver

--stats give some file-transfer stats
--timeout=TIME set I/O timeout in seconds

--backup-dir - make backups into this directory
--bwlimit=KBPS - limit I/O bandwidth, KBytes per second
--delete - delete files that don't exist on the sending side
--delete-after - receiver deletes after transferring, not before
--daemon run as an rsync daemon
--address=ADDRESS bind to the specified address
--exclude=PATTERN exclude files matching PATTERN
--exclude-from=FILE exclude patterns listed in FILE
--include=PATTERN don't exclude files matching PATTERN
--include-from=FILE don't exclude patterns listed in FILE
--min-size=SIZE don't transfer any file smaller than SIZE
--max-size=SIZE don't transfer any file larger than SIZE


# --numeric-ids:
Tells rsync to not map user and group id numbers local user and group names
# --delete:
Makes server copy an exact copy of the source by removing any files that have been removed on the remote machine

Creating large empty files in Linux / UNIX

To create large empty files in Linux or UNIX:

# dd if=/dev/zero of=filename bs=1024 count=desired

Example to create a 1GB file:

dd if=/dev/zero of=file_1GB bs=1024 count=1000
/or/
dd if=/dev/zero of=file_1GB bs=4096 count=250
/or/
dd if=/dev/zero of=file_1GB bs=2048 count=500

Example to create a 2GB file:

dd if=/dev/zero of=file_2GB bs=2048 count=1000
/or/
dd if=/dev/zero of=file_2GB bs=1024 count=2000

Example to create a 512MB file:

dd if=/dev/zero of=file_512MB bs=1024 count=500
/or/
dd if=/dev/zero of=file_1GB bs=512 count=1000

either use

# mkfile size myfile

where can be in KB, MB or GB using k, m or g as suffix. To create a 10 GB file, use

# mkfile 10240m myfile

If you run # mkfile -n 10240m myfile the file will be created, but the disk blocks will not get allocated, only when writing data into the file.

Zettabyte File System (ZFS) in Solaris 10

ZFS is an advanced modern filesystem from Sun Microsystems, originally designed for Solaris/OpenSolaris.

ZFS is a new file system in Solaris 10 OS which provides excellent data integrity and performance compared to other file systems (considering the enterprise storage scenario). Unlike previous file systems, it's a 128-bit file system, which means it can scale up to accommodate very large data. It is perhaps the world's first 128-bit file system. But why do we need so much scalability? The reason is simple. In an enterprise, data is continuously stored on servers and it keeps on increasing. Enterprises want to keep as much of this data live as possible, so that it can be quickly retrieved when required.

ZFS has many features which can benefit all kinds of users - from the simple end-user to the biggest enterprise systems:

• Provable integrity - it checksums all data (and metadata), which makes it possible to detect hardware errors (hard disk corruption, flaky IDE cables, etc...)
• Atomic updates - means that the on-disk state is consistent at all times, there's no need to perform a lengthy filesystem check after forced reboots or power failures
• Instantaneous snapshots and clones - it makes it possible to have hourly, daily and weekly backups efficiently, as well as experiment with new system configurations without any risks
• Built-in (optional) compression
• Highly scalable
• Pooled storage model - creating filesystems is as easy as creating a new directory. You can efficiently have thousands of filesystems, each with it's own quotas and reservations, and different properties (compression algorithm, checksum algorithm, etc...)
• Built-in stripes (RAID-0), mirrors (RAID-1) and RAID-Z (it's like software RAID-5, but more efficient due to ZFS's copy-on-write transactional model).
• Many others (variable sector sizes, adaptive endianness, ...)

In traditional file systems, data is stored on a single disk or on a large volume, consisting of multiple disks. In ZFS, a pool of storage model is used, ie every single storage device is part of a single expandable storage pool, irrespective of where the data is being written. Each storage device which resides inside the pool can have different file systems, which helps administrators scale the system in an easy and efficient manner, ie you no longer need to take care of the file system. Just add a storage device to the pool. With this new architecture, each file system that resides under the pool can share the same amount of size and I/O resources as the pool itself. Also ZFS is used for correcting noisy data corruption. For eg, in cases when you've done an I/O operation, the disk returns an error message, say, 'Can't read the specified block.' The second case could be silent data corruption, wherein you do an I/O operation and the system returns corrupted results. ZFS identifies and if possible even corrects these data corruptions, something which existing file systems can't do. Managing existing file systems is also difficult. For example, you upgrade your system after which you find that the file system doesn't support the machine and you have to copy all the data. This would consume a lot of time, but ZFS helps alleviate this. Moreover, existing file systems have limitations in terms of volumes, file size, etc.

ZFS definitely looks like a great engineering achievement and its makers have all rights to be proud of it. In their own words, they've blown away 20 years of obsolete assumptions and now they refer to ZFS as the last word in filesystems.

When ZFS was first announced, I'm sure many Linux hackers had a thought how it would be a great idea to port such a great filesystem to Linux. Unfortunately, ZFS source is distributed under Sun's CDDL license which is (some say deliberatly) incompatible with the GPL license that Linux kernel uses. So, it looks like there will be no native port of ZFS for Linux in the foreseeable future. What a pity.

Simple usage of "tcpdump"

Tcpdump is a really great tool for network security analyst; you can dump packets that flow within your networks into file for further analysis. With some filters you can capture only the interested packets, which it reduce the size of saved dump and further reduce loading and processing time of packets analysis.

Lets start with capturing packets based on network interface, ports and protocols. Let assume I wanna capture tcp packets that flow over eth1, port 6881. The dump file with be save as test.pcap.

tcpdump -w test.pcap -i eth1 tcp port 6881

Simple right? What if at the same time I am interested on getting packets on udp port 33210 and 33220?

tcpdump -w test.pcap -i eth1 tcp port 6881 or udp \( 33210 or 33220 \)

‘\’ is an escape symbol for ‘(’ and ‘)’. Logic OR implies PLUS (+). In plain text is I want to capture tcp packets flows over port 6881 plus udp ports 33210 and 33220. Careful with ‘and’ in tcpdump filter expression, it means intersection. Thats why I put ‘or’ instead of and within udp port 33210 and 33220. The usage of ‘and’ in tcpdump will be illustrate later.

Ok, how about reading pcap that I saved previously?

tcpdump -nnr test.pcap

The -nn is to tell tcpdump not to resolve DNS on IP and Ports, where r is read.

Adding -tttt to makes the timestamp appears more readable format.

tcpdump -ttttnnr test.pcap

How about capture based on IP ?You need to tell tcpdump which IP you are interested in? Destination IP? or Source IP ? Let say I wanna sniff on destination IP 10.168.28.22 tcp port 22,

how should i write?

tcpdump -w test.pcap dst 10.168.28.22 and tcp port 22

So the ‘and’ makes the intersection of destination IP and port.

By default the sniff size of packets is 96 bytes, you somehow can overload that size by specified with -s.

tcpdump -w test.pcap -s 1550 dst 10.168.28.22 and tcp port 22

Some version of tcpdump allows you to define port range. You can as bellow for capturing packets based on a range of tcp port.

tcpdump tcp portrange 20-24

Bare in mind, the line above I didn’t specified -w which it won’t write to a file but i will just print the captured packets on the screen.

About SUID, SGID and Sticky bit

Set user ID, set group ID, sticky bit

In addition to the basic permissions discussed above, there are also three bits of information defined for files in Linux:

* SUID or setuid: change user ID on execution. If setuid bit is set, when the file will be executed by a user, the process will have the same rights as the owner of the file being executed.
* SGID or setgid: change group ID on execution. Same as above, but inherits rights of the group of the owner of the file on execution. For directories it also may mean that when a new file is created in the directory it will inherit the group of the directory (and not of the user who created the file).
* Sticky bit: It was used to trigger process to "stick" in memory after it is finished, now this usage is obsolete. Currently its use is system dependent and it is mostly used to suppress deletion of the files that belong to other users in the folder where you have "write" access to.

Numeric representation

Octal digit Binary value Meaning
0 000 setuid, setgid, sticky bits are cleared
1 001 sticky bit is set
2 010 setgid bit is set
3 011 setgid and sticky bits are set
4 100 setuid bit is set
5 101 setuid and sticky bits are set
6 110 setuid and setgid bits are set
7 111 setuid, setgid, sticky bits are set

Textual representation

SUID, If set, then replaces "x" in the owner permissions to "s", if owner has execute permissions, or to "S" otherwise.

Examples:
-rws------ both owner execute and SUID are set
-r-S------ SUID is set, but owner execute is not set

SGID, If set, then replaces "x" in the group permissions to "s", if group has execute permissions, or to "S" otherwise.

Examples:
-rwxrws--- both group execute and SGID are set
-rwxr-S--- SGID is set, but group execute is not set

Sticky, If set, then replaces "x" in the others permissions to "t", if others have execute permissions, or to "T" otherwise.

Examples:
-rwxrwxrwt both others execute and sticky bit are set
-rwxrwxr-T sticky bit is set, but others execute is not set

Setting the sticky bit on a directory : chmod +t

If you have a look at the /tmp permissions, in most GNU/Linux distributions, you'll see the following:

lokams@tempsrv# ls -l | grep tmp
drwxrwxrwt 10 root root 4096 2006-03-10 12:40 tmp

The "t" in the end of the permissions is called the "sticky bit". It replaces the "x" and indicates that in this directory, files can only be deleted by their owners, the owner of the directory or the root superuser. This way, it is not enough for a user to have write permission on /tmp, he also needs to be the owner of the file to be able to delete it.

In order to set or to remove the sticky bit, use the following commands:

# chmod +t tmp
# chmod -t tmp

Setting the SGID attribute on a directory : chmod g+s

If the SGID (Set Group Identification) attribute is set on a directory, files created in that directory inherit its group ownership. If the SGID is not set the file's group ownership corresponds to the user's default group.

In order to set the SGID on a directory or to remove it, use the following commands:

# chmod g+s directory
# chmod g-s directory

When set, the SGID attribute is represented by the letter "s" which replaces the "x" in the group permissions:

# ls -l directory
drwxrwsr-x 10 george administrators 4096 2006-03-10 12:50 directory

Setting SUID and SGID attributes on executable files : chmod u+s, chmod g+s

By default, when a user executes a file, the process which results in this execution has the same permissions as those of the user. In fact, the process inherits his default group and user identification.

If you set the SUID attribute on an executable file, the process resulting in its execution doesn't use the user's identification but the user identification of the file owner.

For instance, consider the script myscript.sh which tries to write things into mylog.log :

# ls -l
-rwxrwxrwx 10 george administrators 4096 2006-03-10 12:50 myscript.sh
-rwxrwx--- 10 george administrators 4096 2006-03-10 12:50 mylog.log

As you can see in this example, George gave full permissions to everybody on myscript.sh but he forgot to do so on mylog.log. When Robert executes myscript.sh, the process runs using Robert's user identification and Robert's default group (robert:senioradmin). As a consequence, myscript fails and reports that it can't write in mylog.log.

In order to fix this problem George could simply give full permissions to everybody on mylog.log. But this would make it possible for anybody to write in mylog.log, and George only wants this file to be updated by his myscript.sh program. For this he sets the SUID bit on myscript.sh:

# chmod u+s myscript.sh

As a consequence, when a user executes the script the resulting process uses George's user identification rather than the user's. If set on an executable file, the SUID makes the process inherit the owner's user identification rather than the one of the user who executed it. This fixes the problem, and even though nobody but George can write directly in mylog.log, anybody can execute myscript.sh which updates the file content.

Similarly, it is possible to set the SGID attribute on an executable file. This makes the process use the owner's default group instead of the user's one. This is done by:

# chmod g+s myscript.sh

By setting SUID and SGID attributes the owner makes it possible for other users to execute the file as if they were him or members of his default group.

The SUID and GUID are represented by a "s" which replaces the "x" character respectively in the user and group permissions:

# chmod u+s myscript.sh
# ls -l
-rwsrwxrwx 10 george administrators 4096 2006-03-10 12:50 myscript.sh
# chmod u-s myscript.sh
# chmod g+s myscript.sh
# ls -l
-rwxrwsrwx 10 george administrators 4096 2006-03-10 12:50 myscript.sh

About umask

umask command will be used for setting the default file creation permissions.

When a file is created, its permissions are set by default depending on the umask setting. This value is usually set for all users in /etc/profile and can be obtained by typing:

# umask

The default umask value is usually 022. It is an octal number which indicates what rights will be removed by default to all new files. For instance, 022 indicates that write permissions will not be given to group and other.

By default, and with a umask of 000, files get mode 666 and directories get mode 777. As a result, with a default umask value of 022, newly created files get a default mode 644 (666 - 022 = 644) and directories get a default mode 755 (777 - 022 = 755).

In order to change the umask value, simply use the umask command and give it an octal number. For instance, if you want all new directories to get permissions rwxr-xr--- and files to get permissions rw-r----- by default (modes 750 and 640), you'll need to use a umask value which removes all rights to other, and write permissions to the group : 027. The command to use is:

# umask 027

How to forcefully unmount a Linux / AIX /Solaris disk partition?

Linux / UNIX will not allow you to unmount a device that is busy. There are many reasons for this (such as program accessing partition or open file) , but the most important one is to prevent data loss.

To find out the processes which are active on the partition.

[root@tempsrv ~]# lsof | grep "/mnt"
ssh 22883 lokams cwd DIR 253,1 4096 193537 /mnt
vi 22909 root cwd DIR 253,1 4096 193537 /mnt

/** or **/

[root@tempsrv ~]# fuser -mu /mnt
/mnt: 22883c(lokams) 22909c(root)
[root@tempsrv ~]#

Above output tells that users "lokams" and "root" has a "ssh and vi" processes running that is using /mnt. All you have to do is stop those process and run umount again. As soon as that program terminates its task, the device will no longer be busy and you can unmount it with the following command:

# umount /mnt

To unmount /mnt forcefully with out checking which processes are active currently:
# fuser -km /mnt

-k : Kill processes accessing the file.
-m : Name specifies a file on a mounted file system or a block device that is mounted. In above example you are using /mnt

You can also try umount command with –l option:
# umount -l /mnt

-l : Also known as Lazy unmount. Detach the filesystem from the filesystem hierarchy now, and cleanup all references to the filesystem as soon as it is not busy anymore. This option works with kernel version 2.4.11+ and above only.

To unmount a NFS mount point:

# umount -f /mnt

-f: Force unmount in case of an unreachable NFS system

The above can be accomplished with the below command in AIX:
# fuser -kxuc /mnt

The above can be accomplished with the below command in Solaris:
# fuser -ck /mnt

To list the process numbers and user login names of processes using the /etc/passwd file in AIX / Solaris, enter:
# fuser -cu /etc/passwd