"To be a warrior is not a simple matter of wishing to be one. It is rather an endless struggle that will go on to the very last moment of our lives. Nobody is born a warrior, in exactly the same way that nobody is born an average man. We make ourselves into one or the other." --Kokoro
Sed is a Stream Editor used for modifying the files in unix (or linux). Whenever you want to make changes to the file automatically, sed comes in handy to do this. Most people never learn its power; they just simply use sed to replace text. You can do many things apart from replacing text with sed. Here I will describe the features of sed with examples.
Consider the below text file as an input.
1
2
3
4
>cat file.txt
unix isgreat os.unix isopensource.unix isfree os.
learn operating system.
unixlinux which one you choose.
Sed Command Examples
1. Replacing or substituting string
Sed command is mostly used to replace the text in a file. The below simple sed command replaces the word “unix” with “linux” in the file.
1
2
3
4
>sed's/unix/linux/' file.txt
linux isgreat os.unix isopensource.unix isfree os.
learn operating system.
linuxlinux which one you choose.
Here the “s” specifies the substitution operation. The “/” are delimiters. The “unix” is the search pattern and the “linux” is the replacement string.
By default, the sed command replaces the first occurrence of the pattern in each line and it won’t replace the second, third…occurrence in the line.
2. Replacing the nth occurrence of a pattern in a line.
Use the /1, /2 etc flags to replace the first, second occurrence of a pattern in a line. The below command replaces the second occurrence of the word “unix” with “linux” in a line.
3. Replacing all the occurrence of the pattern in a line.
The substitute flag /g (global replacement) specifies the sed command to replace all the occurrences of the string in the line.
1
2
3
4
>sed's/unix/linux/g' file.txt
linux isgreat os.linux isopensource.linux isfree os.
learn operating system.
linuxlinux which one you choose.
4. Replacing from nth occurrence to all occurrences in a line.
Use the combination of /1, /2 etc and /g to replace all the patterns from the nth occurrence of a pattern in a line. The following sed command replaces the third, fourth, fifth… “unix” word with “linux” word in a line.
You can use any delimiter other than the slash. As an example if you want to change the web url to another url as
1
>sed's/http:\/\//www/' file.txt
In this case the url consists the delimiter character which we used. In that case you have to escape the slash with backslash character, otherwise the substitution won’t work.
Using too many backslashes makes the sed command look awkward. In this case we can change the delimiter to another character as shown in the below example.
1
2
>sed's_http://_www_' file.txt
>sed's|http://|www|' file.txt
6. Using & as the matched string
There might be cases where you want to search for the pattern and replace that pattern by adding some extra characters to it. In such cases & comes in handy. The & represents the matched string.
The first pair of parenthesis specified in the pattern represents the \1, the second represents the \2 and so on. The \1,\2 can be used in the replacement string to make changes to the source string. As an example, if you want to replace the word “unix” in a line with twice as the word like “unixunix” use the sed command as below.
The parenthesis needs to be escaped with the backslash character. Another example is if you want to switch the words “unixlinux” as “linuxunix”, the sed command is
The /p print flag prints the replaced line twice on the terminal. If a line does not have the search pattern and is not replaced, then the /p prints that line only once.
1
2
3
4
5
6
>sed's/unix/linux/p' file.txt
linux isgreat os.unix isopensource.unix isfree os.
linux isgreat os.unix isopensource.unix isfree os.
learn operating system.
linuxlinux which one you choose.
linuxlinux which one you choose.
9. Printing only the replaced lines
Use the -n option along with the /p print flag to display only the replaced lines. Here the -n option suppresses the duplicate rows generated by the /p flag and prints the replaced lines only one time.
1
2
3
>sed-n's/unix/linux/p' file.txt
linux isgreat os.unix isopensource.unix isfree os.
linuxlinux which one you choose.
If you use -n alone without /p, then the sed does not print anything.
10. Running multiple sed commands.
You can run multiple sed commands by piping the output of one sed command as input to another sed command.
1
2
3
4
>sed's/unix/linux/' file.txt| sed 's/os/system/'
linux isgreat system.unix isopensource.unix isfree os.
learn operating system.
linuxlinux which one you chosysteme.
Sed provides -e option to run multiple sed commands in a single sed command. The above output can be achieved in a single sed command as shown below.
linux isgreat system.unix isopensource.unix isfree os.
learn operating system.
linuxlinux which one you chosysteme.
11. Replacing string on a specific line number.
You can restrict the sed command to replace the string on a specific line number. An example is
1
2
3
4
>sed'3 s/unix/linux/' file.txt
unix isgreat os.unix isopensource.unix isfree os.
learn operating system.
linuxlinux which one you choose.
The above sed command replaces the string only on the third line.
12. Replacing string on a range of lines.
You can specify a range of line numbers to the sed command for replacing a string.
1
2
3
4
>sed'1,3 s/unix/linux/' file.txt
linux isgreat os.unix isopensource.unix isfree os.
learn operating system.
linuxlinux which one you choose.
Here the sed command replaces the lines with range from 1 to 3. Another example is
1
2
3
4
>sed'2,$ s/unix/linux/' file.txt
linux isgreat os.unix isopensource.unix isfree os.
learn operating system.
linuxlinux which one you choose.
Here $ indicates the last line in the file. So the sed command replaces the text from second line to last line in the file.
13. Replace on a lines which matches a pattern.
You can specify a pattern to the sed command to match in a line. If the pattern match occurs, then only the sed command looks for the string to be replaced and if it finds, then the sed command replaces the string.
(pulled from http://www.folkstalk.com/2012/01/grep-command-in-unix-examples.html)
Grep Command in Unix and Linux Examples
Grep is the frequently used command in Unix (or Linux). Most of us use grep just for finding the words in a file. The power of grep comes with using its options and regular expressions. You can analyze large sets of log files with the help of grep command.
Grep stands for Global search for Regular Expressions and Print.
The basic syntax of grep command is
grep [options] pattern [list of files]
Let see some practical examples on grep command.
1. Running the last executed grep command
This saves a lot of time if you are executing the same command again and again.
1
!grep
This displays the last executed grep command and also prints the result set of the command on the terminal.
2. Search for a string in a file
This is the basic usage of grep command. It searches for the given string in the specified file.
1
grep"Error"logfile.txt
This searches for the string “Error” in the log file and prints all the lines that has the word “Error”.
3. Searching for a string in multiple files.
1
2
grep"string"file1 file2
grep"string"file_pattern
This is also the basic usage of the grep command. You can manually specify the list of files you want to search or you can specify a file pattern (use regular expressions) to search for.
4. Case insensitive search
The -i option enables to search for a string case insensitively in the give file. It matches the words like “UNIX”, “Unix”, “unix”.
1
grep-i"UNix"file.txt
5. Specifying the search string as a regular expression pattern.
1
grep"^[0-9].*"file.txt
This will search for the lines which starts with a number. Regular expressions is huge topic and I am not covering it here. This example is just for providing the usage of regular expressions.
6. Checking for the whole words in a file.
By default, grep matches the given string/pattern even if it found as a substring in a file. The -w option to grep makes it match only the whole words.
1
grep-w"world"file.txt
7. Displaying the lines before the match.
Some times, if you are searching for an error in a log file; it is always good to know the lines around the error lines to know the cause of the error.
1
grep-B2"Error"file.txt
This will prints the matched lines along with the two lines before the matched lines.
8. Displaying the lines after the match.
1
grep-A3"Error"file.txt
This will display the matched lines along with the three lines after the matched lines.
9. Displaying the lines around the match
1
grep-C5"Error"file.txt
This will display the matched lines and also five lines before and after the matched lines.
10. Searching for a sting in all files recursively
You can search for a string in all the files under the current directory and sub-directories with the help -r option.
1
grep-r"string"*
11. Inverting the pattern match
You can display the lines that are not matched with the specified search sting pattern using the -v option.
1
grep-v"string"file.txt
12. Displaying the non-empty lines
You can remove the blank lines using the grep command.
1
grep-v"^$"file.txt
13. Displaying the count of number of matches.
We can find the number of lines that matches the given string/pattern
1
grep-c"sting"file.txt
14. Display the file names that matches the pattern.
We can just display the files that contains the given string/pattern.
1
grep-l"string"file.txt
15. Display the file names that do not contain the pattern.
We can display the files which do not contain the matched string/pattern.
1
grep-l"string"file.txt
16. Displaying only the matched pattern.
By default, grep displays the entire line which has the matched string. We can make the grep to display only the matched string by using the -o option.
1
grep-o"string"file.txt
17. Displaying the line numbers.
We can make the grep command to display the position of the line which contains the matched string in a file using the -n option
1
grep-n"string"file.txt
18. Displaying the position of the matched string in the line
The -b option allows the grep command to display the character position of the matched string in a file.
1
grep-o-b"string"file.txt
19. Matching the lines that start with a string
The ^ regular expression pattern specifies the start of a line. This can be used in grep to match the lines which start with the given string or pattern.
1
grep"^start"file.txt
20. Matching the lines that end with a string
The $ regular expression pattern specifies the end of a line. This can be used in grep to match the lines which end with the given string or pattern.
(pulled from http://www.folkstalk.com/2011/12/101-examples-of-using-find-command-in.html)
Find Command in Unix and Linux Examples
Find is one of the powerful utility of Unix (or Linux)Â used for searching the files in a directory hierarchy. The syntax of find command is
1
find[pathnames][conditions]
Let see some practical exercises on using find command.
1. How to run the last executed find command?
1
!find
This will execute the last find command. It also displays the last find command executed along with the result on the terminal.
2. How to find for a file using name?
1
2
3
find-name"sum.java"
./bkp/sum.java
./sum.java
This will find all the files with name “sum.java” in the current directory and sub-directories.
3. How to find for files using name and ignoring case?
1
2
3
4
find-iname"sum.java"
./SUM.java
./bkp/sum.java
./sum.java
This will find all the files with name “sum.java” while ignoring the case in the current directory and sub-directories.
4. How to find for a file in the current directory only?
1
2
find-maxdepth1-name"sum.java"
./sum.java
This will find for the file “sum.java” in the current directory only
5. How to find for files containing a specific word in its name?
1
2
3
4
5
find-name"*java*"
./SUM.java
./bkp/sum.java
./sum.java
./multiply.java
It displayed all the files which have the word “java” in the filename
6. How to find for files in a specific directory?
1
find/etc-name"*java*"
This will look for the files in the /etc directory with “java” in the filename
7. How to find the files whose name are not “sum.java”?
1
2
3
4
5
find-not-name"sum.java"
.
./SUM.java
./bkp
./multiply.java
This is like inverting the match. It prints all the files except the given file “sum.java”.
8. How to limit the file searches to specific directories?
1
2
3
4
5
6
7
find-name"sum.java"
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java
You can see here the find command displayed all the files with name “sum.java” in the current directory and sub-directories.
a. How to print the files in the current directory and one level down to the current directory?
1
2
3
4
find-maxdepth2-name"sum.java"
./tmp/sum.java
./bkp/sum.java
./sum.java
b. How to print the files in the current directory and two levels down to the current directory?
1
2
3
4
5
find-maxdepth3-name"sum.java"
./tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
./sum.java
c. How to print the files in the subdirectories between level 1 and 4?
1
2
3
4
5
6
find-mindepth2-maxdepth5-name"sum.java"
./tmp/sum.java
./bkp/var/tmp/files/sum.java
./bkp/var/tmp/sum.java
./bkp/var/sum.java
./bkp/sum.java
9. How to find the empty files in a directory?
1
2
find.-maxdepth1-empty
./empty_file
10. How to find the largest file in the current directory and sub directories
1
find.-typef-execls-s{}\;|sort-n-r|head-1
The find command “find . -type f -exec ls -s {} \;” will list all the files along with the size of the file. Then the sort command will sort the files based on the size. The head command will pick only the first line from the output of sort.
11. How to find the smallest file in the current directory and sub directories
1
find.-typef-execls-s{}\;|sort-n-r|tail-1
Another method using find is
1
find.-typef-execls-s{}\;|sort-n|head-1
12. How to find files based on the file type?
a. Finding socket files
1
find.-types
b. Finding directories
1
find.-typed
c. Finding hidden directories
1
find-typed-name".*"
d. Finding regular files
1
find.-typef
e. Finding hidden files
1
find.-typef-name".*"
13. How to find files based on the size?
a. Finding files whose size is exactly 10M
1
find.-size10M
b. Finding files larger than 10M size
1
find.-size+10M
c. Finding files smaller than 10M size
1
find.-size-10M
14. How to find the files which are modified after the modification of a give file.
1
find-newer"sum.java"
This will display all the files which are modified after the file “sum.java”
15. Display the files which are accessed after the modification of a give file.
1
find-anewer"sum.java"
16. Display the files which are changed after the modification of a give file.
1
find-cnewer"sum.java"
17. How to find the files based on the file permissions?
1
find.-perm777
This will display the files which have read, write, and execute permissions. To know the permissions of files and directories use the command “ls -l”.
18. Find the files which are modified within 30 minutes.
1
find.-mmin-30
19. Find the files which are modified within 1 day.
1
find.-mtime-1
20. How to find the files which are modified 30 minutes back
1
find.-not-mmin-30
21. How to find the files which are modified 1 day back.
1
find.-not-mtime-1
22. Print the files which are accessed within 1 hour.
1
find.-amin-60
23. Print the files which are accessed within 1 day.
1
find.-atime-1
24. Display the files which are changed within 2 hours.
1
find.-cmin-120
25. Display the files which are changed within 2 days.
1
find.-ctime-2
26. How to find the files which are created between two files.
1
find.-cnewer f1-and!-cnewer f2
So far we have just find the files and displayed on the terminal. Now we will see how to perform some operations on the files.
27. How to find the permissions of the files which contain the name “java”?
1
find-name"*java*"|xargs ls-l
Alternate method is
1
find-name"*java*"-execls-l{}\;
28.  Find the files which have the name “java” in it and then display only the files which have “class” word in them?
1
find-name"*java*"-execgrep-Hclass{}\;
29. How to remove files which contain the name “java”.
For some reason, I always seem to forget how to do a to z sequences. I always seem to try to use seq a z . Wrong.. this is to help me remember. hehe.
for i in {a..z}; do echo $i; grep ^$i filename >> $i; done
