Monday, August 15, 2011

Unix SED & awk- File processing

http://sed.sourceforge.net/sedfaq3.html

I'd like to skip first two or three fields at the the beginning of a line and print the rest of line. Consider the following input:
This is a test
Giving back more than we take
I want my input file with the following output:
a test
more than we take
How do I printing lines from the nth field using awk under UNIX or Linux operating systems?

You can use the awk command as follows:
 
echo 'This is a test' | awk '{print substr($0, index($0,$3))}'
 
OR
 
awk '{print substr($0, index($0,$3))}' <<< 'This is a test'
 
You can also use the cut command:
 
echo 'This is a test' | cut -d ' ' -f3-
 
OR
 
cut -d ' ' -f3- <<<'This is a test'
 
Finally, process the file using bash while loop:
#!/bin/bash
_input="/path/to/file.name.txt"
while IFS= read -r line
do
    cut -d ' ' -f3- <<<"$line"
    ### same stuff with awk ###
    ### awk '{print substr($0, index($0,$3))}' <<< "$line" ###
done < "${_input}"
 
Please note that you can also use Perl, Python or other shell text processing commands to do the same thing.

Unix Commands ZIP

The syntax is as follows:
zip file.zip file1.txt file2.doc photo.jpg 

To zip resume.doc and all *.c file, enter:
zip bacup.zip resume.doc *.c
To zip data directory, enter:
zip -r backup.zip data