Sed is a stream editor. A stream editor is used to perform basic text transformations on an input stream (can be a file or stdin). While in some ways similar to an editor which permits scripted edits (such as ed), sed works by making only one pass over the input(s), and is consequently more efficient. But it is sed’s ability to filter text in a pipeline which particularly distinguishes it from other types of editors.
In this article we will see quick use of sed to make various tasks easy.
Tools such as head and tail allow us to view the bottom or the top of a file (eg: cat file.txt | head). What if we need to view a section in between the chunks? We can use the following command to do so, this will print line 5 - 10 of file.txt
sed -n '5,10p' file.txt
Also, we can also print the entire file except a certain range. To exclude lines 15 through 20 from file.txt, do :
sed '15,20d' file.txt
We can also chain set of chunks we want to observe via -e flag. Let’s display lines 13-17 and 20-23 from file.txt:
sed -n -e '13,17p' -e '20,23p' file.txt
To replace every instance of the word HE with SHE in file.txt, do:
sed 's/HE/SHE/g' file.txt
Additionally, we can also use gi instead of g to ignore character case:
sed 's/he/she/gi' file.txt
to replace multiple blank spaces with a single space, we will use the output of ip route show and a pipeline:
ip route show | sed 's/ */ /g'
f you’re interested in replacing words only within a line range (30 through 40, for example), you can do:
sed '30,40 s/version/story/g' file.txt
Sometimes configuration files are loaded with comments. While this is certainly useful, it may be helpful to display only the configuration directives sometimes if you want to view them all at a glance.
To remove empty lines or those beginning with # from the Apache configuration file, do:
sed '/^#\|^$\| *#/d' httpd.conf
The caret sign followed by the number sign (^#) indicates the beginning of a line, whereas ^$ represents blank lines. The vertical bars indicate boolean operations, whereas the backward slash is used to escape the vertical bars.
In this particular case, the Apache configuration file has lines with #’s not at the beginning of some lines, so *# is used to remove those as well.
To replace a word beginning with uppercase or lowercase with another word, we can also use sed. To illustrate, let’s replace the word zip or Zip with rar in file.txt:
sed 's/[Zz]ip/rar/g' file.txt
Another use of sed consists in printing the lines from a file that match a given regular expression. For example, we may be interested in viewing the authorization and authentication activities that took place on July 2, as per the /var/log/secure log in a CentOS 7 server.
In this case, the pattern to search for is Jul 2 at the beginning of each line:
sed -n '/^Jul 1/ p' /var/log/secure
With sed, we can also insert spaces (blank lines) for each non-empty line in a file. To insert one blank line every other line in LICENSE, a plain text file, do:
sed G file.txt
To insert two blank lines, do:
sed 'G;G' file.txt
Add an uppercase G separated by a semicolon if you want to add more blank lines.
In the previous tip we used sed to modify a file but did not save the original file. Sometimes it’s a good idea to save a backup copy of the original file just in case.
To do that, indicate a suffix following the -i option (inside single quotes) to be used to rename the original file.
In the following example we will replace all instances of this or This (ignoring case) with that in file.txt, and we will save the original file as file.txt.orig.
Finally, we will use diff utility to identify the differences between both files:
sed -i'.orig' 's/this/that/gi' file.txt
Let’s suppose you have a file containing full names in the format First name, Last name. To adequately process the file, you may want to switch Last name and First name.
sed 's/^\(.*\),\(.*\)$/\, /g' names.txt
Sometimes replacing all instances of a given word, or a random few, is not precisely what we need. Perhaps we need to perform the replacement if a separate match is found.
For example, we may want to replace start with stop only if the word services is found in the same line. In that scenario, here’s what will happen:
We need to start partying at work, but let’s remember to start all services first.
sed '/services/ s/start/stop/g' msg.txt
will do that.
Of course, sed can be combined with other tools in order to create more powerful commands. For example, let’s use the example given in TIP #4 and extract our IP address from the output of the ip route command.
We will begin by printing only the line where the word src is. Then we will convert multiple spaces into a single one. Finally, we will cut the 9th field (considering a single space as field separator), which is where the IP address is:
ip route show | sed -n '/src/p' | sed -e 's/ */ /g' | cut -d' ' -f9
recently one of our teammate solved one CTF chall which includes this sed utility as one part of it.
If you are interested read it HERE
Well that’s it for this article. See ya in next one!
This article is written by Saket Upadhyay and Peer Reviewed by Hrijul Bhatnagar and Smit Sawant. To check profile of every writer of FrigidSec-Blog please visit HERE