Stream EDit
non-interactive edit
sed [-n]
[-e |--expression=scriptStatmemnts ]
[scriptCommands] [-file scriptFile]
[file …]
Copies file† to STDOUT, editing it according to a script of commands.
Examples:
sed s/blue/green/ infile > outfile
sed 's/old/older/; s/new/old' needsMultipleChanges.txt
sed -f bunch-o-cmds.sed afile anotherfile > da-SEDed-file
sed "s/;/;\n/" > myprog.c # Make multi-statment-lines be on seperate lines
Warning:
sed "s/xx/yy/" file > file
results in an empty file!
Use
sed --in-place "s/xx/yy/" file†
Note: sed "s/xx/yy/" file doesn't update file, rather outputs to STDOUT (perhaps the terminal )
-E script
-r script
--regexp-extended scriptuse extended regular expressions
i.e. include ?, + and | also no need to escape ( ) and { }
| -e script
--expression=script add script to the commands
| -f scriptFile
--file=scriptFile add scriptFile to the commands
| -i[suffix]
--in-place[=suffix]
Virtually Make changes to file
Operating details of --in-place.
| --follow-symlinks | follow symlinks when processing in place
| --posix | disable GNU extensions.
| -s
--separate consider input files as separate rather than as a single
continuous long stream.
| --sandbox | operate in sandbox mode.
| -u
--unbuffered flush the output buffers often (Useful for piping).
| -z
--null-data separate lines by NULLs
| -l n
--line-length=n line-wrap length for
l†
| -n
--quiet
--silent suppress outputing of pattern space
| --help | display help and exit
| --version | output version and exit
sed distributed with Mac Os as of Sequoia 15. is very old and has no version option.
To display the version that Mac OS thinks it is using use:
> echo $(sw_vers -productVersion)
15.1
| | | | | | | | | | | | | | |
Normally, sed compiles the entire script then cycles through the input:
reading a line into a WORK† area (unless there is something left after a D ),
deleting trailing 
applying, in sequence, all commands whose address select the WORK area and
writing the resulting WORK area to standard output† including a 
A script consists of editing commands, one per line:
[address] function [arguments]
address:
- Omitting the addresses selects every line,
( unlike
ed and vi which expects %).
lineNumber A particular line by number (looks dangereous)
3i <table> third line, counting across input files, starting from 1
. (dot) current line
$ last line of input
/Context/ selects all lines that match a regular expression like grep
exactly defined by
openGroup†
(not vi)
- /<LI>/ s/LI/li/ select all LIst tags
AddressExpression
-
.+1 s/"/'/ on the next line
range: ( two addresses select the inclusive lines)
12,33 s/xxx/yyy
.,$ s/xxx/yyy/ i.e. thisLine through last InputLine
-
context address range :
from the first line that matches the first address,
through the next line that matches the second address.
/PRE/,/\/PRE/s/<BR>//
i.e. within ALL PRE blocks remove <BR> tags
If the second address is BEFORE the first address, only the line at the
first address is selected.
In a context address, the construction
\¿regular expression¿
where ¿ is any character, is identical to
/regular expression/
So in this context address \xabc\xdefx , the first x is the delimiter,
the second x stands for itself,
i.e. the regular expression is abcxdef.
Use a ! at the end of the address range to select all lines except those that match.
A function includes substitute, append,
change, delete, and others
Some of the functions use a HOLD area to save all or part of
the WORK area for subsequent retrieval.
arguments depend on the function
Notes on regular expressions:
- Not all versions of sed support all features of (extended) regular expressions.
- To specify a TAB , not all versions permit
\t
- To split a line,
\n is not supported in Mac OS, use \ at the end of the line and continue on the next line.
| † | Function | Description
|
| | # comment
| Special case: On the first line #n, the default output will be suppressed.
This corresponds to -n. Since it can be in a command file the -n is unnecessary.
|
| 2
|
s/regexp/replacement/flags
| Substitute the replacement string for instances of the
regexp in WORK.
Character immediating following s will be used as the delimeter ( not necessarily /).
Useful if regexp contains slashes.
/bin/df | sed s:/Volumes/:: See ed(1).
flags are:
g global. Substitute all nonoverlapping instances of regexp
n = 1 - 512. Substitute for just the nth occurrence of the regexp
n.b. using 2 for flags changes the second occurance IN THE RANGE OF LINES,
not the second occurance in every line.
i.e. only one string will be changed unless you use g
u|U change case of pattern match only.
p Print WORK if a replacement was made.
i ignore case (not supported in Mac OSX version)
w file Write WORK appending to file if a replacement was made.
Special strings:
| Bracket Expression |
list ( example: [13579] )
range ( example:[4-9] )
named class ( example: [[:digit:]] )
| | Repetition |
¿\{1,4\}
(in this example: at least 1 and as many as 4 ¿s)
|
Subexpression
|
¿
¿\(regexp1\)
¿
¿\(regexp2\)
¿
¿
\Ň in the replacment is a backreferene, i.e. \1 inserts the first subexpression, \2 inserts the second subexpression, …
Example:s/\(temperature\)\(pressure\)/\2 or \1/
pressure temperature
| & | in the replacment signifies the matched regexp
Example:
s/<td/& align=right/
changes
<td to <td align=right>
|
|
| 2 |
n W → STDOUT
in → W
next. Output WORK ; read the next line of input to WORK.
| | 2 |
N W + \n + in → WAppend a new-line and the Next line of input to WORK.
Resulting in an embedded new-line, that can be substituted by null, joining the
current WORK and the next input line.
Example: Join continuation of a span tag had been split onto next line
/<span$/N; s/<span\n/<span/
Example: Join 3 lines as in the result from head -n1 * into one line:
>head -n1 mys*
==> mysql <==
if [ $# -lt 1 ] ; then
==> mysql.server <==
#!/bin/sh
==> mysql.server.vvvqq <==
#!/bin/sh
==> mysqladmin.vvvqqq <==
echo /usr/local/mysql/bin/mysqladmin $*
> head -n1 *|sed "N;s/\n//;N;s/\n//"
==> mysql <==if [ $# -lt 1 ] ; then
==> mysql.server <==#!/bin/sh
==> mysql.server.vvvqq <==#!/bin/sh
| | 2 |
G W + H → W
WORK Gets [newline] and HOLD appended .
| | 2 |
h W → H
| HOLD gets contents of WORK .
| | 2 |
H H+W → H
| Append WORK to HOLD .
| | 2 |
g H → W
| contents of the HOLD get put to WORK
| | 2 |
x W ← X → H
Exchange the contents of WORK and HOLD .
| | 2 | c \
text
change. Delete WORK. Output text. Start the next cycle.
| | 2 |
P W→ stdout | Print. Copy WORK to the standard output.
(useful if -n option was used to supress normal output.)
| | 2 | p W(1-nl)→ stdout
| print. Copy the initial segment of the
WORK area through the first new-line to the standard output.
| | 1 | i \
text |
insert text before outputting the selected line.
( it seems that functions that insert should be before functions that reference the entire file.,
'</body>'i\
<style>\
td {text-align:right}\
</style>
|
| 1 |
a \
text
append text after outputting the selcted line.
If the text is several lines, all (but the last) must end with \ .
After the head tag, append a title tag, "nice document" and the ending title tag:
/<head>/a \
<title>\
nice document\
<title> |
| | 2
| d W | delete WORK. Start the next cycle.
| | 2 |
D W(1-nl)
| Delete the initial segment of WORK through the first new-line. Start the next cycle.
| 1† r file
| read file
file must be the last thing on the line and be preceeded by exactly one blank.
| | 1 | w file
| write. Append WORK to file
file must be the last thing on the line and be preceeded by exactly one blank.
The first occurrence of w causes file to be cleared. Each time sed is used, file is overwritten.
Each w file is created before processing begins.
There can be at most 10 distinct w file arguments.
| | |
| | 1 |
= | output the current line number of input.
(good for debugging) (The current line number changes.)
|
| 0 | :label
| define a label for b and t commands to branch to.
| | 2 | b [label]
Branch to the command bearing the :label.
If label is omitted, branch to the end of the script.
| | 2 | t [label]
| test. Branch to the command bearing the
:label if any substitutions have been made since the most
recent reading of an input line or execution of a t.
If label is omitted, branch to the end of the script.
| | 2 |
{ commands }Execute commands only when WORK is selected.
| | 2
| ! function
Don't apply the function (or group, if function is {…) only to lines not selected by the address(es).
| | 2 |
y/string1/string2/
Transform. Replace all occurrences of characters
in string1 with the corresponding characters in
string2.
string1 and string2 must have the same number of characters.
example:
to change all uppercase to lowercase
y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/>
to change all lowercase to uppercase
y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
Change only first character:
s/\(.\)/\u\1/
| 2 | l (ell)
| output WORK area in
an unambiguous form.
Non-printable characters are displayed in
octal notation and
long lines are folded.
EF 82 B7 are NOT altered
Consider the strings command instead.
| | 1 | q | quit.
| | | | | | | | | | | | | | |
Example:
change
Dec 31 19:37
to
19:37 Dec 31
| command | | results
|
|---|
s/... .. ..:../& &/ | duplicates the entire "date time" string
| Dec 31 19:37 Dec 31 19:37
s/... ..// deletes the first Dec 31 | 19:37 Dec 31 19:37
s/..:..//2deletes the 2nd 19:37 | 19:37 Dec 31
| | | | |
(I expect you could use the X command)
Take lines with message followed by target of message and put target at end of message:
Host abt.info not found: 3(NXDOMAIN)
abt
Host ach.info not found: 3(NXDOMAIN)
ach
acj
sed "/NX/N;/NX/s/\n//"
Host abt.info not found: 3(NXDOMAIN)abt
Host ach.info not found: 3(NXDOMAIN)ach
acj
Change 123,456 to 123456
substitute from 1 to 3 characters followed by a comma to the same 1 to 3 characters comma and a zz
substitute ,zz to null
|
s/.\{1,3\},/&zz/ ; s/,zz// |
The new-line escape sequence, \n, matches a new-line embedded in WORK.
s/\n>/>/ joins a line with a leading > to the previous by omitting the new-line.
A period . does not match the ending new-line of WORK.
Use the negation function ! to
have commands applied only to non-selected WORK areas .
Backslashes in text are treated like backslashes in the replacement
string of an s command, and may be used to protect initial blanks and
tabs against the stripping that is done on every script line.
command list example:
s/xxx/yyy/ {
n
b top
}
- Not all versions of
sed handle [tab]s as \t
Pre process files TRanslating tabs to spaces;
tr '\t' ' ' < $1 |sed -f ~/xml.sed
or y/tab/ /
- This command replaces both tags on BSD i.e. Mac OSX
echo " <head></head>" |sed -E "s/<\/{0,1}[^/]+>/<+>/g"
- On single-line patterns,
. (dot) matches character.
It does not match the [NewLine] at the end of the line because the newline is removed when the line is put into the work space.
sed adds a newline when the work space is output.
On multi-line patterns obtained with N or G , dot matchs a newline in the middle of the pattern space.
If there are 3 lines in the pattern space, s/.*// deletes all 3 lines.
Example using of a "bracketed expression" character classes s/[[:digit:]]/N/
- Replacing 3byte strings of leftQuote, rightQuote, apostrophe and dash special UTF-8 characters using
bash ANSI-C Quoting
These occur after copying/pasting a web page viewable using hexdump -C infile
00000230 6b 65 73 2e 20 54 68 61 74 e2 80 99 73 20 77 68 |kes. That...s wh|
00000240 79 20 74 68 65 72 65 20 69 73 20 61 6e 20 65 72 |y there is an er|
00000250 61 73 65 72 20 6f 6e 20 65 76 65 72 79 20 70 65 |aser on every pe|
00000260 6e 63 69 6c 2e e2 80 9d 20 e2 80 94 20 4a 61 70 |ncil.... ... Jap|
00000270 61 6e 65 73 65 20 70 72 6f 76 65 72 62 0a 0a 54 |anese proverb
e2 80 9c | e2 80 99 | e2 80 9d | e2 80 94 |
sed --in-place s!$'\xe2\x80\x9c'\/\"\/g\ ;
s\/$'\xe2\x80\x99'\/\'\/g\ ;
s\/$'\xe2\x80\x9d'\/\"\/g\ ;]
s\/$'\xe2\x80\x94'\/\-\/g wierdFile.txt
without the escapes it would look like :
s/e2809c/"/g ; s/e28099/s/'/g ; s/e2809d/"/;g s/e28094/-/g
-
Replace the 2 byte string C2 A0 with a +. (occured from an .XLXS excel file; seen in vi as |
|
grep 88306,013016 CL.csv|hexdump -C
000 38 38 33 30 36 2c 30 31 33 30 31 36 c2 a0 c2 a0 |88306,013016....|
010 c2 a0 c2 a0 2c 31 32 32 35 31 2c 53 74 61 74 65 |....,12251,State|
sed s\/$'\xc2\xa0'\/\+\/g < CL.csv >CLT.csv
- Change a line beginning with hours:minutes (where hours may be one or two digits) to be zero filled to two digits if it was only one digit:
Green defines a subexpression, brown uses the subexpression
> echo "8:27 stuff stuff" |sed "s/^\(.\):/0\1:/" # Take first any character, folllowed by a colon, substitute a 0 before it and a colon
08:27 stuff stuff
> echo "19:42 stuff stuff" |sed "s/^\(.\):/0\1:/"
19:42 stuff stuff # no change as there are 2 characters between the start of the line and the colon
See also:tr (translate single characters), awk, ed,
grep(includes regular expression discussion),
environ
9 May 10, copy pasted list from a PDF and ended up with x'EF 82 B7 ' . Only way I could get them out was to copy the file and
delete everything except the x'EF 82 B7 ' then prefix it with s/ and suffix it with /zzz/g )
Gotta be a better way! (see Replacing 3byte strings above
see also sed-3.02, see also sedFAQ,
gnu.org/sed
Some information taken from SunOS 5.4 man page of 14 Sep 1992
As of 3/19/13 the current version of sed is 4.2.1.
Most of this is based on the 3.02 version and still is applicable.
Mac OS uses old version of sed which does not support -v and among other things,
does not support \n as a new-line in string2
of the substitute command! (see sedFAQ4.6
Error message like:
sed: -e expression #1, char 1: …
might be due to omitting the -f to specify a script file.
sed file.sed targetFile or sed -e file.sed targetFile
should be
sed -f script.sed targetFile
There is no verbose option.
Just becasue I look here: matching various characters until x: /dsn.*/ this is a greedy match,
i.e. if thre are multiple dnsxxx on the line * matches them all!
For vi use {-} .
perl : (PREX( ? to ungreedy the match.