Stream EDit

non-interactive edit

sed [-n] scriptCommands] [-f scriptFile] [-e |--expression=scriptStatmemnts ] [file …]

Copies file† to STDOUT, editing it according to a script of commands.

Examples: 

sed s/blue/green/ infile > outfile    
sed 's/old/older/; s/new/old'  needsMultipleChanges.txt 
sed -f bunch-o-cmds.sed  afile anotherfile > da-SEDed-file
sed "s/;/;\n/"  > myprog.c # Make multi-statment-lines be on seperate lines

Warning: sed "s/xx/yy/" file > file results in an empty file! Use sed --in-place "s/xx/yy/" file
Note: sed "s/xx/yy/" file    doesn't update file, rather outputs to STDOUT (perhaps the terminal )

-n no default output.
-f scriptFile Commands are in scriptFile
-e script
--expression=script		 add script to the commands
-f scriptFile
--file=scriptFile		 add scriptFile to the commands
-i[suffix]
--in-place[=suffix] 		Virtually make changes to file
Actually outputs a temp file sedxxxxx then deletes original and renames sedxxxxx
With suffix retains original as backup . so there must be enough space for sedxxxxx.
WARNING: If no changes are made the original is still deleted and still renames sedxxxxx which causes the input file(s) to have the current date! -in-place operating details
--follow-symlinks follow symlinks when processing in place
--posix disable GNU extensions.
-E
-r
--regexp-extended 		use extended regular expressions
-s
--separate 		consider input files as separate rather than as a single
continuous long stream.
--sandbox operate in sandbox mode.
-u
--unbuffered 		flush the output buffers often (Useful for piping).
-z
--null-data 		separate lines by NULLs
-l n
--line-length=n 		line-wrap length for l
-n
--quiet
--silent		 suppress automatic outputing of pattern space
--help display help and exit
--version output version and exit

Normally, sed compiles the entire script then cycles through the input:

  • reading a line into a WORK† area (unless there is something left after a D ),
  • deleting trailing ␤
  • applying, in sequence, all commands whose address select the WORK area and
  • writing the resulting WORK area to standard output† including a ␤

    A script consists of editing commands, one per line: 

    [address]   function    [arguments]

    Notes on regular expressions:

    † Function Description
      # comment Special case: On the first line #n, the default output will be suppressed.
    This corresponds to -n. Since it can be in a command file the -n is unnecessary.
    2 s/regexp/replacement/flags

    		Substitute the replacement string for instances of the regexp in WORK.

    Character immediating following s will be used as the delimeter ( not necessarily /).
    Useful if regexp contains slashes. 

      /bin/df | sed s:/Volumes/::
    See ed(1).

    flags are:

    • g global. Substitute all nonoverlapping instances of regexp
    • n = 1 - 512. Substitute for just the nth occurrence of the regexp
      n.b. using 2 for flags changes the second occurance IN THE RANGE OF LINES,
      not the second occurance in every line.
      i.e. only one string will be changed unless you use g
    • u|U change case of pattern match only.
    • p Print WORK if a replacement was made.
    • i ignore case (not supported in Mac OSX version)
    • w file Write WORK appending to file if a replacement was made.

    Special strings:

    Bracket Expression [ … ] list (ex: [13579], range (ex:[4-9]) and named (ex: [[:digit:]]
    Repetition ¿\{1,4\} (in this example: at least 1 and as many as 4 ¿s)
    Subexpression
    		¿ ¿\(regexp1\) ¿ ¿\(regexp2\) ¿ ¿
    \Ň in the replacment is a backreferene, i.e. \1 inserts the first subexpression, \2 inserts the second subexpression, …
    Example:
    s/\(temperature\)\(pressure\)/\2 or \1/ 
pressure temperature
     &  in the replacment signifies the matched regexp Example:
     s/<td/& align=right/
    changes
     <td to <td align=right>

    2 n       W → STDOUT
       in → W     		next. Output WORK ; read the next line of input to WORK.
    2 N   W + \n + in → WAppend a new-line and the Next line of input to WORK.
    Resulting in an embedded new-line, that can be substituted by null, joining the current WORK and the next input line.
    Example: Join continuation of a span tag had been split onto next line
    /<span$/N; s/<span\n/<span/

    Example: Join 3 lines as in the result from head -n1 * into one line: 

    >head -n1 mys*  
==> mysql <==
if [ $# -lt 1 ] ; then 

==> mysql.server <==
#!/bin/sh

==> mysql.server.vvvqq <==
#!/bin/sh

==> mysqladmin.vvvqqq <==
echo /usr/local/mysql/bin/mysqladmin $*


> head -n1 *|sed "N;s/\n//;N;s/\n//" 
==> mysql <==if [ $# -lt 1 ] ; then 
==> mysql.server <==#!/bin/sh
==> mysql.server.vvvqq <==#!/bin/sh
    2 G   W + H → W WORK Gets [newline] and HOLD appended .
    2 h   W → H HOLD gets contents of WORK .
    2 H   H+W → H Append WORK to HOLD .
    2 g   H → W contents of the HOLD get put to WORK
    2 x   W ← X → H Exchange the contents of WORK and HOLD .
    2 c \
    text     		change. Delete WORK. Output text. Start the next cycle.
    2 P W→ stdout Print. Copy WORK to the standard output.
    (useful if -n option was used to supress normal output.)
    2 p W(1-nl)→ stdout print. Copy the initial segment of the WORK area through the first new-line to the standard output.
    1 a \
    text     		append text after outputting the selcted line.
    If the text is several lines, all (but the last) must end with \ .

    After the head tag, append a title tag, "nice document" and the ending title tag:

    /<head>/a \
    <title>\
    nice document\
    <title>
    1 i \
    text         		insert text before outputting the selected line. ( it seems that functions that insert should be before functions that reference the entire file.,
    '</body>'i\
    <style>\
    td {text-align:right}\
    </style>
    2 d Wdelete WORK. Start the next cycle.
    2 D W(1-nl) Delete the initial segment of WORK through the first new-line. Start the next cycle.
    1† r file read file
    file must be the last thing on the line and be preceeded by exactly one blank.
    1 w file write. Append WORK to file
    file must be the last thing on the line and be preceeded by exactly one blank.
    The first occurrence of w causes file to be cleared. Each time sed is used, file is overwritten.

    Each w file is created before processing begins.
    There can be at most 10 distinct w file arguments.

    1 = output the current line number of input.
    (good for debugging) (The current line number changes.)
    0 :label define a label for b and t commands to branch to.
    2 b [label] Branch to the command bearing the :label.
    If label is omitted, branch to the end of the script.
    2 t [label] test. Branch to the command bearing the :label if any substitutions have been made since the most recent reading of an input line or execution of a t.
    If label is omitted, branch to the end of the script.
    2 { commands }Execute commands only when WORK is selected.
    2 ! function Don't apply the function (or group, if function is {…) only to lines not selected by the address(es).
    2 y/string1/string2/

    		Transform. Replace all occurrences of characters in string1 with the corresponding characters in string2.
    string1 and string2 must have the same number of characters.
      example:
    to change all uppercase to lowercase 
    y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/>
    to change all lowercase to uppercase 
    y/abcdefghijklmnopqrstuvwxyz/ABCDEFGHIJKLMNOPQRSTUVWXYZ/
    Change only first character:
     s/\(.\)/\u\1/
    2 l (ell) output WORK area in an unambiguous form.
    Non-printable characters are displayed in octal notation and
    long lines are folded.
    EF 82 B7 are NOT altered

    Consider the strings command instead.

    1 q quit.

    Example:
    change
    Dec 31 19:37
      to
    19:37 Dec 31

    command  results
    s/... .. ..:../& &/ duplicates the entire "date time" string
    		 Dec 31 19:37 Dec 31 19:37
    s/... ..// deletes the first Dec 31 19:37 Dec 31 19:37
    s/..:..//2deletes the 2nd 19:37 19:37 Dec 31
    (I expect you could use the X command)

    Take lines with message followed by target of message and put target at end of message: 

    Host abt.info not found: 3(NXDOMAIN)
abt
Host ach.info not found: 3(NXDOMAIN)
ach
acj

sed "/NX/N;/NX/s/\n//" 

Host abt.info not found: 3(NXDOMAIN)abt
Host ach.info not found: 3(NXDOMAIN)ach
acj
    Change 123,456 to 123456
    substitute from 1 to 3 characters followed by a comma to the same 1 to 3 characters comma and a zz substitute ,zz to null
    s/.\{1,3\},/&zz/ ; s/,zz//
    The new-line escape sequence, \n, matches a new-line embedded in WORK.

    A period . does not match the ending new-line of WORK.
    Use the negation function ! to have commands applied only to non-selected WORK areas .
    Backslashes in text are treated like backslashes in the replacement string of an s command, and may be used to protect initial blanks and tabs against the stripping that is done on every script line.

    command list example: 

    s/xxx/yyy/ {
n
b top
}

    See also:tr (translate single characters), awk, ed, grep(includes regular expression discussion), environ

    9 May 10, copy pasted list from a PDF and ended up with x'EF 82 B7 ' . Only way I could get them out was to copy the file and delete everything except the x'EF 82 B7 ' then prefix it with s/ and suffix it with /zzz/g )
    Gotta be a better way! (see Replacing 3byte strings above see also sed-3.02, see also sedFAQ, gnu.org/sed

    Some information taken from SunOS 5.4 man page of 14 Sep 1992 As of 3/19/13 the current version of sed is 4.2.1.
    Most of this is based on the 3.02 version and still is applicable.

    Mac OS uses old version of sed which does not uspport -v and among other things, does not support \n as a new-line in string2 of the substitute command! (see sedFAQ4.6

    Error message like:
    sed: -e expression #1, char 1: …
    might be due to omitting the -f to specify a script file.
    sed file.sed targetFile or sed -e file.sed targetFile
    should be
    sed -f script.sed targetFile