Stream EDit non-interactive edit

As of 3/19/13 the current version of sed is 4.2.1.
Version 4.2.1 (Aug 24, '10) is a minor version that includes these bug fixes:
A fix to a parsing problem.
A fix to ensure the security contexts are preserved by -i under SELinux.
A fix that ensures the temporary files for sed -i are not made group or world-readable until they are complete.
Version 4.1.5 is a minor bug fix release which fixes the handling of a negative character class when it doesn't include a closed bracket.

Most of this is based on the 3.02 version and still is applicable.

sed [-n] [-e script [-e script]] [-f sfile] [file ...]

sed copies file to STDOUT, editing it according to a script of commands.

sed "s/blue/green/" infile > outfile
sed -e "s/old/older/" -e "s/new/old" needsMultipleChanges.txt
sed -f bunch-o-cmds.sed afile anotherfile > da-SEDed-file
sed "s/;/;\n/" > myprog.c make multi-statment-lines be on seperate lines)

Warning: sed "s/xx/yy/" file > file results in an empty file!

note: sed "s/xx/yy/" file doesn't update file, rather outputs to STDOUT i.e terminal

-n no default output.
-excute command
-f scriptfile

Normally, sed compiles the entire script then cycles through the input :

  • reading a line into a WORK area (unless there is something left after a D ),
  • deletes trailing \n
  • applying, in sequence, all commands whose address select the WORK area , and
  • writing the resulting WORK area to standard output including a \n

    A script consists of editing commands, one per line:

    [address [,address] ]     function    [arguments]

    A function includes substitute, append, change, delete, and others
    Some of the functions use a HOLD area to save all or part of the WORK area for subsequent retrieval.

    arguments depend on the function

    Notes on regular expressions:

    Function Description
    2 s/regexp/replacement/f

    Substitute the replacement string for instances of the regexp in WORK.

    The character immediating following the s will be used as the delimeter( instead of /). Useful if string contains slashes.

      /bin/df | sed s:/Volumes/::
    For a fuller description see ed(1).

    flags are:

    • g Global. Substitute all nonoverlapping instances of regexp.

    • n = 1 - 512. Substitute for just the nth occurrence of the regexp.
      n.b. using 2 for flags changes
      the second occurance IN THE RANGE OF LINES,
      not the second occurance in every line.
      i.e. only one string will be changed unless you use g

    • p Print WORK if a replacement was made.

    • i ignore case (not supported in Mac OSX version)

    • w file Write the WORK area, appending to file if a replacement was made.

    Special strings:

    • Bracket Expression [] list (ex: [13579], range (ex:[4-9]) and named (ex: [[:digit]]
    • Repitition \{1,4\}
    • Subexpression /chapter \(.*\) / matches chapter number as subexpression \1 on the RHS as a backreferene.
    • & in the replacment signifies the matched expression
      s/<td/& align=right/
      <td to <td align=right>

    2 n       W STDOUT
       in W
    next. Output WORK ; read the next line of input to WORK.
    2 N   W + \n + in WAppend a new-line and the Next line of input to WORK.
    Resulting in an embedded new-line, that can be substituted by null, joining the current WORK and the next input line.
    Example: Join continuation of a span tag had been split onto next line
    /<span$/N; s/<span\n/<span/

    Example: Join 3 lines as in the result from head -n1 * which output filename, 1 line and a blank line.

    head -n1 *|cut -c1-10 |sed "N;s/\n//;N;s/\n//"
    2 G   W + H W WORK Gets [newline] and HOLD appended .
    2 h   W H HOLD gets contents of WORK .
    2 H   H+W H Append WORK to HOLD .
    2 g   H W contents of the HOLD get put to WORK
    2 x   W X H Exchange the contents of WORK and HOLD .
    2 P Print. Copy WORK to the standard output.
    (useful if -n option was used to supress normal output.)
    2 p print. Copy the initial segment of the WORK area through the first new-line to the standard output.
    1 a text append text after outputting the selcted line.
    If the text is several lines, all (but the last) must end with \ .

    After the head tag, append a title tag, "nice document" and the ending title tag:

    /<head>/a <title>\
    nice document\
    1 i text insert text before outputting the selected line.
    '</body>'i <style>\
    td {text-align:right}\
    2 d delete WORK. Start the next cycle.
    2 c\ text change. Delete WORK. Output text. Start the next cycle.
    2 D Delete the initial segment of WORK through the first new-line. Start the next cycle.
    1 r file read file and output it.
    file must be the last thing on the line and be preceeded by exactly one blank.
    1 = output the current line number of input.
    (good for debugging) (The current line number changes.)
    2 b label Branch to the command bearing the :label.
    If label is empty, branch to the end of the script.
    2 t label test. Branch to the command bearing the :label if any substitutions have been made since the most recent reading of an input line or execution of a t.
    If label is empty, branch to the end of the script.
    0 :label define a label for b and t commands to branch to.
    1 q quit.
    1 w file write. Append WORK to file .
    file must be the last thing on the line and be preceded by exactly one blank.
    The first occurrence of w causes file to be cleared.
    Each time sed is used, file is overwritten.

    Each w file is created before processing begins.
    There can be at most 10 distinct w file arguments.

    2 { Execute the following commands through a matching } only when WORK is selected.
    2 ! function Don't apply the function (or group, if function is {) only to lines not selected by the address(es).
      # comment Special case: On the first line #n, the default output will be suppressed.
    2 y/string1/string2/

    Transform. Replace all occurrences of characters in string1 with the corresponding characters in string2.
    string1 and string2 must have the same number of characters.
    to change all uppercase to lowercase

    2 l (ell) output WORK area in an unambiguous form.
    Non-printable characters are displayed in octal notation and
    long lines are folded.
    consider the strings command.

    create a file with control V, C, and M using:
    ex -c p funnyfile

    now see the control characters using sed:
    sed "l" funnyfile

    \026 is octal ^V
    \003 is octal ^C
    \r is the ^M (return character)
    \n is the linefeed
    $ is the

    Dec 31 19:37
    19:37 Dec 31

    command  results
    s/... .. ..:../& &/ duplicates the entire "date time" string
    Dec 31 19:37 Dec 31 19:37
    s/... ..// deletes the first Dec 31 19:37 Dec 31 19:37
    s/..:..//2deletes the 2nd 19:37 19:37 Dec 31
    (I expect you could use the X command)

    Take lines with message followed by target of message and put target at end of message:

    Host not found: 3(NXDOMAIN)
    Host not found: 3(NXDOMAIN)
    sed "/NX/N;/NX/s/\n//" 
    Host not found: 3(NXDOMAIN)abt
    Host not found: 3(NXDOMAIN)ach
    Change 123,456 to 123456
    substitute from 1 to 3 characters followed by a comma to the same 1 to 3 characters comma and a zz substitute ,zz to null
    s/.\{1,3\},/&zz/ ; s/,zz//
    The new-line escape sequence, \n, matches a new-line embedded in WORK.

    A period . does not match the ending new-line of WORK.
    Use the negation function ! to have commands applied only to non-selected WORK areas .
    Backslashes in text are treated like backslashes in the replacement string of an s command, and may be used to protect initial blanks and tabs against the stripping that is done on every script line.

    command list example:

    s/xxx/yyy/ { n b top }

    See also:tr (translate single characters), awk, ed, grep(includes regular expression discussion), environ

    9 May 10, copy pasted list from a PDF and ended up with x'EF 82 B7 ' . Only way I could get them out was to copy the file and delete everything except the x'EF 82 B7 ' then prefix it with s/ and suffix it with /zzz/g )
    Gotta be a better way! (see Replacing 3byte strings above

    Modified to Real HTML by Dennis German 12 Mar 2002 see also sed-3.02, see also sedFAQ,

    Some information taken from SunOS 5.4 man page of 14 Sep 1992

    Mac OSX uses old version of sed which, among other things, does not support \n as a new-line in string2 of the substitute command! (see sedFAQ4.6