Stream EDit non-interactive edit

As of 3/19/13 the current version of sed is 4.2.1.
Version 4.2.1 (Aug 24, '10) is a minor version that includes these bug fixes:
A fix to a parsing problem.
A fix to ensure the security contexts are preserved by -i under SELinux.
A fix that ensures the temporary files for sed -i are not made group or world-readable until they are complete.
Version 4.1.5 is a minor bug fix release which fixes the handling of a negative character class when it doesn't include a closed bracket.

Most of this is based on the 3.02 version and still is applicable.

sed [-n] [-e script [-e script]] [-f sfile] [file ...]

sed copies file to STDOUT, editing it according to a script of commands.

Examples:
sed "s/blue/green/" infile > outfile
sed -e "s/old/older/" -e "s/new/old" needsMultipleChanges.txt
sed -f bunch-o-cmds.sed afile anotherfile > da-SEDed-file
sed "s/;/;\n/" > myprog.c make multi-statment-lines be on seperate lines)

Warning: sed "s/xx/yy/" file > file results in an empty file!

note: sed "s/xx/yy/" file doesn't update file, rather outputs to STDOUT i.e terminal

-n no default output.
-excute command
-f scriptfile

Normally, sed compiles the entire script then cycles through the input :

  • reading a line into a WORK area (unless there is something left after a D ),
  • deletes trailing \n
  • applying, in sequence, all commands whose address select the WORK area , and
  • writing the resulting WORK area to standard output including a \n

    A script consists of editing commands, one per line:

    [address [,address] ]     function    [arguments]

    A function includes substitute, append, change, delete, and others
    Some of the functions use a HOLD area to save all or part of the WORK area for subsequent retrieval.

    arguments depend on the function


    Notes on regular expressions:

    Function Description
    2 s/regexp/replacement/f

    Substitute the replacement string for instances of the regexp in WORK.

    The character immediating following the s will be used as the delimeter( instead of /). Useful if string contains slashes.

      /bin/df | sed s:/Volumes/::
    For a fuller description see ed(1).

    flags are:

    • g Global. Substitute all nonoverlapping instances of regexp.

    • n = 1 - 512. Substitute for just the nth occurrence of the regexp.
      n.b. using 2 for flags changes
      the second occurance IN THE RANGE OF LINES,
      not the second occurance in every line.
      i.e. only one string will be changed unless you use g

    • p Print WORK if a replacement was made.

    • i ignore case (not supported in Mac OSX version)

    • w file Write the WORK area, appending to file if a replacement was made.

    Special strings:

    • Bracket Expression [] list (ex: [13579], range (ex:[4-9]) and named (ex: [[:digit]]
    • Repitition \{1,4\}
    • Subexpression /chapter \(.*\) / matches chapter number as subexpression \1 on the RHS as a backreferene.
    • & in the replacment signifies the matched expression
      Example:
      s/<td/& align=right/
      changes
      <td to <td align=right>

    2 n       W STDOUT
       in W
    next. Output WORK ; read the next line of input to WORK.
    2 N   W + \n + in WAppend a new-line and the Next line of input to WORK.
    Resulting in an embedded new-line, that can be substituted by null, joining the current WORK and the next input line.
    Example: Join continuation of a span tag had been split onto next line
    /<span$/N; s/<span\n/<span/

    Example: Join 3 lines as in the result from head -n1 * which output filename, 1 line and a blank line.

    head -n1 *|cut -c1-10 |sed "N;s/\n//;N;s/\n//"
    2 G   W + H W WORK Gets [newline] and HOLD appended .
    2 h   W H HOLD gets contents of WORK .
    2 H   H+W H Append WORK to HOLD .
    2 g   H W contents of the HOLD get put to WORK
    2 x   W X H Exchange the contents of WORK and HOLD .
    2 P Print. Copy WORK to the standard output.
    (useful if -n option was used to supress normal output.)
    2 p print. Copy the initial segment of the WORK area through the first new-line to the standard output.
    1 a text append text after outputting the selcted line.
    If the text is several lines, all (but the last) must end with \ .

    After the head tag, append a title tag, "nice document" and the ending title tag:

    /<head>/a <title>\
    nice document\
    <title>
    1 i text insert text before outputting the selected line.
    '</body>'i <style>\
    td {text-align:right}\
    </style>
    2 d delete WORK. Start the next cycle.
    2 c\ text change. Delete WORK. Output text. Start the next cycle.
    2 D Delete the initial segment of WORK through the first new-line. Start the next cycle.
    1 r file read file and output it.
    file must be the last thing on the line and be preceeded by exactly one blank.
    1 = output the current line number of input.
    (good for debugging) (The current line number changes.)
    2 b label Branch to the command bearing the :label.
    If label is empty, branch to the end of the script.
    2 t label test. Branch to the command bearing the :label if any substitutions have been made since the most recent reading of an input line or execution of a t.
    If label is empty, branch to the end of the script.
    0 :label define a label for b and t commands to branch to.
    1 q quit.
    1 w file write. Append WORK to file .
    file must be the last thing on the line and be preceded by exactly one blank.
    The first occurrence of w causes file to be cleared.
    Each time sed is used, file is overwritten.

    Each w file is created before processing begins.
    There can be at most 10 distinct w file arguments.

    2 { Execute the following commands through a matching } only when WORK is selected.
    2 ! function Don't apply the function (or group, if function is {) only to lines not selected by the address(es).
      # comment Special case: On the first line #n, the default output will be suppressed.
    2 y/string1/string2/

    Transform. Replace all occurrences of characters in string1 with the corresponding characters in string2.
    string1 and string2 must have the same number of characters.
      example:
    to change all uppercase to lowercase
    y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/

    2 l (ell) output WORK area in an unambiguous form.
    Non-printable characters are displayed in octal notation and
    long lines are folded.
    consider the strings command.

     example:
    create a file with control V, C, and M using:
    ex -c p funnyfile
    ---^V,,,^C+++^M===

    now see the control characters using sed:
    sed "l" funnyfile
    ---\026,,,\003+++\r===\n$

    \026 is octal ^V
    \003 is octal ^C
    \r is the ^M (return character)
    \n is the linefeed
    $ is the

    Example:
    change
    Dec 31 19:37
      to
    19:37 Dec 31

    command  results
    s/... .. ..:../& &/ duplicates the entire "date time" string
    Dec 31 19:37 Dec 31 19:37
    s/... ..// deletes the first Dec 31 19:37 Dec 31 19:37
    s/..:..//2deletes the 2nd 19:37 19:37 Dec 31
    (I expect you could use the X command)

    Take lines with message followed by target of message and put target at end of message:

    Host abt.info not found: 3(NXDOMAIN)
    abt
    Host ach.info not found: 3(NXDOMAIN)
    ach
    acj
    
    sed "/NX/N;/NX/s/\n//" 
    
    Host abt.info not found: 3(NXDOMAIN)abt
    Host ach.info not found: 3(NXDOMAIN)ach
    acj
    Change 123,456 to 123456
    substitute from 1 to 3 characters followed by a comma to the same 1 to 3 characters comma and a zz substitute ,zz to null
    s/.\{1,3\},/&zz/ ; s/,zz//
    The new-line escape sequence, \n, matches a new-line embedded in WORK.

    A period . does not match the ending new-line of WORK.
    Use the negation function ! to have commands applied only to non-selected WORK areas .
    Backslashes in text are treated like backslashes in the replacement string of an s command, and may be used to protect initial blanks and tabs against the stripping that is done on every script line.

    command list example:

    s/xxx/yyy/ { n b top }


    See also:tr (translate single characters), awk, ed, grep(includes regular expression discussion), environ

    9 May 10, copy pasted list from a PDF and ended up with x'EF 82 B7 ' . Only way I could get them out was to copy the file and delete everything except the x'EF 82 B7 ' then prefix it with s/ and suffix it with /zzz/g )
    Gotta be a better way! (see Replacing 3byte strings above

    Modified to Real HTML by Dennis German 12 Mar 2002 see also sed-3.02, see also sedFAQ, gnu.org/sed


    Some information taken from SunOS 5.4 man page of 14 Sep 1992

    Mac OSX uses old version of sed which, among other things, does not support \n as a new-line in string2 of the substitute command! (see sedFAQ4.6