Stream EDit

non-interactive edit

sed [-n] [[-e] script] [-f sfile] [file]

Copies file to STDOUT, editing it according to a script of commands.

sed s/blue/green/ infile > outfile
sed -e "s/old/older/" -e "s/new/old" needsMultipleChanges.txt
sed -f bunch-o-cmds.sed afile anotherfile > da-SEDed-file
sed "s/;/;\n/" > myprog.c Make multi-statment-lines be on seperate lines)

Warning: sed "s/xx/yy/" file > file results in an empty file! Use sed --in-place "s/xx/yy/" file
Note: sed "s/xx/yy/" file doesn't update file, rather outputs to STDOUT (perhaps the terminal )

-n no default output.
-excute command
-f scriptfile
suppress automatic printing of pattern space
-e script
add the script to the commands to be executed
-f script-file
add the contents of script-file to the commands to be executed
--follow-symlinks follow symlinks when processing in place
in place edit (makes backup if suffix is supplied)
-l n
specify the desired line-wrap length for the l command
--posix disable all GNU extensions.
use extended regular expressions in the script (for portability use POSIX -E).
consider files as separate rather than as a single
continuous long stream.
--sandbox operate in sandbox mode.
load minimal amounts of data from the input files and flush the output buffers more often
separate lines by NUL characters
--help display this help and exit
--version output version information and exit

Normally, sed compiles the entire script then cycles through the input:

  • reading a line into a WORK area (unless there is something left after a D ),
  • deleting trailing
  • applying, in sequence, all commands whose address select the WORK area and
  • writing the resulting WORK area to standard output including a

    A script consists of editing commands, one per line:

    [address[,address] ]   function    [arguments]

    A function includes substitute, append, change, delete, and others
    Some of the functions use a HOLD area to save all or part of the WORK area for subsequent retrieval.

    arguments depend on the function

    Notes on regular expressions:

    Function Description
    2 s/regexp/replacement/f

    Substitute the replacement string for instances of the regexp in WORK.

    Character immediating following s will be used as the delimeter ( not necessarily /). Useful if regexp contains slashes.

      /bin/df | sed s:/Volumes/::
    See ed(1).

    flags are:

    • g Global. Substitute all nonoverlapping instances of regexp
    • n = 1 - 512. Substitute for just the nth occurrence of the regexp
      n.b. using 2 for flags changes the second occurance IN THE RANGE OF LINES,
      not the second occurance in every line.
      i.e. only one string will be changed unless you use g
    • p Print WORK if a replacement was made.
    • i ignore case (not supported in Mac OSX version)
    • w file Write the WORK area, appending to file if a replacement was made.

    Special strings:

    • Bracket Expression [ … ] list (ex: [13579], range (ex:[4-9]) and named (ex: [[:digit:]]
    • Repetition ¿\{1,4\} (in this example: at least 1 and as many as 4 ¿s)
    • Subexpression /chapter \(.*\)/ matches chapter numbers (or letters);
      subexpression \1 on the RightHandSide (RHS) as a backreferene.
    •  &  in the replacment signifies the matched regexp Example:
      s/<td/& align=right/
      <td to <td align=right>

    2 n       W STDOUT
       in W
    next. Output WORK ; read the next line of input to WORK.
    2 N   W + \n + in WAppend a new-line and the Next line of input to WORK.
    Resulting in an embedded new-line, that can be substituted by null, joining the current WORK and the next input line.
    Example: Join continuation of a span tag had been split onto next line
    /<span$/N; s/<span\n/<span/

    Example: Join 3 lines as in the result from head -n1 * into one line:

    >head -n1 mys*  |cut -c1-50
    ==> mysql <==
    if [ $# -lt 1 ] ; then 
    ==> mysql.server <==
    ==> mysql.server.vvvqq <==
    ==> mysqladmin.vvvqqq <==
    echo /usr/local/mysql/bin/mysqladmin $*
    > head -n1 *|cut -c1-50 |sed "N;s/\n//;N;s/\n//" 
    ==> mysql <==if [ $# -lt 1 ] ; then 
    ==> mysql.server <==#!/bin/sh
    ==> mysql.server.vvvqq <==#!/bin/sh
    2 G   W + H W WORK Gets [newline] and HOLD appended .
    2 h   W H HOLD gets contents of WORK .
    2 H   H+W H Append WORK to HOLD .
    2 g   H W contents of the HOLD get put to WORK
    2 x   W X H Exchange the contents of WORK and HOLD .
    2 P Print. Copy WORK to the standard output.
    (useful if -n option was used to supress normal output.)
    2 p print. Copy the initial segment of the WORK area through the first new-line to the standard output.
    1 a text append text after outputting the selcted line.
    If the text is several lines, all (but the last) must end with \ .

    After the head tag, append a title tag, "nice document" and the ending title tag:

    /<head>/a <title>\
    nice document\
    1 i text insert text before outputting the selected line. ( it seems that functions that insert should be before functions that reference the entire file.,
    '</body>'i <style>\
    td {text-align:right}\
    2 c\ text change. Delete WORK. Output text. Start the next cycle.
    2 d delete WORK. Start the next cycle.
    2 D Delete the initial segment of WORK through the first new-line. Start the next cycle.
    1 r file read file
    file must be the last thing on the line and be preceeded by exactly one blank.
    1 w file write. Append WORK to file
    file must be the last thing on the line and be preceeded by exactly one blank.
    The first occurrence of w causes file to be cleared. Each time sed is used, file is overwritten.

    Each w file is created before processing begins.
    There can be at most 10 distinct w file arguments.

    1 = output the current line number of input.
    (good for debugging) (The current line number changes.)
    2 b [label] Branch to the command bearing the :label.
    If label is omitted, branch to the end of the script.
    2 t [label] test. Branch to the command bearing the :label if any substitutions have been made since the most recent reading of an input line or execution of a t.
    If label is omitted, branch to the end of the script.
    0 :label define a label for b and t commands to branch to.
    1 q quit.
    2 { Execute the following commands through a matching } only when WORK is selected.
    2 ! function Don't apply the function (or group, if function is {…) only to lines not selected by the address(es).
      # comment Special case: On the first line #n, the default output will be suppressed.
    2 y/string1/string2/

    Transform. Replace all occurrences of characters in string1 with the corresponding characters in string2.
    string1 and string2 must have the same number of characters.
    to change all uppercase to lowercase

    2 l (ell) output WORK area in an unambiguous form.
    Non-printable characters are displayed in octal notation and
    long lines are folded.
    EF 82 B7 are NOT altered

    Consider the strings command instead.

    Dec 31 19:37
    19:37 Dec 31

    command  results
    s/... .. ..:../& &/ duplicates the entire "date time" string
    Dec 31 19:37 Dec 31 19:37
    s/... ..// deletes the first Dec 31 19:37 Dec 31 19:37
    s/..:..//2deletes the 2nd 19:37 19:37 Dec 31
    (I expect you could use the X command)

    Take lines with message followed by target of message and put target at end of message:

    Host not found: 3(NXDOMAIN)
    Host not found: 3(NXDOMAIN)
    sed "/NX/N;/NX/s/\n//" 
    Host not found: 3(NXDOMAIN)abt
    Host not found: 3(NXDOMAIN)ach
    Change 123,456 to 123456
    substitute from 1 to 3 characters followed by a comma to the same 1 to 3 characters comma and a zz substitute ,zz to null
    s/.\{1,3\},/&zz/ ; s/,zz//
    The new-line escape sequence, \n, matches a new-line embedded in WORK.

    A period . does not match the ending new-line of WORK.
    Use the negation function ! to have commands applied only to non-selected WORK areas .
    Backslashes in text are treated like backslashes in the replacement string of an s command, and may be used to protect initial blanks and tabs against the stripping that is done on every script line.

    command list example:

    s/xxx/yyy/ { n b top }

    See also:tr (translate single characters), awk, ed, grep(includes regular expression discussion), environ

    9 May 10, copy pasted list from a PDF and ended up with x'EF 82 B7 ' . Only way I could get them out was to copy the file and delete everything except the x'EF 82 B7 ' then prefix it with s/ and suffix it with /zzz/g )
    Gotta be a better way! (see Replacing 3byte strings above

    Modified to Real HTML by Dennis German 12 Mar 2002 see also sed-3.02, see also sedFAQ,

    Some information taken from SunOS 5.4 man page of 14 Sep 1992 As of 3/19/13 the current version of sed is 4.2.1.
    Most of this is based on the 3.02 version and still is applicable.

    Version 4.2.1 (Aug 24, '10) is a minor version that includes these bug fixes:
    A fix to a parsing problem.
    A fix to ensure the security contexts are preserved by -i under SELinux.
    A fix that ensures the temporary files for sed -i are not made group or world-readable until they are complete.
    Version 4.1.5 is a minor bug fix release which fixes the handling of a negative character class when it doesn't include a closed bracket.

    Mac OSX uses old version of sed which, among other things, does not support \n as a new-line in string2 of the substitute command! (see sedFAQ4.6