see also sed.html

The sed FAQ

Table of Contents
see also sedFAQ

3. TECHNICAL

3.1. More detailed explanation of basic sed

3.1.1. Regular Expressions *
LeftHandSide
s/LHS/rrr/

3.1.2. Escape characters*

RightHandSide

s/lll/RHS/

3.1.3. Substitution switches

N - Nth match, if omitted, first match.
g - Global aka: ALL.ex: %s/xdx/xx/g
p - Print.
w file - Write pattern space to file if a replacement was done.

3.2. Common useful one-line sed scripts

———»

3.3. Addressing and address ranges

3.4. Address ranges in GNU sed and HHsed

3.5. Debugging sed scripts

  the "=" command, prints the source line number

 p
 \i
 +++ script point #4 +++
  sd sed debugger

3.6. Notes about s2p, the sed-to-perl translator

3.7. GNU/POSIX extensions to regular expressions

3.2. Common useful one-line sed scripts

   # for UNIX: convert DOS newlines (CR/LF) to Unix format
   sed 's/^M$//' file 
   # for DOS: convert Unix newlines (LF) to DOS format
   sed 's/$//' file         or            sed -n p file                    

   # Delete leading whitespace (spaces | tabs)  
   # (this aligns all text flush left). ^t represents a true tab character.
   sed 's/^[ ^t]*//' file
				   # trailing 
				   sed 's/[ ^t]*$//' file                

   # Delete leading and trailing whitespace
   sed 's/^[ ^t]*//;s/[ ^]*$//' file   

   # Substitute "foo" with "bar" ONLY for lines which contain "baz"
   sed '/baz/s/foo/bar/g' file

   # Delete all CONSECUTIVE null lines from file except the first.
   #  also deletes all nulllines from top and end of file. (emulates "cat -s")
   sed '/./,/^$/!d' file       #  allows 0 null line at top, 1 at EOF
   sed '/^$/N;/\n$/D' file     #  allows 1 null line at top, 0 at EOF

   # Delete all leading null lines 
   sed '/./,$!d' file

   # Delete all trailing null lines 
   sed -e :a -e '/^\n*$/{$d;N;};/\n$/ba' file

   # If a line ends with a backslash, join the next line to it.
   sed -e :a -e '/\\$/N; s/\\\n//; ta' file

   # If a line begins with an equal sign, append it to the previous
   # line (and replace the "=" with a single space).
   sed -e :a -e '$!N;s/\n=/ /;ta' -e 'P;D' file



4. EXAMPLES

ONE-CHARACTER QUESTIONS

4.01. insert a newline into the RHS* of a substitution

4.02. representng control-codes or non-printable characters

4.03. convert files with toggle characters,

like +this+, to look like [i]this[/i]

CHANGING STRINGS

4.10. perform a case-insensitive search

4.11. match only the first occurrence of a pattern in a file.

4.12. parse a comma-delimited (CSV) data file

4.13. handle fixed-length, columnar data

4.14. commify a string of numbers (i.e.1234 -> 1,234)

GNU sed:
sed ':a ; s/\B[0-9]\{3\}\>/,&/i ; ta' *
other seds: sed -e :a -e 's/\(.*[0-9]\)\([0-9]\{3\}\)/\1,\2/;ta'

4.15. prevent regex expansion on substitutions

4.16. convert a string to all lowercase or capital letters

CHANGING BLOCKS (consecutive lines)

4.20. change only one section of a file

4.21. delete or change a block of text if the block contains a

certain regular expression

4.22. locate a paragraph of text if the paragraph contains a

certain regular expression

4.23. match a block of specific consecutive lines

4.23.1. "/range/, /expression/"

4.23.2. "multi-line\nexpression"

4.23.3. block of "literal strings"

4.24. address all the lines between RE1 and RE2,

excluding the lines themselves

4.25. join two lines if line #1 ends in a [certain string]

4.26. join two lines if line #2 begins in a [certain string]

4.27. change all paragraphs to long lines

SHELL AND ENVIRONMENT

4.30. read environment variables with sed ...

4.31.1. on Unix platforms

4.31.2. on DOS platforms

4.32. export or pass variables back

4.32.1. on Unix platforms

4.32.2. on DOS platforms

4.33. handle shell quoting in sed

FILES, DIRECTORIES, AND PATHS

4.40. read (insert/add) a file at the top of a textfile

4.41. make substitutions in every file in a directory, or

in a complete directory tree

4.41.1. ssed solution

4.41.2. Unix solution

4.41.3. DOS solution

4.42. replace "/some/ UNIX/path"

4.43. replace "C:\SOME\DOS\PATH"

4.44. includes

5. WHY ISN'T THIS WORKING?

6. OTHER ISSUES

5.1. variables don't get expanded

5.2. 'p'outputs duplicate lines sometimes.

5.3. DOS version quits

5.4. RE isn't matching/deleting

(Or, "Greedy vs. stingy pattern matching")

5.5. CSDPMI*B.ZIP

5.6. man pages for GNU sed

5.7. version display

5.8. exit code

5.9. 'r' command isn't inserting the file

5.10. match newline using the \n

5.11. "event not found"

6.1. Where to get help

6.2. compare with awk, perl, and other utilities

6.3. When to use sed*

6.4. When NOT to*

6.5. When to use awk or perl instead

6.6. limitations

6.7. incompatibilities

6.7.1. command line

6.7.2. Using comment*

6.7.3. Special syntax in REs

6.7.4. Word boundaries

6.7.5. GNU N*operates differently

7. KNOWN BUGS AMONG SED VERSIONS

7.2. GNU sed v4.0 - v4.0.5

7.3. GNU sed v3.02.80

7.4. GNU sed v3.02

7.5. GNU sed v2.05

7.6. GNU sed v1.18

7.7. GNU sed v1.03

7.1. ssed v3.59

7.8. sed v1.6 (Briscoe)

7.9. sed v1.5 (Helman)

7.10. sedmod v1.0 (Chen)

7.11. HP-UX sed

7.12. SunOS sed v4.1

7.13. SunOS sed v5.6

7.14. Ultrix sed v4.3

7.15. Digital Unix sed

1. GENERAL INFORMATION

 

1.1. Introduction - How this FAQ is organized

1.2. Latest version of the sed FAQ

1.3. FAQ revision information

1.4. add a question/answer to the sed FAQ?

1.5. FAQ abbreviations

1.6. Credits and acknowledgements

1.7. Standard disclaimers

2. BASIC SED

2.1. What is sed?

2.2. versions of sed , and where to get them

2.2.1. Free versions

2.2.1.1. Unix platforms

2.2.1.2. OS/2

2.2.1.3. Microsoft Windows (3x, 9x, NT, 2K)

2.2.1.4. MS-DOS

2.2.1.5. CP/M

2.2.1.6. Macintosh v8 or v9

2.2.2. Shareware and Commercial versions

2.2.2.1. Unix platforms

2.2.2.2. OS/2

2.2.2.3. Windows 9x, NT, 2k

2.2.2.4. MS-DOS

2.3. learn to use sed

2.3.1. Books

2.3.2. Mailing list

2.3.3. Tutorials, electronic text

2.3.4. General web and ftp sites


RegularExpression RE
. any 1 char ? 0 or 1 char
* any number of chars The RE '.*' is designed to be "greedy" (i.e., matching as many characters as possible).
X.*Y matches X followed by a char and stuff thru a Y
^ anchor at beginning, $ anchor at end .
[a-h] a single char if in range a-h ;
[0-9] a digit
[0-9][0-9] 2 digits
[bBrReExyz] 1 char b or B or r or R or e or E or z or y or z
[^rwx] any 1 char except r or w or x
[:alnum:] [:alpha:] [:lower:] [:upper:] --,
[:digit:] [:punct:] 1 char
[:space:] [:blank:] --'
\1 or \2 or \3 ... \8 or \9 is a backref
use \ to escape a . or / or ^ or $ or [ ...
ex: / [1-9]\/[ 0123][:digit:]\/0[0-4] a date from \" 1/ 0/00\" thru \" 9/39/04\" (yuck)
Archive-Name: editor-faq/sed Posting-Frequency: irregular Last-modified: 10 March 2003 Version: 015
URL: http://sed.sourceforge.net/sedfaq.html
Maintainer: Eric Pement (pemente@northpark.edu)
links to http://www.student.northpark.edu/pemente/sed/sedfaq.html
Hacked by Dennis German