Frequently Asked Questions about
sed, the stream editor
version 014*
Fri, 28 Apr 2000
from FAQS.org Severly hacked into Real HTML by hand
and broken into 2 parts by
Dennis German 28 July 2002 second part
1. GENERAL INFORMATION
1.1. Introduction - How this FAQ is organized
1.2. Latest version of the sed FAQ
1.3. FAQ revision information
1.4. How do I add a question/answer to the sed FAQ?
1.5. FAQ abbreviations
1.6. Credits and acknowledgements
1.7. Standard disclaimers
2. BASIC SED
2.1. What is sed?
2.2. What versions of sed are there, and where can I get them?
2.2.1. Free versions
2.2.1.1. Unix platforms
2.2.1.2. OS/2
2.2.1.3. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)
2.2.1.4. MS-DOS
2.2.1.5. CP/M
2.2.2. Shareware and Commercial versions
2.2.2.1. Unix platforms
2.2.2.2. OS/2
2.2.2.3. Windows 95/98, Windows NT, Windows 2000
2.2.2.4. MS-DOS
2.3. Where can I learn to use sed?
2.3.1. Books
2.3.2. Mailing list
2.3.3. Tutorials, electronic text
2.3.4. General web and ftp sites
This FAQ assumes you have already read the man page for sed
(even if you didn't exactly understand it).
substitution features.
Sed is often used as a find-and-replace tool.
sed 's/Glenn/Harold/g' oldfile >newfile
will replace every occurrence of "Glenn" with the word "Harold",
wherever it occurs in the file. The "find" portion is a regular
expression ("RE"), which can be a simple word or may contain
special characters to allow greater flexibility (for example, to
prevent "Glenn" from also matching "Glennon").
to add 8 spaces to the left edge of a
file, so when printed , the printing wouldn't begin at the
absolute left edge of a piece of paper.
sed 's/^/ /' myfile >newfile # my first sed script
sed 's/^/ /' myfile | lp # my next sed script
display only one paragraph of a file,
beginning at the phrase "and where it came" and ending at the
phrase "for all people" by using address ranges of patterns to be matched.
sed -n '/and where it came/,/for all people/p' myfile
Sed's normally prints (i.e., display or show on screen)
the entire file, including the parts that haven't been altered,
unless you use the -n switch. The "-n" stands for "no output".
used in conjunction with a 'p' command
, to print only the sections of the file that
have been specified. The -n switch with the 'p' command allow for
parts of a file to be printed (i.e., sent to the console).
only lines 12-18 of a
file
sed -n 12,18p myfile # the 'p' i.e. print
everything else BUT particular
lines, without changing the file:
sed 12,18d myfile # the 'd' i.e. delete
double-space a file :
sed G myfile >newfile
many editing commands (for deleting, adding,
substituting, etc.) can be put into a separate file and all of the commands in
the file applied to file being edited:
sed -f script.sed myfile # 'script.sed' is the file of commands
# 'myfile' is the file being changed
for good tutorials, see section 2.3.
second part of the FAQ
2.2. What versions of sed are there, and where can I get them?
2.2.1. Free versions
Note: "Free" does not mean "public domain" nor does it necessarily
mean you will never be charged for it. All versions of sed in this
section except the CP/M versions are based on the GNU general
public license and are "free software" by that standard (for
details, see http://www.gnu.org/philosophy/free-sw.html). This
means you can get the source code and develop it further.
At the URLs listed in this category, sed binaries or source code
can be downloaded and used without fees or license payments.
2.2.1.1. Unix platforms
GNU sed v3.02.80
Now a,i,c commands can accept a string after them. Range syntax now
supports "/RE/,+n" (next n lines) or "/RE/,~n" (till the next line
which is a multiple of n). NULs permitted in regexes; \n, \t, \a,
\f, \xHH hex codes supported on LHS and RHS; more changes. An alpha
test release which (if found bug-free) will become GNU sed version
3.03. Supersedes GNU sed-3.02a.
ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz
GNU sed v3.02a
Interim version with most of what is now gsed-3.02.80 (above),
which supersedes it.
GNU sed v3.02
This is the latest official version of GNU sed
ftp://ftp.gnu.org/pub/gnu/sed/sed-3.02.tar.gz
GNU sed v2.05
This version is superseded by v3.02 and v3.02.80, above.
GNU mirror sites. A list of mirror sites is at:
http://www.ensta.fr/internet/unix/GNU-archives.html
Precompiled versions:
GNU sed v3.02-4
source code and binaries for Debian GNU/Linux
http://www.debian.org/Packages/unstable/base/sed.html
GNU sed v3.02-1
source code and binaries for Debian GNU/Linux
http://www.debian.org/Packages/stable/base/sed.html
The 4.4BSD version of sed is available from any 4.4BSD-Lite2 mirror
site:
ftp://ftp.ntua.gr/pub/bsd/4.4BSD/usr/src/usr.bin/sed/
For some time, the GNU project <http://www.gnu.org> used Eric S.
Raymond's version of sed (ESR sed v1.1), but eventually dropped it
because it had too many built-in limits. In 1991 Howard Helman
modified the GNU/ESR sed and produced a flexible version of sed
v1.5 available at several sites (Helman's version permitted things
like \<...\> to delimit word boundaries, \xHH to enter hex code and
\n to indicate newlines in the replace string). This version did
not catch on with the GNU project and their version of sed has
moved in a similar but different direction.
sed v1.3, by Eric Steven Raymond (released 4 June 1998)
http://earthspace.net/~esr/sed-1.3.tar.gz
Eric Raymond <esr@snark.thyrsus.com> wrote one of the earliest
versions of sed. On his website <http://www.tuxedo.org/~esr/> which
also distributes many freeware utilities he has written or worked
on, he describes sed v1.1 this way:
"This is the fast, small sed originally distributed in the GNU
toolkit and still distributed with Minix. The GNU people ditched it
when they built their own sed around an enhanced regex package --
but it's still better for some uses (in particular, faster and less
memory-intensive)." (Version 1.3 fixes an unidentified bug and adds
the L command to hexdump the current pattern space.)
2.2.1.2. OS/2
GNU sed v3.02.80 |
http://www2s.biglobe.ne.jp/~vtgf3mpr/gnu/sed.htm |
GNU sed v2.05 (requires 'emxrt.zip', below)
http://oak.oakland.edu/pub/os2/editors/gnused.zip
http://oak.oakland.edu/pub/os2/emx09c/emxrt.zip
GNU sed v1.06
http://oak.oakland.edu/pub/os2/editors/sed106.zip
2.2.1.3. Microsoft Windows (Win3x, Win9x, WinNT, Win2K)
GNU sed v3.02.80
32-bit binaries and docs, using DJGPP compiler. For details on new
features, see Unix section, above.
http://www.cornerstonemag.com/sed/sed3028a.zip # DOS binaries
ftp://alpha.gnu.org/pub/gnu/sed/sed-3.02.80.tar.gz # source
GNU sed v3.02
32-bit binaries and source, using DJGPP compiler. Requires 80386 SX
or better. Also requires 3 CWS*.EXE extenders if run under MS-DOS.
See section 5.5 ("What is CSDPMI*B.ZIP and why do I need it?"),
below. This version will run under Windows or under MS-DOS.
The binary archive (sed302b.zip) contains 2 executables, sed.exe
and gsed.exe. sed.exe was compiled with the DJGPP regex library,
which is POSIX.2-compliant and usually runs faster; gsed.exe was
compiled with the GNU regex library, which though it runs slower
and is almost POSIX.2-compliant, it has a richer set of regexs and
will run faster on certain complex regexs which cause the DJGPP
sed.exe to run extremely slowly.
ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/
ftp://ftp.cdrom.com/.27/simtelnet/gnu/djgpp/v2gnu/
ftp://ftp.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/
ftp://ftp.cdrom.com/.27/simtelnet/gnu/djgpp/v2gnu/
GNU sed v2.05
32-bit binaries, no docs. Requires 80386 DX (SX will not run) and
must be run in a DOS window or in a full screen DOS session under
Microsoft Windows. Will not run in MS-DOS mode (outside Win/Win95).
We recommend using GNU sed v3.02 (above) instead.
http://www.simtel.net/pub/simtelnet/win95/prog/gsed205b.zip
ftp://ftp.cdrom.com/.27/simtelnet/win95/prog/
GNU sed v1.03
modified by Frank Whaley.
ftp://ftp.itribe.net/pub/virtunix/gnused.zip
Again, we recommend avoiding versions of GNU sed other than version
3.02 or 3.02.80. However, this version appears to be built on gsed
v1.03 beta as a base and then augmented farther. The authors did
not give this sed its own version number or name. Gsed v1.03 is
offered in the "Virtually UN*X" set of Win32 utilities at
<http://www.itribe.net/virtunix/>. It supports Win 95/98/NT long
filenames, and runs in a DOS session or DOS window under Microsoft
Windows, but does not run in DOS mode. This version of sed supports
hex, decimal, binary, and octal representation in expressions.
The Cygwin toolkit:
http://sourceware.cygnus.com/cygwin/
Formerly know as "GNU-Win32 tools." According to their home page,
"The Cygwin tools are Win32 ports of the popular GNU development
tools for Windows NT, 95 and 98. They function through the use of
the Cygwin library which provides a UNIX-like API on top of the
Win32 API." The version of sed used is GNU sed v3.02.
Minimalist GNU-Win32 (Mingw32):
ftp://agnes.dida.physik.uni-essen.de/home/janjaap/mingw32/binaries/sed-2.05.zip
http://agnes.dida.physik.uni-essen.de/~janjaap/mingw32/download.html
According to their home page, "The Minimalist GNU-Win32 Package (or
Mingw32) is simply a set of header files and initialization code
which allows a GNU compiler to link programs with one of the C
run-time libraries provided by Microsoft. By default it uses
CRTDLL, which is built into all Win32 operating systems." The
download page says Mingw32 programs "behave like you would expect
from a Windows application. They support drive letters, for
example. A side effect of using CRTDLL is that Mingw32 is
thread-safe, while Cygwin32 is not." The version of sed used is GNU
sed v2.05.
sed v1.5 (a/k/a HHsed), by Howard Helman
Compiled with Mingw32 for 32-bit environments described above. This
version should support Win95 long filenames.
http://www.dbnet.ece.ntua.gr/~george/sed/sed15.exe
http://www.cornerstonemag.com/sed/sed15exe.zip
2.2.1.4. MS-DOS
sed v1.5 (a/k/a HHsed), by Howard Helman
uncompiled source code (Turbo C)
ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15.zip
ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15.zip
ftp://oak.oakland.edu/pub/simtelnet/msdos/txtutl/
ftp://uiarchive.uiuc.edu/pub/systems/pc/simtelnet/msdos/txtutl/
DOS executable and documentation
ftp://ftp.simtel.net/pub/simtelnet/msdos/txtutl/sed15x.zip
ftp://ftp.cdrom.com/pub/simtelnet/msdos/txtutl/sed15x.zip
ftp://oak.oakland.edu/pub/simtelnet/msdos/txtutl/
ftp://uiarchive.uiuc.edu/pub/systems/pc/simtelnet/msdos/txtutl/
sedmod v1.0, by Hern Chen
http://www.ptug.org/sed/SEDMOD10.ZIP
http://www.cornerstonemag.com/sed/sedmod10.zip
ftp://garbo.uwasa.fi/pc/unix/
CompuServe DTPFORUM, "PC DTP Tools" library, file SEDMOD.ZIP
GNU sed v3.02.80
See section 2.2.1.3 ("Microsoft Windows"), above.
GNU sed v3.02
See section 2.2.1.3 ("Microsoft Windows"), above.
GNU sed v2.05
Does not run under MS-DOS.
GNU sed v1.18
32-bit binaries and source, using DJGPP compiler. Requires 80386 SX
or better. Also requires 3 CWS*.EXE extenders on the path. See
section 5.5 ("What is CSDPMI*B.ZIP and why do I need it?"), below.
We recommend using GNU sed v3.02 (above) instead.
http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118b.zip
ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/
http://www.simtel.net/pub/simtelnet/gnu/djgpp/v2gnu/sed118s.zip
ftp://ftp.cdrom.com/pub/simtelnet/gnu/djgpp/v2gnu/
GNU sed v1.06
16-bit binaries and source. Should run under any MS-DOS system.
simtel.net/pub/simtelnet/gnu/gnuish/sed106.zip
ftp://ftp.cdrom.com/pub/simtelnet/gnu/gnuish/
2.2.1.5. CP/M
ssed v2.2, by Chuck A. Forsberg
http://oak.oakland.edu/pub/cpm/txtutl/ssed22.lbr
Written for CP/M, ssed (for "small/stupid stream editor) supports
only the a(ppend), c(hange), d(elete) and i(nsert) options, and
apparently doesn't support regular expressions. It does have a -u
option to "unsqueeze" compressed files and was used mainly in
conjunction with dif.com for source code maintenance.
change, by Michael M. Rubenstein
http://oak.oakland.edu/pub/cpm/txtutl/ttools.lbr
Rubenstein probably felt that "sed" was an obscure name, so he
renamed it CHANGE.COM (the TTOOLS.LBR archive member CHANGE.CZM is
a "crunched" file). Unlike ssed, change supports full RE's except
for grouping and backreferences, and its only function is for
global substitution.
2.2.2. Shareware and Commercial versions
2.2.2.1. Unix platforms
2.2.2.2. OS/2
Hamilton Labs:
hamiltonlabs.com/cshell.htm
A sizable set of Unix/C shell utilities designed for OS/2.
2.2.2.3. Windows 95/98, Windows NT, Windows 2000
Hamilton Labs:
http://www.hamiltonlabs.com/cshell.htm
A sizable set of Unix/C shell utilities designed for Win9x, WinNT,
and Win2K. Price is $350 in the US, $395 elsewhere, with FedEx
shipping, unconditional guarantee, unlimited support and free
updates. A demo version of the suite can be downloaded from this
site, but a stand-alone copy of sed is not available.
Interix:
interix.com
Interix (formerly known as OpenNT) is advertised as "a complete
UNIX system environment running natively on Microsoft Windows NT",
and is licensed and supported by Softway Systems. It offers over
200 Unix utilities, and supports Unix shells, sockets, networking,
and more. A single-user edition runs about $200. A free demo or
evaluation copy will run for 31 days and then quit; to continue
using it, you must purchase the commercial version.
MKS NuTCRACKER Professional
datafocus.com/products/nutc/
A different, yet related product line offered by MKS (Mortice Kern
Systems, below); the awkward spelling "NuTCRACKER" is intentional.
Various packages offer hundreds of Unix utilities for Win32
environments. Sed is not available as a separate product.
UnixDos:
unixdos.com
UnixDos is a suite of 82 Unix utilities ported over to the Windows
environments. There are 16-bit versions for Win 3.1 and 32-bit
versions for WinNT/Win95. It is distributed as uncrippled shareware
for the first 30 days. After the test period, the utilities will
not run and you must pay the registration fee of $50.
Their version of sed supports "\n" in the RHS of expressions, and
increases the length of input lines to 10,000 characters. By
special arrangement with the owners, persons who want a licensed
version of sed *only* (without the other utilities) may pay a
license fee of $10.
U/WIN:
research.att.com/sw/tools/uwin/
U/WIN is a suite of Unix utilities created for WinNT and Win95
systems. It is owned by AT&T, created by David Korn (author of the
Unix korn shell), and is freely distributed only to educational
institutions, AT&T employees, or certain researchers; all others
must pay a fee after a 90-day evaluation period expires. U/WIN
operates best with the NTFS (WinNT file system) but will run in
degraded mode with the FAT file system and in further degraded mode
under Win95. A minimal installation takes about 25 to 30 megs of
disk space. Sed is not available as a separate file for download,
but comes with the suite.
2.2.2.4. MS-DOS
Mix C/Utilities Toolchest |
http://www.mixsoftware.com/product/utility.htm
According to their web page, "The C/Utilities Toolchest adds over
40 powerful UNIX utilities to your MS-DOS operating system. The
result is an environment very similar to UNIX operating systems,
yet 100% compatible with MS-DOS programs and commands." The
toolchest costs $19.95, with source code available for an
additional fee. Mix C's version of sed is not available separately.
MKS (Mortice Kern Systems) Toolkit
http://www.mks.com
Sed comes bundled with the MKS Toolkit, which is distributed only
as commercial software; it is not available separately.
Thompson Automation Software
teleport.com/~thompson/
The Thompson Toolkit contains over 100 familiar Unix utilities,
including a version of the Unix Korn shell. It runs under MS-DOS,
OS/2, Win 3.0/3.1, Win95, and WinNT. Sed is one of the utilities,
though Thompson is better known for its version of awk for DOS,
TAWK. The toolkit runs about $150; sed is not available separately.
2.3. Where can I learn to use sed?
2.3.1. Books
_Sed & Awk, by Dale Dougherty & Arnold Robbins
(Sebastopol, Calif: O'Reilly and Associates, 1997)
ISBN 1-56592-225-5
About 40 percent of this book is devoted to sed, and maybe 50
percent is devoted to awk. The other 10 percent is given to regular
expressions and concepts which are common to both tools. If you
prefer hard copy, this is definitely the best single place to learn
to use sed, including its advanced features.
The first edition is also very useful. Several typos crept into the
first printing of the first edition (though if you follow the
tutorials closely, you'll recognize them right away). A list of
errors from the first printing of _sed & awk_ is available at
sedawk.txt, and errors in
the 2nd are at sedawk2.txt
though most of these were corrected in later printings. The second
edition tells how POSIX standards have affected these tools and
covers the popular GNU versions of sed and awk. Price is about (US)
$30.00
-----
_Mastering Regular Expressions_, by Jeffrey E. F. Friedl
(Sebastopol, Calif: O'Reilly and Associates, 1997)
ISBN 1-56592-257-3
http://www.oreilly.com/catalog/regex/
http://enterprise.ic.gc.ca/~jfriedl/regex/index.html
Knowing how to use "regular expressions" is essential to effective
use of most Unix tools. This book focuses on how regular
expressions can be best implemented in utilities such as perl, vi,
emacs, and awk, but also touches on sed as well. Friedl's home page
(above) gives links to other sites which help students learn to
master regular expressions. His site also gives a Perl script for
determining a syntactically valid e-mail address, using regexes:
http://enterprise.ic.gc.ca/~jfriedl/regex/email-opt.pl
-----
_Awk und Sed_, by Helmut Herold. (Bonn: Addison-Wesley, 1994)
ISBN 3-89319-685-4
VVA-Nr. 563-00685-8
The text of this book is in German. Now out of print.
-----
_Linux-Unix-Profitools: awk, sed, lex, yacc und make_, by Helumt
Herold. (Bonn: Addison-Wesley, 1998)
ISBN 3-8273-1448-8
addison-wesley
The text of this book is in German. (Comments from German-speaking
reviewers appreciated!)
2.3.2. Mailing list
The informal "seders" mailing list has changed to a Majordomo
mailing list called "sed-users". Regular and digest versions are
available. Average mail volume is 12-25 messages per week. For more
information, address mail to "majordomo@jpusa.org" with any subject |
line and the following in the message body: |
info sed-users yourname@your.site |
To subscribe, mail to "majordomo@jpusa.org" with any subject line |
and one of the following in the message body: |
subscribe sed-users yourname@your.site
subscribe sed-users-digest yourname@your.site
2.3.3. Tutorials, electronic text
The original users manual for sed, by Lee E. McMahon, from the
7th edition UNIX Manual (1978), with the classic "Kubla Khan"
example and tutorial, in formatted text format:
The source code to the preceding manual. Use "troff -ms sed" to print this file properly:
http://plan9.bell-labs.com/7thEdMan/vol2/sed
http://cm.bell-labs.com/7thEdMan/vol2/sed
"Do It With Sed", by Carlos Duarte
http://www.dbnet.ece.ntua.gr/~george/sed/sedtut_1.html
U-SEDIT2.ZIP, by Mike Arst (16 June 1990)
http://wuarchive.wustl.edu/systems/ibmpc/garbo.uwasa.fi/editor/u-sedit2.zip
ftp://ftp.cs.umu.se/pub/pc/u-sedit2.zip
ftp://ftp.uni-stuttgart.de/pub/systems/msdos/util/unixlike/u-sedit2.zip
ftp://sunsite.icm.edu.pl/vol/d2/garbo/pc/editor/u-sedit2.zip
ftp://ftp.sogang.ac.kr/.1/msdos_garbo/editor/u-sedit2.zip
U-SEDIT3.ZIP, by Mike Arst (24 Jan. 1992)
u-sedit3
CompuServe DTPFORUM, "PC DTP Utilities" library, file SEDDOC.ZIP
Another sed FAQ
DreamWeaver Sed AFQ
sed-tutorial, by Felix von Leitner
tutorial
"Manipulating text with sed," chapter 14 of the SCO OpenServer
"Operating System Users Guide"
BOOKCHAPTER-14
_Manipulating_text_with_sed
"Combining the Bourne-shell, sed and awk in the UNIX environment
for language analysis," by Lothar M. Schmitt and Kiel T.
Christianson. This basic tutorial on the Bourne shell, sed and awk
downloads as a 71-page PostScript file (compressed to 290K with
gzip). You may need to navigate down from the root to get the file.
ftp://ftp.u-aizu.ac.jp/u-aizu/doc/Tech-Report/1997/97-2-007.tar.gz
available upon request from Lothar Schmitt <lothar@u-aizu.ac.jp>
2.3.4. General web and ftp sites
http://seders.icheme.org/ # Casper Boden-Cummins |
http://www.cis.nctu.edu.tw/~gis84806/sed/ # Yao-Jen Chang
http://www.math.fu-berlin.de/~guckes/sed/ # Sven Guckes
http://www.math.fu-berlin.de/~leitner/sed/ # Felix von Leitner
http://www.dbnet.ece.ntua.gr/~george/sed/ # Yiorgos Adamopoulos
http://www.cornerstonemag.com/sed/ # Eric Pement
http://spacsun.rice.edu/FAQ/sed.html
ftp://algos.inesc.pt/pub/users/cdua/scripts/ (Carlos Duarte)
ftp://algos.inesc.pt/pub/users/cdua/scripts/ (sed & shell script)
"Handy One-Liners For Sed", compiled by Eric Pement. A large list
of 1-line sed commands which can be executed from the command line.
http://www.cornerstonemag.com/sed/sed1line.txt
http://www.dbnet.ece.ntua.gr/~george/sed/1liners.html
The Single UNIX Specification, Version 2 (technical man page)
http://www.opengroup.org/onlinepubs/7908799/xcu/sed.html |
Getting started with sed
http://ftp.uni-klu.ac.at/sed/sed.html
Comments in sed
http://www.bluesky.com.au:457/OSUserG/_Comments_in_sed.html
"Using sed"
http://www.multisoft.it:457/OSUserG/_Using_sed_main.html
masm to gas converter
http://www.delorie.com/djgpp/faq/converting/asm2s-sed.html
AltaVista results: "sed script" (744+)
http://www.altavista.com/cgi-bin/query?pg=q&kl=XX&stype=stext&q=%22sed+script%22
Google results: "sed script" (668+)
http://www.google.com/search?q=%22sed+script%22
HotBot results: "sed script" (190+)
http://www.hotbot.com/?MT=%22sed+script%22&SM=MC&DV=0&LG=any&DC=10&DE=2
mail2html.zip
http://hiwaay.net/~crispen/src/mail2html.zip
customize VIM to aid writing sed scripts
http://www.fys.uio.no/~hakonrk/vim/syntax/sed.vim
sample uses of sed in batch files and scripts (Benny Pederson)
http://users.cybercity.dk/~bse26236/batutil/help/SED.HTM