UNIX Unleashed, System Administrator's Edition
by Rachel and Robert Sartin, and Robin Burk
Welcome to the world of UNIX. Once the domain of wizards and gurus, today UNIX has spread beyond the university and laboratory to find a home in global corporations and small Internet servers alike. This ability to scale up or down, to accommodate small installations or complex corporate networks with little or no modification, is only one of the characteristics that have won UNIX its popularity and widespread use.
As we'll see through the course of this book, UNIX is a rich and complex system built upon simple, powerful elements. Although many more recent operating systems have borrowed concepts and mechanisms from UNIX, those who are most familiar with legacy mainframe environments, or whose experience is mostly limited to single-user personal computers, may find UNIX to be a bit intimidating at first. The best advice I can give is to take it slowly, but don't give up. As you read through these chapters and begin to use some of the features and utilities described in this book, you'll find that once-foreign ideas have taken clear and concrete shape in your mind.
NOTE: One distinctive characteristic of UNIX compared to other operating systems is the fact that there are several flavors, or variants, of the operating system. Because the source code of the early versions was made available to a variety of computer manufacturers and third parties, many slightly different forms of UNIX co-exist. Some are specific to a given hardware manufacturer; others differ in the utilities, configuration methods or user interfaces they offer. In this book, we will call your attention to the differences among the most commonly used UNIX variants, including:
Other UNIX variants we will examine in these two volumes include:
TIP: Throughout these two volumes, you will find specific details regarding how to accomplish tasks in each of the most popular versions of UNIX. In addition, Part VII in the second volume, UNIX Unleashed, Internet Edition, contains one chapter each of answers to Frequently Asked Questions for each major UNIX flavor.
At its base UNIX is both simple and elegant, with a consistent architecture that, in turn, underlies and guides the design of its many application programs and languages. If you are new to UNIX, I want you to know that I'm a bit jealous of the fun you'll have as you begin to explore this fascinating environment for the first time. If you are a more experienced UNIX user, administrator, or programmer, this revised edition of UNIX Unleashed contains a wealth of information that can help you extend your UNIX use to Internet and World Wide Web applications, guard against hackers and other unauthorized intruders, and fine-tune your system management skills.
UNIX is:
Let's take these one at a time. To begin with, UNIX is a trademark, which means that there is intellectual property associated with UNIX that is not in the public domain. Some versions of UNIX require a paid license for their use.
The term UNIX also refers to a powerful multitasking, multiuser operating system.
Once upon a time, not so long ago, everyone knew what an operating system (OS) was. It was the complex software sold by the maker of your computer system, without which no other programs could function on that computer. It spun the disks, lit the terminals, and generally kept track of what the hardware was doing and why. Application (user) programs asked the operating system to perform various functions; users seldom talked to the OS directly.
Today those boundaries are not quite so clear. The rise of graphical user interfaces, macro and scripting languages, suites of applications that can exchange information seamlessly, and the increased popularity of networks and distributed data--all of these factors have blurred the traditional distinctions. Today's computing environments consist of layers of hardware and software that interact together to form a nearly organic whole.
At its core (or, as we say in UNIX, in the kernel), however, UNIX does indeed perform the classic role of an operating system. Like the mainframe and minicomputer systems that came before, UNIX enables multiple people to access a computer simultaneously and multiple programs and activities to proceed in parallel with one another.
Unlike most proprietary operating systems, however, UNIX has given birth to a whole family of related, or variant, systems. Some differ in functionality or origin; others are developed by computer vendors and are specific to a given line of machines; still others were developed specifically as shareware or even freeware. Although these various flavors of UNIX differ from one another to some degree, they are fundamentally the same environment. All offer their own version of the most common utilities, application programs, and languages. Those who use awk, grep, the Bourne shell, or make in one version of UNIX will find their old favorites available on other UNIX machines as well.
Those who do not care much for these programs, however, will find themselves free to substitute their own approach for getting various computing jobs done. A salient characteristic of UNIX is that it is extensible and open. By extensible, I mean that UNIX allows the easy definition of new commands, which can then be invoked or used by other programs and terminal users. This is practical in the UNIX environment because the architecture of the UNIX kernel specifically defines interfaces, or ways that programs can communicate with one another without having been designed specifically to work together.
An operating system is an important part of a computer system. You can view a computer system as being built from three general components: the hardware, the operating system, and the applications. (See Figure 1.1.) The hardware includes pieces such as a central processing unit (CPU), a keyboard, a hard drive, and a printer. You can think of these as the parts you are able to touch physically. Applications are why you use computers; they use the rest of the system to perform the desired task (for example, play a game, edit a memo, send electronic mail). The operating system is the component that on one side manages and controls the hardware and on the other manages the applications.
Figure 1.1.
Computer system components.
When you purchase a computer system, you must have at least hardware and an operating system. The hardware you purchase is able to use (or run) one or more different operating systems. You can purchase a bundled computer package, which includes the hardware, the operating system, and possibly one or more applications. The operating system is necessary in order to manage the hardware and the applications.
When you turn on your computer, the operating system performs a series of tasks, presented in chronological order in the next few sections.
One of the first things you do, after successfully plugging together a plethora of cables and components, is turn on your computer. The operating system takes care of all the starting functions that must occur to get your computer to a usable state. Various pieces of hardware need to be initialized. After the start-up procedure is complete, the operating system awaits further instructions. If you shut down the computer, the operating system also has a procedure that makes sure all the hardware is shut down correctly. Before turning your computer off again, you might want to do something useful, which means that one or more applications are executed. Most boot ROMs do some hardware initialization but not much. Initialization of I/O devices is part of the UNIX kernel.
After the operating system completes hardware initialization, you can execute an application. This executing application is called a process. It is the operating system's job to manage execution of the application. When you execute a program, the operating system creates a new process. Many processes can exist simultaneously, but only one process can actually be executing on a CPU at one time. The operating system switches between your processes so quickly that it can appear that the processes are executing simultaneously. This concept is referred to as time-sharing or multitasking.
When you exit your program (or it finishes executing), the process terminates, and the operating system manages the termination by reclaiming any resources that were being used.
Most applications perform some tasks between the time the process is created and the time it terminates. To perform these tasks, the program makes requests to the operating system, and the operating system responds to the requests and allocates necessary resources to the program. When an executing process needs to use some hardware, the operating system provides access for the process.
To perform its task, a process may need to access hardware resources. The process may need to read or write to a file, send data to a network card (to communicate with another computer), or send data to a printer. The operating system provides such services for the process. This is referred to as resource allocation. A piece of hardware is a resource, and the operating system allocates available resources to the different processes that are running.
See Table 1.1 for a summary of different actions and what the operating system (OS) does to manage them.
| Action | OS Does This |
| You turn on the computer | Hardware management |
| You execute an application | Process management |
| Application reads a tape | Hardware management |
| Application waits for data | Process management |
| Process waits while other process runs | Process management |
| Process displays data on screen | Hardware management |
| Process writes data to tape | Hardware management |
| You quit, the process terminates | Process management |
| You turn off the computer | Hardware management |
From the time you turn on your computer until you turn it off, the operating system is coordinating the operations. As hardware is initialized, accessed, or shut down, the operating system manages these resources. As applications execute, request, and receive resources, or terminate, the operating system takes care of these actions. Without an operating system, no application can run and your computer is just an expensive paperweight.
The previous section looked at operating systems in general. This section looks at a specific operating system: UNIX. UNIX is an increasingly popular operating system. Traditionally used on minicomputers and workstations in the academic community, UNIX is now available on personal computers, and the business community has started to choose UNIX for its openness. Previous PC and mainframe users are now looking to UNIX as their operating system solution. This section looks at how UNIX fits into the operating system model.
UNIX, like other operating systems, is a layer between the hardware and the applications that run on the computer. It has functions that manage the hardware and functions that manage executing applications. So what's the difference between UNIX and any other operating system? Basically, two things: internal implementation and the interface that is seen and used by users. For the most part, this book ignores the internal implementation. If you wish to know these details, many texts exist that cover them. The interface is what this book describes in detail. The majority of UNIX users need to be familiar with the interface and need not understand the internal workings of UNIX.
The UNIX system is actually more than strictly an operating system. UNIX includes the traditional operating system components. In addition, a standard UNIX system includes a set of libraries and a set of applications. Figure 1.2 shows the components and layers of UNIX. Sitting above the hardware are two components: the file system and process control. Next is the set of libraries. On top are the applications. The user has access to the libraries and to the applications. These two components are what many users think of as UNIX, because together they constitute the UNIX interface.
Figure 1.2.
The layers of UNIX.
The part of UNIX that manages the hardware and the executing processes is called the kernel. In managing all hardware devices, the UNIX system views each device as a file (called a device file). This allows the same simple method of reading and writing files to be used to access each hardware device. The file system (explained in more detail in Chapter 4, "The UNIX File System") manages read and write access to user data and to devices, such as printers, attached to the system. It implements security controls to protect the safety and privacy of information. In executing processes (see Chapter 18), the UNIX system allocates resources (including use of the CPU) and mediates accesses to the hardware.
One important advantage that results from the UNIX standard interface is application portability. Application portability is the ability of a single application to be executed on various types of computer hardware without being modified. This can be achieved if the application uses the UNIX interface to manage its hardware needs. UNIX's layered design insulates the application from the different types of hardware. This allows the software developer to support the single application on multiple hardware types with minimal effort. The application writer has lower development costs and a larger potential customer base. Users not only have more applications available, but can rely on being able to use the same applications on different computer hardware.
UNIX goes beyond the traditional operating system by providing a standard set of libraries and applications that developers and users can use. This standard interface allows application portability and facilitates user familiarity with the interface.
How did a system such as UNIX ever come to exist? UNIX has a rather unusual history that has greatly affected its current form.
In the mid-1960s, AT&T Bell Laboratories (among others) was participating in an effort to develop a new operating system called Multics. Multics was intended to supply large-scale computing services as a utility, much like electrical power. Many people who worked on the Bell Labs contributions to Multics later worked on UNIX.
In 1969, Bell Labs pulled out of the Multics effort, and the members of the Computing Science Research Center were left with no computing environment. Ken Thompson, Dennis Ritchie, and others developed and simulated an initial design for a file system that later evolved into the UNIX file system. An early version of the system was developed to take advantage of a PDP-7 computer that was available to the group.
An early project that helped lead to the success of UNIX was its deployment to do text processing for the patent department at AT&T. This project moved UNIX to the PDP-11 and resulted in a system known for its small size. Shortly afterward, the now famous C programming language was developed on and for UNIX, and the UNIX operating system itself was rewritten into C. This then radical implementation decision is one of the factors that enabled UNIX to become the open system it is today.
AT&T was not allowed to market computer systems, so it had no way to sell this creative work from Bell Labs. Nonetheless, the popularity of UNIX grew through internal use at AT&T and licensing to universities for educational use. By 1977, commercial licenses for UNIX were being granted, and the first UNIX vendor, Interactive Systems Corporation, began selling UNIX systems for office automation.
Later versions developed at AT&T (or its successor, Unix System Laboratories, now owned by Novell) included System III and several releases of System V. The two most recent releases of System V, Release 3 (SVR3.2) and Release 4 (SVR4; the most recent version of SVR4 is SVR4.2) remain popular for computers ranging from PCs to mainframes.
All versions of UNIX based on the AT&T work require a license from the current owner, UNIX System Laboratories.
In 1978, the research group turned over distribution of UNIX to the UNIX Support Group (USG), which had distributed an internal version called the Programmer's Workbench. In 1982, USG introduced System III, which incorporated ideas from several different internal versions of and modifications to UNIX, developed by various groups. In 1983, USG released the original UNIX System V, and thanks to the divestiture of AT&T, was able to market it aggressively. A series of later releases continued to introduce new features from other versions of UNIX, including the internal versions from the research group and the Berkeley Software Distribution.
While AT&T (through the research group and USG) developed UNIX, the universities that had acquired educational licenses were far from inactive. Most notably, the Computer Science Research Group at the University of California at Berkeley (UCB) developed a series of releases known as the Berkeley Software Distribution, or BSD. The original PDP-11 modifications were called 1BSD and 2BSD. Support for the Digital Equipment Corporation VAX computers was introduced in 3BSD. VAX development continued with 4.0BSD, 4.1BSD, 4.2BSD, and 4.3BSD, all of which (especially 4.2 and 4.3) had many features (and much source code) adopted into commercial products.
Because of the multiple versions of UNIX and frequent cross-pollination between variants, many features have diverged in the different versions of UNIX. With the increasing popularity of UNIX in the commercial and government sector came the desire to standardize the features of UNIX so that a user or developer using UNIX could depend on those features.
The Institute of Electrical and Electronic Engineers (IEEE) created a series of standards committees to create standards for "An Industry-Recognized Operating Systems Interface Standard based on the UNIX Operating System." The results of two of the committees are important for the general user and developer. The POSIX.1 committee standardizes the C library interface used to write programs for UNIX. (See UNIX Unleashed, Internet Edition, Chapter 6, "The C and C++ Programming Languages.") The POSIX.2 committee standardizes the commands that are available for the general user.
In Europe, the X/Open Consortium brings together various UNIX-related standards, including the current attempt at a Common Open System Environment (COSE) specification. X/Open publishes a series of specifications called the X/Open Portability. The MOTIF user interface is one popular standard to emerge from this effort.
The United States government has specified a series of standards based on XPG and POSIX. Currently, FIPS 151-2 specifies the open systems requirements for federal purchases.
Various commercial consortia have attempted to negotiate UNIX standards as well. These have yet to converge on an accepted, stable result.
Many mainframe and workstation vendors make a version of UNIX for their machines. We will be discussing several of these variants (including Solaris from SunSoft, AIX from IBM and HP-UX from Hewlett Packard) throughout this book.
Thanks to the great popularity of personal computers, there are many UNIX versions available for Intel platforms. Choosing from the versions and trying to find software for the version you have can be a tricky business because the UNIX industry has not settled on a complete binary standard for the Intel platform. There are two basic categories of UNIX systems on Intel hardware: the SVR4-based systems and the older, more established SVR3.2 systems.
SVR4 vendors include NCR, IBM, Sequent, SunSoft (which sells Solaris for Intel), and Novell (which sells UnixWare). The Santa Cruz Operation (SCO) is the main vendor in the SVR3.2 camp.
Several versions of UNIX and UNIX-like systems have been made that are free or extremely cheap and include source code. These versions have become particularly attractive to the modern-day hobbyist, who can now run a UNIX system at home for little investment and with great opportunity to experiment with the operating system or make changes to suit his or her needs.
An early UNIX-like system was MINIX, by Andrew Tanenbaum. His book Operating Systems: Design and Implementations describes MINIX and includes a source listing of the original version of MINIX. The latest version of MINIX is available from the publisher. MINIX is available in binary form for several machines (PC, Amiga, Atari, Macintosh, and SPARCStation).
The most popular source version of UNIX is Linux (pronounced "lin nucks". Linux was designed from the ground up by Linus Torvalds to be a free replacement for UNIX, and it aims for POSIX compliance. Linux itself has spun off some variants, primarily versions that offer additional support or tools in exchange for license fees. Linux has emerged as the server platform of choice for small to mid-sized Internet Service Providers and Web servers.
Many people considering making the transition to UNIX have a significant base of PC-based MS-DOS and Microsoft Windows applications. There have been a number of efforts to create programs or packages on UNIX that would ease the migration by allowing users to run their existing DOS and Windows applications on the same machine on which they run UNIX. This is a rapidly changing marketplace as Microsoft evolves its Windows and Windows NT operating systems.
As described in the section "The History of UNIX," UNIX has its roots in a system that was intended to be small and supply orthogonal common pieces. Although most UNIX systems have grown to be fairly large, and monolithic applications are not uncommon, the original philosophy still lives in the core commands available on all UNIX systems. There are several common key items throughout UNIX:
For detailed information on commands and connecting them together, see the chapters on shells (Chapters 8-13) and on common commands (Chapters 5--9).
The original UNIX systems were very small, and the designers tried to take every advantage of those small machines by writing small commands. Each command attempted to do one thing well. The tools could then be combined (either with a shell script or a C program) to do more complicated tasks. One command, called wc, was written solely to count the lines, words, and characters in a file. To count all the words in all the files, you would type wc * and get output like that in Listing 1.1.
$ wc * 351 2514 17021 minix-faq 1011 5982 42139 minix-info 1362 8496 59160 total $
To turn the simple, orthogonal commands into a powerful toolset, UNIX enables the user to use the output of one command as the input to another. This connection is called a pipe, and a series of commands connected by pipes is called a pipeline. For example, to count the number of lines that reference MINIX in all the files, one would type grep MINIX * | wc and get output like that in Listing 1.2.
$ grep MINIX * | wc 105 982 6895 $
Each command has actions that can be controlled with options, which are specified by a hyphen followed by a single letter option (for example, -l). Some options take option arguments, which are specified by a hyphen followed by a single letter, followed by the argument (for example, -h Header). For example, to print on pages with 16 lines each all the lines in the file minix-info that mention Tanenbaum, you would enter wc minix-info | pr -l 16 and get output like that in Listing 1.3.
$ grep Tanenbaum minix-info | pr -l 16 Feb 14 16:02 1994 Page 1 [From Andy Tanenbaum <ast@cs.vu.nl> 28 August 1993] The author of MINIX, Andrew S. Tanenbaum, has written a book describing Author: Andrew S. Tanenbaum subjects.ast (list of Andy Tanenbaum's Andy Tanenbaum since 1987 (on tape) Version 1.0 is the version in Tanenbaum's book, "Operating Systems: Design $
The bad news is that some UNIX commands have some quirks in the way they handle options. As more systems adopt the standards mentioned in the section "The History of UNIX," you will find fewer examples of commands with quirks.
UNIX pays no attention to the contents of a file (except when you try to run a file as a command). It does not know the difference between a spreadsheet file and a word processor file. The meaning of the characters in a file is entirely supplied by the command(s) that uses the file. This concept is familiar to most PC users, but was a significant difference between UNIX and other earlier operating systems. The power of this concept is that any program can be used to operate on any file. The downside is that only a program that understands the file format can fully decode the information in the file.
UNIX has a long history as an open development environment. More recently, it has become the system of choice for both commercial and some personal uses. UNIX performs the typical operating system tasks, but also includes a standard set of commands and library interfaces. The building-block approach of UNIX makes it an ideal system for creating new applications.
©Copyright,
Macmillan Computer Publishing. All rights reserved.
