Chapter 30

Writing CGI Scripts in REXX

CONTENTS

• What Is REXX?
• A Brief History of REXX
• Getting Input to the Script
• Decoding Input from a Form
• Sending the Document Back to the Client
• Diagnostics and Reporting Errors
• Security Concerns
• A Simple REXX CGI Script
• Summary

This chapter is aimed at readers who want to write their own WWW executable scripts in the REXX language using WWW's Common Gateway Interface (CGI). This chapter assumes that you have a knowledge of HTML, forms, and cgi-bin programming. It also assumes that you have programming experience, some familiarity with REXX, or access to a REXX interpreter or documentation.

This chapter gives you a brief overview of the highlights of the REXX programming language, as well as its history. It then explains how to read input from various sources into your REXX script, how to decode the input, how to send the document back to the client (Web browser), and how to report diagnostics and errors. This chapter also identifies security issues you should be aware of and how to code for them. Finally, the chapter provides a simple but complete REXX script that can run in your Web server, with explanations of how to install it and make it accessible to your Web server.

What Is REXX?

REXX is a procedural language that enables you to write programs in a clear and structured way. It has powerful arithmetic and character-manipulation facilities, and because it often is executed by an interpreter, it permits rapid program development.

REXX uses a minimum of boilerplate, required punctuation, special escape characters, notations in general, and reserved words (keywords are reserved only in context). It has only one data type, the character string, so no declarations are needed. Also, there are no inherent limits on the size of strings. REXX allows the creation of simple programs with minimum overhead. As a result, it is easy to use and remember by both computing professionals and casual users.

At the same time, REXX provides a rich set of control constructs (such as IF_THEN_ELSE, DO_END, WHILE, UNTIL, FOR, SELECT_WHEN_OTHERWISE, ITERATE, LEAVE, and so on), internal and external subroutines and functions, associative variables (often referred to as stem variables in REXX), dynamic variable scoping, powerful string parsing, data-extraction features, and character- and word-manipulating facilities. REXX can access information from the host's environment and can issue commands to the host environment or programs written in other languages. REXX programs are highly portable across a wide variety of hardware platforms (from mainframes to PCs and Macs) and operating systems (from MVS/VM through UNIX and VMS to OS/2, PC/DOS, and MacOS).

For these reasons, REXX is an ideal language for writing CGI scripts.

A Brief History of REXX

REXX originally was specified and implemented in 1979 by Mike Cowlishaw of IBM. During its first five years of life, it was developed by a single individual (with feedback from hundreds of early users); the result was a very coherent language. In 1983, the REXX interpreter was included as part of IBM's VM/System Product operating system for IBM mainframes. The first non-IBM implementation became available from Mansfield Software in 1985. Also in 1985, the definition of the language was published in The REXX Language, A Practical Approach to Programming, by M.F. Cowlishaw (Prentice Hall, 1985). The first REXX compiler was made available to IBM customers in 1989.

In 1990, the first international REXX Symposium for Developers and Users was organized by and held at the Stanford Linear Accelerator Center in California; it now is held annually. Work toward an ANSI REXX standard began in 1991, and a draft standard was forwarded to ANSI at the end of 1994. The public review period ended May 3, 1995. IBM released an Object version of REXX for OS/2 in 1995.

For beginning users, Sams Publishing offers Teach Yourself REXX in 21 Days. You also can find some good sources of further information on REXX by looking at the URLs http://www.yahoo.com/Computers/Languages/Rexx/ or http://rexx.hursley.ibm.com/rexx. These sources include pointers to answers to frequently asked questions, news on REXX, REXX implementations (including freeware/shareware and commercial), REXX products, and lists of REXX books and manuals.

Getting Input to the Script

The input can be sent to the script in several ways, depending on the client's URL or an HTML form. The most important ways are via environment variables and standard input.

QUERY_STRING Environment Variable

The QUERY_STRING is anything that follows the first question mark (?) in the URL. This information can be added by an HTML ISINDEX document or by an HTML form (using the METHOD="GET" action). It also can be embedded manually in an HTML hypertext link. This string usually is an information query-for example, what the user wants to search for in databases or perhaps the encoded results of your feedback form.

You can access the QUERY_STRING input in REXX via

Input=GETENV('QUERY_STRING')

This input is encoded by the user's browser in the standard URL format. It changes spaces to plus signs (+) and encodes special ASCII characters (such as a semicolon) with a %XX hexadecimal encoding, where XX is the ASCII hexadecimal representation of the character. The hexadecimal code for an ASCII semicolon is 3B, for example, so any semicolons after encoding appear as %3B (or %3b for some browsers). See the ASCII table in Appendix B, "HTML Language Reference," for a complete set of ASCII character codes. You can convert the plus signs back to spaces by using the REXX TRANSLATE command. For example, the REXX code

Input=TRANSLATE(GETENV('QUERY_STRING'),' ','+')

converts any plus signs (+) in the QUERY_STRING to spaces and places the result in Input.

Listing 30.1 provides an example of how to use REXX to decode hexadecimal-encoded characters in the input.

Listing 30.1. REXX code to decode ASCII % hexadecimal-encoded characters.

/******************************************** */
/*Most browsers insert ASCII codes (preceded  */
/*by a %) for some characters such as space or*/
/*+. The following converts encoded characters*/
/*in Input to the equivalent ASCII characters.*/
/******************************************** */

DO WHILE INDEX(Input,'%')/=0
   PARSE VAR Input Pre'%'+1 Char +2 Input
   IF VERIFY(TRANSLATE(Char),'0123456789ABCDEF')=0 THEN
      Input=Pre||X2C(Char)||Input
   ELSE Input=Pre||X2C('27')||Char||Input
END
Input=TRANSLATE (Input, '%',X2C('27'))

If your server is not decoding results from a form, the information provided in QUERY_STRING also is provided on the command line. It therefore is available via the REXX PARSE ARG command. For example, for the URL

http://www.my.box/cgi-bin/foo?hello+world

if you use the REXX command

PARSE ARG Arg1 Arg2

then Arg1 contains "hello" and Arg2 contains "world" (note that the plus sign in the URL is replaced by a space).

PATH_INFO Environment Variable

PATH_INFO comes from the "extra" information after the path of your CGI script in the URL. This information is not encoded by the server in any way. Suppose that you have a CGI script called foo, which is accessible to your server. When users access foo and want to tell foo that they currently are in the English language directory rather than the Spanish directory, they could access your script in an HTML document by using this URL:

http://www.my.box/cgi-bin/foo/lang=english

Before executing foo, the server sets the PATH_INFO environment variable to contain /lang=english, and foo can decode this and act accordingly.

The PATH_INFO can be accessed in REXX via the PATH_INFO environment variable, for example, by using the following REXX command:

Path=GETENV('PATH_INFO')

Standard Input

If an HTML form that calls your CGI script has METHOD="POST" in its FORM tag, your CGI script receives the encoded form input in standard input (for example, in stdin in UNIX). The server does not send you an EOF (End Of File) at the end of the data; instead, you should use the environment variable CONTENT_LENGTH to determine how much data you should read from standard input. Listing 30.2 shows you how you can read the standard input in REXX.

Listing 30.2. REXX code to read standard input from an HTML form's POST method.

/******************************************** */
/*Read HTML FORM POST input from standard     */
/*input. Note that we preserve or save the    */
/*Input in case we need to send it to another */
/*script. If so we can restore the stdin for  */
/*the called command by  using the command:   */
/*ADDRESS UNIX script '<' StdinF              */
/******************************************** */

StdinF='/tmp/stdin'_GETPID() /*Get unique file name   */
       /* The uni-REXX function _GETPID() returns the */
       /* process id for the current process.         */
    
IF GETENV('REQUEST_METHOD')="POST" THEN DO
   Input=CHARIN(,1,GETENV('CONTENT_LENGTH'))
   IF Input='' THEN DO
      SAY 'Null input from POST!'
      EXIT
   END
   IF CHAROUT(StdinF,Input,1) /=0 THEN DO
      SAY 'Unable to write out all POST chars!'
   END
   Fail=CHAROUT(StdinF)   /*Close the file*/
END

Listing 30.3 provides a summary of how to read all the preceding forms of input in REXX.

Listing 30.3. How to read the various possible sources of input to your REXX CGI script.

/* ************************************************************ */
/* Read and display the input from the various possible sources */
/* ************************************************************ */
PARSE ARG Parms
SAY 'Command line input="'Parms'".'

SAY 'Standard Input="'CHARIN(,1,GETENV('CONTENT_LENGTH'))'".'

SAY 'PATH_INFO="'GETENV('PATH_INFO')'".'
SAY 'QUERY_STRING="'GETENV('QUERY_STRING')'".'

If you were to execute the code in Listing 30.3 for the URL http://www.my.box/cgi-bin/foo/map=england?451+371, the output would appear as this:

Command line input="451 371".
Standard Input="".
PATH_INFO="/map=england".
QUERY_STRING="451+371".

Decoding Input from a Form

When you write an HTML form, each of your input items has a NAME tag. When the user places data in these items in the form, that information is encoded into the form data. The value that each of the input items is given by the form is called its VALUE.

Form data is a stream of NAME=VALUE pairs separated by the ampersand (&) character. Each NAME=VALUE pair is URL encoded; spaces are changed to plus signs (+) and some characters are encoded into hexadecimal. Listing 30.4 shows how you can decode the NAME=VALUE pairs in REXX.

Listing 30.4. Decoding NAME=VALUE pairs provided by an HTML form.

/* ************************************* */
/* Data  from a FORM comes in the form:  */
/* name1=value1&name2=value2             */
/* Here we decode the Input into an      */
/* array of names and values.            */
/* ************************************* */

DO I=1 BY 1 UNTIL Input=''
   PARSE VAR Input Name.I'='Value.I'&'Input
END I

Sending the Document Back to the Client

CGI scripts can return many document types. To tell the server what kind of document you are sending back, CGI requires you to place a short ASCII header on your output. This header indicates the MIME type of the following document. A couple of common MIME types relevant to WWW are

• A "text" Content-type to represent textual information. The two most likely subtypes are
  text/plain  Text with no special formatting requirements
  text/html  Text with embedded HTML commands
• An "application" Content-type that is used to transmit application data or binary data. For example,
  application/postscript  The data is in PostScript and should be fed to a PostScript interpreter

The first line of the output from your CGI script should read

Content-type: type/subtype

where you replace type and subtype with the MIME type and subtype for your output. Next, you have to send a blank line. After these two lines have been output, your server knows to begin the actual output. You can output these lines in REXX by using the SAY command. Listing 30.5 shows how you might use REXX to set the Content-type based on the file type.

Listing 30.5. Setting the Content-type of the document based on the file type.

FileName='/u/sf/cottrell/public_html/cgi.html'
/********************************************* */
/*Code fragment to set the Contentype subtype  */
/*based on the  file type (as determined by the*/
/*characters after the last . in the filename) */
/********************************************* */
L=LASTPOS('.',Filename); Type=''
IF L>0 THEN
  IF LENGTH(FileName)>L THEN
     Type=TRANSLATE(SUBSTR(FileName,L+1))

SELECT
   WHEN Type='HTM' | Type='HTML' THEN
     SAY 'Content-type: type/html'
   WHEN Type='PS'                THEN
     SAY 'Content-type: application/postscript'
   WHEN FIND('TXT RXX PL C',Type)/=0 | Type=''
     THEN SAY 'Content-type: type/text'
   OTHERWISE DO
     SAY 'Content-type: type/html'; SAY ''
     SAY 'Unknown Content-type="'Type'"'
     EXIT
   END
END
SAY '' /*Don't forget the second line*/

Diagnostics and Reporting Errors

Because the standard output is included in the document sent to the browser, diagnostics output with the REXX SAY command appear in the document. This output needs to be consistent with the Content-type: type/subtype mentioned in the preceding section. Listing 30.6 shows how you can report diagnostics in a REXX CGI script.

Listing 30.6. Reporting diagnostics in a REXX CGI script.

/* ************************************** */
/* Code fragment for reporting CGI Script */
/* diagnostics                            */
/* ************************************** */
PARSE SOURCE . . Fn . /* Get the filename of the script*/
...
Debug=1
SAY "Content-type: text/html";  SAY ''
...
IF Debug>0 THEN SAY Fn': PATH_INFO="'GETENV('PATH_INFO')'"<br>'

If errors are encountered (for example, no input is provided, invalid characters are found, too many arguments are specified, you requested an invalid command to be executed, or an invalid syntax appears in the REXX script), the script should provide detailed information on what is wrong. It might be helpful to the user to provide information on the settings of various WWW environment variables. Listing 30.7 gives an example of how you might report errors in your CGI script.

Listing 30.7. Reporting errors in a REXX CGI script.

/****************************************** */
/*Code Fragment for REXX CGI Error Reporting*/
/****************************************** */
  ADDRESS 'COMMAND'; SIGNAL ON SYNTAX
  PARSE Arg Parms
  ...
  IF GETENV('QUERY_STRING')='' THEN
     CALL Exit 400,'No query string given!<br>'
  ...
/*******************************************/
/*REXX will jump to this error exit if a   */
/*syntax error occurs. It returns the      */
/*contents of the line where the error was */
/*discovered in the script.                */
/*******************************************/
Syntax:
   PARSE SOURCE . . Fn .
   CALL Exit 501, 'Syntax error on line',
      SIGL 'of' Fn'. Line="'SOURCELINE(SIGL)'"<br>'
  ...


Exit: PROCEDURE EXPOSE Debug Parms
/* *************************************** */
/* Exit - Assumes Content-type: text/html  */
/* *************************************** */
PARSE ARG Code, Msg
SAY '<title>'GETENV('SCRIPT_NAME')'</title>'
SAY '<h2>'GETENV('SCRIPT_NAME') 'error Code' Code'.</h2>'
SAY 'The WWW utility on'
SAY '<tt>'GETENV('SERVER_NAME')'</tt>'
SAY 'reports the following error:'
IF Msg/='' THEN SAY '<hr><h1><code>'Msg'</code></h1>'
IF Debug>0 THEN DO; SAY,
   '<hr>Complete environment follows:<p><pre>'
   ADDRESS Unix "set"
       /* "set" is a Unix shell command   */
       /* to print the contents of all the environment */
       /* variables currently defined.                 */
   SAY 'Command line input="'Parms'".'
   SAY '</pre>'
END
SAY '<hr><a HREF="/suggestion/cottrell">Suggestions</a>'
IF Code=0 THEN RETURN; ELSE EXIT 24

Security Concerns

Any time a script interacts with a client (such as a Web browser) via a server (such as a Web server), it is possible that the client may attack the server to gain unauthorized access or deny service. Even the most innocent script can be dangerous to the integrity of your system. The following sections highlight some of the pitfalls to avoid.

Beware of the REXX INTERPRET or ADDRESS UNIX Statement

The REXX INTERPRET or ADDRESS UNIX statement can be very dangerous. Consider the following statements in a REXX script:

INTERPRET TRANSLATE(GETENV('QUERY_STRING'),' ','+')

or

ADDRESS UNIX TRANSLATE(GETENV('QUERY_STRING'),' ','+'))

These clever one-liners take the QUERY_STRING and convert it into a command to be executed by the Web server. Unfortunately, the user easily can put a command in the QUERY_STRING to delete all accessible files. So you must restrict what command(s) the system is allowed to execute in response to the input.

If a set of commands must be executed, you might want to set up a table containing the acceptable commands that can be executed in response to the user's requests. Setting up a table is fairly simple and allows much flexibility later. Listing 30.8 shows a table of requests and the commands they execute. This list is formatted in a fashion suitable for use with the code in Listing 30.9.

Listing 30.8. A sample list of rules that map URL requests to UNIX commands.

# List of rules that map Requests to the Full commands to
# be executed, and also provide restrictions to be applied to
# the commands.
# The format of this file is
#    Request = Full-command [; security rule] [#comment]
#    [#comment]
# The only security rules implemented at the moment are:
#   SLAC which means the
#   command is only valid from clients in the SLAC IP
#   domain (134.79.) and % which means the script is prepared
#   to handle % encoded characters.
man         = /usr/ucb/man
finger      = /usr/ucb/finger
whois       = /usr/local/bin/whois
trace       = /usr/local/bin/traceroute; SLAC

Listing 30.9 gives an example of REXX code that can be used to read the table in Listing 30.8, check that the request is allowed for, and apply some restrictions.

Listing 30.9. REXX code to map the URL to the UNIX command to be executed.

/* ************************************************* */
/* Check the Request versus the rules file to see if */
/* it is OK and to get the Full command to execute.  */
/* ************************************************* */
IF LINES(Rulesfile)=0 THEN DO
   SAY Rulesfile 'is empty or does not exist!<br>'
   EXIT
END

OK=0
PARSE VAR Request Command'+'Args
DO L = 1 TO LINES(Rulesfile)
   Line.L=LINEIN(Rulesfile)
   PARSE VAR Line.L Line.L '#' Comment /* Remove comments*/
   IF Line.L='' THEN ITERATE L
   PARSE VAR Line.L Pattern . '=' Full . ';' Rule
   IF Full='' THEN DO
       SAY 'Line #' L 'in' Rulesfile 'is incomplete!<br>'
       ITERATE L
   END
   IF WORD(Pattern,1) = WORD(Command,1) THEN DO
      /* Check whether Command is restricted to SLAC IP domain*/
      IF INDEX(Rule,'SLAC') /=0 THEN DO
         IF SUBSTR(GETENV('REMOTE_ADDR'),1,7) /='134.79.' THEN DO
            SAY Request 'restricted to SLAC nodes!<br>'
            EXIT
         END
      END
      /* Add % to valid list of characters, if specified */
      /* in Rule (see listing 30.1 for more on Valid) */
      IF INDEX(TRANSLATE(Rule),'%') /=0 THEN Valid=Valid||'%'
      OK=1; LEAVE L
   END
END L

IF OK = 0 THEN DO
   SAY Request 'not validated by Rules list.<br>'
   EXIT
END

SAY 'Will execute "'Full TRANSLATE(Args,' ','+')'".<br>'

If Rulesfile has the file name of the file that contains Listing 30.8, and if Request='finger+cottrell', then executing Listing 30.9 results in the following:

Will execute "/usr/ucb/finger cottrell".

Escaping Dangerous Characters

A well-behaved client escapes any characters that have special meaning to the system in a query string. A well-behaved client, for example, replaces special characters such as a semicolon (;) or a greater than sign (>) with %XX, where XX is the ASCII code for the character in hexadecimal. This helps avoid problems with your script misinterpreting the characters passed from the client when they are used to construct the arguments of a command (such as the UNIX finger command) to be executed (for example, via the REXX ADDRESS UNIX command) in the server's command environment (for example, the Bourne Shell in UNIX).

A mischievous client, however, might bypass the ASCII hexadecimal encoding and use special characters to confuse your script and gain unauthorized access. Your CGI script therefore must be careful to accept only the subset of characters that does not confuse your script. A reasonable subset for UNIX is the alphanumeric characters (0-9, a-z, A-Z), the minus sign (-), the underscore (_), the period (.), the slash (/), and the at symbol (@). Any other characters should be treated with care and be rejected in general.

Listing 30.10 shows how you can use REXX to check for valid characters in the input.

Listing 30.10. REXX code to verify that the characters in the string are restricted to a valid subset.

/* ******************************************* */
/* REXX code fragment to check that the        */
/* characters in Input are restricted to a     */
/* Valid set of characters.                    */
/* ******************************************* */

Valid=' abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
Valid=Valid||'0123456789-_/.@'

V=VERIFY(Input,Valid)
IF V/=0 THEN DO
   SAY 'Bad character('SUBSTR(Input,V,1)')in:"'Input'"'
   EXIT 99
END

The same goes for escaped characters after they have been converted. If you need to pass such characters in a string for the system to execute, then your CGI script should treat these characters with care. If your script passes a string to be executed by the UNIX shell, for example, then you need to insert a backslash before each character that has a special meaning to the UNIX shell before it is passed to the UNIX shell. Listing 30.11 shows how you might accomplish this in REXX.

Listing 30.11. REXX code to escape special characters before passing them on to be executed by the UNIX Bourne Shell.

/* *************************************************** */
/* The UNIX Bourne shell treats some characters in a   */
/* command's argument list as having a special meaning.*/
/* This could result in the shell executing unwanted   */
/* commands. This code escapes the special characters  */
/* in String by prefixing them with the \ character.   */
/* *************************************************** */
Esc=';&|>*?' /*List of chars to be escaped*/

DO UNTIL Esc=''/*Check for chars to be escaped*/
  PARSE VAR Esc Char 2 Esc
  P=POS(Char,String)
  DO WHILE P /=0
     Pre=SUBSTR(String,1,P-1) /*Get text before char*/
     Post=SUBSTR(String,P+1)  /*and after    */
     String=Pre||'\'||Char||Post
     P=POS(Code,String)
  END /*DO WHILE P/= 0*/
END /*DO UNTIL Esc='' */

Restricting Distribution of Information

The IP address of the client is available to the CGI script in the environment variable REMOTE_ADDR. This can be used by the script to refuse the request if the client's IP address does not match some requirements. Listing 30.12 shows how you can use the REMOTE_ADDR environment variable in a REXX script to restrict access.

Listing 30.12. REXX code to restrict access to an IP domain.

/* *********************************************** */
/* REXX code fragment to check whether the request */
/* came from a client whose address is in the      */
/* SLAC IP domain (134.79.)                        */
/* *********************************************** */
IF SUBSTR(GETENV('REMOTE_ADDR'),1,7)/='134.79.' THEN
  CALL Exit 403, 'Access restricted to SLAC nodes!<br>'

Testing the Script

You should remember to test the script before getting the WWW server to execute it. However obvious this tip might sound, it is very easy for an untested script to cause the server problems. If the script mistakenly asks for input from the console by executing a REXX PULL command with nothing on the stack, for example, or by executing a REXX TRACE ?R command, the process on the server can stall. Or the script may go into an infinite loop or continuously spawn new processes and use up all the server's process slots.

A Simple REXX CGI Script

You are now in a position to write a simple REXX script. Listing 30.13 shows an example of a complete but simple REXX CGI script that enables the user to execute a UNIX finger command for a user ID specified in the URL. A user can invoke this script by using this URL:

http://www.my.box/cgi-bin/foo?cottrell

where it is assumed that the script is called foo.

Listing 30.13. A REXX CGI script.

#!/usr/local/bin/rxx
/* Sample CGI Script in Uni-REXX, invoke from*/
/* http://www.slac.stanford.edu/cgi-bin/finger?cottrell        */

SAY "Content-type: text/plain"; SAY ''

Input=TRANSLATE(GETENV('QUERY_STRING'),' ','+')

DO WHILE INDEX(Input,'%')/=0
   PARSE VAR Input Pre'%'+1 Char +2 Input
   IF VERIFY(TRANSLATE(Char),'0123456789ABCDEF')=0 THEN Input=Pre||X2C(Char)||Input
   ELSE DO; SAY 'Invalid %'Char' ASCII encoding!<br>'; EXIT; END
END

Valid=' abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ'
Valid=Valid||'0123456789-_/.@'
V=VERIFY(Input,Valid)
IF V/=0 THEN DO
   SAY 'Bad char('SUBSTR(Input,V,1)') in:"'Input'"'
   EXIT 99
END

IF SUBSTR(GETENV('REMOTE_ADDR'),1,7)/='134.79.' THEN DO
  SAY 'Access restricted to SLAC nodes!<br>'; EXIT
END'

ADDRESS UNIX 'finger' Input

Naturally, the next thing you should do is test this script in your Web server. To get your Web server to execute a CGI script, you must do the following:

Write the script and save it somewhere.

Move the file to a valid area, as defined by the server software, and make sure that it is executable. The procedures to accomplish this step may differ from site to site. You should contact your local Web master to help you with this.

Summary

In this chapter, you learned that REXX is an effective language for writing CGI scripts. You also examined REXX examples of how to handle most of the functions required by a CGI script. You learned how to retrieve input from the environment variables, standard input, and the command line. You learned how to decode NAME=VALUE pairs from a form, how to send the output back to the client, and how to report errors and diagnostics. You also learned about common security pitfalls and how to avoid them. Finally, you reviewed a complete REXX CGI script and learned how to make it accessible to your Web server.

You also might want to take a look around http://www2.hursley.ibm.com/goserve/ for examples of using REXX to implement a WWW server and providing e-mail support via an HTML form. The site at http://www.slac.stanford.edu/slac/www/resource/how-to-use/cgi-rexx/ also provides access to one site's guide to how to write CGI scripts in REXX.