From CometWiki

Revision as of 14:55, 12 February 2014 by Jim (Talk | contribs)
(diff) ← Older revision | Latest revision (diff) | Newer revision → (diff)
Jump to: navigation, search

XAP Gateway Internals

There are 2 major parts to the XAP technology. The first part is what we call the CGI server and the second is what we call the XAP Gateway.

The CGI Server

The CGI server listens on the port configured for it. It expects to receive connections using the http protocol. It then performs the following steps:

   It receives and assembles a whole browser request.
   It parses the individual parts of this request, splitting out the following components:
   The CGI Environment variables.
   Any Cookies that the browser may have sent.
   Any query strings that the browser may have sent.
   It then writes these items into a Keyed File* (called the XAP environment file), so that a user program can access them quickly.
       CGI environment variables are written to the keyed file such that the key name is the environment variable name with dots instead of underlines separating words of the name, ie HTTP_USER_AGENT would become HTTP.USER.AGENT.
       Cookie names are preceded with a "C." so that a cookie by the name of "my cookie" would be written as "C.MY COOKIE".
       Likewise, query strings are preceded with a "Q." so that a query string by the name of "credit card number" would be written as "Q.CREDIT CARD NUMBER".
   If an appropriate cookie was present in the request, the CGI server establishes the context data for the connection.
   It then launches the Internet Basic program designated by the SCRIPT.NAME environment variable. The program inherits a connection to the browser through the XAP gateway, any context items established in step 4 above, and the keyed file produced in step 3 above.

Note: Any mention of "files" in this document does not necessarily mean "disk files". Files may reside in memory or disk depending on performance/sharing issues.

The XAP Gateway

The XAP Gateway acts in response to commands included in the Internet Basic program launched by the CGI Server. The interface is very easy to learn because it contains relatively few basic commands:

   Write – the program may write data to the browser through the gateway.
   If no previous data has been written to the XAP Gateway, the XAP Gateway will generate a standard HTML header before sending the data to the browser. The program is free to modify this behavior through a control explained below. Otherwise, the data will be sent directly to the browser. All data is buffered for best performance.
   Control – the program may send various controls that cause the gateway to perform actions meaningful to the program. These will be described below.
   Status – the program can get various pieces of information from the gateway by the use of a simple status function.
   Close – When the LUN pointing to the XAP Gateway is closed, all buffered data is sent to the browser, Context data is filed away for future connections, and the connection to the browser is gracefully closed.
   The XAP environment file is also used as part of the interface between the program and the XAP gateway. The program can read browser request data from this file as explained above. It may also write information that will eventually be written to the browser as explained below.

XAP Gateway Controls

Most of the power of the XAP Gateway is embedded in the various Controls that the program may send to it. The Control statement looks like a standard function in the language:


Result-string = CONTROL(LUN,Control.String [,EXCP=exception.routine])

The CONTROL statement sends the specified control-string to the specified logical unit number. In the case of the XAP Gateway, the Logical Unit Number or "LUN" is always 0 (zero).

The CONTROL statement also reads the result field from the input buffer and places it in the result-string. A plus sign (+) in the first byte of the result-string indicates that the control function was performed successfully. A minus sign (-) in the first byte indicates that the control was not performed successfully. The remaining bytes of the result-string contain text describing the success or failure of the control function.

The CONTROL statement performs the same function as the following statements:

FILE (LUN) CTL=control-string

INPUT (LUN) result-string

Here are the controls that may be sent to the XAP gateway. Where string constants are used in the examples, string expressions may also be used.

   TYPE – sets a MIME type to the browser. As explained above if no TYPE control is sent, the type is assumed to be HTML and the appropriate header is sent to the browser..
   Result-string = CONTROL (0,"TYPE 0") notifies the XAP gateway that the program will subsequently write its own MIME type header.
   result-string = CONTROL(0,"TYPE 1") tells the XAP Gateway to set the mime type to "text/html"
   result-string = CONTROL(0,"TYPE 2") tells the XAP Gateway to set the mime type to "text/plain"
   Result-string = CONTROL (0,"TYPE 3") tells the XAP Gateway to set the mime type to "text/plain" and supply automatic carriage return line feeds following every write.
   COOKIE – sends a cookie to the browser. This control must be sent before any other data is sent to the browser because the cookie information is part of the header.
   The syntax is as follows:
   A$ = CONTROL (0,'COOKIE cookie-name=cookie-value,time-period')
   Where time-period is the specified length of time for the cookie to be stored in the browser.
   The format for time period is:
   +nM = n minutes into the future
   +nH = n hours into the future
   +nD = n days into the future
   For example, to make the CUSTOMERNUMBER=123456789 cookie stay in the browser system for 25 days, you would use the following statement:
   REDIRECT – Sends the appropriate commands to the browser so that it will automatically connect to the URL contained in the REDIRECT control. This control must be sent before any other data is sent to the browser because the redirect information is part of the header.
   For example, if you wanted your program to redirect the browser to, you would use the following statement:
   A$ = CONTROL(0,'REDIR')
   OPEN – performs a "test open" of the file contained in the control to validate whether it is present. This file may be a template file, html file, image file or any file. The named file is opened, and the handle is saved for further operations using that file.
   For example, if you wanted your program to send the image contained in the file "c:\images\logo.jpg" to the browser, you would use the following command:
   After the execution of this control, the result string A$ would contain "+OK" if the file was successfully opened, or "-ERR error opening path" if the file could not be found.
   SEND – Sends the file referred to in the control to the browser. The file is sent to the browser in the fastest way possible. The SEND control may specify a path pointing to the file to be sent --
   or optionally, if no path is specified --
   A$ = CONTROL (0,'SEND')
   the XAP gateway will send a file previously opened by the OPEN control as specified above.
   PUSH – Flushes the output buffer contained in the XAP gateway to the browser for debugging purposes or to implement "server push" actions.
   MERGE – Merges data from a template file named in the control with data contained in the CGI Environment file, Then sends the resultant merged data stream to the browser. Merge is the most powerful of all of the controls. It merges selected data from the CGI environment file with a template file (or a section of a template file) and sends the resulting web page to the browser.

The syntax for the MERGE control may take several forms. The general description of the merge command is:

A$ = CONTROL (LUN,’MERGE Path-name |"." [,SECTION Section-name])

As you can see, the MERGE command takes an optional path name and an optional section name. Here is how Comet processes a MERGE command.

       Comet first determines if there is a path name specified, or a dot, meaning use the last file either opened from a previous MERGE command or an OPEN command. If a Path name was supplied, that file is opened.
       Comet then determines whether a SECTION name was supplied. If a Section name was specified, Comet searches the file for a string of characters such as:
       <!—BEGIN Section-nameà
       If found, the file pointer is set just beyond this string, causing Comet to read data from this point on. If no such string was found, Comet starts reading the e file from the first byte.
       If a section was specified in the control statement, Comet then searches the file for an end-of-section string. This string is of the form:
       <!—END Section-nameà
       If such a string is found, a pointer is set to the character just prior to the opening character of this string. This pointer marks the last character sent from the file on this Merge command. If no such string is found, the ending pointer is set to the last character of the file.
       Comet then searches the file starting at the current file pointer for an indicator marking data to be inserted. At the current time, this marker is a pair of stars (**).
       If such a marker is found, all data is streamed from the file to the Browser up to this marker.
       Comet then searches the file for an ending indicator. At the current time this is also a pair of stars (**). If such a marker is found, all characters between the beginning and ending markers (including the **’s) is assembled into a Key to be used in a keyed read on LUN 1. If the read is successful, the data record resulting from this keyed read is streamed to the browser.
       If the keyed read was unsuccessful, the first * of the beginning marker is sent to the browser and the process is repeated from step d. above.
       Once the entire file or section has been streamed to the browser, the control command is completed.

It should be noted that the syntax used for the begin and end indicators denotes HTML comments. Thus the markers would not be visible in the html page if the browser displayed them.

It should also be noted that even though the operations above look like operations on a file, the data is really read into memory and streamed from there for performance reasons.

Following are two lessons from a tutorial which deal with the MERGE command:

This lesson and the next one are the most important lessons in the eComet tutorial. In this lesson, you will learn how to merge live Comet data into stored HTML documents.

Let us repeat that last part (for emphasis): you can merge live Comet data into a stored HTML document in order to make a dynamic return page.

Here's how it works:

First, create an XAP-ready HTML document. In the locations where you want to include a merge field, surround the field name with double asterisks.

Here's a sample HTML file. We've included one merge field named **TODAY**.




Sample merge document




This is a sample document.

Today's date is **TODAY**.



For the sake of this example, let's suppose we saved this file on drive C: and named the file TODAY.HTM.

Next, write an Internet Basic program that does two things:

1.Writes the merge field(s) to the CGI Environment File, using the double-asterisk field name as the key

2.Merges the HTML document with the CGI Environment File via the MERGE command

Here's some sample code to show you how this works:



TODAY$ = '06/08/2000'

WRITE (1,CGIDATA) KEY='**TODAY**' ! Write merge field

A$ = CONTROL(0,'MERGE C:\TODAY.HTM') ! Merge the HTML file

The WRITE statement writes a record to the CGI Environment file (remember, this file is automatically opened on Logical Unit Number 1 when the XAP program is started). The key is the same as the merge field name in the HTML document (i.e.,**TODAY**).

The MERGE command merges the HTML document (C:\TODAY.HTM) with any merge field(s) contained in the CGI Environment File. In this example, there's only one merge field.

The result is a web page that contains "06/08/2000" in the exact location where "**TODAY**" was contained in the stored document.

Here are some important notes:

The MERGE command is very similar to the SEND command (i.e., it sends a stored document from the eComet system to the browser.

The merge field(s) can appear anywhere in the stored document. This means that you can merge Comet data into places where you want data to appear, and you can also merge HTML commands into a stored document (to change the appearance of a web page for a particular user, for example).

Here's another example. Remember the program (from Lesson 2) that displayed several Comet system variables? Well, we've rewritten that example using the MERGE command.

First, we created an HTML document and included several merge fields. The "raw" document is named systvar.htm. Take a look at the HTML source code and notice the merge fields (**PARTITION** and the others).

Next, we created an Internet Basic program that writes the desired merge records to the CGI Environment File. Look at the following source code and notice the WRITE statements.



! *** Main program


A$ = DSTAT('CL1') ! Refresh TIME$


OPEN (10) '#CGICTRL' ! Open CGI control file



CLOSE (10)


! *** Write merge fields to CGI file


OUTPUT$ = PARTITION$ ! Partition #

WRITE (1,OUTPUT) KEY='**PARTITION**' ! Write record


OUTPUT$ = TIME$ ! System time

WRITE (1,OUTPUT) KEY='**TIME**' ! Write record


OUTPUT$ = DATE$ ! System date

WRITE (1,OUTPUT) KEY='**DATE**' ! Write record


OUTPUT$ = LONGYEAR$ ! 4-digit year

WRITE (1,OUTPUT) KEY='**LONGYEAR**' ! Write record


OUTPUT$ = VERSION$ ! Version #

WRITE (1,OUTPUT) KEY='**VERSION**' ! Write record


! *** Merge the CGI file with the HTML file, and send the results


MERGESTRING$ = 'MERGE '+HTMLPATH$ + 'systvar.htm' ! Build MERGE string





The MERGE command may also be executed via the FILE statement, as shown here:

FILE (0) CTL='MERGE ______________'

HTML file name

In Lesson 3, we showed you how to send multiple stored documents to the web browser with the SEND command. The MERGE command can be used to do the same thing.

For example, here's a program segment that sends 3 stored documents via the MERGE command:

A$ = CONTROL(0,'MERGE c:\ecomet\html\sample3x.htm')

A$ = CONTROL(0,'MERGE c:\ecomet\html\sample3y.htm')

A$ = CONTROL(0,'MERGE c:\ecomet\html\sample3z.htm')

Please note these differences between SEND and MERGE:

The MERGE command invokes the merge feature, while the SEND command does not

There's another very important difference between SEND and MERGE. The MERGE command can send sections of an HTML document. The sections are user-defined portions of the HTML file.

These sections are defined via the HTML comment tag and some XAP-specific keywords within each comment. Here's an example:

This is a section of HTML code. This

section includes text as well as HTML


This whole section is defined by the BEGIN and END statements, which are contained within HTML comments. The browser ignores the BEGIN and END statements, since they're contained within HTML comments. However, the MERGE command uses the BEGIN and END commands to defined a section of the file. This section is called ABC123. Assume that you saved the above HTML file as: c:\ecomet\html\test.htm Here's how you would send this section of the HTML file via the MERGE command: A$ = CONTROL(0,'MERGE c:\ecomet\html\test.htm SECTION ABC123') Some important notes: There is no limit to the number of sections you may define The first section in an HTML document does not require a statement The final section does not require an statement. No blank spaces are allowed at the beginning of these commands (i.e., inside of the " statement and an statement. Content outside of those boundaries will not be sent to the browser. The next example is based on the sample program you saw in Lesson 4. In this case, we will be merging data from a Comet data file, the DLRS file, into a stored HTML document. The HTML document will be divided into 3 sections, named TOP, MIDDLE, and BOTTOM. The TOP section will contain the initial HTML commands and text for the top portion of the web page. Since this is the first section, we don't need a statement. <HTML> <HEAD> <TITLE> Comet Dealers </TITLE> </HEAD> <BODY BGCOLOR="FFFFFF">

Comet Dealers

Here is a list of Comet dealers.

This data is contained in a

Comet keyed file named DLRS.

The DEMO11B program reads records

from that file and displays the

complete list of names.

The MIDDLE section will be very short. In this section, we will include a merge field named **DEALER**, followed by a line break tag (

    • DEALER**

By itself, the MIDDLE section does not look very impressive. Just wait until you see the way we use Internet Basic to merge data into this section! The BOTTOM section will contain the final HTML commands for our sample web page: </BODY> </HTML> We'll save the above file and name it: c:\ecomet\html\dealers.htm Now all we have to do is write an Internet Basic program that sends the 3 sections (and merged data) to the web browser. The first MERGE command will send SECTION TOP: A$ = CONTROL(0,'MERGE c:\ecomet\html\dealers.htm SECTION TOP') The next part of the Internet Basic program will READ data from the DLRS file, WRITE a key to the CGI Environment File, and MERGE SECTION MIDDLE. These steps will be repeated for all of the records in the DLRS file: OPEN (10) 'DLRS' ReadMore: READ (10,DEALERINFO) EXCP=AllDone WRITE (1,DEALERINFO) KEY='**DEALER**' A$ = CONTROL(0,'MERGE . SECTION MIDDLE') GOTO ReadMore Notice that the key value (**DEALER**) matches the merge field in the HTML file. Also notice that the MERGE command contains a different format than before. If you are merging more than one section of an HTML file, you only have to specify the file name once (on the first MERGE command). After that, you can use the period (.) to represent the HTML file name. The period is both a coding shortcut and a way to get faster performance from the MERGE command. Once an HTML file has been opened and read into memory, there's no need to re-open and re-read it. Thus, the period refers to a document that is already in memory. The final section the program closes the data file and sends the BOTTOM section of the HTML file: AllDone: CLOSE (10) A$ = CONTROL(0,'MERGE . SECTION BOTTOM') KILL PARTITION$ Here's something to Think About As we mentioned above, merge fields can appear anywhere in an HTML file, including HTML commands and parameters. For example, you could include a merge field where you would normally include a hard-coded parameter, as in the BACKGROUND portion of the BODY tag: <BODY BACKGROUND="**PICTURE**"> Then, from within an Internet Basic program, you could define a specific background graphic simply by writing the **PICTURE** key to the CGI Environment File and merging it with your HTML file. PICTURE: FORMAT PICTURE$ . PICTURE$ = "SAMPLE.GIF" WRITE (1,PICTURE) KEY="**PICTURE**" A$ = CONTROL(0,'MERGE HTML-document SECTION section-name') You could use this technique to good effect, for example, by displaying a unique background graphic for each customer who uses your web site. You could determine who's who by reading a persistent cookie (assuming you had already sent one to the browser), getting unique data from that cookie (a customer number, let's say), and using that data to read a GIF or JPG file name from a keyed file. Then you could write that file name to the CGI Environment File and merge it into an HTML file, as shown in the code segment above. This would result in a very personalized web site for each of your customers.
  • Keyed Files
Keyed files provide direct access to the data records. Comet stores data for keyed files in two DOS disk files. One DOS file stores the data records in their entered order, and a "key tree" file stores the record keys in ascending ASCII order. An individual record may be retrieved using this identifier using the KEY= parameter of the READ (or associated input) statement. A keyed file may also be read in key-sequential order; records are retrieved in ascending sorted key order. The index for a keyed file is a string field from 1 to 64 characters long. When the file is created, the key length is specified. So is the record length; this value ranges from 1 to 1,024 bytes, and is fixed for each record in the file. Because of the ease of use and advantages of direct access, keyed files are the most common types of data files used in Internet Basic applications. When an Internet Basic program opens a keyed file, the Comet operating system assigns a unique file pointer to the file. Each user opening the file is assigned a unique pointer, allowing multiple users to access data from the same file at the same time. To avoid data integrity problems when more than one user is accessing a file, Internet Basic provides a record locking mechanism. The EXTRACT statement is used to read and lock individual data records. As a user accesses records in a keyed file, the file pointer keeps track of the current location. When the file is opened, the pointer is located at the record associated with the first key. If the records are retrieved without an index, the pointer moves along in ascending ASCII order of the key values (i.e., in key-sequential order). When a record is accessed directly using an index, the pointer is moved to the associated position. If no record is found matching the specified index, an exception occurs, but the pointer is located at the index following the requested index value. From this point, the program could retrieve records in sequence (without an index), or retrieve another record from the file using another index value. Keyed files may also be accessed with a numeric index. In this case, the index value is used to specify the record number in the file in record number sequence (rather than in key sequence). Application notes: Since Key trees are built using "Balanced B Tree" structures, access to specific records is very fast. Depending on key length, access to a single record in a file of several million records may only take 3 or 4 reads from the underlying file system. A keyed file provides the most convenient way to randomly access a file. Therefore, most Internet Basic programs use keyed files to store data records. When a record is deleted from a keyed file, the vacated space is automatically reused by the file to store subsequent records. This advantage is another reason why keyed files are predominant in Internet Basic applications.
Personal tools