John Forkosh Associates ®
E D S   D e s i g n   D o c u m e n t s
Copyright © 1986-2001, John Forkosh Associates, Inc.
All rights reserved.

Click for:   homepage,   resume



  CCCCCCC    OOOOOO   NN      NN TTTTTTTTTT EEEEEEEEEE NN      NN TTTTTTTTTT   SSSSSSS 
 CCCCCCCCC  OOOOOOOO  NNN     NN TTTTTTTTTT EEEEEEEEEE NNN     NN TTTTTTTTTT  SSSSSSSSS
CC      CC OO      OO NNNN    NN TT  TT  TT  EE     EE NNNN    NN TT  TT  TT SS        
CC         OO      OO NN NN   NN     TT      EEEEEE    NN NN   NN     TT      SSSSSSS  
CC         OO      OO NN   NN NN     TT      EEEEEE    NN   NN NN     TT       SSSSSSS 
CC      CC OO      OO NN    NNNN     TT      EE     EE NN    NNNN     TT             SS
 CCCCCCCCC  OOOOOOOO  NN     NNN    TTTT    EEEEEEEEEE NN     NNN    TTTT    SSSSSSSSS 
  CCCCCCC    OOOOOO   NN      NN    TTTT    EEEEEEEEEE NN      NN    TTTT     SSSSSSS  

   1: tcpdsgn.doc
   2: kbddsgn.doc
   3: dbudsgn.doc
   4: affdsgn.doc



TTTTTTTTTT   CCCCCCC  PPPPPPPP   DDDDDDDD     SSSSSSS    GGGGGG   NN      NN           
TTTTTTTTTT  CCCCCCCCC PPPPPPPPP  DDDDDDDDD   SSSSSSSSS  GGGGGGGG  NNN     NN           
TT  TT  TT CC      CC  PP     PP  DD     DD SS         GG         NNNN    NN           
    TT     CC          PPPPPPPP   DD     DD  SSSSSSS   GG    GGG  NN NN   NN   .       
    TT     CC          PPPPPPP    DD     DD   SSSSSSS  GG   GGGGG NN   NN NN  ...      
    TT     CC      CC  PP         DD     DD         SS GG      GG NN    NNNN .....     
   TTTT     CCCCCCCCC PPPP       DDDDDDDDD  SSSSSSSSS   GGGGGGGG  NN     NNN  ...      
   TTTT      CCCCCCC  PPPP       DDDDDDDD    SSSSSSS     GGGGGG   NN      NN   .       



==============================================================================

            TERMINAL CONTROL PROGRAM DESIGN AND PROGRAMMING NOTES

==============================================================================


                                TABLE OF CONTENTS
                                -----------------

I.      INTRODUCTION
        1. Purpose
        2. Source code
        3. Processing Overview
        4. TCP Structure Chart

II.     EDS_DEVDEF.H DATA STRUCTURES
        1. Purpose
        2. DCPDEF Structure
        3. Additional DEVDEF Data Elements

III.    EDS_TCPMAIN PROGRAM
        1. Overall Program Structure
        2. Pseudocode for EDS$_ACKMSG Requests
        3. Pseudocode for EDS$_MSG Requests
        4. Pseudocode for tcp_ast() Routine
        5. Programming Notes
 

I. INTRODUCTION
===============

        1. Purpose
        ----------
        The functionality supported by the terminal control program (TCP)
        is as follows:
                (a) reads ttydef.txt file and creates the device table
                    global section containing the specified devices,
                (b) allocates, assigns, and setmode's each device,
                (c) mails initialization messages for each successfully
                    assigned device to the CRT process,
                (d) issues read qio's whenever the CRT ack's a device,
                (e) issues write qio's for each output buffer mailed
                    to TCP by the CRT.

        2. Source Code
        --------------
        All terminal control program (TCP) source code is contained in
        EDS_TCPMAIN.C, which resides in directory [.TCP] from the
        current root.  The device structure created for each device is
        defined by EDS_DEVDEF.H and EDS_DCPDEF.H in [.LIB].
        TCPMAIN_BLD.COM in [.TCP] links tcpmain (and other library modules)
        into the executable image.

        3. Processing Overview
        ----------------------
        The TCP is responsible for two initialization and three runtime tasks.
        During initialization the TCP does the following:
            o   Reads its ttydef.txt file (filename is 3rd argument on command
                line), and builds the device table global section for each
                device.  The readfile() routine parses the ttydef.txt input
                file, and passes the device information down to storetcp(),
                which is a storestruct()-type routine.
            o   For each successfully-assigned device, TCP mails an
                EDS$_INITMSG to its parent CRT process.
        After initialization the TCP waits for mailbox messages from its
        parent CRT:
            o   Each EDS$_ACKMSG from the CRT causes tcp to call readasynch()
                for the acked device, thus arming its qio for the next read.
            o   Each EDS$_MSG from the CRT contains an output buffer that is
                written to the device by a call to wrtasynch().
            o   Each read completion asynchronously calls tcp_ast() which mails
                an EDS$_MSG containing the qio input buffer to the CRT.

        The EDS$_INITMSG originally sent to the CRT always (for terminal-type
        devices) results in an EDS$_ACKMSG reply.  This reply arms the
        first qio to the device.  If the init msg is not acked reads are
        never issued to the device.

        Subsequent qio's are issued only after each message is acked.
        Therefore no device can have more than one outstanding ast at any
        particular moment.  This tactic prevents a "noisy" device from
        monopolizing the CRT by filling its mailbox.  Each device's typeahead
        buffer thus automatically queues messages to be serviced,
        rather than queueing them in the CRT's single-threaded mailbox.
 

        4. TCP Structure Chart
        ----------------------
        The routines comprising the TCP process are organized as shown
        in the following diagram.


                                   +---------+
                                   | TCPmain |
                                   +----+----+
                                        |
                                        |
          +--------+----------------+---+----------------+-----------+
          |        |                |                    |           |
          |   +----------+     +----------+         +--------+  +---------+
          |   | readfile |--+  |readasynch|         |readmbxw|  |wrtasynch|
          |   +----------+  |  +----------+         +--------+  +---------+
          |        |        |       |                      |        |
          |        |        |      AST interrupt -----+    |        |/|
          |   ------------  |            |            |    |          |
          |    ttydef.txt   |  +-------------------+  |    |        +----+
          |   ------------  |  | AST Address Table |  |    |      +----+ |
          |                 |  | ----------------- |  |    |    +----+ |-+
         +------+-----+     |  | 1. tcp_ast        |  |    |    |DEVs|-+
         |      |     |     |  | 2.                |  |    |    +----+
     +--------+ |     |     |  | 3. (no other      |  |    | (note-readasynch
     |groupmbx| |     |     |  | 4. ast routines   |  |    | also connects to
     +--------+ |     |     |  | 5. defined)       |  |    | the same devices)
           +--------+ |     |  | 6.                |  |    |
           | crembx | |     |  | 7.                |  |    |
           +--------+ |     |  +-------------------+  |    |
                 +--------+ |   |            |        |    |
                 | crmpsc | |   |         +--+--+     |    |
                 +--------+ |   |         |     |     |    |
                        |   |   |    +--------+ |     |    |
                        |   |   |    | wrtmbx | |     |    |
                        |   |   |    +--------+ |     |    |
                        |   |   |        | +----------+    |
                        |   |   |        | |readasynch|    |
                        |   |   |        | +----------+    |
                        |   |   |        |                 |
                        |   |   |        |                 |
                       +---------+     +---------------------+
                       | DEVTBL  |     |                     |
                       |  Global |-----|     CRT Process     |
                       | Section |     |                     |
                       +---------+     +---------------------+
 

II. EDS_DEVDEF.H DATA STRUCTURES
================================

        1. Purpose
        ----------
        Every EDS device has an associated data structure that specifies
        generic information relevant to the qio's issued on its behalf,
        as well as application-dependent data that maintains information
        necessary to support the functionality associated with the device.
        Generic qio information is stored in a dcptable structure, which is
        usually included in a larger structure containing both the dcptable
        and the application-dependent data.  For TCP/CRT devices, the
        embedded dcptable is used by TCP to carry all the VMS arguments
        necessary for issuing qio's.  Other data is used by CRT to control
        keyboard translation, cluster connections, etc.

        2. DCPDEF Structure
        -------------------
        All the dcptable parameters are used in a straightforward way by TCP
        as arguments to readasynch() and wrtasynch() for issuing qio's.
        The termtbl[] string is re-initialized every time a keyboard change
        is executed, and this must be done by CRT before issuing the ack that
        re-arms the ast.

        The dcptable structure has the following form:
        struct  dcptable_struct {
        /* --- Device Initialization Information for assign and setmode --- */
        unsigned char   devnam[16];             /* name of device */
        int             chan;                   /* assigned channel */
        int             acmode;                 /* access mode */
        int             baud_rate;              /* speed for setmode QIO */
        /* --- QIO Parameters required by DCP --- */
        int             efn;                    /* efn to set on completion */
        int             astadr;                 /* addr of AST routine */
        int             astprm;                 /* AST parameter */
        int             maxmsg;                 /* maxmsg size for qio read */
        int             timeout;                /* timeout count for qio */
        short           iosb[4];                /* i/o status block for qio */
        unsigned char   termtbl[32];            /* terminator table for qio */
        unsigned char   *qio_buff;              /* ptr to qio message buffer */
        /* --- Various other Device Control Parameters --- */
        int             devnum;                 /* -1 or global sect index */
        int             assigned;               /* TRUE if device assigned */
        int             posted;                 /* TRUE when qio is pending */
        int             re_arm;                 /* TRUE if AST rearms itself */
        int             ast_count;              /* #pending AST's for device */
        int             device_type;            /* Type of device */
        int             device_state;           /* State of device */ }
 

        3. Additional DEVDEF Data Elements
        ----------------------------------
        During initialization, TCP's storestruct routine moves information
        from ttydef.txt into the device initialization area of devtable.
        This includes device_type (terminal,video), entitlements (control,
        display,update), etc.  Data needed for keyboard translation includes
        an operatormsg structure containing the previously-entered nesid,
        the page number, etc.  Flags identifying the show, show type, etc
        are used by the CRT itself.

        The devtable structure has the following form:
        struct  devtable_struct {
        /* --- Buffers and Parameters needed by TCP to issue QIO's --- */
        struct dcptable_struct dcptable;        /* generic device table */
        unsigned char   qio_buff[80];           /* qio buffer in devtable */
        /* --- Additional Device Initialization Information --- */
        unsigned char   devmode[32];            /* device mode for CRT */
        int             device_state;           /* Initial state of device */
        int             device_class;           /* Terminal/Video/Control */
        int             device_type;            /* Type of device in class */
        unsigned char   control_unit;           /* control unit (cluster) id */
        unsigned char   device_ident;           /* twin for tty, or vid-a/b */
        char            entitlements[8];        /* YN control/update/display */
        int             keyboard;               /* keyboard number (0-4) */
        char            *kbd_sect;              /* crt ptr to global kbd tbl */
        /* --- Buffers and Parameters needed by kbdxlate routine --- */
        int             msgsiz;                 /* actual size of msg_buff */
        unsigned char   msg_buff[80];           /* complete msg to crt */
        unsigned char   select_opcode;          /* opcode for page# resets */
        int             prefix_opcode;          /* opcode_tbl index for pref */
        struct crtmsghdr_struct crtmsg;         /* mbx msg, see eds_mbxdef.h */
        struct operatormsg_struct operatormsg;  /* xlated operator msg */ 
        /* --- Buffers and Parameters needed by CRT --- */
        int             devindx;                /* device index in dev glbl*/
        char            partner_id;             /* tcp identification */
        unsigned char   ctrlw;                  /* refresh screen flag */
        unsigned char   show;                   /* Show using this device*/
        unsigned char   show_type;              /* Election or Sports */ }
 

III. EDS_TCPMAIN PROGRAM
========================

        1. Overall Program Structure
        ----------------------------
        Tcpmain is in the form of a shell, i.e., it was originally copied
        from [.LIB]EDS_DCPSHELL.C and then adapted to support its required
        functionality.  It therefore contains a large while(TRUE) loop where
        it first waits for a mailbox message to wake it up, and then does
        a switch on the mailbox message's opcode byte.  Two cases of this
        switch, EDS$_ACKMSG and EDS$_MSG constitute TCP's specific
        functionality.  EDS$_ACKMSG posts another qio read to a device, and
        EDS$_MSG writes a buffer to a device.

        The asynchronous part of TCP's functionality is provided by its
        ast routine, tcp_ast().  Every time a read completes, tcp_ast mails
        the qio buffer to its CRT parent process.  However, it does not re-arm
        the ast.  Rather, TCP waits for CRT to ack the buffer, and then
        tcpmain re-arms it itself.

        2. Pseudocode for EDS$_ACKMSG Requests
        --------------------------------------
             o  call sect_ptr() to get pointer to device table structure
                in the global section,
             o  extract pointer to dcptable from devtable,
             o  call readasynch() to re-issue the qio, using the devtable
                structure address as the argument passed to the ast routine.

        3. Pseudocode for EDS$_MSG Requests
        -----------------------------------
             o  call sect_ptr() to get pointer to device table structure
                in the global section,
             o  extract pointer to dcptable from devtable,
             o  call wrtasynch() to write the mailbox message buffer
                to the device.

        4. Pseudocode for tcp_ast() Routine
        -----------------------------------
             o  extract pointer to dcptable from devtable argument,
             o  write a masilbox message to the CRT containing the
                device number, so that CRT can read the qio buffer
                from the global section.

        5. Programming Notes
        --------------------
             o  You may essentially ignore all the variables (and associated
                functionality) referring to ast_count, total_asts, etc.  These
                were coded before the ack protocol was implemented, and in that
                environment they prevented a device from accumulating too many
                unanswered asts.  This situation is now logically impossible so
                that code is vestigial.  It was left in place just in case some
                situation ever arose where its functionality might be useful,
                but it should be mentally ignored when reading the code.
             o  In tcp_ast, the if(total_asts < MAXAST_COUNT) test is always
                satisfied (see note above), so only the first half of the code
                is relevant.  It simply mails the input buffer to the CRT.



KKKK    KK BBBBBBBB   DDDDDDDD   DDDDDDDD     SSSSSSS    GGGGGG   NN      NN           
KKKK   KK  BBBBBBBBB  DDDDDDDDD  DDDDDDDDD   SSSSSSSSS  GGGGGGGG  NNN     NN           
 KK   KK    BB     BB  DD     DD  DD     DD SS         GG         NNNN    NN           
 KKKKKK     BBBBBBBB   DD     DD  DD     DD  SSSSSSS   GG    GGG  NN NN   NN   .       
 KKKKK      BBBBBBBB   DD     DD  DD     DD   SSSSSSS  GG   GGGGG NN   NN NN  ...      
 KK  KKK    BB     BB  DD     DD  DD     DD         SS GG      GG NN    NNNN .....     
KKKK   KK  BBBBBBBBB  DDDDDDDDD  DDDDDDDDD  SSSSSSSSS   GGGGGGGG  NN     NNN  ...      
KKKK    KK BBBBBBBB   DDDDDDDD   DDDDDDDD    SSSSSSS     GGGGGG   NN      NN   .       



==============================================================================

             KEYBOARD LEXICAL SCANNER DESIGN AND PROGRAMMING NOTES

==============================================================================


                                TABLE OF CONTENTS
                                -----------------

I.      INTRODUCTION
        1. Purpose
        2. Source code
        3. Processing Overview
        4. Flow Diagram

II.     EDS_KBDINIT AND EDS_STOKBD
        1. Purpose
        2. Data Structures and Usage
        3. Keyboard.txt Syntax
        4. Processing Overview
        5. Programming Notes

III.    EDS_RDCRTMBX
        1. Overall Program Structure
        2. Pseudocode for rdcrtmbx
        3. Programming Notes

 

I. INTRODUCTION
===============

        1. Purpose
        ----------
        The functionality supported by the Keyboard Translation Subsystem
        is as follows:
            (a) initialize the keyboard global section,
            (b) translate sequences of operator keystrokes into
                EDS command strings,
            (c) interpret numeric strings as one or two numbers, e.g.,
                for menu burst command,
            (d) maintain ongoing nesid from state,race,source,etc, keys,
            (e) maintain page number.

        2. Source Code
        --------------
        Several sets of routines comprise the keyboard subsystem, all of which
        reside in [.TCP] with corresponding .H files in [.LIB]:
             o  EDS_KBDINIT.C and EDS_STOKBD.C in [.TCP], together with
                EDS_OPSDEF.H and EDS_OPS2DEF.H in [.LIB] comprise the source
                code that creates the keyboard global section used to
                translate operator keystrokes.  Of course, the keyboard.txt
                text file is used to determine the actual keystroke
                translation.
             o  EDS_RDCRTMBX.C (linked into the CRT) captures all mailbox
                messages to the CRT.  If the mailbox message opcode is
                EDS$_MSG then it contains an operator command buffer and
                is run through the translation algorithms.  Otherwise,
                the message is from someone else, and it is passed back to
                the CRT unchanged.
             o  EDS_KBDXLATE.C translates keystrokes using the keyboard
                global section, and accumulates TCP buffers until a
                terminating opcode (indicating a completed command) is
                detected.
             o  EDS_KBDNUM.C and EDS_KBDNES.C are used by kbdxlate to
                maintain the nesid, page#, etc, after a completed command
                is detected.
 

        3. Processing Overview
        ----------------------
        Consider a TCP/CRT pair that has been brought up, and is currently idle
        (i.e., no messages are being generated).  Now imagine that an operator
        enters a series of keystrokes ending with a (qio) command terminator.
        The following steps will take place (see the diagram's corresponding
        step numbers):

        Step# Program   Description of Processing

        (#1)  CRT       The CRT completes the previous task and calls rdcrtmbx
                        which has been waiting for mail from the CRT's mailbox.

        (#2)  tcp_ast   The operator presses a qio-terminator key and VMS
                        interrupts the TCP, executing tcp_ast to handle the
                        qio.

        (#3)  tcp_ast   The ast routine formats and writes a mailbox message to
                        the CRT and then exits, and the TCP continues.

        (#4)  rdcrtmbx  Wakes up (see step 1) and reads the AST's message.
                        If the message is not from a tty device, i.e., if it
                        doesn't require translation, it is passed back to CRT
                        immediately (i.e., go to step 8).

        (#5)  kbdxlate  Messages requiring translation are passed from rdcrtmbx
                        to kbdxlate which fills in the operatormsg structure
                        fields before returning. It first determines the opcode
                        and then calls the appropriate routine(s) to translate
                        the message (see kbdnum and kbdnes below).

        (#6)  kbdnum    This routine translates a message containing one (or
                        more) numeric field(s).

        (#7)  kbdnes    This routine translates a message consisting of
                        an nes key.

        (#8)  rdcrtmbx  Gets back the fully translated message from kbdxlate,
                        and returns it to CRT for processing.

        (#9)  CRT       Processes the command, constructs one (or more) output
                        buffers which it mails to the TCP to be written to
                        the device(s).

        (#10) CRT       Finally, the CRT writes an < ACK > message to
                        the TCP's mailbox, and this message causes the next
                        qio to be posted, thus repeating the command cycle.
 

        4. Flow Diagram
        ---------------
        The diagram below depicts the structure of TCP/CRT communications
        and keyboard message translation, and the 10 numbered steps demonstrate
        the flow of a typical transaction (see explanatory notes for each step
        above).

                                   write                                  
                                 the msg     +---------------------+      
        devices              +-<-------------< MSG >-+          TCP|
           +----+    outbound|    re-arm AST |       |--+          |      
         +----+ |         msg|  <------------< ACK >-+  |          |      
       +----+ |-+ <====//    |               +----------|----------+      
       |TTYs|-+       //====>|                 |        |                 
       +----+                |              all|    +-------+             
                      inbound|             done|    |TCP mbx|             
                   (#2)   msg|   +--------+    |    +-------+             
                             +->-|AST trap|-->-+        |                 
                                 +--------+             |                 
                                     | (#3)             |
       +------------+ to DBS         |                  |                 
       |< recv msgs >--<-------------|-------------+    |                 
       | OTHER PGMS | from AUP       |             |    |                 
       |< send msgs >---->-----+     |          +-----------+              
       +------------+          |     |          | WRTMBXMSG |              
                              +-------+         +-----------+              
                              |CRT mbx|            |      |               
                              +-------+          (ACK)  (MSG)
                                  | (#4)           |      |               
           +--------+(#5) +-------|-------+   (#1) |      |formatted      
           |KBDXLATE|---------(from AST)  |--<--+  |      |output msg     
           +--------+     |       RDCRTMBX|     |  |(#10) |(#9)           
               |          +---------------+   +-----------------+      
           +---+----+             | (#8)      |    |      |  CRT|      
       (#6)|    (#7)|             +----->-----|-Process command |      
       +------+ +------+         xlated input |                 |      
       |KBDNUM| |KBDNES|                      +-----------------+      
       +------+ +------+                                                  
 

II. EDS_KBDINIT AND EDS_STOKBD
==============================

        1. Purpose
        ----------
        The "front end" of keyboard translation is table driven by data in a
        global section that maps every keystroke onto a string.  For instance,
        if 'A' maps onto "THIS IS " and 'B' maps onto "A TEST", then if the
        operator presses AB (and assuming the qio terminates) we want the
        CRT to ultimately behave as if the string "THIS IS A TEST" had been
        entered.
        
        2. Data Structures and Usage
        ----------------------------
        The global section containing the keyboard mapping strings is organized
        as follows.  These structures are defined in [.LIB]EDS_GLBDEF.H.

        The first 42 bytes of the global section contains the following
        generic header:
        struct glbhdr_struct
                { char  sectnam[16];    /* name assigned to this section */
                int     pagcnt;         /* # 512-byte pages to be allocated */
                int     recsz;          /* #bytes allocated per getsect call */
                int     maxused;        /* max index of a record in use */
                int     maxrecs;        /* max index that fits in section */
                int     offset1;        /* #bytes to 1st byte of 1st record */
                int     lockflag;       /* set me if you're writing to sect */
                char    process_id[2]; }
        Most of the generic header is unused for the keyboard table.  Only
        lockflag has any meaning, in that it lets kbdinit know when the
        first CRT has mapped to the section.  As soon as this happens,
        kbdinit exits.

        After these 42 bytes there are 16 (note that NKBDS is defined as 16
        in eds_glbdef.h) structures, one for each of 16 possible keyboard
        translation tables.  Each such structure contains 548 bytes:
        struct kbdtbl_struct
                { int   maxmsg;                 /* max chars in qio read */
                unsigned char termtbl[32];      /* qio terminator table */
                int     offset[128]; }          /* offsets to equivalence str */
        Each qio issued by TCP terminates either on maxmsg or on one of the
        characters specified in termtbl, whichever comes first.  TCP sends
        the untranslated string to CRT where translation is done by kbdxlate.
        Kbdxlate simply uses each char as an index into the offset area.
        If the offset is 0 that means that the key is untranslated.  Otherwise,
        offset is used as a byte offset (from the very beginning of the global
        section), and the approrpiate string is retrieved.  Then either the
        untranslated key or the string translation is concatanated together
        with all the other keystrokes to form the fully translated operator
        message.

        After the 42-byte header and the sixteen 548-byte tables is the string
        translation area.  This area is addressed by the offset arrays in the
        kbdtbl structures above.  Both are built on the fly as stokbd reads the
        keyboard.txt file and copies translation strings into the global
        section.  Each string is separated from the next by a null.
 

        3. Keyboard.txt Syntax
        ----------------------
        When kbdinit comes up it first initializes the 16 kbdtbl structs
        by setting maxmsg to 8, all 32 termtbl bytes to zero, and all 128
        offsets to zero.  Thus, if nothing else happened, every qio would
        terminate after 8 bytes, and there would be no translation.
        This default situation is modified by stokbd which reads the
        keyboard.txt file and resets the global section accordingly.

        After this initialization, kbdinit calls readfile (which
        further calls stokbd, the storestruct entry for kbdinit) to process
        the keyboard.txt file.  This file specifies the keyboard translation
        for all 16 keyboards with the following three card types:
                KEYBOARD / kbd#
                MAXMSG / maxbytes_for_qio
                keystroke(s) / translation

        The first card type has uppercase KEYBOARD as its first token and
        the keyboard number 1-16 as its second.  All cards following a
        KEYBOARD card refer to that keyboard until another keyboard card
        is encountered, etc.  If a KEYBOARD card is not the first card in
        the file, then keyboard#1 is assumed until a KEYBOARD card occurs.
        The MAXMSG card sets the maxmsg word in the kbdtbl structure.
        If more than one appears (for the same keyboard), the last one
        encountered takes effect.  The keystrokes/translation card requires
        a bit more explanation than the others:

        Any card not containing KEYBOARD or MAXMSG as its first token is
        interpreted as a keystrokes/translation card.  The first token
        is a series of keystrokes, each of which will be translated to the
        same string, i.e., the string in the second token.  Thus, for instance,
        Aa/HELLO will translate both uppercase and lowercase A's to HELLO.
        (Note: it's more efficient to use this form instead of A/HELLO and
        a/HELLO since the combined format will add only one string to the
        string area, and then point both offsets to it.)  For special keys,
        e.g., carriage return and erase, you may use the construction
        \012/translation or \127/translation or ABC\012\127DEF/translation,
        etc.  The backslash ALWAYS takes exactly the three chars following it
        and does an atoi on them, using the result to set the corresponding
        offset.  Note that our backslash's arguments are thus in base 10.
        Also, this same backslash construction may appear on the translation
        side of the card if you want the keystroke to translate to a string
        containing non-printable characters.

        One final, and important, note about the keystroke/translation card.
        It is not totally true that the backslash must ALWAYS be followed
        by exactly three chars.  The exceptions refer to the EDS opcodes
        (like control mode, select raw, and all our other qio-terminating
        delimiters) as follows.  Any backslash (on the translate side of the
        card) followed by one of the chars in the "Escape Mnemonic"
        column of the table below accomplishes two things.  First,
        it adds the defined value to the translation string.  Secondly, it adds
        the keystroke(s) to termtbl.  To understand this consider the following
        example.  Suppose we have the card !/\a where (looking down the
        mnemonics column) \a means VIDA key.  This means we need the qio to
        terminate when the operator presses !, so that char must be added to
        the termtbl.  Any keystroke that contains one of these backslash
        mnemonics anywheres in its translate string (usually at the end)
        is added to the termtbl for its keyboard.
 

        The special opcode mnemonics are reproduced below from EDS_OPSDEF.H:

        /* --- State Changes ---        --- Escape Mnemonic --- */
        #define EDS$CTRLMODE_OP '\001'          /* f */
        #define EDS$DISPMODE_OP '\002'          /* d */
        #define EDS$UPDTMODE_OP '\003'          /* u */
        /* --- Select Race(Source)/List --- */
        #define EDS$SLCTRAW_OP '\004'           /* R */
        #define EDS$SLCTVPA_OP '\005'           /* V */
        #define EDS$SLCTDLG_OP '\006'           /* G */
        #define EDS$SLCTNUM_OP '\007'           /* N */
        /* --- Control Commands --- */
        #define EDS$CLRWORK_OP '\010'           /* W */
        #define EDS$ADDMENU_OP '\011'           /* A */
        #define EDS$INSMENU_OP '\012'           /* I */
        #define EDS$DELMENU_OP '\013'           /* D */
        #define EDS$CLRMENU_OP '\014'           /* M */
        #define EDS$VIDA_OP '\015'              /* a */
        #define EDS$VIDB_OP '\016'              /* b */
        #define EDS$BURST_OP '\017'             /* B */
        /* --- Special Codes (e.g., enter string) --- */
        #define EDS$STRING_OP '\020'            /* S */
        #define EDS$STRING2_OP '\021'           /* T */
        #define EDS$STRING3_OP '\022'           /* U */
        #define EDS$STRING4_OP '\023'           /* Q */
        #define EDS$CHGTTY_OP '\024'            /* Y */
        #define EDS$ERSKBD_OP '\025'            /* k */
        #define EDS$CHGKBD_OP '\026'            /* K */



DDDDDDDD   BBBBBBBB   UUUU  UUUU DDDDDDDD     SSSSSSS    GGGGGG   NN      NN           
DDDDDDDDD  BBBBBBBBB  UUUU  UUUU DDDDDDDDD   SSSSSSSSS  GGGGGGGG  NNN     NN           
 DD     DD  BB     BB  UU    UU   DD     DD SS         GG         NNNN    NN           
 DD     DD  BBBBBBBB   UU    UU   DD     DD  SSSSSSS   GG    GGG  NN NN   NN   .       
 DD     DD  BBBBBBBB   UU    UU   DD     DD   SSSSSSS  GG   GGGGG NN   NN NN  ...      
 DD     DD  BB     BB  UU    UU   DD     DD         SS GG      GG NN    NNNN .....     
DDDDDDDDD  BBBBBBBBB    UUUUUU   DDDDDDDDD  SSSSSSSSS   GGGGGGGG  NN     NNN  ...      
DDDDDDDD   BBBBBBBB      UUUU    DDDDDDDD    SSSSSSS     GGGGGG   NN      NN   .       



==============================================================================

                DATABASE UPDATE DESIGN AND PROGRAMMING NOTES

==============================================================================


                                TABLE OF CONTENTS
                                -----------------

I.      INTRODUCTION
        1. Purpose
        2. Source code
        3. Applicable Documents
        4. Processing Overview

II.     EDS_DBS2DEF.H DATA STRUCTURES
        1. Purpose
        2. Msgfmt Structure Array
        3. Tokfmt Structure Array
        4. Fldfmt Structure Array

III.    DBSTRUCT.H DATA STRUCTURE
        1. Purpose
        2. Description of Fields

IV.     EDS_DISTCVT PROGRAM AND DATA STRUCTURES
        1. Purpose
        2. Description of Algorithms
        3. Eds_distcvt.h Structure

V.      EDS_DBUPDATE MAIN PROGRAM
        1. Overall Program Structure
        2. Pseudocode for EDS$_UPDMSG Requests
        3. Programming Notes

VI.     EDS_NESDEBLOCK
        1. Overall Structure
        2. Programming Notes

VII.    EDS_NESPARSER
        1. Overall Structure
        2. Pseudocode
        3. Programming Notes
 

I. INTRODUCTION
===============

        1. Purpose
        ----------
        The functionality supported by the database update process (DBU)
        is as follows:
            (a) it receives nes or vpa message blocks from the PCP,
            (b) reads the corresponding records from the race database,
            (c) updates these races in core, and
            (d) mails the updated records to the database server (DBS)
                to be rewritten into the database.

        2. Source Code
        --------------
        Three application source modules comprise the database update (DBU)
        process.  These are EDS_DBUPDATE.C, EDS_NESDEBLOCK.C,
        and EDS_NESPARSER.C, which reside in directory [.DBS] from the
        current root.  The table-driven functionality of nesparser is
        supported by EDS_DBS2DEF.H in [.LIB].  DBUPDATE_BLD.COM in [.DBS]
        links these modules (and other library modules) into the executable
        image.

        3. Applicable Documents
        -----------------------
        The external format of nes and vpa messages is described in the
        following documents:
                o ASYNC COMPUTER CODED SPECS, May 24 1986, distributed by NES
                o Notes on 370-PDP Records, 1/18/82, see Chuck Browne

        4. Processing Overview
        ----------------------
        Update messages from PCP come blocked and delimited in one of
        three ways:
                (message1)(message2)...(last message)        (from nes)
                < IRS >message1< IRS >...                    (from vpa)
                < EOM >message1< EOM >...                    (not used)
        Because the KMW converter translates IRS's, # replaces IRS in
        messages sent by vpa.  And vpa sends nes ()-messages in the form
        #(message)#(etc)#, so () must always take precedence over #.
        These violations of the documented protocol are discussed
        in the notes below.

        Regardless of the envelope, each message is deblocked by nesdeblock
        and passed to nesparser in a separate call, completely stripped of all
        delimiters, etc by the time that call is issued.  Nesparser then does
        all the functional work required, returning an updated core image
        of the race database record corresponding to the input message.
        This image is then mailed to DBServer to be rewritten.
 

II. EDS_DBS2DEF.H DATA STRUCTURES
=================================

        1. Purpose
        ----------
        Parsing nes/vpa messages in nesparser is driven with the aid of
        three arrays of structures declared in EDS_DBS2DEF.H.  The first
        array is msgfmt[] which contains one structure for each report type
        (e.g., N, S, V, D, C) to be parsed.  Msgfmt[] specifies the sequence
        of tokens comprising each message by pointing nesparser into the
        tokfmt[] structure array.  Tokfmt[] specifies various parameters about
        each token needed to continue parsing.  One of these is a pointer into
        the fldfmt[] structure array.  Whenever the contents of a token updates
        a database field, fldfmt[] specifies the position,length,format
        of the field to be updated.

        2. Msgfmt Structure Array
        -------------------------
        The first byte of every update message, regardless of source,
        is the report type: S for state summaries, V for vpa, etc.
        This character uniquely identifies the format of the message.
        Each structure in the msgfmt[] array of structures refers to one
        report type, identified by msgfmt[].rpttype.  Once the proper
        msgfmt[] is found, msgfmt[].toknum[] is an array of ints indicating
        the sequence of tokens that comprise the message.  For instance,
        an S-report will match msgfmt[2].rpttype, and the sequence of
        tokens is msgfmt[2].toknum[] = 1,2,3,4,8,9,10,11.  Each toknum[]
        is an index into the tokfmt[] array of structures.

        For example, the sixth token in an S-message is the number of precincts
        reporting.  The sixth toknum is 9, and a box is drawn around it
        illustrating how it indexes the tokfmt[] array of structures below.

        The msgfmt[] array of structures has the following form, and for the
        Election '86 it was initialized as follows...
        struct  msgfmt_struct {
        char    rpttype;                /* first char of message is rpttype */
        int     source;                 /* NES=2, VPA=3 (returned to main) */
        int     toknum[16]; }           /* indexes into tokfmt[] for report */
        msgfmt[] =
        {   /* rpttype source toknum[]... */
        /* 0 */ { 'N',  NES,  1,2,3,4,8,9,10,11,-1 },   /* report#9 is pres */
        /* 1 */ { 'V',  VPA,  1,2,3,5,6,7,12,-1 },      /* vpa */
                                        +-+
        /* 2 */ { 'S',  NES,  1,2,3,4,8,|9|,10,11,-1 }, /* nat'l elections */
    +-<---------------------------------+-+
    |   /* 3 */ { 'D',  NES,  1,2,3,4,8,9,10,11,-1 },   /* house */
    |   /* 4 */ { 'C',  NES,  1,2,3,13,14,15,9,10,11,-1 } /* county races */
    |   /* ----------------------------------------- */}
 
    |(continued from previous page)
    |
    |   3. Tokfmt Structure Array
    |   -------------------------
    |   A tokfmt[] structure specifies the token delimiter, which field
    |   in the database is updated by the token, etc.  Thus, once
    |   msgfmt[].toknum[] determines a sequence of tokfmt[] indices,
    |   nesparser can parse the message one token at a time and update
    |   the appropriate database field.
    |
    |   Continuing the example above, tokfmt[9] describes the precincts
    |   reporting token, which is stored in the database.  Exactly where in
    |   the database is specified by fldfmt[13] which is indexed by tokfmt.
    |
    |   The tokfmt[] array of structures has the following form, and for the
    |   Election '86 it was initialized as follows...
    |   struct  tokfmt_struct {
    |   int     fldpos;                 /* + = dbs field#, or - = key offset */
    |   int     algorithm;              /* usage: switch (algorithm) {...}  */
    |   int     maxlen;                 /* max length (max #bytes) of token */
    |   int     maxreps;                /* max repetitions (1 except votes) */
    |   char    delim;                  /* this char terminates token */
    |   } tokfmt[] =
    |   {      /* fld algor maxlen reps delim */
    |   /* 0 */ {  0,  0,      1,  1,  '\0' },  /* rptid not used, skip col1 */
    |   /* 1 */ { -4, ELECTN,  1,  1,  '\0' },  /* elctn/party */
    |   /* 2 */ { -3,  0,      1,  1,  '\0' },  /* race/office */
    |   /* 3 */ { -1,  0,      2,  1,  '/'  },  /* state id # */
    |   /* 4 */ { -5, RDRACE,  3,  1,  '/'  },  /* district# (3 chars) ... */
    |   /* 5 */ { -6, RDRACE,  2,  1,  '\0' },  /* ... but for House 2 chars */
    |   /* 6 */ {  9, VPAWIN,  1,  1,  '\0' },  /* status/win posted */
    |   /* 7 */ {  8, CPYTOK,  1,  1,  '\0' },  /* ok to display vpa */
    |   /* 8 */ {  0,  0,      3,  1,  ' '  },  /* county # */
    +---------->-+--+
        /* 9 */ {|13|,CPYTOK,  6,  1,  ' '  },  /* #precincts rpted */
    +-<----------+--+
    |   /*10 */ { 14, SETPRC,  6,  1,  ' '  },  /* #precincts juris */
    |   /*11 */ { 21, CPYVOT,  8, 15,  ' '  },  /* rawvote totals */
    |   /*12 */ { 22, CPYVOT,  2, 15,  '\0' },  /* vpa = 6 sets of 2 chars */
    |   /* --- special tokens for county-level fields in "C" report --- */
    |   /*13 */ { -5,  0,      3,  1,  '/'  },  /* district# for county */
    |   /*14 */ {  0, SETCNT,  3,  1,  ' '  },  /* county # determines cd */
    |   /*15 */ {  0,  0,      2,  1,  '\0' }   /* correction plus space */
    |   /* ------------------------------- */ }
 
    |(continued from previous page)
    |
    |   4. Fldfmt Structure Array
    |   -------------------------
    |   To update a database field, tokfmt[].fldpos contains a positive
    |   integer that indexes a final strcuture, fldfmt[].  The fldfmt[]
    |   indexed by tokfmt[].fldpos specifies the position, length, and format
    |   of the field in the database record.
    |
    |   The fldfmt[] array of structures has the following form, and for the
    |   Election '86 it was initialized as follows...
    |   struct  fldfmt_struct {
    |   int     fldtype;                /* 0=hdr field, 1=cand, 2=vote */
    |   int     offset;                 /* #bytes to beginning of field */
    |   int     fldlen;                 /* 0=4-byte int, non-0=ascii fld len */
    |   } fldfmt[] =
    |   /*fld*/ { /* fldtype offset-to-fld      fldlen  */
    |   /* 0 */ { HDR_FLD,      _NESID,         8       },
    |   /* 1 */ { HDR_FLD,      _NOFCAND,       0       },
    |   /* 2 */ { HDR_FLD,      _NESMAX,        3       },
    |   /* 3 */ { HDR_FLD,      _VPAMAX,        3       },
    |   /* 4 */ { HDR_FLD,      _RAWORDER,      MAXCAND },
    |   /* 5 */ { HDR_FLD,      _VPAORDER,      MAXCAND },
    |   /* 6 */ { HDR_FLD,      _RAWPART,       5       },
    |   /* 7 */ { HDR_FLD,      _VPAPART,       5       },
    |   /* 8 */ { HDR_FLD,      _IDVOK,         1       },
    |   /* 9 */ { HDR_FLD,      _IDWIN,         1       },
    |   /*10 */ { HDR_FLD,      _IDVPAR,        1       },
    |   /*11 */ { HDR_FLD,      _TSTAMP,        8       },
    |   /*12 */ { HDR_FLD,      _DBPREC,        3       },
    +-->/*13 */ { HDR_FLD,      _PRECRPT,       0       },
        /*14 */ { HDR_FLD,      _PRECTOT,       0       },
        /*15 */ { HDR_FLD,      _PRIORITY,      1       },
        /*16 */ { CAND_FLD,     _NAME,          NAMLEN  },
        /*17 */ { CAND_FLD,     _NESSEQ,        3       },
        /*18 */ { CAND_FLD,     _VPASEQ,        3       },
        /*19 */ { CAND_FLD,     _NESRANK,       3       },
        /*20 */ { CAND_FLD,     _VPARANK,       3       },
        /*21 */ { VOTE_FLD,     _RAWVOTE,       RAWLEN  },
        /*22 */ { VOTE_FLD,     _VPAPERC,       3       },
        /*end*/ { -99,          0,              0       }
        /* ---------------------------------------------- */ }
 

III. DBSTRUCT.H DATA STRUCTURE
==============================

        1. Purpose
        ----------
        Each race (or game for the Sports system) is represented by an
        indexed RMS record in the EDS database.  Records are indexed by
        a 7-byte key stored in bytes 0-6 of each record, and followed by
        a null byte so that "C" may treat the keys as character strings.
        All key bytes are printable ascii characters (except some county
        races, see distcvt below) laid out as follows:
                bytes   description
                0-1     State number, e.g., "33" for New York,
                2       Office, e.g., "P" for president,
                3       Party, e.g., "O" for general election,
                4-6     Congressional District, e.g., "000" for statewide races
                7       always '\000', but not used as part of RMS key.
        Note: for the Sports Database, bytes 0-1 are the Game Number "01"-"99",
        and bytes 2-6 are hard-coded as "SO000" followed by a null.

        2. Description of Fields
        ------------------------
        Note: Fields preceded by an asterisk (*) below are initialized by the
        database build program, and not altered by database update during
        election coverage.  Fields preceded by a dollar sign ($) are updated
        by database update.

        /* --- Record Layout Consists of Header + up to 15 Candidates --- */
          struct rc_entry  {
          struct racehdr rc_hdr;                /* record header */
          struct candentry rc_cand[MAXCAND];  } /* candidate data area */

        /* --- Database Record Header Layout --- */
          struct        racehdr {
        * char  nesid[8];               /* SSOPDDD (state/office/party/dist) */
        * int   nofcand;                /* number of candidates */
        * char  nesmax[3];              /* highest nes vote field needed */
        * char  vpamax[3];              /* highest vpa vote field needed */
        $ char  raworder[MAXCAND];      /* RAW candidate rank list */
        $ char  vpaorder[MAXCAND];      /* VPA candidate rank list */
        * char  rawpart[5];             /* #candidates / RAW display page */
        * char  vpapart[5];             /* #candidates / VPA display page */
          /* VPA display flags... */
        $ char  idvok;                  /* ok to display vpa */
        $ char  idwin;                  /* winner posted */
        $ char  idvpar;                 /* VPA data is non zero */
        $ unsigned char tstamp[8];      /* VMS-formatted time of last update */
        $ char  dbprec[3];              /* %precints reporting (for display) */
        $ int   precrpt;                /* #precincts reporting */
        $ int   prectot;                /* total #precincts in jurisidction */
        * char  priority;               /* priority candidate */
          unsigned char filler[16];  }  /* unused */
        
        /* --- Candidate Data Layout (one for each candidate in a race) --- */
          struct        candentry {
        * char  name[NAMLEN];           /* name (20 char max) */
        * char  nesseq[3];              /* incoming nes vote field number */
        * char  vpaseq[3];              /* incoming vpa vote field number */
        $ char  nesrank[3];             /* NES rank */
        $ char  vparank[3];             /* VPA rank */
        $ char  rawvote[RAWLEN];        /* raw vote (8 chars) */
        $ char  vpaperc[3];             /* vpa %  */
          unsigned char filler[8];  }   /* unused */
 

IV. EDS_DISTCVT PROGRAM AND DATA STRUCTURES
===========================================

        1. Purpose
        ----------
        County-level races are not directly accommodated by the 7-byte RMS
        key described above, so the following strategy has been implemented
        to handle the few county races carried by EDS.  First of all, note
        that there are two cases requiring separate algorithms:
             o  For non-district races (office not H) we know the district
                number will be "000", so we can simply replace bytes 4-6
                by the county number (but the nesid must somehow tell us that
                this replacement has occurred, or we'll mis-interpret the
                county number information as a district number).
             o  For H-races being reported by county we use byte 4 to represent
                the district and bytes 5-6 for the county as discussed below.

        2. Description of Algorithms
        ----------------------------
        Firstly, whenever a county race is stored in the database, it is
        given a unique state number to distinguish it from races where
        bytes 4-6 represent the district.  The new state number is determined
        by a lookup table.  This table is an int array called cstate[],
        and you use the "normal" state number (after calling atoi) as an index
        into cstate.  You get back the new state number to be used.  But if you
        get back -1, that means EDS is not carrying county-level races for
        the state (and you should report an error).

        After state conversion, for non-house races (office is not H) the
        county number simply is placed in the district field, bytes 4-6.
        If we are storing an H-race then the following strategy is used
        instead.  The district number (after calling atoi) is used as an
        index into the keys[] array, which returns a single character.
        That character is placed into byte 4.  Bytes 5-6 store the hex
        representation (in character form, e.g., 255 becomes "FF") of the
        county.  Thus, for example, looking at the tables below:
            81SO255 means Illinois, Senate race, county 255, and
            81HOAFF means Illinois, House race, district 10, county 255.

        3. Eds_distcvt.h Structure
        --------------------------
        Note that Illinois (State#14) is mapped into State#81, and that no
        other state has county-level races being carried by EDS.

        /* ------------------------------------------------------------------
        Conversion is indexed into the table below (-1 for illegal "C" state)
        ------------------------------------------------------------------- */
        int     cstate[] = {
        /* -----    -0-   -1-   -2-     -3-     -4-     -5-     -6-     -7- */
        /*  0- 7 */ -1,   -1,   -1,     -1,     -1,     -1,     -1,     -1,
        /*  8-15 */ -1,   -1,   -1,     -1,     -1,     -1,     81,     -1,
        /* 16-23 */ -1,   -1,   -1,     -1,     -1,     -1,     -1,     -1,
        /* 24-31 */ -1,   -1,   -1,     -1,     -1,     -1,     -1,     -1,
        /* 32-39 */ -1,   -1,   -1,     -1,     -1,     -1,     -1,     -1,
        /* 40-47 */ -1,   -1,   -1,     -1,     -1,     -1,     -1,     -1,
        /* 48-55 */ -1,   -1,   -1,     -1,     -1,     -1,     -1,     -1,
        /* 56-63 */ -1,   -1,   -1,     -1,     -1,     -1,     -1,     -1  }
        /* -----    -0-   -1-   -2-     -3-     -4-     -5-     -6-     -7- */

        /* ------------------------------------------------------------------
        Numbers 0-63 are translated to/from the following characters.
        ------------------------------------------------------------------- */
        char    keys[] =
        "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz$.";
 

V. EDS_DBUPDATE MAIN PROGRAM
============================

        1. Overall Program Structure
        ----------------------------
        Dbupdate is in the form of a shell, i.e., it was originally copied
        from [.LIB]EDS_CLISHELL.C and then adapted to support its required
        functionality.  It therefore contains a large while(TRUE) loop where
        it first waits for a mailbox message to wake it up, and then does
        a switch on the mailbox message's opcode byte.  Two cases of this
        switch, EDS$_PUTMSG and EDS$_UPDMSG, are handled identically.  They
        constitute dbupdate's specific functionality.  (The duplicate PUT
        and UPD options were historically intended to handle different
        functionality that turned out never to be needed.)

        ( During the original design of EDS, concerns over the efficiency
        of a core-based database versus the flexibility of an rms disk-based
        database needed to be answered so that the correct tradeoff
        could be selected.  These concerns were addressed by a lot of
        timing code built into dbupdate.  All such code is in dbupdate
        itself and is ignorable when reading the program; no timing code
        is in nesdeblock or nesparser. )

        2. Pseudocode for EDS$_UPDMSG Requests
        --------------------------------------
             A. Determine msgtype by checking each char in buffer until
                an (, < IRS >, or < EOM > is found.  If it's an < IRS >,
                check the next char for a ( which takes precedence.
             B. while ( nesdeblock() returns another message from the block )
                        { call nesparser() to construct an updated record
                          mail the updated record to DBServer
                        }
             C. continue while(TRUE) loop, waking up when PCP sends more mail.

        3. Programming Notes
        --------------------
             o  The msgtype, determined in step 2A above, indexes an array
                of strings, *nesdelims[], passed to nesdeblock.  Each string
                contains three characters: message prefix delim, postfix,
                and if the third char is non-null then the first message
                in a block is not preceded by a prefix.
             o  Before calling nesdeblock for the first time, dbupdate sets
                the msgseqnum of the mailbox message containing the buffer
                to zero.  This is a kludge allowing nesdeblock to keep track
                of how many chars in the buffer have been deblocked, using
                a convenient int that's coupled to the buffer in the calling
                sequence.
             o  On return from nesparser, the opcode to DBServer mailbox
                is set to EDS$_NES, _VPA, or _WIN.  This is for AutoUPdate's
                benefit, so that only menu's containing the updated "side"
                of a race need be rewritten.
 

VI. EDS_NESDEBLOCK
==================

        1. Overall Structure
        --------------------
        Nesdeblock has three principal sections of code:
            (a) it first finds the next stx (i.e., ( or # or < EOM >)
                in the buffer,
            (b) copies all chars after this till an etx
                (i.e., ) or # or < EOM >) to the output message, and
            (c) resets its pointers into the buffer and returns the message
                size.

        2. Programming Notes
        --------------------
             o  During initialization, nesdeblock concatanates a prefix to
                each county message which is stored while deblocking the
                first message.  Thus, nesparser always receives complete
                messages regardless of nes conventions.
             o  All white space chars ( < '\040' or > '\177') are stripped
                from the block.  This automatically takes care of < CR >'s,
                < LF >'s, and various other "annoyance" chars sent by nes.
             o  Again, note that msgseqnum is reset before returning, to
                keep track of where to pick up deblocking on the next call.
 

VII. EDS_NESPARSER
==================

        1. Overall Structure
        --------------------
        Nesparser contains three major sections of code:
            (a) Initialization where the report type is determined
                and variables are set up.
            (b) Perhaps 90% of the code resides in a large while loop
                that picks up info about each token from toknum[],
                and then parses it from the message.  After that, a large
                switch handles any special-purpose processing required,
                on a token-by-token basis.
            (c) After the last vote field is processed, the while loop
                completes and the various sorts and rankings are
                performed before returning.

        2. Pseudocode
        -------------
             A. Check first char of message for valid report type.
             B. while ( msgfmt[report_type].token[itoken++] > 0 )
                  { /* --- we got tokens to process until token[] = -1 --- */
                  o copy chars from message to token until delim or maxlen,
                  o if token is part of key, store it in key and continue,
             C.   switch ( type of token )
                    { /* --- precincts reporting, vote field, etc --- */
                    case READ_DATABASE_RECORD: /* got the key so get the rec */
                        o issue the rms read
                        o set up array of nes or vpa sequence numbers
                    case COPY_TOKEN_TO_DATABASE_HEADER: /* any old token */
                        o look up field number in tokfmt[]
                        o check fldfmt[] for alpha or int, convert if needed,
                        o calculate offset from beginning of database record,
                        o move the chars
                    case COPY_VOTE_FIELD_TO_CANDIDATE: /* votes are special */
                        o check ordering of vote fields vs. candidates,
                        o calculate offset into appropriate candidate area,
                        o strip leading zeroes, check numeric,
                        o move the chars
                    case PRECINCTS_REPORTING: case VPA_CALLED_WINNER: etc:
                        there are several other cases requiring some special
                        processing, but the above three are the major cases.
                    } /* end of switch */
                  } /* end of while */
             D. Recalculate horserace order.

        3. Programming Notes
        --------------------
             o  Between the while loop and the switch is another loop
                from 1 to max_repetitions.  Max_repetitions is always 1
                except for the vote tokens when it's 15.  This permits
                msgfmt[] to contain only one token index for the 15 possible
                vote fields, and also saves some initialization overhead.
             o  The string _racerptrnf suppresses record-not-found (rnf)
                error messages unless the report_type is a char in this string.



     AA    FFFFFFFFFF FFFFFFFFFF DDDDDDDD     SSSSSSS    GGGGGG   NN      NN           
    AAAA   FFFFFFFFFF FFFFFFFFFF DDDDDDDDD   SSSSSSSSS  GGGGGGGG  NNN     NN           
   AA  AA   FF     FF  FF     FF  DD     DD SS         GG         NNNN    NN           
  AA    AA  FFFFFF     FFFFFF     DD     DD  SSSSSSS   GG    GGG  NN NN   NN   .       
 AAAAAAAAA  FFFFFF     FFFFFF     DD     DD   SSSSSSS  GG   GGGGG NN   NN NN  ...      
AAAAAAAAAA  FF         FF         DD     DD         SS GG      GG NN    NNNN .....     
AA      AA FFFF       FFFF       DDDDDDDDD  SSSSSSSSS   GGGGGGGG  NN     NNN  ...      
AA      AA FFFF       FFFF       DDDDDDDD    SSSSSSS     GGGGGG   NN      NN   .       



==============================================================================

                AFFILIATE PROCESS DESIGN AND PROGRAMMING NOTES

==============================================================================


                                TABLE OF CONTENTS
                                -----------------

I.      INTRODUCTION
        1. Purpose
        2. Source code
        3. Processing Overview

II.     AFF_PROCESS_MSG MAIN AFFILIATE SWITCH
        1. Overall Structure

III.    AFF_FMTENTCTR1 FORMAT CONTROL ROUTINE
        1. Overall Structure
        2. Programming Notes

IV.     AFF_NESBUILD MESSAGE FORMATTING ROUTINE
        1. Overall Structure
        2. Programming Notes
 

I. INTRODUCTION
===============

        1. Purpose
        ----------
        The functionality supported by the affiliate process (AFF)
        is as follows:
                (a) read the show race table and build a screen table for
                    the affiliate, containing every race he is permitted
                    to view (separately determined for nes and vpa "sides"
                    of each race),
                (b) accept keyboard requests for a "full refresh",
                (c) accept autoupdate requests to refresh particular races,
                (d) in either case, read the race database and generate
                    messages in nes- or vpa-format containing the current
                    vote data,
                (e) block the messages and ship them to the affiliate.

        2. Source Code
        --------------
        The affiliate process (AFF) uses the same EDS_CRT.C main program source
        module as the regular CRT process.  It also contains three other
        affiliate-specific routines that reside in [.CRT] along with all
        the other CRT code.  These are AFF_PROCESS_MSG.C, AFF_FMTENTCTR1.C,
        and AFF_NESBUILD.C.  The short file EDS_AFFDEF.H defines
        several symbols needed by AFF.  The command file AFFA005BLD.COM links
        these source routines (and various other library modules) into the
        executable image.  AFFA005BLD is supported by the make procedure
        which uses AFFA005.DAT as input.  All these file reside in [.CRT].

        3. Processing Overview
        ----------------------
        All messages to AFF are first captured and processed by by the main
        program, eds_crt, identically to the normal CRT process.  In both
        cases these messages are ultimately passed down to the process_msg
        routine.  This entry point, process_msg(), has been totally
        rewritten for AFF, and resides in file AFF_PROCESS_MSG.C as mentioned
        above.  The calling sequence is identical for both versions of the
        routine (as it must be since both are called from the same eds_crt
        main program).  Two mailbox opcodes, EDS$_MSG and EDS$_UPDMSG,
        are recognized by AFF's version of process_msg.  EDS$_MSG indicates
        an operator request, and EDS$_UPDMSG is an autoupdate.  In addition,
        EDS$_MSG indicates affiliate initialization if the STATE_DEFAULT bit
        has been set in the modestate word addressed by the cluster pointer,
        i.e., if (cls_ptr->mstate & STATE_DEFAULT) is TRUE.

        During affiliate initialization, process_msg goes through the show
        race table and builds a screen table for the affiliate.  The screen
        table contains every race in the show, usually twice, i.e., once
        for the nes side of the race and once for the vpa side.  But before
        either side is added, the display permission field for that side of the
        race is checked for Y.  Once the screen table is built, the
        initialization code resets the mailbox opcode to force a download
        of all the races.  That is, we "refresh" all the affiliate's races
        when we first come up.

        In any case, process_msg always passes the information in its calling
        sequence down to affentctr1, which is the single control routine
        available to AFF.  (The function names here have followed CRT
        conventions for control routines, even though AFF has only one such
        routine.)  Affentctr1 then builds a list of nesid's corresponding
        to the request it has been passed.  For each nesid in the list,
        it calls nesbuild which returns a character string containing
        an nes- (or vpa-) formatted message for that race.  Affentctr1 then
        blocks these messages by report type, and finally ships the blocks
        off to TCP to be written to the affiliate's device.
 

II. AFF_PROCESS_MSG MAIN AFFILIATE SWITCH
=========================================

        1. Overall Structure
        --------------------
        Process_msg has two principal purposes: to initialize the affiliate's
        screen tables as described above, and to set up a call to affentctr1
        (which will actually format and ship off the message blocks).
        Cls_ptr->opcode discriminates among these cases, and a large switch
        on it surrounds most of process_msg's code to control the program
        flow.

        For an initialization EDS$_MSG, process_msg loops through the
        affiliate's race table, which has been set up by eds_crt when the
        process came up.  If permission to view either side of the race is
        set, then that race is added to the screen table.  After the loop
        is completed, the number of races is stored in cls_ptr->size[0].
        Thus, in effect, the screen table is set up as though the user
        had "menu burst" every displayable race in his race table onto his
        screen from control mode.  This will make autoupdate send the
        affiliate crt an update message whenever any of these races is
        updated.

        For an operator refresh request or for an autoupdate EDS$_UPDATE
        message, the opcode byte in the operatormsg structure is set
        accordingly, and affentctr1 is called to ship off messages
        for the updated races.  Upon return from affentctr1, process_msg
        simply returns to eds_crt which waits for the next mailbox
        message.
 

III. AFF_FMTENTCTR1 FORMAT CONTROL ROUTINE
==========================================

        1. Overall Structure
        --------------------
        The first task affentctr1 performs is to determine the subset of
        races in the screen table that need to be sent to the affiliate.
        A refresh request means that all races are sent.  An autoupdate
        request only sends those races in the screen table whose altered
        flag has been set to AUTOUP (presumably by autoupdate).  A sieve
        in affentctr1 checks each race in the screen table to see if it
        satisfies the corresponding condition.

        For those races that do, nesbuild is called with the nesid to
        format a message.  The returned message must now be added to
        the appropriate message block.  This block is determined by the
        report type, which is the first char of the returned message.
        This char is also used to determine whether or not the block
        contains a common prefix (for county races).  If it does, the
        prefix chars are stripped off the message returned by nesbuild
        before the race message is added to the block.

        Filled blocks are shipped off to TCP to be output to the affiliate,
        and the next race in the screen table is examined by the sieve.
        After the screen table is exhausted, any partially filled blocks
        are shipped off to the TCP, and affentctr1 returns to process_msg.

        2. Programming Notes
        --------------------
             o  Blocking behavior is determined by the three arrays
                rpt_type[8], rpt_buff[8] and prefchar[8], although prefchar
                is only relevant for county races.  Each report type
                returned by nesbuild is compared to the rpt_type array for
                a match, and the corresponding rpt_buff is the buffer number
                where the message will be blocked.  You'll note that S and D
                reports are blocked together (in buffer 2), and all other
                report types go in their own blocks.  This table-driven
                behavior is thus easily adjustable should different
                blocking be required.
             o  The prefchar array is used for county messages as follows.
                It defines the prefix by "saying" that all chars preceding
                prefchar constitute the prefix.  Thus, everything preceding
                the first / in county messages is the prefix.  This prefix
                also affects blocking since every message in the block
                must have the same prefix.  As soon as affentctr1 sees a
                county race (which it wants to put in the county block
                buffer) with a different prefix, it immediately ships off
                the current block (no matter how short it is) and starts
                a new block.
 

IV. AFF_NESBUILD MESSAGE FORMATTING ROUTINE
===========================================

        1. Overall Structure
        --------------------
        Nesbuild is essentially the reverse process performed by nesparser,
        i.e., nesparser constructs a database record from an update message
        whereas nesbuild constructs a message from a record.  The tables
        in EDS_DBS2DEF.H that are described in section II of DBU documentation
        are therefore also used by nesbuild to drive the message building
        process.

        Two major sections of code comprise nesbuild.  An initialization
        section first reads the race from the database and determines
        what report type (N,S,D,V,C) corresponds to the race.  Recall that
        the report type is the first char of the update message which will
        be constructed.  Moreover, it is the report type that's used to
        determine which structure in the msgfmt[] array of structures
        specifies the sequence of tokens comprising the message.  Until we
        know this, we can't reconstruct the message from the database record.
        After we determine the report type, a large loop reconstructs the
        message one token at a time, using msgfmt[] and tokfmt[] to select
        information from the database record.  The completed message is
        finally returned to affentctr1.

        Processing performed during nesbuild initialization is the following.
        Called with the nesid, nesbuild first reads the race record from
        the race database.  It then loops through the candidates, picking
        up their nes or vpa sequence numbers (depending on which side of
        the race has been requested) so that the vote fields can later be
        constructed in the proper order.  Next, the report type is determined
        by looking at the office byte in the nesid (e.g., office H means
        report type D), although if the state is "00" report type N (national
        president) is assumed.  The report type is now used to select the proper
        msgfmt[] structure array, used to table-drive the sequence of
        tokens that will build the message.  Finally, if the state number is
        greater than 63, we're dealing with a county race whose nesid has been
        transformed by distcvt, so distcvt is called again to retrieve the
        original information.  

        After the above initialization steps, nesbuild loops over the tokens
        specified by msgfmt[report_type].  It picks up the corresponding data
        from the race record and concatanates it into the message, one token
        at a time.  When the vote token is encountered (it's always the last
        token in the message), a special section of code loops through the
        candidates in the nes- or vpa-sequence order accumulated during
        initialization for this purpose.  This completes the message which
        is returned to affentctr1 to be blocked and mailed to the affiliate.

        2. Programming Notes
             o  A define symbol, ZERO_VPA, is set to TRUE.  With this setting,
                vpa percentages are forced to zero unless both of the
                following conditions are met: a winner must be selected and
                percents must be displayable.  That is, even if we have vpa
                data in the record, it will not be built into the message.
                (Note: I received a telephone call from WBBM regarding
                this fact because they were unaware of it, and wanted to
                know why they weren't getting vpa percentages.  I explained
                these criteria which satisfied them technically, although they
                seemed unhappy about being denied the data.  By re-defining
                ZERO_VPA as FALSE, you can make WBBM happy.)
             o  For vpa messages, the party designator for general elections
                is the number 0 rather than the nes standard letter O.
                During update, nesparser replaces the vpa zero with oh,
                and nesbuild reverses the process replacing oh with zero for
                vpa messages.  WBBM called again, being unaware of vpa's
                non-standard convention for this field.  The "case ELECTN:"
                inside the switch(algorithm) can be easily modified to
                accommodate the affiliates.
             o  Several message tokens are discarded by nesparser when a race
                is updated because EDS doesn't store the corresponding
                information, e.g., the county number for a non-county race,
                and the correction indicator for county races.  These tokens
                cannot be validly reconstructed since the original data was
                discarded.  They are either blank- or zero-filled, and
                placed in the message so that the affiliate can pasre it
                properly.


Copyright © 1986-2001, John Forkosh Associates, Inc.   All rights reserved.
email: john@forkosh.com