Wapp

Wapp - A Web-Application Framework for TCL
========================================

1.0 Introduction
----------------

Wapp is a simple and lightweight framework that strives to simplify the
construction of web application written in TCL. The same Wapp application
can be launched in multiple ways:

  *  From the command-line (ex: <tt>tclsh app.tcl</tt>).  In this mode,
     The wapp-app find an available TCL port on localhost, starts an
     in-process web server listening on that port, and then launches the 
     users default web browser directed at localhost:$port

  *  As a CGI program

  *  As an SCGI program

  *  As a stand-alone web server

All four methods of launching the application provide the same interface
to the user.

1.0 Hello World!
----------------

Wapp is designed to be easy to use.  A hello-world program is as follows:

>
    package require wapp
    proc wapp-default {} {
       wapp "<h1>Hello, World!</h1>\n"
    }
    wapp-start $::argv

The application defines one or more procedures that accept HTTP requests.
For an HTTP request where the initial portion of the URI is "abcde", the
procedure named "wapp-page-abcde" will be invoked to construct the reply.
If no such procedure exists, "wapp-default" is invoked instead.

The procedure generates a reply using one or more calls to the "wapp"
command.  Each "wapp" command appends new text to the reply.

The "wapp-start" command starts up the built-in web server.

To run this application, but the code above into a file named "main.tcl"
and then run type "<tt>tclsh main.tcl</tt>" at the command-line.  That
should cause the "Hello, World!" page to appear in your web browser.


### 1.1 A Slightly Longer Example

Information about each HTTP request is encoded in the global ::wapp()
array variable.  The following sample program shows the information
available in ::wapp().

>
    package require wapp
    proc wapp-default {} {
      global wapp
      wapp "<h1>Hello, World!</h1>\n"
      wapp-unsafe "<p>See the <a href='$wapp(BASE_URL)/env'>Wapp "
      wapp "Environment</a></p>"
    }
    proc wapp-page-env {} {
      wapp "<h1>Wapp Environment</h1>\n"
      wapp "<p><ul>\n"
      foreach var [lsort [array names ::wapp]] {
        wapp <li>
        wapp-escape-html "wapp($name) = $::wapp($name)\n"
      }
      wapp "</ul></p>"
    }
    wapp-start $::argv

In this application, the default "Hello, World!" page has been extended
with a hyperlink to the /env page.  The "wapp-unsafe" command works exactly
the same as "wapp" (it appends its argument text to the web page under
construction) except that the argument to "wapp-unsafe" is allowed to contain
TCL variable and command expansions.  The "wapp" command will generate a
warning if its argument contains TCL variable or command expansions, as a
defense against accidental XSS vulnerabilities.

The /env page is implemented by the "wapp-page-env" proc.  This proc
generates an HTML unordered list where each element of the list describes
a single value in the global ::wapp() array.  The "wapp-escape-html"
command is like "wapp" and "wapp-unsafe" except that "wapp-escape-html"
escapes HTML markup so that it displays correctly in the output.

### 1.2 The ::wapp() Global Variable

To better understand how the ::wapp() array works, try running the previous
sample program, but extend the /env URL with extra path elements and query
parameters.  For example:
[http://localhost:8080/env/longer/path?q1=5&title=hello+world%21]

Notice how the query parameters in the input URL are decoded and become
elements of the ::wapp() array.  The same thing occurs with POST parameters
and cookies - they are all converted into entries in the ::wapp() array
variable so that the parameters are easily accessible to page generation
procedures.

The ::wapp() variable contains additional information about the request,
roughly corresponding to CGI environment variables.  To prevent environment
information from overlapping and overwriting query parameters, all the
environment information uses upper-case names and all query parameters
are required to be lower case.  If an input URL contains an upper-case
query parameter (or POST parameter or cookie), that parameter is silently
omitted from the ::wapp() variable

The ::wapp() variable contains the following environment values:

  +  **HTTP_HOST**  
     The hostname (or IP address) and port that the client used to create
     the current HTTP request.  This is the first part of the request URL.

  +  **HTTP_USER_AGENT**  
     The name of the web-browser or other client program that generated
     the current HTTP request.

  +  **HTTPS**  
     If the HTTP request arrived of SSL, then this variable has the value "on".
     For an unencrypted request, the variable does not exist.

  +  **REMOTE_ADDR**  
     The IP address and port from which the HTTP request originated.

  +  **SCRIPT_NAME**  
     In CGI mode, this is the name of the CGI script in the URL.  In other
     words, it is the initial part of the URL path that identifies the
     CGI script.  For other modes, this variable is an empty string.

  +  **PATH_INFO**  
     The part of the URL path that follows the SCRIPT_NAME.  For all modes
     other than CGI, this is exactly the URL pathname.

  +  **REQUEST_URI**  
     The URL for the inbound request, without the initial "http://" or
     "https://" and without the HTTP_HOST.

  +  **REQUEST_METHOD**  
     "GET" or "HEAD" or "POST"

  +  **BASE_URL**  
     The text of the request URL through the SCRIPT_NAME.  This value can
     be prepended to hyperlinks to ensure that the correct page is reached by
     those hyperlinks.

  +  **PATH_HEAD**  
     The first element in the PATH_INFO.  The value of PATH_HEAD is used to
     select one of the "wapp-page-XXXXX" commands to run in order to generate
     the output web page.

  +  **PATH_TAIL**  
     All of PATH_INFO that follows PATH_HEAD.
 

### 1.3 Additional Wapp Commands

The following utility commands are available for use by applications built
on Wapp:

  +  **wapp-start** _ARGLIST_  
     Start up the application.  _ARGLIST_ is typically the value of $::argv,
     though it might be some subset of $::argv if the containing application
     has already processed some command-line parameters for itself.

  +  **wapp** _TEXT_  
     Add _TEXT_ to the web page output currently under construction.  _TEXT_
     must not contain any TCL variable or command substitutions.

  +  **wapp-unsafe** _TEXT_  
     Add _TEXT_ to the web page under construction even though _TEXT_ does
     contain TCL variable and command substitutions.  The application developer
     must ensure that the variable and command substitutions does not allow
     XSS attacks.

  +  **wapp-encode-html** _TEXT_  
     Add _TEXT_ to the web page under construction after first escaping any
     HTML markup contained with _TEXT_.

  +  **wapp-encode-url** _TEXT_  
     Add _TEXT_ to the web page under construction after first escaping any
     characters so that the result is safe to include as the value of a
     query parameter on a URL.

  +  **wapp-mimetype** _MIMETYPE_  
     Set the MIME-type for the generated web page.  The default is "text/html".

  +  **wapp-reply-code** _CODE_
     Set the reply-code for the HTTP request.  The default is "200 Ok".

  +  **wapp-redirect** _TARGET-URL_  
     Cause an HTTP redirect to the specified URL.

  +  **wapp-reset**  
     Reset the web page under construction back to an empty string.

  +  **wapp-set-cookie** \[-path _PATH_\] \[-expires _DAYS_\] _NAME_ _VALUE_  
     Cause the cookie _NAME_ to be set to _VALUE_.

  +  **wapp-send-hex** _HEX_  
     Cause the HTTP reply to be binary that is constructed from the
     hexadecimal text in the _HEX_ argument.  Whitespace in _HEX_ is ignored.
     This command is useful for returning small images from a pure script
     input.  The "wapp-file-to-hex" command can be used at development time
     to generate appropriate _HEX_ for an image file.

  +  **wapp-cache-control** _CONTROL_  
     The _CONTROL_ argument should be one of "no-cache", "max-age=N", or
     "private,max-age=N", where N is an integer number of seconds.

  +  **wapp-etag** _ETAG_  
     Set the expiration tag for the web page.

  +  **wapp-send-file** _FILENAME_  
     Make the content of the file _FILENAME_ be the HTTP reply.

  +  **wapp-send-query** _DB_ _SQL_  
     Run the SQLite query _SQL_ on the _DB_ database connection and make the
     HTTP reply be the value of the first column of the first row in the result.

  +  **wapp-set-csp** _POLICY_  
     Set the Content Security Policy for the application.  This command only
     works for command-line and server modes.  This command is a no-op for CGI
     and SCGI since there is no standard way of communicating the desired
     content security policy back to the server in those instances.

  +  **wapp-debug-port** _PORT_  
     For debugging use only: open a listening TCP socket on _PORT_ and run
     an interactive TCL shell on connections to that port.  This allows for
     interactive debugging of a running instance of the Wapp server.  This
     command is a no-op for short-lived CGI programs, obviously.  Also, this
     command should only be used during debugging, as otherwise it introduces
     a severe security vulnerability into the application.

  *  **wapp-safety-check**  
     Examine all TCL procedures in the application and report errors about
     unsafe usage of "wapp".

### 1.4 Design Rules

All global procs and variables used by Wapp begin with the four character
prefix "wapp".  Procs and variable intended for internal use begin with
the seven character prefix "wappInt".

# Copyright (c) 2017 D. Richard Hipp
# 
# This program is free software; you can redistribute it and/or
# modify it under the terms of the Simplified BSD License (also
# known as the "2-Clause License" or "FreeBSD License".)
#
# This program is distributed in the hope that it will be useful,
# but without any warranty; without even the implied warranty of
# merchantability or fitness for a particular purpose.
#
#---------------------------------------------------------------------------
#
# Design rules:
#
#   (1)  All identifiers in the global namespace begin with "wapp"
#
#   (2)  Indentifiers intended for internal use only begin with "wappInt"
#
#   (2)  Assume single-threaded operation
#
#   (3)  Designed for maintainability
#
proc wapp {txt} {
  global wapp
  append wapp(.reply) $txt
}
proc wapp-unsafe {txt} {
  global wapp
  append wapp(.reply) $txt
}
proc wapp-escape-html {txt} {
  global wapp
  append wapp(.reply) [string map {& &amp; < &lt; > &gt;} $txt]
}
proc wapp-reset {} {
  global wapp
  set wapp(.reply) {}
}
proc wapp-mimetype {x} {
  global wapp
  set wapp(.mimetype) $x
}
proc wapp-reply-code {x} {
  global wapp
  set wapp(.reply-code) $x
}

# This is a safety-check that is run prior to startup
#
# Examine the bodys of all procedures in this program looking for
# unsafe calls to "wapp".  Issue w
proc wapp-safety-check {} {
  foreach p [info procs] {
    set ln 0
    foreach x [split [info body $p] \n] {
      incr ln
      if {[regexp {[;\n] *wapp +\[} $x] ||
          [regexp {[;\n] *wapp +"[^\n]*[[$]} $x]} {
        puts "$p:$ln: unsafe \"wapp\" call: \"[string trim $x]\"\n"
      }
    }
  }
}

proc wapp-start {args} {
  set mode auto
  set port 0
  set n [llength $args]
  for {set i 0} {$i<$n} {incr i} {
    switch -- [lindex $args $i] {
      -port {incr i; set port [lindex $args $i]}
      -mode {incr i; set mode [lindex $args $i]}
      default {error "unknown option: [lindex $args 1]"}
    }
  }
  if {$mode=="auto" && [info exists env(GATEWAY_INTERFACE)]
        && $env(GATEWAY_INTERFACE)=="CGI/1.0"} {
     wappInt-cgi-request
  }
  if {$mode=="server"} {
    wappInt-start-listener $port 0 0
  } else {
    wappInt-start-listener $port 1 1
  }
}

# Start up a listening socket.  Arrange to invoke wappInt-new-connection
# for each inbound HTTP connection.
#
#    localonly   -   1 to listen on 127.0.0.1 only
#
#    browser     -   1 to launch a web browser pointing to the new server
#
proc wappInt-start-listener {port localonly browser} {
  if {$localonly} {
    set x [socket -server wappInt-new-connection -myaddr 127.0.0.1 $port]
  } else {
    set x [socket -server wappInt-new-connection $port]
  }
  if {$browser} {
    set port [chan configure $x -sockname]
    set url http://[lindex $port 1]:[lindex $port 2]/
    puts "exec firefox $url"
  }
}

# Accept a new inbound HTTP request
#
proc wappInt-new-connection {chan ip port} {
  global wappInt
  set wappInt($chan,REMOTE_HOST) $ip:$port
  set wappInt($chan,header) {}
  fconfigure $chan -blocking 0 -translation binary
  fileevent $chan readable "wappInt-readable $chan"
}

# Close an input channel
#
proc wappInt-close-channel {chan} {
  global wappInt
  foreach i [array names wappInt $chan,*] {unset wappInt($i)}
  close $chan
}

# Process new text received on an inbound HTTP request
#
proc wappInt-readable {chan} {
  if {[catch "$wappInt-readable-unsafe $chan"]} {
    wappInt-close-channel $chan
  }
}
proc wappInt-readable-unsafe {chan} {
  global wappInt
  set n [gets $chan line]
  if {$n>0} {
    if {[regexp {^\s+} $line]} {
      append wappInt($chan,header) $line
    } else {
      append wappInt($chan,header) \n$line
    }
    if {[string length $wappInt($chan,header)]>100000} {
      error "HTTP request header too big - possible DOS attack"
    }
  } elseif {$n==0} {
    wappInt-parse-header $chan
    if {$wappInt($chan,REQUEST_METHOD)=="POST"
           && [info exists wappInt($chan,CONTENT_LENGTH)]
           && [string is integer -strict $wappInt($chan,CONTENT_LENGTH)]} {
      set wappInt($chan,toread) $wappInt($chan,CONTENT_LENGTH)
      fileevent $chan readable [list wappInt-read-post-data $chan]
    } else {
      wappInt-handle-request $chan
    }
  }
}

# Read in as much of the POST data as we can
#
proc wappInt-read-post-data {chan} {
  global wappInt
  
}