5.2 Wish List

This page contains the wish list that was included in the original 1992 FunnelWeb Hacker's Manual. As hardly any changes have been made since then, I thought I'd just include it here verbatim.

Documentation

An official example: A official example program written using FunnelWeb should be constructed and made available.

Index program: In order to typeset the documentation, a portable index sorting program is required. One should be written and added to the distribution kit. Perhaps this could be the official example!

Command Line Interface

Buffer length: Currently the FunnelWeb shell uses the COMLINE_MAX constant to size its internal command line buffers. This is untenable. The maximum length of a shell command line should not be machine dependent.

Antiquated Features: As the FunnelWeb language develops, it is likely that some changes will have to be made that will render one or more language constructs obsolete. When this happens, it may be necessary to add a command line option that has the power to turn on and off warnings or errors flagging antiquated features.

Shell Interpreter

SETALL command: When writing FunnelWeb scripts, it is sometimes desirable to set all  of FunnelWeb's options to some particular value so that the script is not vulnerable to changes in FunnelWeb's default values which might occur from time to time. To this end, it may be worth creating a "SETALL" command that is identical to the "SET" command except that it will generate an error if the value of an option is not specified explicitly.

Recursion test: A test should be added to test for recursive invocation in a shellscript.

Diagnostic counting: The code for counting diagnostics in the script interpreter is rather messy and perhaps even buggy. It should be cleaned up and commented.

Argument counting: In the command.c module, there is a variable that counts the arguments to a command. Currently it takes the value of the number of parameters including the command verb. This has turned out to make the code less readable. It should be changed to be the number of arguments to the command verb.

Make facility: It may be worth building some sort of make facility into the script language so as to support machines such as the Macintosh that do not already have this facility.

Signature file: One problem with using FunnelWeb in conjunction with an external Make facility is that a user might change a FunnelWeb source file without making changes that will affect the files that it generates. If FunnelWeb is then run and the "+D" option is on, then the output files will be deleted (to avoid further Make propagations). If Make then has a production linking the .fw file to its output files, then it may conclude that the output files are still out of date. To solve the problem, FunnelWeb could be changed to write a .sig file whenever it processes a .fw file. The Make production could then be wired up to link the .fw file to the .sig file instead of to the output files.

Language Design

Some proposed changes to FunnelWeb do not correspond to any particular component of FunnelWeb and are really to do with the design of the input language.

Output or file?: The "@O" special sequence for defining an output file is somewhat non-mnemonic and can be confused with zero ("0"). Perhaps it should be replaced by the "@F" sequence.

Syntax of section names: Currently section names use the same syntax as macro names. For example "@". It can be argued that angle brackets should be reserved only for macro names and that some other syntax should be found for delimiting section names. This is not a clear issue. It could also be argued that they are both names, and that because sections can inherit their names from the macros they contain, that the names are of the same "type".

One macro per section: One particular style of using FunnelWeb is to have at most one macro definition per section. It may be worth adding a pragma that instructs FunnelWeb to enforce this.

Should @\{ suppress EOL?: When defining a macro in FunnelWeb, it seems to be rule rather than the exception that the "@\{" be followed by "@-". This suppresses the EOL on the definition line, allowing the first line of the macro to be placed immediately above and in line with the other lines without introducing an EOL at the start of the macro text. One option is to introduce a pragma to determine whether to suppress EOLs following "@\{".

Pragma syntax: It is not clear how "loose" the syntax of pragmas should be. Perhaps they should be case insensitive.

Conditionals: Depending on demand, it may be worth reintroducing some sort of conditional feature into FunnelWeb. However, it would have to be very simple to compete with the various ways in which conditionals can already be fudged within FunnelWeb as it stands.

File markers: It might be worth modifying the language so that a special syntactical marker is required at the start and end of files. This will assist in detecting truncations and other corruptions.

Formal parameter lists: It might be worth changing over to a syntax for formal parameter lists that does not require the @( and @). However, they could be retained as optional for backward compatibility.

Scanner/Mapper

All non-contiguous mappings: Currently FunnelWeb requires that all input files be mapped into a contiguous lump of memory. This caused problems for two reasons. First, to do this, one has to allocate the memory first, and to do that, you have to know how long the file is, and it turns out that finding out the length of a file in a portable manner is very inefficient. Second, although IBM PC compatibles may have megabytes of memory, it is segmented into blocks of at most 64K. This means that FunnelWeb currently cannot read a file longer than 64K on a PC. These problems could be avoided if the mapper and scanner were reorganized to allow input files to be read in and stored as a linked list of chunks of text rather than a contiguous block.

EOL is unspecifiable: FunnelWeb uses ASCII character decimal ten (10) internally to represent logical end-of-line and is currently organized so that if one of these is inserted into the text by the user using a "@^D(10)", it will be written out as a logical end of line, rather than as a single ASCII character 10. This should be fixed.

Allow mnemonics for unprintables: FunnelWeb allows users to insert unprintable characters into the output using the "@^D(ddd)" special sequence. Perhaps it would be changed so that it understands ASCII standard mnemonics such as "LF" as well as ASCII numbers.

Version pragma: A "version" pragma should be added that allows the user to specify in the input file the version of FunnelWeb that was around when the input file was created. At a later date, such a construct would be very useful for determining how an input file should be updated if the FunnelWeb language has changed between versions.

Parser

There are no proposals to change the parser except as a consequence of other proposals.

Analyser

Recursion detection: Currently the FunnelWeb analyser flags, with an error, all macros with an infinite expansion. This would be best changed to flagging all macros that are directly or indirectly recursive. To do this, Tarjan's algorithm[Tarjan72] for the detection of strong components should be installed.

Once only macros: By default FunnelWeb prevents a macro from being called more than once unless it has a "@M" associated with it. However, FunnelWeb does allow a macro that calls such a macro to be called more than once. Perhaps this "loophole" should be plugged somehow.

Tangle

The Tangler is one of the cleanest components of FunnelWeb, as basically all it has to do is expand some very well-defined macros.

Text indentation: Currently FunnelWeb supports no indentation  and blank indentation . A third form could be added if it was considered necessary. Text indentation  is the same as blank indentation  except that instead of indenting with blanks, FunnelWeb would indent with the text to the left of the called macro. This facility could be useful for commenting out large bodies of text in languages that do not have multi-line comments (e.g. Ada). A discussion of the pros and cons of this form of indentation appears earlier.

Weave

Perhaps FunnelWeb's weakest aspect is its typesetting facility.

Align table of contents: When FunnelWeb generates a table of contents, the section numbers are not quite aligned with the start of the controlling heading above them.

Typesetting strength: It should be possible to specify the level of typesetting strength for headings so that short documents do not look overdone when typeset. A new pragma would be good for this.

Typeset a portion: Sometimes it is desirable to typeset just a portion of a program. A command line option could be added to do this. The option could accept as its argument, a string containing a list of section numbers or heading names.

Generic typesetter option: In addition to building in a number of different versions of Weave, one for each popular typesetter, it would be possible to add a special generic format where the typeset output is expressed in terms of FunnelWeb macros . The user could then specify macro definitions for a non-supported typesetter and run the output through FunnelWeb Tangle to get a typeset file in a format suitable for the unsupported typesetter.

Suppression of include files: It should be possible to specify in the input file that particular included files not appear in the typeset output. Currently, the fact that an inclusion has occurred is not even represented in the typeset output. Suppression of inclusions is particularly necessary where a library of macros has been included at the top of each of a group of source files.

Cross reference of identifiers: WEB provides a list of identifiers and a list of all the definitions in which they are used. A similar feature could be added to FunnelWeb.

Support for non-printables: Currently FunnelWeb does not provide support for typesetting the special "@\circumflex(num)" sequences. This should be added.

Support for @+ sequences: Currently Weave does not see "@+" sequences as such. Instead it perceives them as ordinary EOLs. However, there are arguments for typesetting them specially.

Typeset text in macro bodies: One of the much-loved features of WEB is the way that it allows the user to switch recursively between document and program formats. FunnelWeb does not allow this, but should. In FunnelWeb, the delimiters "@{" and "@}" are already used consistently to delimit macro text. The "@[" and "@]" sequences have been reserved for the delimitation of documentation text.

Non-tt typesetting: The current version of FunnelWeb sets all its macro text in tt font. This is both a blessing and a curse. It is a blessing because it connects the reader directly to the code, with no complicated intermediary. It is a curse because it looks ugly compared to the beautifully typeset programs produced by other literate programming tools.

The difficulty with adding such beautiful typesetting is that it is necessarily language-specific. Keywords and syntax differ from language to language and it would not be easy to come up with a set of language independent rules.

One approach is to write a set of Weave back-ends, one for each language. Another approach is to generate  back ends. This is the approach taken in the Spider  system[Ramsey89]. In the Spider  system, the programmer writes production rules for converting lexical components of the program text into typesetter instructions. The Spider  program reads these rules and generates a new version of WEB suited for the target language.

For FunnelWeb a slightly different system is proposed in which Spider-like rules appear in the input file and are used directly by Weave to perform the typesetting. An intermediate abstract typesetting language could be used so that the productions can be made language specific, but not typesetter specific.

Lister

Glue factor: A glue factor could be added that determines how many lines can be in between two diagnostics in the listing before the two groups of lines are joined together in the listing with no intervening ellipsis.

Diagnostics

Advisory information: Some of FunnelWeb's diagnostics provide a detailed explanatory paragraph. While this information might be useful the first time, it has the capacity to clog up a listing file if the user has made the same error many times. To solve this problem, FunnelWeb could be modified so that such explanations are only displayed the first time the error occurs.

Abort after n errors: A facility could be added to prevent FunnelWeb's scanning, parsing, and analysing phases from continuing if a certain number of errors have already been issued.

Speed

Measurement of speed: Although FunnelWeb can generate a breakdown of where it is spending its time, it does not give a final rating in lines per minute. This should be added.

Find the hot spots: Although FunnelWeb has been designed to allow high speed, not much effort has so far been made to make it fast. This should be done.

Change some declarations: FunnelWeb is full of variable declarations where the variables are wider than they need be. Replacing these might speed it up.

Correctness

Bounds analysis: Not much effort has gone into the design of FunnelWeb's input boundaries. An analysis should be made of FunnelWeb's behaviour when the following quantities are stretched:

• Input line length.
• Input file size.
• Number of macros.
• Length of macro.

In particular, FunnelWeb's behaviour at 32K and 64K boundaries should be observed.

Stack detection: Macintosh THINK-C provides just 6K of memory for the stack. It might be worth adding checks to make sure that the stack is not being blown.

Test Suite

The following tests should be added to the test suite:

Lister
------
   LR01: Test with a full listing with no diagnostics.
   LR02: Test with a full listing with diagnostics.
   LR03: Test abbreviated listing with no diagnostics.
   LR04: Test abbreviated listing with diagnostics.
   LR05: Test error context with nearby diagnostics.

Boundary Cases
--------------
Static analysis might preclude the need for
most of these tests.
   BC01: Test what happens when memory runs out.
   BC02: File with a single line of a megabyte.
   BC03: File of a megabyte of EOLs.
   BC04: Output file with an extremely long line.
   BC05: Output file with one million lines.
   BC06: Test on a file with very many macros.

General
-------
   GN01: A large legal input file exercising
         as many features as possible.
         1. Test listing file.
         2. Test output files.
         3. Test typeset file.
   GN... A selection of ten real-life FunnelWeb files.

Platform-Specific Changes

Icon for the Macintosh: Currently no icon is supplied for the Macintosh version of FunnelWeb. An icon depicting a spider or a funnelled web of some kind would seem appropriate.