Milestones in the Nonpareil Project =================================== Phase 1 - Planning (1996 - 2002) Phase 2 - Compiler front end (January-August 2003) Phase 3 - Compiler (November 2004 - July 2005) Phase 4 - Compiler continued (November 2006 - June 2007) Phase 5 - KMT (June 2007 - September 2007) Phase 6 - KMT continued (February 2008 - June 2008) This document records when certain stages were completed, but this should not be taken to mean that they were worked on only since the previous deadline, or that they were never revised afterwards. Jeffrey H. Kingston Phase 1 - Planning (1996 - 2002) ================================ November 1996. I completed my paper, "The future of document formatting". I also began thinking seriously around this time about the design of the system now called Nonpareil. February - November 2002. My student Bradley Baetz carried out the first implementation of Nonpareil, for his Honours thesis: "Nonpareil - a strongly typed object oriented functional programming language". A fairly detailed but unwritten language design had evolved by the time he started, with syntax diagrams. 27 August 2002. On this day I left my job to devote myself full time to the Nonpareil project, although it then took about six weeks of finishing odd jobs and setting up an office and computer at home before I was able to actually start work. 24 December 2002. My paper, "Prospectus for Nonpareil", was completed and placed on my web site, and comments were solicited from members of the Lout mailing list. This paper presented a precise syntax, attempted a type system, and discussed some document formatting issues, including the format of literal documents, requirements for the user interface including automatic construction using introspection, and locating mouse clicks. These ideas had been worked out prior to 2002 but not written down. January 2003. On holidays, during which time I had a good idea for modelling selections and locating mouse clicks, which involves inserting references in the display object to the originating content. As the location search goes down the display tree, it accumulates the references it passes through into a list, and this list becomes the selection. Although not fully automatic as originally intended, it's simple, viable, and comprehensible by the programmer. Phase 2 - Compiler front end (January-August 2003) ================================================== 24 January 2003. Returned from holidays and commenced coding the Nonpareil compiler. 24 February 2003. Completed coding of the lexer, parser, system, module, and renaming parts of the Nonpareil compiler, and released the first version of my paper "Nonpareil Language Specification", which laid out the rules of these parts of the language. 23 March 2003. Completed coding of the manifesting part of the compiler, which propagates inherited features into child classes and identifies what each name refers to. This included introducing the NAMED type, a union of everything that can be named (except modules which have their own name space), plus tightening up a lot of the previous code, including revising renaming to ensure that each module has at most one view of each class. Also sorted out many details concerning class extensions, builtin classes, and norename classes, and defined and handled the builtin classes object, int, real, list, tuple2 .. tuple12. 3 April 2003. Completed revised specification of functions, variables, named parameters, default parameters, and currying, influenced by careful reading of Cardelli and Wegner for typing, and the OCaml reference manual for named and default parameters. Worked out how to implement these features efficiently, including passing default markers for default value parameters, the funn, funn_rep, and funn_m classes for currying, and int_ref etc. with coercion functions for automatic boxing/unboxing (read Breazu-Tannen et al). Revised the language specification document to include all this. 19 April 2003. Completed a revision of view handling, which unifies views and operator tables into contexts and ensures that each class and feature knows its own name in each module, and that each class knows its own feature view in each module. Also modified name handling so that when names are printed in error messages a full trace of their renaming history is given. 28 June 2003. After two months' work from the preceding milestone (!), produced a type checker which successfully checked all the classes I have so far, including the bodies of the map and fold functions from the nlist class. Defined the set of feature signatures of a meet type formally. Local type inference of actual generic parameters is done by introducing unification variables constrained to be above and below given types, as in Pierce and Turner (1999) but differing in some details. 7 August 2003. Over a month from the preceding milestone (!), completed a rewrite of the type system which treats user-defined coercion functions as introducing elements into the subtype relation, not just introducing features into classes. The main problem was that this allowed cycles of equivalent classes in the subtype relation, which required a large rethink since I was using the ancestor set as the type representation, but with cycles there is no longer a unique map back from ancestor sets to types. Have also refined the local type inference algorithm further, introducing `range types', to overcome the problem of how unification variables and overloading interact (one needs to be able to undo the effects of a type check if it is just one hypothesis from a set of overloaded functions, but unification variables don't naturally lend themselves to undo). Wrote an Appendix to the User's Guide entitled `The Nonpareil Type System' explaining all this in detail, and posted the Version 1.1 Language Specification document including this appendix on the Nonpareil home page. =========================================================================== August 2003 - October 2004. During this period I worked on my other research project, timetable construction, producing a paper for submission to PATAT04, and subsequently the KTS web site (see http://www.it.usyd.edu.au/~jeff). =========================================================================== Phase 3 - Compiler (November 2004 - July 2005) ============================================== 2 November 2004. For the last week or so I have been working on the design of NPD, the Nonpareil format for literal documents, as now described in the relevant appendix of the Nonpareil Language Specification. Today I start coding again with the following short-term goals: * Conversion of lexer output from 32-bit Unicode scalar values to 16-bit Unicode characters, basically because I've realized the cost of finding properties of 32-bit quantities. * An ADT for reading Unicode property files, converting to two-level optimized property tables, and reading, writing, and querying those tables. * Lexing and parsing of NPD files. 13 November 2004. Well, today I really started coding again. I've gone back to 32-bit Unicode scalar values, basically because they are going to happen eventually (ISO/EIC 10646 is a 31-bit code), accessed via a four-level table, and done a lot of work sorting out how the various Unicode properties and algorithms work. Also lost some time to dentistry in this period. 17 November 2004. Am now able to write out a binary character properties table and associated string pool. The two binary (.data) files are loadable in virtually no time, vs. 2 seconds to load and convert the text files. Here are the file sizes: 60332 Nov 17 15:10 char_pool.data 355280 Nov 17 15:10 char_trie.data 15386 Nov 13 16:25 SpecialCasing.txt 897402 Nov 13 16:25 UnicodeData.txt The UChar type, after this initialization, can return the Unicode General Category of a character, its combining class, etc. - all the properties in UnicodeData.txt and SpecialCasing.txt except character name and a few conditional casings that this code doesn't handle; plus a Nonpareil lexical class. 19 November 2004. Lexical analyser now fully Unicode compliant, with Unicode definitions of identifiers, digits, and punctuation. Added infixr alongside infix - implementation was wonderfully easy, just add 1 to infixr precedences before scanning the right shell. The interpreter seems to be working as well as before. Next is code generation. 22 November 2004. Am now generating typedefs (including special ones for builtin types) and structs (including anonymous bit fields to span gaps caused by multiple inheritance). 26 November 2004. Got to generating function headers but realized that it was getting horrible. Have realized that the "system and view" model of the documentation is the right one but is not implemented by the current compiler, which has problems around MFEATURE and MFEATSIG, these being basically the same thing. The problem arose because I was feeling my way to the implementation before. So I am embarking on a major rewrite. I may be gone some time. 10 December 2004. Rewrite appears to be working today. It took two weeks, and not everything I thought could be simplified could be, but everything is a lot clearer now and it was worth it. Now have to get back to code generation. 19 December 2004. Have code generation framework going well, the main omissions at the moment are call expressions, which have to link to the correct parameters etc. Found a clever method of introducing hidden parameters into let and anonymous functions, and spent what feels like it was about a week (!) fiddling around with case expressions. Now need to do call expressions and then start auditing just what else is missing. Clone is missing and needs a special implementation invoking memcmp, since although it is a normal feature, to implement it that way would produce a zillion parameters. Caching is also missing. Default values for parameters are partly done. 22 December 2004. A small rewrite removed the FUNCTION type, which on reflection was an abstract supertype and not a real type, separated PARAMETER into PARAMETER_VIEW and PARAMETER, and merged EXPR_FUN into EXPR_LET (since anonymous functions are simple cases of let expressions). 24 December 2004. Last day of work before one month's summer holiday. Have introduced CFUNCT and CPARAM types to represent C functions and their parameters, and successfully ported a lot of code over to this design. Currently have just implemented CFunctCallCodeGen which is supposed to generate one call to a C function. Have linked it in to expr_call.c - to the targeted call case only so far, but it should handle them all. The code is compiling but I ran out of time to test and it is failing an assertion. 29 January 2005. Started work again today after the summer holiday. 20 February 2005. Since starting again I did some more code generation, then things got messy so I did a major rewrite, introducing a FUNCTION class from which everything that's a function inherits. This rewrite took two weeks up to today; I'm now back working on code generation, basically where I left off before. 23 March 2005. Still working on code generation. Have completed all the basic stuff: creation functions, dynamically dispatched features, etc. Am now working on the refinements (arrays, predefined objects, that sort of thing). Had a major setback three days ago, when I realized that I could not easily determine which object fields were pointers and which were immediate values, in those cases where the field type is a type variable. In particular, when an object creation occurs within a generic class, no static scheme will work. This information is needed when swizzling heap data so that it can be stored in a file and read in for a fast start-up. Am still investigating this issue but the options at present seem to be: * Use mmap(), which requires no swizzling and would be very fast. However, as far as I know it only exists on Unix, and the manual entry does not guarantee that it will reload into the same virtual address that it stored from, although you can ask for it and there seems to be no reason why it wouldn't. [Tim Cooper is using equivalent functions in Windows, but he is not attempting to reload into the same virtual address.] * Run the initialization code twice; those fields that differ in the two runs must be pointers to objects. Put swizzle bits into objects where there is doubt, to record the results of this, and consult those bits when unswizzling but not otherwise. If floating point operations are non-deterministic (which I seem to remember that they can be, although I can't find any info about it), the two runs may diverge, causing this method to fail. Speed is not such a big issue because it's all in an off-line run. Anyway even the mmap() solution might require two runs, the first to find out how much memory is going to be needed, the second to use that memory. * Include full type information in the type tag of every object, rather than just the class as at present. Essentially this is full-blown introspection, which I am not ready to do yet, and anyway I don't really know how to do it in the case of object creations lying within generic classes and functions, and so I am hoping that this "full introspection" will never be needed. Also I am not sure how the sequencing would go: what about the introspection objects themselves? Perhaps they are not generic. 24 March 2005. Last day before taking Easter break. Have begun working on swizzling. The plan is to run initialization twice, compare object-by-object, set swizzle bits, and write the file. Then read back and unswizzle. It will be an assert error if the comparison shows that the two initializations took different paths. 31 March 2005. Swizzling code done, but initialization altogether has quite a lot of other stuff, still to do. Got sick of wondering where everything is kept and how it fits together, so have assembled everything into a single directory, the compiler in one subdirectory and the system in another. A single "make" will now build and install a working Nonpareil system from scratch (or it will, when the compiler is finished), with everything in the right place, and everybody knowing what those places are. 6 April 2005. Have "np -init" working today - writing out a file of swizzled predefined objects. But all_predefined arrays not done yet. 7 April 2005. Added all_predefined arrays. Not bad for one day's work! However the current design of all_predefined requires initialization to do a prerequisite search through the whole system, not module by module as the current implementation does, because an all_predefined array of a class in one module can receive contributions from class extensions in later modules. So I need to redo the initialization code. 20 April 2005. Got side-tracked into making an abstract backend, which turned out to be a two-week job! It's finished now: the compiler is compiling, and C code it generates is compiling cleanly, but I have not yet attempted to run that C code. 25 April 2005. Sorted out a quite long list of smallish issues, then struck clone, which turned out to be another rock in the path. It needs to be partitioned, but the partitioning depends on having already validated every type in the entire system, since classes connected by meet types or inheritance must lie in the same partition. 28 April 2005. Have sorted out how to do clone, but during the implementation I struck problems with incorrect types, and then when I tried to debug the problem I got into a terrible mess between ordinary parameters and forwarded parameters. This has prompted yet another refactoring, which I began today, in which there are separate Nonpareil functions with Nonpareil parameters, and C functions with C parameters. The Nonpareil functions refer to the corresponding C functions, and the Nonpareil parameters refer to the corresponding C parameters. This gives a concrete model of the forwarded parameters: they are exactly the parameters that will appear in the C function. This was always the case, but it will be much clearer now. 1 May 2005. Have more or less finished the latest refactoring, and it is indeed much clearer. Have introduced new CREATION_FN_VIEW, CREATION_FN, INVARIANT_VIEW, and INVARIANT classes, and certainly cleared up a lot of things that were confused before (e.g. inner letdefs, and invariants), so the refactor has been a big success and will probably stick - at last. 8 May 2005. Heaven knows where the last week went, but I am still finishing off the same refactoring, I've been debugging it today, and now have a complete compile to C again, although the generated C code is giving a few error messages, not many, when compiled. I did lose a day or so to feeling off colour, and there has been some detailed work in strengthening the C_FUNCT abstraction, which has turned out well. 14 May 2005. C compile now working properly. Spent some time sorting out old problems with let definitions. These are now working, although at the cost of refusing to compile certain complex kinds. Also sorted out initialization of string literals, there was a bug there. Above all, clone is now working. On my list of things to do begun on 31 March there are 92 items, with 23 remaining to be done. The principal tasks remaining in this pure compiler phase of the project are function inlining, fast reload, enumerated types, and a problem that dawned on me the other day: I am inserting coercions before type range variable unification is complete, so they can't always be the right coercions. Have reorganized the "Implementation Notes" report into a structure that seems likely to stick. 17 May 2005. Successfully generating the "char" class today, and reading it. Now have to implement enumerated types. 22 May 2005. Have carried out quite a lot of the implementation of enumerated types, and also completed an excursion into integral types, setting up byte, short, int, ubyte, ushort, and uint, with coercions and other conversions between them. But now have realized that my approach to enumerated types, involving two classes for each enumerated type, was wrong. I should rather keep the implementation much closer to the back end, leaving the front end untouched except that enumerated classes inherit ultimately from enum_object rather than object. 9 June 2005. Have just returned from a week's bushwalking. Now well into the revised implementation of enumerated classes, currently implementing legal_code and with_code. After that I will need to write the code to build the trie, and enumerated classes should be done. Also added mathematical functions to the real class today. 12 June 2005. Have completed code generation of all_predefined and all_enumerated arrays, including inheritance. Have changed to an initialization order more compatible with building tries. Still have to check this order, and still have to initialize tries. 23 June 2005. Have just returned from a week's holiday. I completed trie building before I left (a very tricky algorithm indeed), and today I have enhanced the back end module so that it checks whether C files have not changed since the previous compile, and skips the compile in that case. This involves saving backup copies of all generated files, and comparing them character by character with the current versions. A C file's generated header files must also compare equal, and there must be an object file, if skipping is to be done. The next step is to sort out the ordering of class-level predefined object initializations. 25 June 2005. Have finally completed a definitive description of predefined object initialization, and designed an algorithm to do it. Next step: implementation. 1 July 2005. Have completed the implementation of predefined object initialization. Along the way I realized that the initialization algorithm basically traverses back-end functions, to see what they call. This inspired me to make a major renaming to identify all the back-end functions clearly. I then carried out an audit to match up code generation and predefined object initialization, so initialization checks what code generation generates. Very clean. 2 July 2005. Finished predefined object initialization and set to work on generating two binaries: "np_init", which initializes predefined objects from scratch and saves them in a checkpoint file; and "np", which loads predefined objects quickly from the checkpoint file. Compiler is running right through without crashing, but still have to audit the generated C files. There are #include files to add, plus must read through it all carefully. 3 July 2005. Now have a clean compile of the generated C code, except for a couple of small, older problems. Have to read through the code carefully (but I bet it's right), then go on and work out system_load(). 7 July 2005. Now generating system_load() and compiling it cleanly. That is, I now have complete working "np_init" and "np" binaries. My list of things to do currently has 17 items, quite substantial ones but all either reorganizations, bug fixes, or optimizations, nothing basically new to add now. 13 July 2005. Working through the list of things remaining to do, currently 12 of them left. Implemented function inlining today (not in general, just two common special cases). The most substantial remaining problem is that of coercions being inserted before the types they coerce to have been finalized. Otherwise it's routine stuff from here on. ===================================================================== Worked on my other project, timetable construction, for most of August 2005 to October 2006. I did do some planning, however. I discovered OpenGL during this period, and realized what a great boon it will be for Nonpareil, both in running time and program development time. No need to fill my own polygons! I also pondered the deficiencies of the current state and worked out how to improve the NPD format and how to do introspection properly. 1 February 2006. Began work on the Nonpareil model of documents, displays, and selections. Commenced work on "The Nonpareil Document and Display Models". Only spending about 50% of my time on this, the rest is going to timetabling. ===================================================================== Phase 4 - Compiler ctd. (November 2006 - September 2007) ======================================================== November 2006: Redesigned NPD, first reflection design December 2006: Syntax revisions, second reflection design January 2007: On holiday February 2007: Odds and ends, largely a waste of time March 2007: Lout; rewrote type system incl. subtyping and coercion April 2007: Reflection; wrote cgen submodule; converted to wchar_t May 2007: Reorganized clients of cgen; reflection code generation June 2007: Completed reflection; started work on KMT 1 November 2006. Since 25 October I have been reviewing Linux documentation on character sets and internationalization, reviewing the Unicode web site, etc. Today I exhumed a design for documents in Nonpareil, written last February, and expanded it into the beginnings of a second major document (the language spec being the first) describing Nonpareil's use as a document formatting language. 10 November 2006. Placed a document called "Nonpareil Document Formatting" on the web today. It includes a well-developed model of documents and scenes, including a workable plan for selection, plus a revised and much simpler specification of the NPD format. I have to get another Lout release out now, but after that I intend to set to work to add reflection to the Nonpareil compiler. 26 November 2006. Completed a design for reflection and posted on the web a revised language specification containing it. The new spec is a few pages smaller than the old, thanks to the simplification that comes with reflection. Also made quite a few other small changes. None of these new things have been implemented yet; have to do that now. 30 November 2006. Started coding a few days ago, found several disturbing bugs in the current implementation, and also that the makefile documentation did not conform with reality, and was not particularly well designed even so. So I have been tidying that up. Did I stop work in the middle of something? Strange. Anyway it's all working today, so now I start on the changes for Version 1.04, the revised language specification published on 26 November (under a wrong version number). 1 December 2006. Made various minor syntactic changes today, principally removing infixr and adding prefun. 4 December 2006. Had a couple of days away. Today I implemented the revised syntax of anonymous functions, plus a couple of other very small changes. 8 December 2006. Have new parser and new error handling approach, both currently untested. The new error handler will keep going, and I'm currently checking the consequences of that for each kind of error. 12 December 2006. Done all checking, and have revised manifesting including a substantial rewrite of manifesting call expressions, now complete although not yet tested. Have to add PREFUN too. 13 December 2006. Added prefun calls. Sorted out a problem with the module name table (formerly operator table - that was the problem). It now eventually accumulates every name in its module, and checks the module-wide name consistency rule properly. Previously it was correctly allowing prefix and infix names to clash, but incorrectly allowing operator names and ordinary names not to clash. 14 December 2006. Did some testing and everything is now working up to the first invocation of a class name, which obviously needs reinterpreting as a type object. So am beginning the coding of reflection. I may be gone some time. 17 December 2006. Am now creating type classes. Moved all_predefined, all_enumerated, legal_code, and with_code to type class. 22 December 2006. Abandoned type classes - having an extra class for every real class is too messy to implement. Have found a simpler approach that gives everything I need, documented it, and am now working on the implementation. The front end seems to be all done today; I'm generating code again, but with both the old type tag and the new type field at the moment. 24 December 2006 -- 27 January 2007. On holidays, did nothing. 29 January 2007. Sorting out manifesting and code generation of default values of creation features, which were not done properly before. The front end is now accepting the right language, but the code generator still has to be fixed. 31 January 2007. Now handling default values of creation features correctly; the details are in the implementation document. Have begun the back end handling of type objects by generating one function for each class which creates a type object of that class. Next step is to add type parameters to front-end functions as first-class parameters; the corresponding back-end parameters will have type np_type. 1 February 2007. Have realized that reflection leads naturally to unifying TYPE_VAR with FEFN_PARAM: struct type_var_rec { KIND_TAG kind_tag; FILE_POS file_pos; NAME name; int seq_num; int occurrences; int swizzle_index; TYPE resolved_to; TYPE lower_constraint; TYPE upper_constraint; VAR_STATE state; }; struct fefn_param_rec { KIND_TAG kind_tag; FILE_POS file_pos; NAME name; TYPE_VARS type_vars; ARRAY_FEFN_PARAM parameters; TYPE result_type; BEFN_PARAM befn_param; PARAM_KIND param_kind; BOOLEAN is_private; FEFN_FEATURE feature_view; FEFN hidden_value; }; and TYPE with EXPR_CALL: struct vtype_rec { KIND_TAG kind_tag; FILE_POS file_pos; TYPE_VAR variable; }; struct ctype_rec { KIND_TAG kind_tag; FILE_POS file_pos; USTRING uninstantiated; CLASS class; ARRAY_TYPE generics; }; struct expr_call_rec { KIND_TAG kind_tag; FILE_POS file_pos; TOKEN param_token; TYPE type; BOOLEAN large_scale; BACKEND_OBJ be_var; SYNTAX_TYPE syntax_type; USTRING value; ARRAY_TYPE generics; NAME op_name; ARRAY_EXPR parameters; TYPE class_type; ARRAY_EXPR sorted_actuals; FEFN function; }; In this way types become expressions. The type of a type expression is np_type, and its value is the np_type object, although type checking proceeds at compile time on the type expressions themselves, with no need for an explicit value. A meet type can be another kind of expression (not needed!). This has various advantages, one leading one being that there is no longer any need for separate lists of formal type and ordinary parameters, or separate lists of actual type and ordinary parameters. However, formal parameters must be marked as type parameters, by their kind flag or otherwise. Instead of FEFN function, the call refers to an arbitrary NAMED entity. Initially, when uninstantiated, this entity is just a name definition stored in the name table. But later it gets overwritten with the FEFN, class view, etc. We may also be able to merge sorted_actuals with parameters. 2 February 2007. Took the first step towards the above plan by separating TYPE_VAR off from TYPE etc. The old type.c had 3891 lines; the new type.c and type_var.c files have 2821 and 1433 lines respectively, totalling 4254 lines. The next step is a separate class for meet types. 5 February 2007. Have separate types and files for variable invocations, class types, and meet types; line counts are 1621 type.c 470 vtype.c 704 ctype.c 537 mtype.c 557 itype.c 3889 total which is almost exactly the old total size, except that we are not counting the 1433 lines of type_var.c (!); but it's all nicely partitioned and much easier to follow now. 9 February 2007. Made BEFN more abstract, by hiding all its subtypes from the rest of the compiler, and doing some refactoring to clean up the resulting messy interface. The CLASS type is below the BEFN interface; it is very back-end-oriented, and really does belong there. Now need to get back to the hard job of merging types as above. It seems possible to replace "FEFN function" in expr_call.c by "BEFN function"; the only thing we will really lose is the name, and we gain the ability to say what the interpretation really is. When this is done it may make sense to share this field with the CLASS field of ctype; also this would mean that references to type variables point to the back-end type parameter, not the front-end parameter, so the correspondence is: struct type_var_rec { KIND_TAG kind_tag; FILE_POS file_pos; NAME name; int seq_num; int occurrences; int swizzle_index; TYPE resolved_to; TYPE lower_constraint; TYPE upper_constraint; VAR_STATE state; }; struct befn_param_rec { KIND_TAG kind_tag; BACKEND_TYPE be_type; BACKEND_OBJ be_obj; ARRAY_BEFN inner_functs; ARRAY_BEFN_PARAM parameters; BACKEND_OBJ inline_be_obj; BOOLEAN utilized; PARAM_KIND param_kind; FEFN_PARAM np_param; int index; BEFN_FEATURE creation_feature; }; It's no better or worse than linking with front-end parameters, and it may actually be advantageous for renaming etc. It links type checking closely to the back end, which is good because type checking goes on things, not names of things. If CLASS gets overloaded with BEFN, we might as well unify it with BEFN_CREATION, so that it really means something in function terms, but can be taken as a class as well. If this merging is done, then the type_vars of CLASS correspond to the parameters of the creation function (some of them anyway). It's interesting that we've felt obliged to store a set of names and a pointer back to the front end in CLASS but not BEFN. 10 February 2007. Puzzling over how befns come to get names, for debugging and the backend object. Realized that the great majority of them actually store the fefn they are derived from. This suggests that we make an FEFN be a part of every BEFN; it can have a null value in the few cases (initialization functions) that have no corresponding fefn. Furthermore, when we import a feature or class view, we can change the FEFN so that it now refers to the version in the current class. In this way, names in error messages deriving from back-end entities will use the most appropriate front-end names. In particular, we will no longer store a set of names in CLASS, just the current class view. We also need to make CLASS_VIEW a FEFN and CLASS a BEFN. The function represented can be the type object creation function. It exists for all classes. 11 February 2007. A bug led me to start revising the handling of system views and systems. The previous arrangement was a mess. 14 February 2007. Just completed the revision of system views and systems. No longer a mess. Code now checks that imports and inheritance occur only between class views from the same module. And now have unified CLASS with BEFN and CLASS_VIEW with FEFN. 15 February 2007. Folded FEFN_CREATION back into CLASS_VIEW, and BEFN_CREATION back into CLASS. It would seem that the logical next step is to unify TYPE_VAR and FEFN_PARAM/BEFN_PARAM. 17 February 2007. Made manifested expression calls link to a BEFN rather than an FEFN; did this some time in the last few days. Revised the reflection model into a hierarchy of three types, np_type inherited by np_class_type and np_meet_type, which makes more sense, and added them to the language core module. Introduced the "typeobj" expression to return the np_type object corresponding to a given type; it disambiguates type expressions, which can also denote creation functions and selfless calls. Not implemented yet. Did some design work on the problem of the = operator, which looks like it will have to be a global function, outside any class. Added a section about it to the language specification, but have set aside the implementation for now. 2 March 2007. Wasted some time on version 2007_03_02_nogood which I have archived and put away. Returning to the archived version of 17 February now. The failed stuff had to do with trying to unify TYPE_VAR and FEFN_PARAM/BEFN_PARAM. This proved troublesome and I have since realized that front-end classes contain signatures which inevitably contain back-end types, hence there does not seem to be a way to fully separate front-end classes from back-end classes. So the plan now is to leave TYPE_VAR as is but to link it to a BEFN_PARAM, and to link the BEFN_PARAM back to the typevar. This puts TYPE_VAR in a similar position to FEFN_PARAM, but we are not going to attempt to unify the two, because it's a swamp. Instead, BEFN_PARAM just has to be prepared to go back to either a FEFN_PARAM or a TYPE_VAR, depending on which of the two pointers is non-NULL. Have also lost about a week to other stuff; took a few days off, worked on Lout for a few days, revised timetabling papers for two days. Did a couple of hours today, then away for another break. 9 March 2007. Had a few days off, then did more other stuff (revised KTS papers, released Lout Version 3.34) and am back at work on Nonpareil today. Implemented the "typeobj" expression, now need to use it in creation calls, for example converting nlist{int}(3) to nlist_create(typeobj nlist{int}, 3) which matches the backend fn. 16 March 2007. Am nearing the end of a long detour, in which I am revising the type system and in particular replacing Ordinary, Unify, and Range variables by ordinary variables on the one hand, and "range types" on the other, the latter being either unfrozen or frozen. It's basically the same but more principled; I'm investigating the implementation of the operations more formally than previously. 18 March 2007. Just completed a rewrite of the type systems documentation, both in the language specification and in the implementation guide. Have to implement it now, but it is very concrete and close to what I already have, so that should not be too hard. 20 March 2007. Began implementing the new version of the type system today. Also wrote up a manifesto for moving the type system into a separate library module, as a way of generalizing it and improving the interface to the rest of the compiler. I won't do that though until I have reflection working. 22 March 2007. The new version of the type system is compiled and tested today. This is about 12 days after I started the revision. The next problem, causing a core dump today, is that range types get frozen but are left unresolved at the end of MatchFunction, so we can't be sure about coercions and run-time types at that point; we have to defer those until we descend again, during code generation. So there is some re-thinking and rewriting to do there. 23 March 2007. Sorting out the relationship between coercions and subtype tests. This has led to a realization that subtypes have to be tested twice: once when manifesting, without asking for coercions because they are unavailable in the presence of range types, and once during code generation after range types have all been resolved, at which point an error could occur. Also privatised expr_rec further, hiding it from its own subtypes. 28 March 2007. Finally finished working on the relationship between coercions and subtype tests. There were two problems: when the upper constraint is a meet type, and when the lower constraint is a variable. Solutions to both have now been documented, implemented and tested. The next step is to insert subtype calls, and the resulting coercions, during code generation. 29 March 2007. Cleaned up expression classes (again) ready for code generation changes. Also worked out that I need to insert coercion calls now basically everywhere that the current code inserts casts. 1 April 2007. Hit a few snags but am now inserting coercion calls correctly. Am storing in each expression not just its own type but also the type expected by the surrounding context. This turned out to be a mess to get right in every case, and it may fail yet, on the branches of features. So we'll live with it for now, for the sake of getting on, and review it later, hopefully returning to the more consistent plan in which the enclosing type is a parameter of the code gen call. May need to do the type system library first though, to get easy access to actual types of matched signatures, which was the sticking point this time. Next step: codegen of type variables. 2 April 2007. Got right through to the end of code generation today. Now I have to make sure that the right code is being generated. 13 April 2007. Took a week off for Easter, then spent a couple of days preparing and releasing Version 3.35 of Lout. Back at work on Nonpareil today, fixing little bugs all day. Have a clean compile of quite a lot more code than I had before, and am generating code for invoking type variables now. 20 April 2007. In the middle of a major diversion, completely redesigning and reimplementing the code generator. The design is all done and it's great; have to implement it now. I have already implemented the raw syntax generator. At the same time as this I am changing from my own USTRING to C's wchar_t *. My own WCS module supplementing what C offers for wchar_t * is done. 23 April 2007. Still in the diversion; have a SYMTBL module now that uses wchar_t * rather than USTRING, and am in the middle of implementing the code generator. A good achievement today was encoding statements with their peculiarities into sgen constructs. 26 April 2007. Now have a clean compile on an apparently complete code generation module. Totally untested at this point. 27 April 2007. Have introduced submodules into the compiler, made the code generator one of them and the utilities another, and am in the middle of converting to the new code generator and simultaneously converting everything to wide character strings. 29 April 2007. Have converted all expression and type classes to the new code generator and to wide character strings. Also converted all utility modules to wide character strings, which involved throwing large chunks of them away. And am now halfway through converting the befn classes to the new code generator and wide characters. 30 April 2007. Got to the end of all befn classes, including class.c, except there are a few things to fix up still in class.c. 4 May 2007. Working towards one befn object for *every* backend function. At present the "_load" functions are bundled into the corresponding "_init" functions, and the "_swizzle" and "_unswizzle" functions are represented only by their names, in their class record. Have to elaborate these into proper befn types: BEFN_LOAD, BEFN_SWIZZLE, BEFN_UNSWIZZLE. It will be better thus; for one thing, there will be less code in class.c, which is currently handling all of these. Should probably return to a separate befn for creation too, not double up on the class record, however tempting it was. Also requiring each BEFN to supply BEFnCodeGenFunctionNames and BEFnCodeGenImplementation functions, not currently written but very similar to stuff we already have. It's good to leave the generation of the implementation to be dynamically dispatched, since predefs are not functions. Will probably end up dynamically dispatching the interface generation, for the same reason, and also the generation of parameter names, since not all of these can be deduced from a corresponding front end function or type variable. The old ClassCodeGenFunctionsMainModule function mainly calls functions that generate using the old Phase2Body calls. These will transfer easily to the new method. The exceptions are: NPClassTypeCreateFnCodeGen(c, cgw); ArrayForEach(c->orig_predefs, f) BEFnFeaturePredefDeclare(f, cgw); These will have to be investigated and brought into the new system; each needs to be a befn (the predefs already are) and added to BEMOD_MAIN. 9 May 2007. Had three days off. Today I got a clean compile of the rewrite that began somewhere between 13 and 20 April. Have noticed a problem with the way cgen writers are accessed before they are linked to any module. Summary of uses of cgw before there is any module: * To set builtin types (ClassCodeGenInitializeBuiltinTypes); * To make global and struct names (quite safe, obviously, but not well modelled - surely the cgen_sys itself would be a better choice here); * To set inline_be_obj fields, both in builtin functions and in functions implementable by object access; this is the most problematic of all the uses, since the constructs could be either C or intermediate code constructs, and the decision is not made until some module is opened; so they can't be retrieved so early from the writer. Will leave it for now, but later on when fefn/befn is re-united will have to redo it. 15 May 2007. Took a few days for initiating the design of KMT, the object-oriented type and module system that I intend to spin off from the Nonpareil compiler. Started work on a document describing the design, discussing the issues, and citing the literature. That seemed to go well. Yesterday I picked up the clean compile from 9 May and started testing. I quickly realized that the cgen interface was badly designed. Today I've redone it; the next step is to redo the use of it within the compiler to match the new interface; have done system.c and system_view.c so far. 16 May 2007. Now have a clean compile again using the revised cgen interface. So only lost a couple of days. 17 May 2007. Had to redo the structure of code generation, it was hard to follows the steps as it was. Now all nicely structured and the whole thing documented above SystemCodeGen(). Since testing is so boring I am time-sharing with the specification of the KMT module and type system, which continues to go well. I believe I've sorted out the representation of types and the essence of the distinction between type parameters and term parameters. 20 May 2007. Finally sorted out the C context of code generation, by adding a SYNTAX_KIND parameter to the code gen routines which indicates whether a C expression, statement, sequence of statements, or sequence of declarations followed by sequence of statements may be generated. Although res_asst was distinguishing between expressions and statements, nothing was distinguishing the last three cases, and I saw several points in the code which looked like they would generate invalid C in unusual circumstances. Also got the compiler to run to the end of code generation. The generated C code does not compile, but at least it's generated. 22 May 2007. Now working on getting the generated C code to compile. Hit a deep bug which was preventing backend features from being inserted into classes when they had to be imported then inherited. Apparently there was none of that going on before, but adding an explicit type parameter in class "object" made an example of it, since classes in module "algebra" then had to inherit it. Have patched the problem for now but the ultimate solution will have to wait (not long, I hope) for KMT. Have reached a point where I can seriously start work on the new reflective back end. 25 May 2007. Working steadily through the reflective back end. Have removed all trace of the "object" type; it was useless, even for getting a "type" field into all objects, since it was not an ancestor of the enumerated types. 28 May 2007. Had the weekend off. Currently revising array and string creation, partly to replace the create1 - create2 thing by a single call, and partly to add a type parameter. 30 May 2007. Slogging through the various revisions to the generated code. Am just about to regenerate creation functions. 1 June 2007. Going steadily. Added the ffile module to cgen, to further improve the C code formatting by doing proper line breaking. Have written code for creating the five initial reflection objects, and am now generating the bodies of the typecreate routines, which invoke them. Next step is creation functions. 3 June 2007. Still going steadily. Fixed various small things today, then did the bodies of creation functions. The main remaining problem seems to be casting, expecially of "self" in dynamically dispatched features, where the expression type and the backend type do not agree. Have to sort that out carefully. 5 June 2007. Slowed down by a cold, but still working. Completed the np_class_type builtin functions today (class_name, is_abstract, and parent_types). Have not tackled the casting problem from 3 June yet. 12 June 2007. Have finally started work on the casting problem from 3 June. The past week has been partly lost to the cold, and the rest of the time went on thinking about the design of KMT, and writing out my ideas, since I did not feel up to detailed code hacking. 14 June 2007. Completed the rewrite which is intended to solve the casting problem. Have clean compile and all seems to be working. Have moved on to other problems with the C compilation, currently generating the type object parameter of curried function calls. 16 June 2007. Now have a clean compile of the generated C code, except for one type of error message, warning about casting from void * to narrow types (specifically "bool", i.e. char). This is benign but it would be good to get rid of it. Also the class unswizzle functions are just stubs at the moment; and there are still two object memory systems, have to get rid of one of those. 18 June 2007. Revised the include files in nonpareil/compiler/lib, in particular replacing the two object memory systems by one. I now have a working compiler again. Hurrah! Next job is KMT. Phase 5: KMT ============ 19 June 2007. Began work on KMT today, although some design work was done earlier (see above). 29 June 2007. Went straight on to KMT. Did quite a lot of literature searching and reading, and have most of the design done, including a document describing it and a fairly complete "kmt.h" file defining the interface. I have begun the implementation. Wrote memory allocation today, previously I had already converted type.c to kmt_type.c. 2 July 2007. Working steadily on converting the Nonpareil type classes to KMT type classes. Have converted the root class, var types, and path types so far. Have changed ITYPES to KMT_GRAPH_TYPE, introduced KMT_ERROR_TYPE for error placeholder types, and KMT_RAW_TYPE for uninterpreted path and var types. All going well. 4 July 2007. Finished converting the type classes, although I have passed over a couple of things, notably keeping track of which range types have been resolved (so as to undo), and KMT_GRAPH_TYPE. Also brought over ARRAY (now KMT_ARRAY) and SYMTBL (now KMT_TABLE), the latter modified to allow multiple entries with the same key, and both simplified by using void * instead of macro-genericity. 5 July 2007. Worked on kmt_interface.c today, got a reasonable first version including the retrieval operations; not inheritance yet. 10 July 2007. Struck a few conceptual problems and decided to do some reading. For example, I found an interesting Java test which shows that an interface can be a component of itself. Have retrieved many papers from the internet and written up several issues. Am tending now towards search-up inheritance rather than copy-down inheritance, which is a major change. 17 July 2007. Continuing with the reading. The current focus is on the implementation of dynamic dispatch using vtables. Have found only one description of how to build vtables, for Java, and am writing my own, with proof of correctness. 23 July 2007. Have written an entire document on how to build vtables, which will probably end up as a chapter of the Nonpareil implementation guide. The outcome of that whole large exercise is that KMT should offer only search-up inheritance and function subtyping operations; these are enough to support a back-end vtable building algorithm, and the back end is the right place for it. This finishes my two-week design detour; I should be able to get back to implementing KMT now. 1 August 2007. Proved to my satisfaction that meet types and repeated inheritance do not mix, thus opening the way to using the Nonpareil implementation of subtyping in KMT. Have converted topological sort, and today I did most of signature matching. No obvious obstacles now. 3 August 2007. Implemented InferBegin and InferEnd with their backups and restores of range types, all carefully specified and documented. May be able to improve the type operations now this is done. May even decide to get rid of tightening. 5 August 2007. Changes to range types have motivated a re-analysis of tightening, now reinterpreted as indicating that short-cuts are possible because only one last type operation will be performed on the range type. May now be able to automate the infer action function, giving one less thing for the user to worry about. 8 August 2007. Finished implementing the changes to range types, with tightening changed to finalizing. Will now look again at the need for an infer action function, and the information it needs. 10 August 2007. Have more or less worked out the design for retrieval, and begun implementing it. Did a complete implementation of renaming today. Also optimized type copying to copy only when the type contains a variable; if not, KmtTypeCopy returns self. 12 August 2007. Have settled on a larger, better structure for the KMT user's guide; today I'm busy documenting signature matching, as carefully planned but not yet implemented. 17 August 2007. User's Guide more or less done now; the two main omissions are access control and function overriding. 18 August 2007. Reorganized kmt.h today to follow the new structure of the User's Guide, including replacing signatures with function types. Also re-implemented the type show functions along the way. All done and have a clean compile, ready to go on. 19 August 2007. Wrote User's Guide section on renaming and type printing, in the course of which I had a few minor new ideas about renaming. 20 August 2007. User's Guide further reorganized today. Now has a place for nearly everything, although a few sections are still not written: type equality, subtyping (including overriding), join, and meet; and access control. Currently no place for interpretation, nor for inserting closure parameters. 35 pages. 24 August 2007. More work on the User's Guide, with a much improved treatment of function types, including a serious list of their policies for controlling function matching and subtyping. Also removed the whole concept of global policies, and distributed them to the objects that need them. 50 pages. 29 August 2007. Went into something of a black hole on raw types and interpretation. Have just finished implementing the new structure, with documentation; but the code for actually interpreting is not done yet, have to do that next. 31 August 2007. Have completed interpretation now, breaking it into two stages (conversion and validation), all except the actual retrieval of interfaces from the context. 4 September 2007. Hit a rock in assigning meaning to renaming, had to disallow retrieval of a set of name syntaxes (but managed to argue that this was not useful anyway), and now have everything sorted out again. The bad news, however, is that a visit_num integer in each interface is not enough; one needs not to prevent retrieval from searching the same interface twice, but rather to prevent it searching for the same name and syntax in the same interface twice. 5 September 2007. Now have a reasonable implementation of retrieval, which revisits interfaces but not if there is a cycle. Need to add lexical search, the new Prev operation (which returns an interface, an inherit relation, and a rename entry), and do the type substitution stuff too. 6 September 2007. Handled cycles in the interpretation of inherit types, by adding another callback to the interpret policies, marking inherit relations on entry, and unmarking on exit. The plan from here is to keep ploughing through type conversion and validation until it's done; then the only major features unimplemented will be access control and class extensions. 9 September 2007. Sorted out the mutual interaction of retrieval and conversion (each needs to pass along the other's parameters), wrote the code, and went on to complete the conversion of raw path types into non-raw variable and path types. Hurrah! 10 September 2007. Rampaged through the whole conversion plus validation stuff, now have code to carry out the four steps of wisdom: conversion, build ancestor sets, topological sort, and validation. 13 September 2007. Lost a couple of days to illness, today I tightened up the implement of the interpretation code so that it refuses to interpret anything outside its subtree. This makes it much easier to explain to the user. Last day of work on KMT for the time being. ==================================================================== During this period I worked on timetable construction, specifically on resource assignment, and produced a paper about it for PATAT08. Also had my usual summer holiday in January 2008. ==================================================================== Phase 6: KMT continued ====================== 5 February 2008. Resumed work on KMT today. Decided, more or less, to replace the boolean "is_typic" by the enumeration "system". 6 February 2008. Changed name of "system" to "genus", and now each interface has a "genus" and "species". Working out the consequences through the documentation today. 7 February 2008. Updated code to handle genus. Still have some changes to make to the documentation, and also have to check that genus is being checked when interpreting function and range types. 15 February 2008. Spent a week redesigning the interface of KMT, generalizing type expressions into expressions. This should not change the implementation very much, except for function matching, but it is a much better model, because more unified. 17 February 2008. Have more or less finalized the redesign of the KMT interface. Went back on the plan to rename "types" into "expressions" - it only caused confusion. But have a much clearer idea now of the relationship between type expressions, module expressions, and term expressions, and have documented this in the User's Guide. 19 February 2008. Completed the changes to the User's Guide to reflect the revised design. Now to implement. Also started an "Issues and Examples" appendix to the User's Guide, which I intend will eventually contain all non-trivial uses of KMT from the Nonpareil compiler, when they are done. Moved to previously written sections, on name syntax and overriding respectively, to this appendix. 21 February 2008. Working through the implementation, most of it is done. Have completed a first rewrite of the function matching code, it is quite tricky and will need another careful look. 24 February 2008. Had a couple of days off, during which time I planned a reorganization of the KMT User's Guide into a more object-oriented style, that is, grouping operations closer to their objects. Carried out this reorganization today. 26 February 2008. Reorganized the code to follow the new structure, also tidied up some things. Have a clean compile, and checked that every declared function has an implementation, even if only a stub. So ready now to finish off the implementation. 27 February 2008. Re-documented function matching, now ready to check and complete its implementation. 29 February 2008. D'Oh to me, have just realized that function types require that types KMT_INTERFACE (for parameters) and FORMAL_PARAM be the same type; or anyway that variable types refer to a FORMAL_PARAM from which the upper bound etc. is available. 2 March 2008. Completed a redesign which handles the unfortunate discovery I made on 29 February. The User's Guide has the new design in full; now I have to implement it in KMT itself. 3 March 2008. Implemented the new design in KMT, down to the point where I have a clean compile. 5 March 2008. Have started work on the implementation. Looking over interpretation generally, added KmtTypeInterpret today. 7 March 2008. Interpretation is in good shape, as is retrieval now except that result types (with variable substitutions) are not being calculated yet, and the function application code is not yet being used when validating path types. Added default interpretation policies today, also clarified and documented the relationship between the genus and kind policies of parameters. 9 March 2008. Clarifying my understanding of how function matching works, including realizing that the user has to pass a default type when a default value is permitted. Worked out and documented a lot of details in this area, to the point where the code can probably be written correctly now (some doubts about currying). 11 March 2008. Added parameter and variable default types to code and documentation. Sorted out the semantics of copying types, and checked over and corrected the implementation, including adding an implementation of KmtFunTypeCopy(). Made a start on the real implementation of function matching. 12 March 2008. Hurrah! Wrote KmtFunMatch() properly today, all except the examination of range types in the result type. 13 March 2008. Reviewed what to do about range types at the end of the match, and decided (at least for now) to just set their finalizing flags to TRUE. Documented this and the reasons why in the User's Guide. Did a few other small things. Begun to replace every assert() by KmtUsageAssert() or KmtInternalAssert(). 16 March 2008. Had a couple of days off. Today I completed the replacement of assert() by KmtUsageAssert() or KmtInternalAssert() and began checking that every call to KmtUsageAssert() is documented in the user's guide as "This function aborts if..." Completed this check for kmt_*_type.c. 20 March 2008. Have been looking into the relationship between paths and function types. Developed a new formal model of the KMT type system as a result of this, but doubtful whether I will change very much because of it. Still, it's been useful. 31 March 2008. For about two weeks now I have been trying to understand the relationship between paths and function types. I finally cracked it yesterday, but the result, sadly, will be some pretty big changes to the system. I have a new design now, based on just function types and record types, which works for the big things. I am currently exploring the smaller things, mainly printing and evaluation. 2 April 2008. Close to beginning the implementation today. Am working on a formal presentation of evaluation; I have a formal syntaxes for raw and evaluated types. 4 April 2008. Wrote a beautiful "Big picture" section for users to read. It lays out the scope of the whole project in three lucid pages. 6 April 2008. Working on a revised User's Guide, which looks great. Also redesigned display to use printf-style format strings. 8 April 2008. About four chapters done, including a seven-page description of function matching which is absolutely fabulous. 10 April 2008. All but two chapters done (record types and access types). 12 April 2008. All chapters done; a few minor blemishes only remain. 17 April 2008. Had three days off, back at work today fine-tuning the user's guide to KMT. Have already redone kmt.h, a few days ago. 18 April 2008. Clean compile of kmt.h, kmt_interns.h, and kmt_type.c. 21 April 2008. In the throes of yet another revision, removing raw types as separate types, and also removing the need for estimating the number of elements of sequences. Have revised the memory allocator to handle extensible arrays very efficiently, rewritten the user's guide, and updated kmt.h. Have clean compile of kmt.h, kmt_interns.h, and kmt_type.c again - now going on to other types. 22 April 2008. The plan is to go through all the eight kinds of types, implementing most things (notably not evaluation), then redesign and implement the remainder. Got all type files compiling today except kmt_record_type.c and kmt_access_type.c. 23 April 2008. Done kmt_access_type.c, and laid out kmt_record_type.c, but then got side-tracked into redoing the symbol table module to use the new extensible arrays. Finished that now. Trying to reduce the dependence on the utils module, with the aim of making KMT free of it eventually. 24 April 2008. Did some work on kmt_record_type.c (including kmt_ancestor_set.c and kmt_rename.c). Also cleared up some tricky issues today (evaluation must be given an evaluated context, printing of record types) and documented them. 27 April 2008. Working steadily. Completed display of record types, which naturally was a mess. Tried to generalize and unify but that would have required creating display objects which would use memory, so it seemed better in the end to hack through, given that the function is peripheral anyway. Now have a clean compile, of course missing many things, principally evaluation, and will take an off-site backup of what I have now. Has just started auditing what's missing; added evaluation policy handling code. 29 April 2008. Currently rewriting retrieval. Have completed both retrieval from a type and retrieval from a context. But have not attempted to do evaluate-as-you-go yet. Made some simplifications - got rid of visit numbers and active paths, without any significant loss of efficiency, and generalized the retrieve path to cover everything passed through, very nicely. Rewrites are good. 3 May 2008. Got hung up over genus stuff, did some redesigning. Today I have rewritten display so that it does not use genus; for parameters it uses kind instead, which works better. 5 May 2008. Finished rewrite of function matching. 9 May 2008. Took a couple of days off, had to mull over how the stages of evaluation were going to work now that call types and access types are no longer part of the result of evaluation. Decided to gather two lists during the first stage of evaluation, one of raw record types needing a Stage 2 (topological sorting), the other of raw types needing a Stage 3 (validation). All done today except record types. 11 May 2008. Still puzzling over evaluation, took some time off to write a tester, with a lexer, parser, and command executer. All written, compiled, and documented today. Also added functions for explicit upcast and downcast of types. 14 May 2008. Completed a rewrite which allowed for more flexibility in building abstract supertypes of various things, and also made a queue of types module for storing types pending evaluation. 21 May 2008. Completed evaluation (including detours through retrieval) today. This is two months since about 17 March 2008, when I started work on trying to understand the duplication of functions in the design, leading to the current redesign; and today I am about back where I was, with some small things still to do now as they were still to do then. An immense rewrite. 24 May 2008. Spent the last few days designing access control, and have begun implementing it today. So far, have added access control attributes to components and modified retrieval so that it passes the original context of the retrieval right through the retrieval, including into the retrieve test function. From here it should be straightforward to implement the access rules. 25 May 2008. Completed the implementation of access control today. Struck one awkward thing: in some cases you have to do a kind of retrieval to check access, and then you might possibly close an evaluation cycle. All handled and documented though. 27 May 2008. Tidied up a few things, but basically I'm having an alarmingly slack couple of days. Had a look at the whole User's Guide today: it's currently 80 pages, not counting the Appendix. 31 May 2008. Have been slogging through function subtyping for the last few days. I now have a definition, and everything is documented. I have just completed some changes to function matching which were needed for function subtyping, and am ready to implement function subtyping proper. 2 June 2008. Implemented equality and subtype testing for function types today. Only took 2.5 hours, but that was after days of struggling to understand it. Join and meet still to do. 5 June 2008. Reworked the definitions of equality and subtype testing so that they can share the parameter matching algorithm with join and meet, and reimplemented them. Join and meet still to do, but no major roadblocks now. 7 June 2008. Progressing well with join and meet. Now have clean code for finding the join or meet version of two corresponding parameters, depending on their variance etc., all nicely documented. Just have to assemble the join and meet function types now. 10 June 2008. Completed function type operations and documentation today, about 2 weeks after I started work on them (before 31 May). Also added the index to the User's Guide today. The only features not yet implemented are closure parameters and private parameters; plus I have to test everything, then rewrite the Nonpareil compiler to use KMT. 13 June 2008. Completed a careful review of the User's Guide. I did it because I don't feel like testing, but it was useful. 14 June 2008. Revised the testing code today, and have just started testing. Found and fixed one bug so far. 18 June 2008. Had most of the last few days off, doing odd jobs. Am back at work testing today; spent some time making the testing code more pleasant to run interactively. I've fixed several small bugs, and at the moment everything being tested is working, including some quite complex function subtype tests. 28 June 2008. Struck a rock during testing, which was that leaving upper constraints unevaluated failed when the function type was copied later during evaluation. Did quite a long analysis and am now replacing unevaluated upper constraints by "transient types", which are evaluated types that contain a substitution, a context, and an unevaluated type - very similar to record types. Their unevaluated type gets evaluated later, the same way the internals of records do. Am sharing most of the code with record types. Documented, implemented, and compiling today; still to test it. 30 June 2008. Still testing and bringing out a menagerie of odd cases. Have made a list of those errors that are detected late, examined all of them closely. I managed to move one or two forward, and I have just dealt with detecting problems with a meet type after it has already been copied. 1 July 2008. Spent most of today tidying up the User's Guide, by making and using a Lout package for formatting KMT expressions. Also reviewed the grammar, added renaming, and tested it; it seems to be working, first time, which is a nice change. I may call a halt here for a while and do something else. Most of KMT is tested now, the main omission being the detailed behaviour of function matching, including range types.