Introduction
Run-time Structure
Overall compilation model
Launching Petit Larceny
User interface
Building Petit Larceny
Building variants of Petit Larceny
Petit Larceny Source Files
Run-time system files
Compiler files
Assembler files
System internals
Code generation
Startup
Runtime architecture
Dispatch loop
Generic arithmetic
Architecture Dependencies
Word size
Endianness
Status as of 4 June 1999
Portability problems
Nonportable code
Copmpiler and operating system idiosyncrasies
Operating systems support
Unix
MacOS
Generic operating systems
Petit Larceny should run on most 32-bit architectures and is largely operating-system independent. In particular, it has been tested on several variants of Unix, on the Macintosh Operating System, and on Windows 95. Opertaing-system independece comes at a cost, however, since not all features provided by Larceny can be provided by operating-system independent means. Thus, while Petit Larceny can use the "generic" operating system support (where some features are absent), operating-system dependent units can be fashioned for each particular system Petit Larceny runs on.
We expect to remove word-size dependencies in Petit Larceny before long, allowing it to run on a broader class of systems.
Petit Larceny's reasonably fast interpreter mitigates the compilation problem to some extent, but interpreted code is still significantly slower than compiled code in most cases.
The loading and linking process will differ somewhat from one operating system to another, and as Petit Larceny has portability across operating systems and architectures as one of its goals, a least-common-denominator approach has been adopted. Additional mechanisms may be provided on some operating systems.
A Petit Larceny application consists of: an object file containing run-time system and library code (PETIT.LIB); a heap image containing data for the library code (PETIT.HEAP); a set of files containign data but no code for the application (*.FASL); an object file containing containing the application code (APPLICATION.LIB); and finally an application object file tying these together (APPLICATION.OBJ). This structure, though elaborate, is reasonably portable, does not constrain more advanced techniques on platforms that provide them, and allows applications to be built without requiring the heap image to be rebuilt from scratch.
We can distinguish three classes of C source files:
Petit Larceny is an application built in this fashion. It contains not application code, because all compiled code is already in the libraries, thus the application main file contains an empty code table. The name of the application main file is "petit-larceny.c".
petit-larceny The executable: a stub
petit.heap The heap image: data for the compiled Scheme libraries
petit-larceny.so The shared library: All compiled code for the run-time
system and the compiled Scheme libraries.
Petit Larceny is run by launching petit-larceny; the program dynamically links in petit-larceny.so and loads petit.heap into the run-time heap. There is some operating-system dependent cruft associated with how the executable finds the shared library; this must be resolved on a system-by-system basis.
It is possible to statically link petit-larceny against the library, and to use other heap images by giving the name of the heap image on the command line in standard fashion.
On other operating systems, the structure is similar.
cd Rts make Rts/libpetit.soSecond, build the heap image and the application itself. You do this from Larceny's build system, which is started thusly:
build -petit -host -endiannessHost is the name of the host Scheme system; the allowable hosts are listed at the head of the build program itself. Endianness specifies the endianness of the target platform; it is either little or big. See the section on architecture dependencies below.
Once the build system is up, build the heap image and the executable with the following command:
(make-petit-heap)
If your operating system does not have a C compiler that can be invoked by another program, then you will need to do two things. First, before executing the preceding command, you must select the dummy C compiler in the build environment (FIXME: how to do this?). Second, after the build has completed, you must compile and link the generated C code. (FIXME: A list of the files that need to be compiled and linked needs to be distributed with Petit Larceny but this does not happen at present.)
FIXME: on systems that _do_ have CLI compiler it may still be necessary to do some hacking to get the interface right.
arithmetic.mac Generic arithmetic macros.
config.c Definitions of system name, default heap file name,
and architecture name.
millicode.h Header file for portable millicode implementation.
millicode.c Portable millicode implementation, including
dispatch loop.
multiply.c Portable 32x32->64 multiplication code.
petit-config.h Configuration file for Petit Larceny
petit-hacks.h Preprocessor symbols and macros that are useful
and/or fix gross naming bugs in Larceny that have
not yet been fixed globally. These fixes should
be folded into the system.
syscall.c Syscalls specific to Petit Larceny.
twobit.h Macros for all MAL instructions and primops.
pass5p2.sch MAL code emitter procedures. dumpheap-extra.sch Overriding procedures for heap dumper. asm-switches.sch Assembler switches.
The new LOP files contain segments that are represented as lists with three elements: code vector, constant vector, and list of C functions defined by the code vector. The code vector is a string that contains C code. The constant vector has the same tagged structure is in SPARC LOP files, except that code vectors again are strings containing C code. The list of C function names can be used by a postprocessor to produce prototypes for the functions defined in the code vector. Each entry is a list of length 3: function name, flag for whether the procedure is always defined (some procedures are only defined if a certain calling convention is used), and a flag for whether the procedure is the top-level thunk that's to be run when a bootstrap heap image is loaded.
Every C function in a file is declared "static" and has a name that is unique in the file. When a C file is produced from a LOP file for compilation, forward declarations are added at the beginning of the file, all code vectors are copied from the LOP segments to the file, and finally trampolines are added that can be called at startup time. Each trampoline knows about the initialization procedures in the file, and for each of them, in order, it creates a trivial thunk which it then calls.
When a group of LOP files are dumped into a heap image, temporary C source files are generated as outlined above and compiled individually; simultaneously, all the data from the LOP files are dumped into a heap image that's identical to the standard heap image except that every code vector slot contains #f. The C code always knows which code vector it needs and gets the code vector by using its name. Consequently, the startup-thunk list in the SPARC heap image is replaced by a C procedure that knows the name of all the startup thunks and can just call them.
When a dumped heap is linked with Larceny run-time support, Larceny only needs to know the name of the startup procedure.
The run-time system is available as an archive Rts/libpetit.so. The archive is created by executing "(cd Rts ; make libpetit.so)". In particular, the archive contains a file with a main() function. When a heap is dumped, the resulting C files are compiled and then linked with the archive to produce an executable (and a code-less heap image).
Twobit_start is the result of compiling the startup procedure in Asm/Standard-C/dumpheap-extra.sch. The startup procedure is called with a single argument, the argument vector. Its job is to call all the init thunks in the heap and then call _go_ with the list of symbols to intern and the argument vector. In a native heap, the list of thunks is in a list in the bootstrap procedure's constant vector. In a C heap, the code pointers are not present in the initial heap, so a trick is necessary. The startup procedure calls a special primitive, mc_petit_patch_boot_code, which patches code vectors into the list of thunks. It gets the code pointers from an external array that is created specially for each heap. This array is called twobit_start_procedures and is created by the heap dumper. This is the only change necessary in the boot code, relative to the native boot code.
Heaps can be dumped iff code pointers don't move between program invocations.
Whenever a code vector returns normally to the dispatch loop, it must return a pointer to its continuation (a procedure). The loop then immediately calls that continuation.
However, it's also possible to transfer control via longjump. The procedure call_scheme does this: it saves the current state, creates a new stack frame, sets REG0 to the procedure to call, and does a longjump to the dispatch loop, which grabs the code vector from REG0 and calls it, just like in the startup phase. Normally this is not the way to reach the dispatch loop: only Scheme-to-Scheme-thru-Millicode calls do this. The exit handler is reached if the initial Scheme procedure executes a RETURN instruction.
add( a, b, k )
{
if (both_fixnums( a, b )) {
r, oflo = ADD( a, b );
if (!oflo) {
RESULT = r;
return k;
}
}
/* oflo or not both fixnums */
mc_generic_add( globals, k ); /* does its thing inline or longjmps */
}
Unfortunately, compiling Petit Larceny on a 16-bit platform that provides 32-bit representations may not be straightforward, since native integers on such a platform are most likely 16-bit. While the use of 32-bit data for the object representation is abstracted from the C compiler's representations (for example, the type name "word" is used everywhere for the type that holds a Scheme object pointer), other data used by the run-time system have been declared simply as "int" or "unsigned", and assumptions may have been made about these being at least 32 bits wide.
Thus, to compile Larceny to run under Windows 3.1, MS-DOS, or similar 16-bit operating systems, it may be necessary to use a a system like Win32s or a 32-bit DOS extender.
In contrast, compiling Petit Larceny in 32-bit mode on a DEC Alpha should not cause any problems.
Definition: Little-endian systems store multi-byte data from least significant to most significant byte with the least significant byte at the lowest address.
Endianness dependencies in the system are few and fairly well isolated. If your system has mixed endianness you will almost certainly need to adapt Petit Larceny. For example, systems that store IEEE doubles in big-endian word order but the bytes of each word in little-endian order are neither little-endian nor big-endian by the above definitions, and code that manipulates data layouts must know.
Few parts of the C run-time system have any endianness dependencies; search the code for the string ENDIAN_.
Some Scheme code that manipulates flonums and bignums is endianness-dependent. The dependencies are isolated in the files flonums-el.sch, flonums-be.sch, bignums-el.sch, and bignums-be.sch in Lib/Common.
The development system needs to know about endianness, so the build script must be modified, as must Lib/makefile.sch, Asm/Common/dumpheap.sch, and Util/nbuild.sch.
Petit Larceny currently casts code pointers to integers and back, and relies on two non-portable assumptions in doing so.
The first assumption is that there are integers that are wide enough to hold code pointers; this may not be the case and is not required by Standard C. This problem may or may not be addressed in the future.
The second assumption is that code pointers point to a WORD address. This too is not required by Standard C, and though most modern compilers generate code that is word-aligned when possible for performance reasons, some don't, and in general, many won't. (One compiler that doesn't generate aligned code is LCC-WIN32.) This problem will be fixed.
Some C code relies on compiler-specific semantics of the right shift operator: that a right shift of an unsigned quantity shifts in 0 bits at the high end, and that a right shift of a signed quantity shifts in a copies of the high bit. This problem will be fixed.
Some macros used by the garbage collector (in Rts/Sys/cheney.c) expand to lines much longer than the maximum length guaranteed by Standard C (509 bytes). Thus some Standard C compilers may reject the program. I have not yet encountered a compiler that has this problem, although I have seen standalone C preprocessors that have truncated the output to about 1K.
Some library Scheme code (notably the REPL and the heap dumper) assumes that the operating system supports reasonable file names, i.e., more than 8+3, and are in any case operating-system specific. Names used by these should be computed more carefully. This problem will be fixed.
Pointers to different "objects" (blocks of memory) may be compared using relational operators, possibly wreaking havoc on systems that represent pointers as segment+offset. This should be a minor problem thesedays and will be fixed if encountered.
Extensive use of 'int' assuming it's large enough...
When Petit Larceny was ported to the DEC Alpha, I used the DEC C compiler's -taso and -xtaso_short switches to compile the program in 32-bit mode. However, the ARGV pointer passed to the MAIN function is still a 64-bit pointer, and cannot be cast to a 32-bit pointer! Thus it was necessary to copy ARGV before passing it to the argument-parsing structures. Similarly, a LONG on this architecture is 64 bits, and one cast to LONG in the run-time system therefore had an unintended effect. Other, similar nonportable code may still exist in the run-time system.
FIXME - code is not loaded at the same address every time (MacOS, at least) - compiler requires unique filenames across subdirectories (CodeWarrior) - very large number of function pointers causes problems (TOC on CodeWarrior; what about MIPS conventions?) - operating system has no subdirectories (CP/M) - operating system has short filenames (MS-DOS) - object size limited to 64K or so (malloc and friends on 16-bit systems) requires use of system-specific allocation procedures. One may have to replace calls to malloc() by halloc()
The SPARC foreign function interface ought to work with Petit Larceny on SPARC systems; this has yet to be investigated.
There is a little bit of hair involved here to make sure that Scheme and the operating systems support communicate about the value of the newline character; this appears to be a matter of programming.
Currently there are two tables: twobit_start_procedures and twobit_load_table. Both are in HEAPDATA.c which is linked into the petit-larceny file with all the other object files -- what I propose to place in petit-heap.so. The latter would contain twobit_start_procedures only; the application would supply twobit_load_table. In petit-larceny this would be empty; in an application built for other purposes (to load the compiler, say) it would be nonempty.
The split would speed up the build process because no heap dumping would need to take place for applications -- only compiling and linking the compiled code to an executable and a bunch of FASL files. The FASL files can be loaded, and a complete heap dumped. It's a very localized change. It would work on all architectures that have a notion of libraries, as linking can happen statically as well as dynamically.