TRACING AT PLI 2000 ART Memo 11 (Version 0) Colin Runciman, 28 September 2000 INTRODUCTION At the PLI 2000 meetings we had various discussions with individuals about tracing, and we also convened an ad hoc discussion meeting for people interested in tracing. This memo is a (fragmentary) record of these discussions. DISCUSSION MEETING Those present at the meeting in addition to ourselves were: Carl Seger (Intel) Andy Gill (Galois) Richard Watson (Australia, visiting OGI) Meurig Sage (Glasgow) Ralf Hinze (Bonn) After introductions we began with illustrative demonstrations of three tracing systems (Carl's debugger for Forte, Andy's HOOD, our Hat 1.0). There were also `paper simulations' of algorithmic debugging in Freja, and sequential tracing as implemented by Richard. The simple example program we used for these demonstrations was something like: main = print (f 3) f x = dec x + inc x - 1 where dec x = x - 1 inc x = x + 1 Although all the systems could provide a more or less sensible account of such a simple program, we agreed that *none* of them had yet matured to the stage of a robust and dependable tool ready for use by the developers of serious applications. For applicability in practice, at least three areas need work: (1) handling long-running programs without consuming infeasible amounts of time and/or memory; (2) portably supporting the full repertoire of Haskell 98, not forgetting monadic programs; (3) specifying appropriate strategies for debugging that can be implemented using available tools. However, all the systems we considered showed promise, and the call for `a Haskell debugger' was louder than ever. Those of us working on tracing/debugging tools should be aiming to deliver something in time for the Haskell workshop 2001. It was useful to try several different approaches, but we should also collaborate. There have been many past attempts to develop a functional tracer/debugger but most have ended up as bit-rotted tar files before they were ever widely used. Now is the time to stop the rot! To understand the strengths and weaknesses of a tracer it would be useful to have something like the benchmark suites used by compiler writers to measure their success. In the context of tracing what seems to be needed is a collection of ten or twenty benchmark programs, of various sizes and styles, and for each of these: (1) a clear source of the working program; (2) a set of specific questions about how the program works that are not easily answered just by looking at the source, but which one would hope could be answered by using a tracer; (3) a broken version of the program that still compiles despite the deliberate introduction of bugs; (4) a set of questions about this program (though one could make do with the implicit question `Where are the bugs?', to be answered without direct comparison of the broken and unbroken sources). We should aim to include a preliminary set of such tracing benchmarks in the next release of nhc+hat. To avoid (the charge of) bias the origins of the benchmark programs should be diverse. Finally, besides the obvious practical motivation of a Haskell Debugger, there are interesting questions about traces from the academic point of view. These include: (1) the relative power or information content of different types of trace; (For example, can EDTs be extracted from redex trails?) (2) the design space for trace representations, and for query or programming languages over traces as data structures; (For example, what could the interface to archived redex trails be?) (3) how to evaluate the effect of tracers in terms of an explicit model of cognitive processes in the programmer, or repeatable laboratory experiments examining the success of trace users. INDIVIDUAL DISCUSSIONS As well as leaving piles of ART/Hat Bulletin 1 on the tables of handouts for conference people, we handed some out individually and tried to encourage potential users. In my opinion, we don't need or want a *lot* of users: if we had, say, 100 active users we might spend too much time trying to respond to their requests. However, we do need *some* users: (1) to find and report problems; (2) to make suggestions about what would improve tracing in practice; (3) to provide application-driven pull; (4) to become enthusiasts (hopefully!) who will promote wider use and justify further development of tracers like Hat. The immediate requests to interested parties are: (1) make contact: let us know they are there and interested; (2) try the tracer: give us some kind of feedback; (3) contribute to the Tracing Benchmarks.