Describing Console I/O Behavior for Testing Student Submissions in Haskell

Oliver Westphal
Janis Voigtländer

We present a small, formal language for specifying the behavior of simple console I/O programs. The design is driven by the concrete application case of testing interactive Haskell programs written by students. Specifications are structurally similar to lexical analysis regular expressions, but are augmented with features like global variables that track state and history of program runs, enabling expression of an interesting range of dynamic behavior. We give a semantics for our specification language based on acceptance of execution traces. From this semantics we derive a definition of the set of all traces valid for a given specification. Sampling that set enables us to mechanically check program behavior against specifications in a probabilistic fashion. Beyond testing, other possible uses of the specification language in an education context include related activities like providing more helpful feedback, generating sample solutions, and even generating random exercise tasks.

In Jurriaan Hage: Proceedings Eighth and Ninth International Workshop on Trends in Functional Programming in Education (TFPIE 2019 and 2020), Vancouver, Canada and Krakow, Poland, 11th June 2019 and 12th February 2020, Electronic Proceedings in Theoretical Computer Science 321, pp. 19–36.
Published: 24th August 2020.

ArXived at: bibtex PDF
References in reconstructed bibtex, XML and HTML format (approximated).
Comments and questions to:
For website issues: