ABZ call for papers on the POSIX pilot project in the Grand Challenge

Introduction and motivation

Recent advances in theory and tool support have inspired industrial and academic researchers to join up in an international Grand Challenge (GC) in Verified Software, and have created a sense of urgency in making its objectives a reality. The main motto of the GC is that

Work has started with the creation of a Verified Software Repository (VSR) with two principal aims: The GC work is funded by EPSRC in the UK, and NSF in the USA. Work is also underway in China, Brazil, and other European countries.

The first VSR pilot project experiment took place during 2006. This involved the successful mechanisation of the verification of a commercially sanitised version of the first project to meet the high-assurance standard, ITSEC Level 6: the Mondex smart-card banking application. The original formal specification and design are publicly available, and the mechanisation was accomplished by seven groups using different theories and tools. Their work is being recorded in a special issue of the Formal Aspects of Computing Journal. Further information on VSR can be found here.

In the first conference on Verified Software: Theories, Tools, and Experiments, held in Zurich in 2005, Joshi and Holzmann suggested a new pilot project for the grand challenge: the specification and verification of the POSIX file-store interface of Unix. It involves a small subset of POSIX (which is detailed below) suitable for flash-memory hardware with strict fault-tolerant requirements to be used by forthcoming NASA's Jet Propulsion Laboratory missions. Obviously, we do not stop there: following the GC's motto, we see this as an important research opportunity far beyond JPL's original pilot project.

The challenge proposed for participants at the ABZ 2008 Conference is for them to make a significant contribution to the file-store pilot project. Is the mechanical verification of a practical system within the state of the art? Can high levels of automation be achieved? Can useful guarantees be given about the dependability of the system across different hardware platforms? How do different theories, tools, and technique compare?

A series of informal development workshops will be organised leading up to the ABZ 2008 Conference for those who are going to work on the problem. This is inspired by what we did on the Mondex project, where the workshops gave useful deadlines for people to make progress and to interact. It will also give an opportunity for groups to parcel out the work, so that we might proceed collaboratively rather than competitively. It may even be the case that groups might be able to agree on a common notation to work in.

Project overview

Work has started on this project, following an architecture set out by Intel (see documentation here). This architecture explains how to separate and then orthogonally combine the following aspects of file systems:

The POSIX interface was chosen for the following reasons: This architecture is particularly useful for NAND flash-memory devices, the technology JPL intend to use. It is also amenable to generalisation to other types of hardware, as well as varied fault-tolerance concerns. As a scientific (rather than commercial) challenge, this is important in order to move away from NASA's specific hardware and fault-tolerant requirements. Thus, following this path still keeps us to the Grand Challenge's ideal of scientific purity, generality, and accuracy.

Functional requirements

The functional requirements specification for POSIX file systems are hosted by the Open Group (see the catalogue of documentation), and was agreed as a standard for Unix implementations in 2003. Before this, an early attempt at producing this requirements document used the Z Notation to straighten out unresolved ambiguities (see the resulting Z document). The Open Group requirements document was originally inspired by available Unix file systems formal specifications, such as Morgan and Sufrin's paper on the Unix filing system.

The VSR group at the University of York is working on the functional specification, following an initial subset of the POSIX API discussed with JPL. This minimal POSIX interface focuses on the basic functionality of files and directories, where we want to support at least the following API:

This API commands are documented in the POSIX standard. We agreed with JPL not to provide initial support for: Popular flash-memory file-systems, such as YAFFS2, do not support these features, since they are not typically needed for the embedded environments in which flash devices usually reside. But this should not stop interested researchers tackling these problems: we are just prioritising the work, which must eventually be tackled.

We also agreed with JPL to opt for a very conservative guarantee about concurrent behaviour: that the result of executing concurrent operations is equivalent to executing them in some serial order. This guarantee entails a performance cost, which is being traded for simplicity. But this may be too strict a requirement for a general embedded file-system, so this issue remains open in the context of the grand challenge, and so presents yet another research opportunity to those who want to get involved.

The results obtained between May 2006 and May 2007 are:

Some publications will appear in ICECCS 2007 in Auckland, New Zealand.

Hardware requirements

Although a general Intel architecture integrating the file system functionality with varied hardware is possible, we again turn our attention to one particular target: NAND flash-memory. This type of hardware has been standardised quite recently by ONFI. In collaboration between Trinity College Dublin and the University of York, we have started formalising ONFI's standard.

Fault tolerance issues

Due to the nature of the environment this small functional subset would run in, two important robustness requirements for fault-tolerance were later agreed with JPL:

In recovery from power loss in particular, they require the file system to be reset-reliable in the following sense: if an operation Op is in progress at the time of a power loss, then on reboot, the file system state will be as if Op either has successfully completed or has never started. Work on this front is yet to be done, and contributions from the community are very welcome.

Publication

It is expected that each group working on the problem will submit a paper to ABZ 2008 in the normal way. They will also be asked to deposit their work in the Verified Software Repository, and to submit a longer, more technical paper to a special issue of a leading journal.

Other ongoing pilot projects

There are a series of other pilot projects beside POSIX. We list further information about them in the list below.

Contact details

Dr. Leo Freitas or Prof. Jim Woodcock
Department of Computer Science,
University of York, YO10 5DD, UK




Last updated 1 June 2007, by Leo Freitas
Back to ABZ main page