Implementation of CMM Based Systems

The implementation of CMM based methods that underpin the AURA based systems has been a major research topic for some time within the group. The implementation covers both the software implementation and the hardware implementation.

CMM Software

CMM software consists of the ADAM software library, which is free to use and available to down load. The new software is the AURA C++ library, an on going set of functions to build AURA based systems. This is available to research users for free subject to a license and can be bought through Cybula Ltd. for commercial organisations.

Early work on the ADAM library is reported in [42, 50] where a visual programming environment for associative memories is discussed.

CMM Hardware

The CMM hardware has been developed for some time. The early systems consisted of simple ADAM based systems. We now have a dedicated parallel CMM machine, Cortex-1, for many applications.

ADAM

Initial work on implementation of neural networks evaluated some simple ideas. In [18, 38, 44] the issue of building very large neural networks was investigated, highlighting the need for these types of systems for data analysis. Parallel implementation has always been a theme in the work. [16] looked at how ADAM could be implemented on torroidal mesh networks. This was linked to a simple parallel processor based on 68000 processors [17] and was used as an example for load balancing research in a PhD by G Macharia [t1]. A novel parallel implementation in an early form of FPGA logic was also described in [14] which suggested the compilation of the CMM into a smaller, compact array. Implementation of ADAM on a Transputer array was given in [71] and on a message passing parallel machine in [83], both showed how ADAM could be implemented most efficiently on such systems. Other work on parallel implementation of ADAM was also given in [98].

C-NNAP

The first work on hardware CMMs surrounded a simple ADAM card, which was implemented in a system called the cellular neural network associative processor, C-NNAP. This is described in an internal report [58], some papers [67, 75, 77, 81] and a full paper in an IEEE monograph [ 70]. Application of the system in vision was discussed in [68]. The research showed that it was generally cost effective to build hardware versions of CMMs. One of the major benefits being the ability to implement training on the hardware, difficult to achieve in weighted networks using Back Propagation learning.

VME based PRESENCE implementation of CMMs

The work on a dedicated VME bus based CMM card was first reported in 51, 63], a system developed in collaboration with British Aerospace. The card, still high performance by today's standard, allowed 16Mbyte of CMM memory and allowed parallel evaluation of the CMM (128 bit parallelism). This work was highly successful and obtained a write up in the EPSRC news letter IMPACT [scanned image of the article]. Its use in image processing was discussed in [86].

PCI based PRESENCE implementation of CMMs

After verifying that very high performance implementations of CMMs could be achieved on a VME bus [105], our attention turned to cost effective massively parallel implementation. The best PCI bus was chosen for best cost:performance. In the AURAII grant a PCI based card was developed, which used dynamic RAM rather than static offering us 128Mbyte memory rather than 20Mbyte at a much lower cost. The parallel implementation was based upon FPGA devices, and developed by John Kennedy in a PhD thesis within the AURA I project [t13]. The design of the card [110] demonstrated that massive parallelism could be achieved with such systems. The last paper to call the architecture C-NNAP was [105] since then the system has been called Cortex-1. Details of the PCI card can be found in [125, 142]. Many applications of the hardware are described in the applications pages of this site, one benchmark application has been a k-NN classifier build using the hardware and a software implementation [135] which illustrates good speup for that application.

VLSI implementations of CMMs

Some investigations of VLSI implementation have been looked at [134] but with no firm conclusions as to the efficiency of these implementations.

The Cortex-1 neural associative parallel processor

The latest work within the AURA II research project has been the development of a parallel processor based on the PCI PRESENCE cards. This system is built to undertake large data problems such as the address matcher, text database, trade mark database and computer vision problems. It is a general purpose associative parallel processor. The system is based around 28 PRESENCE cards, each with 128Mbyte of memory. These are hosted in 7 PC nodes each with 768 Mbyte main memory. The system uses conventional ethernet.