Research in the CVPR group

Our research spans a wide range of topics, from theoretical aspects of pattern recognition to the practical application of computer vision. The overall philosophy of the group is to bring the objective principles of pattern recognition to the design of robust and effective algorithms for machine vision.

Some of fundamental questions being asked are:

The mathematical framework for this is provided by information theory (especially Bayesian methods), statistical physics and optimisation theory. Vision tasks under study include face recognition and modelling, polarisation imaging, reflectance modelling, diffusion tensor imaging, shape-from-shading and stereo. We also carry out research on relational and graph descriptions of patterns, including matching, partitioning, embedding, clustering and generation.

A sample of current research topics

Inverse rendering

In order to explain why an image appears the way it does, one can seek a physical explanation. i.e. to decompose an image into geometry, reflectance and illumination. This disentangles extrinsic scene properties from the intrinsic features of objects, potentially easing tasks such as recognition. In addition, it provides a route to physically-based image editing such as relighting or viewpoint changes. This task is ill-posed and so we have approached it from a number of different perspectives. Restricting consideration only to faces allows statistical models of face shape and reflectance properties to be used to constrain the problem. More recently, we have tried to solve scene level inverse rendering using deep learning and self-supervision from multiview images and differentiable rendering.

  1. Y. Yu and W.A.P. Smith. InverseRenderNet: Learning single image inverse rendering. In Proc. CVPR, 2019.
  2. O. Aldrian and W.A.P. Smith. Inverse Rendering of Faces with a 3D Morphable Model. IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 35, Number 5, pp. 1080-1093, 2013.
  3. O. Aldrian and W.A.P. Smith, Inverse Rendering of Faces on a Cloudy Day. In Proc. ECCV, pages 201-214, 2012.
  4. W.A.P. Smith and E.R. Hancock. Estimating Facial Reflectance Properties using Shape-from-shading. International Journal of Computer Vision, Volume 86, Number 2-3, pp. 152-170, 2010.

Graphs and Networks

The modern world generates vast amounts of data on a daily basis, in a wide variety of forms. Some of this data is relational and can be best represented using graphs and networks. Examples include protein interactions, social networks, financial interactions and chemical structures. Understanding large amounts of data in this form is very challenging.

A social network

Our interest is in studying the properties of graphs and networks from the perspective of a data representation. We are interested in both the underlying theory and practical applications. We have published extensively in the following areas

Graph features from paths, walks and spectral graph theory Graph kernels The statistical modelling of graph datasets Graph complexity and entropy Graph Matching

  1. Ye C, Wilson RC, Comin C, F. Costa L and Hancock ER (2014), “Approximate von Neumann entropy for directed graphs”, Physical Review E., 5, 2014. Vol. 89(5) American Physical Society.
  2. Aziz F, Wilson RC and Hancock ER (2013), “Backtrackless Walks on a Graph”, IEEE Transactions on Neural Networks and Learning Systems. Vol. 24(6), pp. 977-989. IEEE Computational Intelligence Society.
  3. Xiao B, Hancock ER and Wilson RC (2009), “A generative model for graph matching and embedding”, Computer Vision and Image Understanding., 7, 2009. Vol. 113(7), pp. 777-789. Academic Press Inc.
  4. Luo B, Wilson RC and Hancock ER (2006), “A spectral approach to learning structural variations in graphs”, Pattern Recognition., 6, 2006. Vol. 39(6), pp. 1188-1198. Elsevier Limited.
  5. Rocha J, Segura J, Wilson RichardC and Dasgupta S (2009), “Flexible structural protein alignment by a sequence of local transformations”, Bioinformatics., 7, 2009. Vol. 25(13), pp. 1625-1631.

Novel Vision Systems

We are also interested in novel imaging techniques such as shape-from-polarisation. We have showed how Fresnel theory can be used to relate the angle of reflection to the polarisation state of light in the diffuse reflection regime. This allows the recovery of surface normals from polarisation images of objects, although the directions are still ambiguous. We have used photometric stereo and polarisation on our light stage to accurately capture both the shape and reflectance models of real objects such as human skin.

Plenoptic imaging

In order to overcome atmospheric distortions in long range imagery, we have used a plenoptic camera system and computational imaging. The aim here is to recover higher resolution imagery undistorted by turbulence from the atmosphere.

  1. Tozza S, Smith WAP, Zhu D, Ramamoorthi R and Hancock ER (2017), “Linear Differential Constraints for Photo-polarimetric Height Estimation”, In 2017 IEEE International Conference on Computer Vision (ICCV)., 7, 2017. IEEE Computer Society Press.
  2. Huynh Cong Phuoc, Robles-Kelly A and Hancock ER (2013), “Shape and Refractive Index from Single-View Spectro-Polarimetric Images”, International Journal of Computer Vision., 1, 2013. Vol. 101(1), pp. 64. Springer Netherlands.
  3. Saman G and Hancock ER (2011), “Refractive Index Estimation Using Polarisation and Photometric Stereo”, In Computer Analysis of Images and Patterns - 14th International Conference, CAIP 2011, Seville, Spain, August 29-31, 2011, Proceedings, Part II. Vol. 6855, pp. 548-554. SPRINGER-VERLAG BERLIN.

Quantum Algorithms for Graphs

We study the use of quantum algorithms, including quantum information and quantum random walks, for extracting information about graphs. We have developed methods of measuring the similarity of graphs with quantum walks, and used quantum information create graph kernels. We have exploited the entropy of quantum systems to quantify the complexity of graphs. We have also explored the use of quantum representations of graphs and of images.

  1. Rossi L, Torsello A and Hancock ER (2015), “Measuring graph similarity through continuous-time quantum walks and the quantum Jensen-Shannon divergence”, Physical Review E., 2, 2015. Vol. 91(2) American Physical Society.
  2. Zhang Y, Lu K, Xu K, Gao Y and Wilson RC (2015), “Local feature point extraction for quantum images”, Quantum Information Processing. Springer New York.
  3. Bai L, Rossi L, Torsello A and Hancock ER (2015), “A quantum Jensen-Shannon graph kernel for unattributed graphs”, Pattern Recognition., 2, 2015. Vol. 48(2), pp. 344-355. Elsevier Limited.
  4. Rossi L, Torsello A, Hancock ER and Wilson RC (2014), “Characterizing graph symmetries through quantum Jensen-Shannon divergence”, Phys. Rev. E.
  5. Ren P, Aleksic T, Emms D, Wilson RC and Hancock ER (2011), “Quantum walks, Ihara zeta functions and cospectrality in regular graphs”, Quantum Information Processing., 6, 2011. Vol. 10(3), pp. 405-417. Springer New York.
  6. Emms D, Wilson RC and Hancock ER (2009), “Graph matching using the interference of continuous-time quantum walks”, Pattern Recognition., 5, 2009. Vol. 42(5), pp. 985-1002. Elsevier Limited.
  7. Emms D, Severini S, Wilson RichardC and Hancock EdwinR (2009), “Coined quantum walks lift the cospectrality of graphs and trees”, Pattern Recognition., 9, 2009. Vol. 42(9), pp. 1988-2002. Elsevier Limited.
  8. Emms D, Wilson RC and Hancock ER (2009), “Graph matching using the interference of discrete-time quantum walks”, Image and Vision Computing., 6, 2009. Vol. 27(7), pp. 934-949. Elsevier Limited.