Department of Computer Science

Testing Robotic Systems: A New Battlefield!

Arnaud Gotlieb

Simula Research Laboratory


Industrial robotics is a field which evolves very fast, with ever growing needs in terms of safety, performance, robustness and reliability. Nowadays, industrial robots are communicating cyber-physical systems which embeds complex distributed multi-core software-systems involving intelligent motion control, anti-collision, advanced force/torque control [1]. This increased complexity makes these robots more fragile and more error-prone that they were previously. Failures can originate from many sources including system and software bugs, communication downtime, CPU overload, robots wear, etc. Hopefully, advanced verification techniques such as constraint-based testing and validation intelligence are employed to cope with specification and development errors and ensure a better quality of delivered robots.

My talk will address the challenges of testing industrial serial robots and will review examples where Artificial Intelligence techniques have been used to ease the automation of some parts of the testing processes. In particular, the usage of Constraint Programming in the automatic generation of test scenarios for an integrated painting system will be presented [2], as well as the deployment of this technology into the real-world continuous integration process of a large robot manufacturing company [3]. Test case selection, scheduling and prioritization is another example where advanced intelligent method based on Reinforcement Learning has been deployed [4]. Finally, the talk will present a recent work where constraint optimization and learning can be combined in order to stress test industrial robots [5]. Obviously, these test optimization methods are meant to be complementary to other strong formal verification techniques such as model-checking and theorem-proving, and only the combination of these methods will lead to an industrial manufacturing world where robots are safer and more reliable.


[1] A. Grau, M. Indri, L. L. Bello, and T. Sauter Industrial robotics in factory automation: From the early stage to the Internet of Things. Proc. Of the 43rd Annual Conference of the IEEE Industrial Electronics Society (IECON 2017), Beijing, China, 2017

[2] Morten Mossige, Arnaud Gotlieb, and Hein Meling. Generating tests for robotized painting using constraint programming. In Int. Joint Conf. on Artificial Intelligence (IJCAI-16) - Sister Conference Best Paper Track, New York City, Jul. 2016.

[3] Morten Mossige, Arnaud Gotlieb, and Hein Meling. Testing robot controllers using constraint programming and continuous integration. Information and Software Technology, Vol. 57, Jan. 2015.

[4] Helge Spieker, Arnaud Gotlieb, Dusica Marijan, and Morten Mossige. Reinforcement learning for automatic test case prioritization and selection in continuous integration. In Proc. of 26th Int. Symp. on Software Testing and Analysis (ISSTA’17), Santa Barbara, CA, USA, Jul. 2017.

[5] Mathieu Collet, Arnaud Gotlieb, Nadjib Lazaar, Morten Mossige: Stress Testing of Single-Arm Robots Through Constraint-Based Generation of Continuous Trajectories. 1 st IEEE Conference on Artificial Intelligence Testing (AITest 2019), San Francisco, USA, Apr. 2019, pp 121-128

Department of Computer Science
Deramore Lane, University of York, Heslington,
York, YO10 5GH, UK
Tel: 01904 325500