Double cycle hybrid testing of hybrid distributed IoT system

Testing heterogeneous IoT applications such as a home automation systems integrating a variety of devices poses serious challenges. Oftentimes requirements are vaguely defined. Consumer grade cyber-physical devices and software may not meet the reliability and quality standard needed. Plus, system behavior may partially depend on various environmental conditions. For example, WI-FI congestion may cause packet delay; meanwhile cold weather may cause an unexpected drop of inside temperature. We surmise that generating and executing failure exposing scenarios is especially challenging. Modeling phenomenons such as network traffic or weather conditions is complex. One possible solution is to rely on machine learning models approximating the reality. These models, integrated in a system model, can be used to define surrogate models and fitness functions to steer the search in the direction of failure inducing scenarios. However, these models also should be validated. Therefore, there should be a double loop co-evolution between machine learned surrogate models functions and fitness functions. Overall, we argue that in such complex cyber-physical systems, co-evolution and multi-hybrid approaches are needed.