<  Back to the Polytechnique Montréal portal

Automated performance deviation detection across software versions releases

Abderrahmane Benbachir, Isnaldo Francisco De Melo, Michel R. Dagenais and Bram Adams

Conference or Workshop Item - Paper (2017)

Accepted Version
Terms of Use: All rights reserved.
Download (400kB)
Cite this document: Benbachir, A., Melo, I. F. D., Dagenais, M. R. & Adams, B. (2017, July). Automated performance deviation detection across software versions releases. Paper presented at 2017 IEEE International Conference on Software Quality, Reliability and Security (QRS), Prague, Czech Republic (8 pages). doi:10.1109/qrs.2017.55
Show abstract Hide abstract


Performance is an important aspect and critical requirement in multi-process software architecture systems such as Google Chrome. While interacting closely with members of the Google Chrome engineering team, we observed that they face a major challenge in detecting performance deviations between releases, because of their very high release frequency and therefore limited amount of data on each. This paper describes a deep analysis on the data distributions followed by a comparative approach using median based confidence interval for software evaluation. This technique is capable of detecting performance related deviations. It is substantially different from the standard confidence interval, in that it can be used in the presence of outliers and random external influences since the median is less influenced by them. We conducted a bottom-up analysis, using stack traces in a very large pool of releases. The results show that our approach can accurately localize performance deviations at a function-level granularity, using a very small number of trace samples, nearby 5 runs.

Uncontrolled Keywords

Tracing, Performance, Deviation, Quality, Confidence Interval

Open Access document in PolyPublie
Subjects: 2700 Technologie de l'information > 2700 Technologie de l'information
2700 Technologie de l'information > 2705 Logiciels et développement
2700 Technologie de l'information > 2720 Logiciel de systèmes informatiques
Department: Département de génie informatique et génie logiciel
Research Center: Non applicable
Grant number: CRDPJ468687-14
Date Deposited: 12 Feb 2018 16:35
Last Modified: 24 Oct 2018 16:12
PolyPublie URL: https://publications.polymtl.ca/2977/


Total downloads

Downloads per month in the last year

Origin of downloads


Repository Staff Only