Frank Brüseke and Henning Wachsmuth and Gregor Engels and Steffen Becker: PBlaman: Performance Blame Analysis based on Palladio Contracts. In Concurrency and Computation Practice and Experience, vol. 26, no. 12, pp. 1975--2004 (2014)
Show Bibtex | Show Abstract | DOI

@article{brueseke2013b, author = {Frank Brüseke and Henning Wachsmuth and Gregor Engels and Steffen Becker}, title = {PBlaman: Performance Blame Analysis based on Palladio Contracts}, journal = {Concurrency and Computation Practice and Experience}, year = {2014}, volume = {26}, number = {12}, pages = {1975--2004} }

In performance-driven software engineering, the performance of a system is evaluated through models before the system is assembled. After assembly, the performance is then validated using performance tests. When a component-based system fails certain performance requirements during the tests, it is important to find out whether individual components yield performance errors or whether the composition of components is faulty. This task is called performance blame analysis. Existing performance blame analysis approaches and also alternative error analysis approaches are restricted, because they either do not employ expected values, use expected values from regression testing, or use static developer-set limits. In contrast, this paper describes the new performance blame analysis approach PBlaman that builds upon our previous work and that employs the context-portable performance contracts of Palladio. PBlaman decides what components to blame by comparing the observed response time data series of each single component operation in a failed test case to the operation's expected response time data series derived from the contracts. System architects are then assisted by a visual presentation of the obtained analysis results. We exemplify the benefits of PBlaman in two case studies, each of which representing applications that follow a particular architectural style.

Frank Brüseke and Steffen Becker and Gregor Engels: Decision Support via Automated Metric Comparison for the Palladio-based Performance Blame Analysis. In Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013), Prague (Czech Republic). ACM New York, NY, USA, pp. 77--88 (2013) ICPE '13 Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering
Show Bibtex | Show Abstract | URL | DOI

@inproceedings{bruBecEng2013-1, author = {Frank Brüseke and Steffen Becker and Gregor Engels}, title = {Decision Support via Automated Metric Comparison for the Palladio-based Performance Blame Analysis}, booktitle = {Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering (ICPE 2013), Prague (Czech Republic)}, year = {2013}, pages = {77--88}, publisher = {ACM New York, NY, USA}, note = {ICPE '13 Proceedings of the 4th ACM/SPEC International Conference on Performance Engineering } }

When developing component-based systems, we incorporate third-party black-box components. For each component, performance contracts have been specified by their developers. If errors occur within the system built from these components, it is very important to find out whether components violate their performance contracts or whether the composition itself is faulty. This task is called performance blame analysis. In our previous work we presented a performance blame analysis approach that blames components based on a comparison of response time values from testing to values derived from the performance contract. In that approach, the system architect needs to manually assess if the test data series shows faster or slower response times than the data derived from the contract. This is laborious as the system architect has to do this for each component operation. In this paper we present an automated comparison of each pair of data series as decision support. In contrast to our work, other approaches do not achieve fully automated decision support, because they do not incorporate sophisticated contracts. We exemplify our performance blame analysis including the automated decision support using the "Common Component Modeling Example" (CoCoME) benchmark.

Frank Brüseke and Yavuz Sancar and Enes Yigitbas: Erfolgsfaktoren von Testprozessbewertungsmodellen. In Proceedings of the Conference on Software & Systems Engineering Essentials 2010 (SEE 2010), Köln (Germany). Technische Universität München, pp. 277-298 (2010)
Show Bibtex | URL

@inproceedings{SEE10Erf, author = {Frank Brüseke and Yavuz Sancar and Enes Yigitbas}, title = {Erfolgsfaktoren von Testprozessbewertungsmodellen}, booktitle = {Proceedings of the Conference on Software & Systems Engineering Essentials 2010 (SEE 2010), Köln (Germany)}, year = {2010}, pages = {277-298}, publisher = {Technische Universität München} }

Frank Brüseke and Steffen Becker and Gregor Engels: Palladio-based performance blame analysis. In Proceedings of the 16th International Workshop on Component-Oriented Programming (WCOP; satellite event of the CompArch 2011), Boulder Colorado, CO (USA). ACM (New York, NY (USA)), pp. 25-32 (2011)
Show Bibtex | Show Abstract | DOI

@inproceedings{brueseke2011a, author = {Frank Brüseke and Steffen Becker and Gregor Engels}, title = {Palladio-based performance blame analysis}, booktitle = {Proceedings of the 16th International Workshop on Component-Oriented Programming (WCOP; satellite event of the CompArch 2011), Boulder Colorado, CO (USA)}, year = {2011}, pages = {25-32}, address = {New York, NY (USA)}, publisher = {ACM} }

Performance is an important quality attribute for business information systems. When a tester has spotted a performance error, the error is passed to the software developers to fix it. However, in component-based software development the tester has to do blame analysis first, i. e. the tester has to decide, which party is responsible to fix the error. If the error is a design or deployment issue, it can be assigned to the software architect or the system deployer. If the error is specific to a component, it needs to be assigned to the corresponding component developer. An accurate blame analysis is important, because wrong assignments of errors will cause a loss of time and money. Our approach aims at doing blame analysis for performance errors by comparing performance metrics obtained in performance testing and performance prediction. We use performance prediction values as expected values for individual components. For performance prediction we use the Palladio approach. By this means, our approach evaluates each component’s performance in a certain test case. If the component performs poorly, its component developer needs to fix the component or the architect replaces the component with a faster one. If no omponent performs poorly, we can deduce that there is a design or deployment issue and the architecture needs to be changed. In this paper, we present an exemplary blame analysis based on a web shop system. The example shows the feasibility of our approach.

Frank Brüseke and Yavuz Sancar and Gregor Engels: Architecture-Driven Derivation of Performance Metrics. In Proceedings of Software-Qualitätsmodellierung und -bewertung (SQMB '10), Paderborn, Germany. Technische Universität München (München, Germany), pp. 22-31 (2010)
Show Bibtex | URL

@inproceedings{bruSanEng2010_sqmb, author = {Frank Brüseke and Yavuz Sancar and Gregor Engels}, title = {Architecture-Driven Derivation of Performance Metrics}, booktitle = {Proceedings of Software-Qualitätsmodellierung und -bewertung (SQMB '10), Paderborn, Germany}, year = {2010}, pages = {22-31}, address = {München, Germany}, publisher = {Technische Universität München} }

Yavuz Sancar and Frank Brüseke and Gregor Engels: Teststufenspezifische Qualitätsattribute für die Qualitätsbewertung von nichtfunktionalen Anforderungen. In Proceedings of Software-Qualitätsmodellierung und -bewertung (SQMB '10), Paderborn, Germany. Technische Universität München (München, Germany), pp. 50-57 (2010)
Show Bibtex | URL

@inproceedings{SQMB10TQ, author = {Yavuz Sancar and Frank Brüseke and Gregor Engels}, title = {Teststufenspezifische Qualitätsattribute für die Qualitätsbewertung von nichtfunktionalen Anforderungen}, booktitle = {Proceedings of Software-Qualitätsmodellierung und -bewertung (SQMB '10), Paderborn, Germany}, year = {2010}, pages = {50-57}, address = {München, Germany}, publisher = {Technische Universität München} }

Yavuz Sancar and Frank Brüseke and Hendrik Voigt and Stefan Sauer and Gregor Engels: Towards Economical Software Release Recommendations. In ECOOP 2010 - Workshop on Testing Object-Oriented Software Systems (ETOOS). , pp. 59-67 (2010)
Show Bibtex

@inproceedings{Etoos2010, author = {Yavuz Sancar and Frank Brüseke and Hendrik Voigt and Stefan Sauer and Gregor Engels}, title = {Towards Economical Software Release Recommendations}, booktitle = {ECOOP 2010 - Workshop on Testing Object-Oriented Software Systems (ETOOS)}, year = {2010}, pages = {59-67}, month = {June} }

Frank Brüseke and Fabian Christ and Stefan Sauer and Andreas Wübbeke: Testen von Software-Produktlinien. s-lab Report , no. 4. University of Paderborn, Software Quality Lab (s-lab) (2011) (t.a.)
Show Bibtex

@techreport{bcsw2010, author = {Frank Brüseke and Fabian Christ and Stefan Sauer and Andreas Wübbeke}, title = {Testen von Software-Produktlinien}, institution = {University of Paderborn, Software Quality Lab (s-lab)}, year = {2011}, type = {s-lab Report }, number = {4}, note = {(t.a.)} }