Automated Program Debugging Research vs. Prac7ce? Franz Wotawa Technische Universität Graz Ins2tute for So7ware Technology Inffeldgasse 16b/2, 8010 Graz, Austria [email protected]
Some ques7ons asked for… •  How usefully can these techniques be transferred from laboratory seJngs to real-­‐life industry environments? •  Are automated debugging techniques actually helping prac22oners? •  Are the assump2ons adopted by research-­‐oriented debugging techniques valid in prac2ce? •  Do we have solid data from rigorous case studies to prove the feasibility and advantage of using exi2ng research prototypes? •  What are the gaps between the best-­‐of-­‐breed prac2ces in industry and the most advanced techniques proposed in academia? •  What are the limita2ons and challenges of current research in program debugging and what are the most urgent needs in prac2ce? IWPD 2014, Naples, Italy …(IN ADDITION) A SHORT PRESENTATION ON EXPERIENCES OBTAIN SO FAR… IWPD 2014, Naples, Italy … AND HERE COMES MY STORY IWPD 2014, Naples, Italy The beginnings… •  Debugging VHDL programs / designs: –  Friedrich, Gerhard and Stumptner, Markus and Wotawa, Franz, Model-­‐based diagnosis of hardware designs, Ar2ficial Intelligence 111(1-­‐2):3-­‐39, 1999. –  Franz Wotawa, On the Rela6onship between Model-­‐Based Debugging and Program Slicing, Ar2ficial Intelligence 135(1-­‐2):
124-­‐143, 2002. –  Franz Wotawa, Debugging Hardware Designs using a Value-­‐
Based Model (ar2cle), Applied Intelligence 16(1):71-­‐92, Kluwer Academic Publishers, 2002 –  Bernhard Peischl and Franz Wotawa, Automated Source Level Error Localiza6on in Hardware Designs, IEEE Design & Test of Computers, Jan-­‐Feb, 2006. IWPD 2014, Naples, Italy .. the 7me in the middle… •  Java debugging: –  Cris2nel Mateis and Markus Stumptner and Dominik Wieland and Franz Wotawa, Model-­‐Based Debugging of Java Programs, Proceedings of the 4th Interna2onal Workshop on Automated and Algorithmic Debugging, AADEBUG '00, 2000. –  Cris2nel Mateis and Markus Stumptner and Franz Wotawa, Modeling Java Programs for Diagnosis, Proceedings of the European Conference on Ar2ficial Intelligence (ECAI), IOS Press, 2000. –  Wolfgang Mayer and Markus Stumptner and Dominik Wieland and Franz Wotawa, Can AI help to improve debugging substan6ally? Debugging experiences with value-­‐based models, Proceedings of the European Conference on Ar2ficial Intelligence (ECAI), pp. 417-­‐421, IOS Press, 2002. IWPD 2014, Naples, Italy ..to some more recent work •  Switching to constraints: –  M. Nica and S.A. Nica and F. Wotawa, On the use of muta6ons and tes6ng for debugging, So7ware: Prac2ce & Experience, 2012. –  F. Wotawa and M. Nica and I.-­‐D Moraru, Automated debugging based on a constraint model of the program and a test case, The journal of logic and algebraic programming 81(4), 2012. •  And spreadsheets: –  Birgit Hofer and André Riboira and Franz Wotawa and Rui Abreu and Elisabeth Getzner, On the Empirical Evalua6on of Fault Localiza6on Techniques for Spreadsheets, Proceedings of the 16th Interna2onal Conference on Fundamental Approaches to So7ware Engineering (FASE 2013), 2013. IWPD 2014, Naples, Italy Characteris7cs of the approach •  Logic or constraint model of statements and their connec2ons with other statements •  Based on the seman2cs of the language and at least one failing test case •  Guarantees to find a sound solu2on (wrt the model) •  Connec2ons to various other techniques –  Slicing –  Spectrum-­‐based fault localiza2on IWPD 2014, Naples, Italy Debugging using constraints 1. begin
2. 
3. 
4. 
5. 
6. end;
Ab(2) ∨ i = 2 * x; Ab(3) ∨ j = 2 * y; Ab(4) ∨ o1 = i + j; Ab(5) ∨ o2 = i * i; i = 2 * x;
j = 2 * y;
o1 = i + j;
o2 = i * i;
x = 1, y = 2, o1 = 8, o2 = 4
Programm execu7on IWPD 2014, Naples, Italy x = 1 y = 2 o1 = 8 o2 = 4 Constraint solving / equa7on solving Finding bugs using constraints Ab(2) ∨ i = 2 * x; Ab(3) ∨ j = 2 * y; Ab(4) ∨ o1 = i + j; Ab(5) ∨ o2 = i * i; x = 1 y = 2 o1 = 8 o2 = 4 Ab(2) ∧ ¬Ab(3) ∧ ¬Ab(4) ∧ ¬Ab(5) j = 2 * 2 = 4 o1 = i + j = 8 = i + 4 → i = 4 o2 = 4 = i * i = 4 * 4 → FAIL!!!! ¬Ab(2) ∧ Ab(3) ∧ ¬Ab(4) ∧ ¬Ab(5) i = 2 * 1 = 2 o1 = 8 = 2 + j → j = 6 o2 = 4 = i * i = 2 * 2 And so on ... finally leading to 2 possible diagnoses statement 3 and statement 4 IWPD 2014, Naples, Italy Experimental results IWPD 2014, Naples, Italy Experimental results IWPD 2014, Naples, Italy Grand challenges •  What dis2nguishes research in (automated) debugging from debugging currently used in prac2ce? •  What are the difficul2es when trying to bring research into prac2ce? IWPD 2014, Naples, Italy Differences •  Research •  Prac7ce –  Simplifying assump2ons –  Do not consider programming environments –  Assume addi2onal knowledge (spec) –  Do not consider applica2on area (development vs. maintenance) –  External programs, libs, etc. –  Programmer knows expected behavior, structure, effects,… –  Needs advice & explana2ons –  Full automa2on would be fine (incl. correc2on) –  Par2al automa2on requires addi2onal effort IWPD 2014, Naples, Italy Example •  There are libraries and sub-­‐libraries etc. o7en used in prac2ce •  Huge amount of code to consider! •  General behavior might not be so difficult to handle •  Modeling of sub-­‐libraries necessary •  Abstrac2on needed •  O7en not really difficult to handle (from a theore2cal perspec2ve) IWPD 2014, Naples, Italy Another example •  Debugging approaches need to be adapted to –  Programming language –  Programming environment •  Both change over 2me! •  Difficult to adapt for basic research (because nothing really changes) •  Strong need for applied research and industry engagement! IWPD 2014, Naples, Italy So what now? •  There is a lot of work to do! •  Applied research dealing with specific applica2on domains •  Bringing together different methods •  Consider the programmer as part of the process!!! IWPD 2014, Naples, Italy THE JOURNEY JUST STARTS IWPD 2014, Naples, Italy