Agile Development & Companies
• Harvard Business Review: “Why the
lean start-up changes everything”
• Agile development applied to
entire business of start-ups
Iterate!
Image: leadnet.org
Simplest
possible
prototype!
Case Study:
Case Study #1
Background
• My PhD thesis: new CAD System for
FPGAs
• Formed company to commercialize
– 4 people initially
• An early customer was Altera
– 10 months to produce a new “placement and
routing” system for Altera’s chips
– Aggressive goal: 10X better than current
system
3
Place and Route Example
4
Project Management
• Quick plan
– 1 day: estimate all tasks & time
– Add time margin (mine: 50%)
• Concrete milestones
– Meet a milestone every 3 months or don’t get
paid!
• Scared
– Turned out to be helpful!
– Avoid any side projects, unnecessary complexity
• Get something working, measure & improve
– Measure our software vs. Altera’s every day
5
Outcome
• Hit all milestones, except first (2 weeks late)
– Focused!
• CAD system exceeded expectations
– 30X less runtime
– 38% faster circuits
– Altera replaced their P & R engine with the
prototype
• Started simple, measured where to improve
– Some simple algorithms still in current Quartus II 
didn’t need more!
6
Case Study 2:
(First Version of) Quartus
Background
• Altera had highly successful CAD system:
Max+PlusII
• Decided to do a complete re-write to a
new CAD system
– Quartus
• Goals:
– C++ (not C)
– Cleaner, more extensible code
– Allow multiple engineers to collaborate on a
project easily
– Allow fast, “incremental” recompiles
8
Project Management
• Planned for a year with no coding
– Too much waterfall  planning paralysis
• Let complexity reign
– Fancy user features
– Fancy language (object oriented C++ to the max)
• No working prototype for ~2 years
– Can’t test new features in full system
• Didn’t understand & measure key metric
– CAD system that optimizes well
– And is ready for next-generation chip
9
Outcome
• Software not ready in time for chip
• Software rushed to market
– Not stable enough, didn’t optimize well
enough
– Very bad customer experiences
• Lost sales: $billions!
• Rewrote & renamed later versions:
– Now very good!
10
Case Study 3:
Hardware + Software (Stratix V)
Big Project!
+
> $200 M
State of the art
for 2 years!
• More planning?
–
–
–
–
–
3 year project
~400 engineers at peak
Yes, plan as far ahead as you can
But don’t paralyze yourself
Accept that plan gets coarse as you go out in time
• Key plan: measurable milestones
–
–
–
–
12
3 year project  need to break up schedule
Project milestones  goals / milestones per team
Clear must have features
Everything else: nice to have
Agile / Iterative Development?
• Build prototype / model of HW + SW system
– Use to evaluate and test new ideas
– Done by smaller team (~10 engineers)
– Will never ship to customers!
• Prototype shows good results?
–
–
–
–
Build real hardware & full software
Team grows to 400
Always keep system working as you improve
Always measure results vs. expected (from prototype)
• HW & SW both in source code control
• Still iterative, but with more structure
Communication?
• Weekly status meeting
1. For sub-teams (~8 engineers)
2. Managers discuss results for ~8 teams (~64 people)
3. Directors meet to discuss overall project status
• Meeting style
– wiki, crisp reporting
– Big picture  progress toward milestones
Outcome
• Iterative development of HW + SW
– Works much better than pure waterfall
– Many companies: out of business by not getting full HW
+ SW system working
• Every 2nd generation: usually late!
– Engineers & marketing put in too much complexity
– Not scared enough, not enough iterative development!
• Late to market is terrible
– Competitor has twice the transistors, 1 year earlier
– Want to optimize, but on time more important
Testing
Testing Philosophy
• “If you didn’t test it, it doesn’t work”
– Assume all code broken until proven
otherwise
– Look at what the program does not what it is
supposed to do
• Scientist: you are testing hypotheses about
program
– Think of every case your program should
handle  cover them all
Testing Time
• Develop tests as or before you code
– Test-driven development
Prototype
Refine
Test &
Evaluate
Test-Driven Development
• The tests are the detailed specification
– And now can be executed, not just read
More tests
Fewer specification documents
Types of Tests
End to End Tests
Unit Tests
End-to-end (system) tests
• Tests like an end user
• Hard to debug a failure  problem could be
anywhere!
• Fragile: often test some exact user input would
produce some exact output text / file
Unit Tests
• Test small pieces of system (single classes / functions)
• Easier to debug failure  start with 1 class / function
• Don’t check user interface (I/O): check API / class
does what it should with code (testfixture)
Types of Tests
Fewer
(but not 0!)
End to End Tests
Integration Tests
Integration Tests
• Bigger unit tests
• But now test multiple classes / bigger
functionality
• Moderate difficulty to debug
Unit Tests
More
Testing Automation
• You will re-test often
– Every time you change the program
– How to speed up?
• System tests: save in files
– Use file redirection
• prog.exe < in.txt > out.txt
• diff out.txt out_good.txt // matched known good result?
– Less fragile
• Write validity checker instead of checking exact output match
• Write scripts to run all tests and check results
Unit Tests
• Want:
– To test individual classes / functions
– How?
• Carefully chosen user input?
 Output
...
No!
struct Point {
float x;
float y;
t_point& operator*=(float rhs);
. . .
Unit Tests
• Write new code
–
–
–
–
To put class in right state
To directly send it some input / make some calls
To immediately check the responses
Test driver
Unit Tests
myCode.cpp
struct Point {
float x;
float y;
t_point& operator*=(float rhs);
. . .
tester.cpp
int test1 () {
Point testme (1, 2);
testme *= 3;
if (testme != Point (3, 6)) {
cout << “uh – oh” << endl;
return (1);
}
return (0);
}
test_main.cpp
int main () {
int error = 0;
error += test1();
. . . // Lots more tests
g++ myCode.cpp main.cpp –o prog.exe
g++ myCode.cpp tester.cpp
test_main.cpp –o test.exe