Run automatically without any human participation beyond launching the test suite (and maybe not even that)
Results should be pass/fail
Test failures need to be blindingly obvious
Test must be reproducible and consistent