admin/git
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Sane use of test_expect_failure

Browse Source 
Originally, test_expect_failure was designed to be the opposite
of test_expect_success, but this was a bad decision.  Most tests
run a series of commands that leads to the single command that
needs to be tested, like this:

    test_expect_{success,failure} 'test title' '
	setup1 &&
        setup2 &&
        setup3 &&
        what is to be tested
    '

And expecting a failure exit from the whole sequence misses the
point of writing tests.  Your setup$N that are supposed to
succeed may have failed without even reaching what you are
trying to test.  The only valid use of test_expect_failure is to
check a trivial single command that is expected to fail, which
is a minority in tests of Porcelain-ish commands.

This large-ish patch rewrites all uses of test_expect_failure to
use test_expect_success and rewrites the condition of what is
tested, like this:

    test_expect_success 'test title' '
	setup1 &&
        setup2 &&
        setup3 &&
        ! this command should fail
    '

test_expect_failure is redefined to serve as a reminder that
that test *should* succeed but due to a known breakage in git it
currently does not pass.  So if git-foo command should create a
file 'bar' but you discovered a bug that it doesn't, you can
write a test like this:

    test_expect_failure 'git-foo should create bar' '
        rm -f bar &&
        git foo &&
        test -f bar
    '

This construct acts similar to test_expect_success, but instead
of reporting "ok/FAIL" like test_expect_success does, the
outcome is reported as "FIXED/still broken".

Signed-off-by: Junio C Hamano <gitster@pobox.com>
This commit is contained in:
Junio C Hamano
2008-02-01 01:50:53 -08:00
parent 6ce8e44a1e
commit 41ac414ea2
48 changed files with 496 additions and 425 deletions
Download Patch File Download Diff File Expand all files Collapse all files

30
t/t0000-basic.sh
View File

@@ -46,13 +46,25 @@ test_expect_success \
'.git/objects should have 3 subdirectories.' \
'test $(wc -l < full-of-directories) = 3'
################################################################
# Test harness
test_expect_success 'success is reported like this' '
:
'
test_expect_failure 'pretend we have a known breakage' '
false
'
test_expect_failure 'pretend we have fixed a known breakage' '
:
'
################################################################
# Basics of the basics
# updating a new file without --add should fail.
test_expect_failure \
'git update-index without --add should fail adding.' \
'git update-index should-be-empty'
test_expect_success 'git update-index without --add should fail adding.' '
! git update-index should-be-empty
'
# and with --add it should succeed, even if it is empty (it used to fail).
test_expect_success \
@@ -70,9 +82,9 @@ test_expect_success \
# Removing paths.
rm -f should-be-empty full-of-directories
test_expect_failure \
'git update-index without --remove should fail removing.' \
'git update-index should-be-empty'
test_expect_success 'git update-index without --remove should fail removing.' '
! git update-index should-be-empty
'
test_expect_success \
'git update-index with --remove should be able to remove.' \
@@ -204,9 +216,9 @@ test_expect_success \
'put invalid objects into the index.' \
'git update-index --index-info < badobjects'
test_expect_failure \
'writing this tree without --missing-ok.' \
'git write-tree'
test_expect_success 'writing this tree without --missing-ok.' '
! git write-tree
'
test_expect_success \
'writing this tree with --missing-ok.' \