Upreckon: testcases.py annotate

annotate testcases.py @ 122:d6fd880207cb

Fixed memory-limit-exceeded false positives on Win32

author	Oleg Oshmyan <chortos@inbox.lv>
date	Sun, 24 Apr 2011 16:48:29 +0100
parents	218b8c28549c
children	42c8f5c152a5

rev	line source
77 69eadc60f4e2 Memory limit is now applied to the RSS when os.wait4 is available Oleg Oshmyan <chortos@inbox.lv> parents: 76 diff changeset	1 # Copyright (c) 2010-2011 Chortos-2 <chortos@inbox.lv>
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	2
43 81f58c938ec5 Fixed the -s option Oleg Oshmyan <chortos@inbox.lv> parents: 41 diff changeset	3 # TODO: copy the ansfile if not options.erase even if no validator is used
81f58c938ec5 Fixed the -s option Oleg Oshmyan <chortos@inbox.lv> parents: 41 diff changeset	4
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	5 from __future__ import division, with_statement
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	6
91 c62c9bfd614a Removed import_error Oleg Oshmyan <chortos@inbox.lv> parents: 90 diff changeset	7 from compat import *
c62c9bfd614a Removed import_error Oleg Oshmyan <chortos@inbox.lv> parents: 90 diff changeset	8 import files, problem, config
c62c9bfd614a Removed import_error Oleg Oshmyan <chortos@inbox.lv> parents: 90 diff changeset	9 from __main__ import options
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	10
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	11 import glob, re, sys, tempfile, time
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	12 from subprocess import Popen, PIPE, STDOUT
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	13
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	14 import os
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	15 devnull = open(os.path.devnull, 'w+')
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	16
85 741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	17 if options.autotime:
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	18 # This is really a dirty hack that assumes that sleep() does not spend
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	19 # the CPU time of the current process and that if clock() measures
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	20 # wall-clock time, then it is more precise than time() is. Both these
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	21 # assumptions are true on all platforms I have tested this on so far,
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	22 # but I am not aware of any guarantee that they will both be true
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	23 # on every other platform.
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	24 c = time.clock()
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	25 time.sleep(1)
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	26 c = time.clock() - c
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	27 if int(c + .5) == 1:
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	28 clock = time.clock
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	29 else:
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	30 clock = time.time
741ae3391b61 Moved clock/time detection into platform-specific modules and testcases Oleg Oshmyan <chortos@inbox.lv> parents: 83 diff changeset	31
108 218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	32 class DummySignalIgnorer(object):
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	33 def __enter__(self): pass
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	34 def __exit__(self, exc_type, exc_value, traceback): pass
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	35 signal_ignorer = DummySignalIgnorer()
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	36
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	37 try:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	38 from win32 import *
72 7520b6bb6636 Windows Error Reporting is now suppressed (at least the dialogs) Oleg Oshmyan <chortos@inbox.lv> parents: 71 diff changeset	39 except Exception:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	40 from unix import *
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	41
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	42 __all__ = ('TestCase', 'load_problem', 'TestCaseNotPassed',
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	43 'TimeLimitExceeded', 'CanceledByUser', 'WrongAnswer',
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	44 'NonZeroExitCode', 'CannotStartTestee',
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	45 'CannotStartValidator', 'CannotReadOutputFile',
81 24752db487c5 Fixed errors in the win32 module Oleg Oshmyan <chortos@inbox.lv> parents: 79 diff changeset	46 'CannotReadInputFile', 'CannotReadAnswerFile',
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	47 'MemoryLimitExceeded', 'CPUTimeLimitExceeded',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	48 'WallTimeLimitExceeded')
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	49
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	50
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	51
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	52 # Exceptions
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	53
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	54 class TestCaseNotPassed(Exception): __slots__ = ()
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	55 class TestCaseSkipped(TestCaseNotPassed): __slots__ = ()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	56 class TimeLimitExceeded(TestCaseNotPassed): __slots__ = ()
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	57 class CPUTimeLimitExceeded(TimeLimitExceeded): __slots__ = ()
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	58 class WallTimeLimitExceeded(TimeLimitExceeded): __slots__ = ()
77 69eadc60f4e2 Memory limit is now applied to the RSS when os.wait4 is available Oleg Oshmyan <chortos@inbox.lv> parents: 76 diff changeset	59 class MemoryLimitExceeded(TestCaseNotPassed): __slots__ = ()
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	60 class CanceledByUser(TestCaseNotPassed): __slots__ = ()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	61
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	62 class WrongAnswer(TestCaseNotPassed):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	63 __slots__ = 'comment'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	64 def __init__(self, comment=''):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	65 self.comment = comment
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	66
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	67 class NonZeroExitCode(TestCaseNotPassed):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	68 __slots__ = 'exitcode'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	69 def __init__(self, exitcode):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	70 self.exitcode = exitcode
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	71
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	72 class ExceptionWrapper(TestCaseNotPassed):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	73 __slots__ = 'upstream'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	74 def __init__(self, upstream):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	75 self.upstream = upstream
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	76
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	77 class CannotStartTestee(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	78 class CannotStartValidator(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	79 class CannotReadOutputFile(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	80 class CannotReadInputFile(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	81 class CannotReadAnswerFile(ExceptionWrapper): __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	82
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	83
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	84
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	85 # Helper context managers
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	86
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	87 class CopyDeleting(object):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	88 __slots__ = 'case', 'file', 'name'
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	89
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	90 def __init__(self, case, file, name):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	91 self.case = case
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	92 self.file = file
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	93 self.name = name
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	94
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	95 def __enter__(self):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	96 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	97 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	98 self.file.copy(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	99 except:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	100 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	101 self.__exit__(None, None, None)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	102 except:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	103 pass
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	104 raise
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	105
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	106 def __exit__(self, exc_type, exc_val, exc_tb):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	107 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	108 self.case.files_to_delete.append(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	109
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	110
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	111 class Copying(object):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	112 __slots__ = 'file', 'name'
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	113
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	114 def __init__(self, file, name):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	115 self.file = file
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	116 self.name = name
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	117
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	118 def __enter__(self):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	119 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	120 self.file.copy(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	121
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	122 def __exit__(self, exc_type, exc_val, exc_tb):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	123 pass
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	124
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	125
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	126
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	127 # Test case types
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	128
f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	129 class TestCase(object):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	130 __slots__ = ('problem', 'id', 'isdummy', 'infile', 'outfile', 'points',
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	131 'process', 'time_started', 'time_stopped',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	132 'realinname', 'realoutname', 'maxcputime', 'maxwalltime',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	133 'maxmemory', 'has_called_back', 'files_to_delete',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	134 'cpu_time_limit_string', 'wall_time_limit_string',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	135 'time_limit_string')
106 aa0378ea1f93 Replaced constant properties with regular attributes Oleg Oshmyan <chortos@inbox.lv> parents: 105 diff changeset	136 has_ansfile = has_iofiles = False
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	137 needs_realinname = True
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	138
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	139 if ABCMeta:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	140 __metaclass__ = ABCMeta
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	141
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	142 def __init__(case, prob, id, isdummy, points):
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	143 case.problem = prob
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	144 case.id = id
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	145 case.isdummy = isdummy
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	146 case.points = points
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	147 case.maxcputime = case.problem.config.maxcputime
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	148 case.maxwalltime = case.problem.config.maxwalltime
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	149 case.maxmemory = case.problem.config.maxmemory
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	150 if case.maxcputime:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	151 case.cpu_time_limit_string = '/%.3f' % case.maxcputime
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	152 else:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	153 case.cpu_time_limit_string = ''
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	154 if case.maxwalltime:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	155 case.wall_time_limit_string = '/%.3f' % case.maxwalltime
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	156 else:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	157 case.wall_time_limit_string = ''
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	158 if not isdummy:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	159 if case.needs_realinname:
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	160 case.realinname = case.problem.config.testcaseinname
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	161 case.realoutname = case.problem.config.testcaseoutname
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	162 else:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	163 if case.needs_realinname:
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	164 case.realinname = case.problem.config.dummyinname
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	165 case.realoutname = case.problem.config.dummyoutname
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	166
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	167 @abstractmethod
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	168 def test(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	169 raise NotImplementedError
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	170
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	171 def __call__(case, callback):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	172 case.has_called_back = False
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	173 case.files_to_delete = []
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	174 case.time_limit_string = case.wall_time_limit_string
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	175 try:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	176 return case.test(callback)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	177 finally:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	178 now = clock()
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	179 if getattr(case, 'time_started', None) is None:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	180 case.time_started = case.time_stopped = now
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	181 elif getattr(case, 'time_stopped', None) is None:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	182 case.time_stopped = now
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	183 if not case.has_called_back:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	184 callback()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	185 case.cleanup()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	186
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	187 def cleanup(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	188 #if getattr(case, 'infile', None):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	189 # case.infile.close()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	190 #if getattr(case, 'outfile', None):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	191 # case.outfile.close()
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	192 if getattr(case, 'process', None) and case.process.returncode is None:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	193 # Try KILLing after three unsuccessful TERM attempts in a row
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	194 for i in range(3):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	195 try:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	196 terminate(case.process)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	197 except Exception:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	198 time.sleep(0)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	199 case.process.poll()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	200 else:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	201 case.process.wait()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	202 break
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	203 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	204 # If killing the process is unsuccessful three times in a row,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	205 # just silently stop trying
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	206 for i in range(3):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	207 try:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	208 kill(case.process)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	209 except Exception:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	210 time.sleep(0)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	211 case.process.poll()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	212 else:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	213 case.process.wait()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	214 break
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	215 if case.files_to_delete:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	216 for name in case.files_to_delete:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	217 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	218 os.remove(name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	219 except Exception:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	220 # It can't be helped
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	221 pass
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	222
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	223 def open_infile(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	224 try:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	225 case.infile = files.File('/'.join((case.problem.name, case.realinname.replace('$', case.id))))
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	226 except IOError:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	227 e = sys.exc_info()[1]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	228 raise CannotReadInputFile(e)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	229
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	230 def open_outfile(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	231 try:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	232 case.outfile = files.File('/'.join((case.problem.name, case.realoutname.replace('$', case.id))))
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	233 except IOError:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	234 e = sys.exc_info()[1]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	235 raise CannotReadAnswerFile(e)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	236
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	237
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	238 class SkippedTestCase(TestCase):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	239 __slots__ = ()
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	240
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	241 def test(case, callback):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	242 raise TestCaseSkipped
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	243
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	244
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	245 class ValidatedTestCase(TestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	246 __slots__ = 'validator'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	247
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	248 def __init__(case, *args):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	249 TestCase.__init__(case, *args)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	250 if not case.problem.config.tester:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	251 case.validator = None
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	252 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	253 case.validator = case.problem.config.tester
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	254
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	255 def validate(case, output):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	256 if not case.validator:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	257 # Compare the output with the reference output
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	258 case.open_outfile()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	259 with case.outfile.open() as refoutput:
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	260 for line, refline in zip_longest(output, refoutput):
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	261 if refline is not None and not isinstance(refline, basestring):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	262 line = bytes(line, sys.getdefaultencoding())
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	263 if line != refline:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	264 raise WrongAnswer
24 c23d81f4a1a3 Score returned by TestCase.__call__() is now normalized to 0..1 Oleg Oshmyan <chortos@inbox.lv> parents: 23 diff changeset	265 return 1
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	266 elif callable(case.validator):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	267 return case.validator(output)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	268 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	269 # Call the validator program
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	270 output.close()
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	271 if case.problem.config.ansname:
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	272 case.open_outfile()
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	273 case.outfile.copy(case.problem.config.ansname)
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	274 try:
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	275 case.process = Popen(case.validator, stdin=devnull, stdout=PIPE, stderr=STDOUT, universal_newlines=True, bufsize=-1)
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	276 except OSError:
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	277 raise CannotStartValidator(sys.exc_info()[1])
108 218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	278 with signal_ignorer:
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	279 comment = case.process.communicate()[0].strip()
26 5bbb68833868 Output text improvements Oleg Oshmyan <chortos@inbox.lv> parents: 25 diff changeset	280 match = re.match(r'(?i)(ok\|(?:correct\|wrong)(?:(?:\s\|_)answer)?)(?:$\|\s+\|[.,!:]+\s)', comment)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	281 if match:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	282 comment = comment[match.end():]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	283 if not case.problem.config.maxexitcode:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	284 if case.process.returncode:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	285 raise WrongAnswer(comment)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	286 else:
24 c23d81f4a1a3 Score returned by TestCase.__call__() is now normalized to 0..1 Oleg Oshmyan <chortos@inbox.lv> parents: 23 diff changeset	287 return 1, comment
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	288 else:
24 c23d81f4a1a3 Score returned by TestCase.__call__() is now normalized to 0..1 Oleg Oshmyan <chortos@inbox.lv> parents: 23 diff changeset	289 return case.process.returncode / case.problem.config.maxexitcode, comment
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	290
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	291
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	292 class BatchTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	293 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	294
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	295 @property
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	296 def has_iofiles(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	297 return (not case.problem.config.stdio or
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	298 case.validator and not callable(case.validator))
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	299
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	300 @property
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	301 def has_ansfile(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	302 return case.validator and not callable(case.validator)
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	303
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	304 def test(case, callback):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	305 case.open_infile()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	306 if case.problem.config.stdio:
54 1914ae9cfdce Bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 50 diff changeset	307 if options.erase and not case.validator or not case.problem.config.inname:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	308 # TODO: re-use the same file name if possible
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	309 # FIXME: 2.5 lacks the delete parameter
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	310 with tempfile.NamedTemporaryFile(delete=False) as f:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	311 inputdatafname = f.name
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	312 contextmgr = CopyDeleting(case, case.infile, inputdatafname)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	313 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	314 inputdatafname = case.problem.config.inname
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	315 contextmgr = Copying(case.infile, inputdatafname)
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	316 with contextmgr:
79 ee8a99dcaaed Renamed configuration variable tasknames to problems Oleg Oshmyan <chortos@inbox.lv> parents: 77 diff changeset	317 with open(inputdatafname) as infile:
83 37c4ad87583c Several small fixes Oleg Oshmyan <chortos@inbox.lv> parents: 82 diff changeset	318 with tempfile.TemporaryFile('w+') if options.erase and (not case.validator or callable(case.validator)) else open(case.problem.config.outname, 'w+') as outfile:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	319 call(case.problem.config.path, case=case, stdin=infile, stdout=outfile, stderr=devnull, universal_newlines=True, bufsize=-1)
62 593ad09cd69b Multiple exit code handling fixes Oleg Oshmyan <chortos@inbox.lv> parents: 61 diff changeset	320 if config.globalconf.force_zero_exitcode and case.process.returncode or case.process.returncode < 0:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	321 raise NonZeroExitCode(case.process.returncode)
103 4e6f231f055f Fixed race condition resulting in calling back twice from TestCase.test Oleg Oshmyan <chortos@inbox.lv> parents: 99 diff changeset	322 case.has_called_back = True
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	323 callback()
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	324 outfile.seek(0)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	325 return case.validate(outfile)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	326 else:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	327 case.infile.copy(case.problem.config.inname)
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	328 call(case.problem.config.path, case=case, stdin=devnull, stdout=devnull, stderr=STDOUT)
62 593ad09cd69b Multiple exit code handling fixes Oleg Oshmyan <chortos@inbox.lv> parents: 61 diff changeset	329 if config.globalconf.force_zero_exitcode and case.process.returncode or case.process.returncode < 0:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	330 raise NonZeroExitCode(case.process.returncode)
103 4e6f231f055f Fixed race condition resulting in calling back twice from TestCase.test Oleg Oshmyan <chortos@inbox.lv> parents: 99 diff changeset	331 case.has_called_back = True
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	332 callback()
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	333 try:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	334 output = open(case.problem.config.outname, 'rU')
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	335 except IOError:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	336 raise CannotReadOutputFile(sys.exc_info()[1])
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	337 with output as output:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	338 return case.validate(output)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	339
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	340
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	341 # This is the only test case type not executing any programs to be tested
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	342 class OutputOnlyTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	343 __slots__ = ()
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	344 needs_realinname = False
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	345
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	346 def cleanup(case):
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	347 pass
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	348
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	349 def test(case, callback):
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	350 case.time_stopped = case.time_started = 0
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	351 case.has_called_back = True
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	352 callback()
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	353 try:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	354 output = open(case.problem.config.outname.replace('$', case.id), 'rU')
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	355 except IOError:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	356 raise CannotReadOutputFile(sys.exc_info()[1])
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	357 with output as output:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	358 return case.validate(output)
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	359
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	360
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	361 class BestOutputTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	362 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	363
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	364
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	365 # This is the only test case type executing two programs simultaneously
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	366 class ReactiveTestCase(TestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	367 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	368 # The basic idea is to launch the program to be tested and the grader
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	369 # and to pipe their standard I/O from and to each other,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	370 # and then to capture the grader's exit code and use it
26 5bbb68833868 Output text improvements Oleg Oshmyan <chortos@inbox.lv> parents: 25 diff changeset	371 # like the exit code of an output validator is used.
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	372
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	373
71 1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	374 class DummyTestContext(problem.TestGroup):
1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	375 __slots__ = ()
1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	376 def end(self):
1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	377 say('Sample total: %d/%d tests' % (self.ncorrect, self.ntotal))
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	378 return 0, 0, self.log
71 1bee3a0beeb5 Added a 'Sample total' line when using test groups Oleg Oshmyan <chortos@inbox.lv> parents: 69 diff changeset	379
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	380 def load_problem(prob, _types={'batch' : BatchTestCase,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	381 'outonly' : OutputOnlyTestCase,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	382 'bestout' : BestOutputTestCase,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	383 'reactive': ReactiveTestCase}):
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	384 # We will need to iterate over these configuration variables twice
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	385 try:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	386 len(prob.config.dummies)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	387 except Exception:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	388 prob.config.dummies = tuple(prob.config.dummies)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	389 try:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	390 len(prob.config.tests)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	391 except Exception:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	392 prob.config.tests = tuple(prob.config.tests)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	393
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	394 if options.legacy:
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	395 prob.config.usegroups = False
58 c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	396 newtests = []
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	397 for i, name in enumerate(prob.config.tests):
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	398 # Same here; we'll need to iterate over them twice
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	399 try:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	400 l = len(name)
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	401 except Exception:
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	402 try:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	403 name = tuple(name)
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	404 except TypeError:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	405 name = (name,)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	406 l = len(name)
58 c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	407 if l > 1:
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	408 prob.config.usegroups = True
58 c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	409 newtests.append(name)
c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	410 if prob.config.usegroups:
c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	411 prob.config.tests = newtests
c95addbe0851 Fixed test group support Oleg Oshmyan <chortos@inbox.lv> parents: 57 diff changeset	412 del newtests
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	413
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	414 # Even if they have duplicate test identifiers, we must honour sequence pointmaps
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	415 if isinstance(prob.config.pointmap, dict):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	416 def getpoints(i, j, k=None):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	417 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	418 return prob.config.pointmap[i]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	419 except KeyError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	420 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	421 return prob.config.pointmap[None]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	422 except KeyError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	423 return prob.config.maxexitcode or 1
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	424 elif prob.config.usegroups:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	425 def getpoints(i, j, k):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	426 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	427 return prob.config.pointmap[k][j]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	428 except LookupError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	429 return prob.config.maxexitcode or 1
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	430 else:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	431 def getpoints(i, j):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	432 try:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	433 return prob.config.pointmap[j]
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	434 except LookupError:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	435 return prob.config.maxexitcode or 1
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	436
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	437 # First get prob.cache.padoutput right,
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	438 # then yield the actual test cases
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	439 for i in prob.config.dummies:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	440 s = 'sample ' + str(i).zfill(prob.config.paddummies)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	441 prob.cache.padoutput = max(prob.cache.padoutput, len(s))
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	442 if prob.config.usegroups:
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	443 if not isinstance(prob.config.groupweight, dict):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	444 prob.config.groupweight = dict(enumerate(prob.config.groupweight))
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	445 for group in prob.config.tests:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	446 for i in group:
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	447 s = str(i).zfill(prob.config.padtests)
2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	448 prob.cache.padoutput = max(prob.cache.padoutput, len(s))
89 3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	449 if prob.config.dummies:
3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	450 yield DummyTestContext()
3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	451 for i in prob.config.dummies:
3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	452 s = str(i).zfill(prob.config.paddummies)
99 9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	453 if (yield _types[prob.config.kind](prob, s, True, 0)):
9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	454 yield
89 3ae6cb69e4ef Sample total is no longer printed if there are no samples Oleg Oshmyan <chortos@inbox.lv> parents: 88 diff changeset	455 yield problem.test_context_end
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	456 for k, group in enumerate(prob.config.tests):
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	457 if not group:
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	458 continue
0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	459 yield problem.TestGroup(prob.config.groupweight.get(k, prob.config.groupweight.get(None)))
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	460 case_type = _types[prob.config.kind]
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	461 for j, i in enumerate(group):
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	462 s = str(i).zfill(prob.config.padtests)
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	463 if not (yield case_type(prob, s, False, getpoints(i, j, k))):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	464 if options.skim:
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	465 case_type = SkippedTestCase
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	466 else:
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	467 yield
39 2b459f9743b4 Test groups are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 27 diff changeset	468 yield problem.test_context_end
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	469 else:
f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	470 for i in prob.config.tests:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	471 s = str(i).zfill(prob.config.padtests)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	472 prob.cache.padoutput = max(prob.cache.padoutput, len(s))
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	473 for i in prob.config.dummies:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	474 s = str(i).zfill(prob.config.paddummies)
99 9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	475 if (yield _types[prob.config.kind](prob, s, True, 0)):
9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	476 yield
76 0e5ae28e0b2b Points are now weighted on a test context basis Oleg Oshmyan <chortos@inbox.lv> parents: 72 diff changeset	477 for j, i in enumerate(prob.config.tests):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	478 s = str(i).zfill(prob.config.padtests)
99 9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	479 if (yield _types[prob.config.kind](prob, s, False, getpoints(i, j))):
9b41934b2e95 Fixed a major regression introduced in changeset 1fb319ec33af Oleg Oshmyan <chortos@inbox.lv> parents: 91 diff changeset	480 yield

Mercurial > ~astiob > upreckon > hgweb

annotate testcases.py @ 122:d6fd880207cb