Upreckon: upreckon/testcases.py annotate

annotate upreckon/testcases.py @ 232:f94f9724c543

Switched from str to bytes for external validator output

author	Oleg Oshmyan <chortos@inbox.lv>
date	Fri, 29 Jun 2012 03:32:49 +0300
parents	65b5c9390010
children

rev	line source
232 f94f9724c543 Switched from str to bytes for external validator output Oleg Oshmyan <chortos@inbox.lv> parents: 218 diff changeset	1 # Copyright (c) 2010-2012 Chortos-2 <chortos@inbox.lv>
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	2
43 81f58c938ec5 Fixed the -s option Oleg Oshmyan <chortos@inbox.lv> parents: 41 diff changeset	3 # TODO: copy the ansfile if not options.erase even if no validator is used
81f58c938ec5 Fixed the -s option Oleg Oshmyan <chortos@inbox.lv> parents: 41 diff changeset	4
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	5 from __future__ import division, with_statement
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	6
146 d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	7 from .compat import *
d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	8 from .exceptions import *
d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	9 from . import files, config
91 c62c9bfd614a Removed import_error Oleg Oshmyan <chortos@inbox.lv> parents: 90 diff changeset	10 from __main__ import options
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	11
146 d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	12 import re, sys, tempfile
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	13 from subprocess import Popen, PIPE, STDOUT
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	14
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	15 import os
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	16 devnull = open(os.path.devnull, 'w+b')
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	17
108 218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	18 class DummySignalIgnorer(object):
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	19 def __enter__(self): pass
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	20 def __exit__(self, exc_type, exc_value, traceback): pass
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	21 signal_ignorer = DummySignalIgnorer()
218 65b5c9390010 With _unix, Escape presses now cancel test data unarchiving Oleg Oshmyan <chortos@inbox.lv> parents: 217 diff changeset	22 def install_escape_handler(): pass
65b5c9390010 With _unix, Escape presses now cancel test data unarchiving Oleg Oshmyan <chortos@inbox.lv> parents: 217 diff changeset	23 def remove_escape_handler(): pass
108 218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	24
146 d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	25 try:
d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	26 from .win32 import *
d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	27 except Exception:
d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	28 from .unix import *
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	29
146 d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	30 __all__ = ('TestCase', 'SkippedTestCase', 'ValidatedTestCase', 'BatchTestCase',
d5b6708c1955 Distutils support, reorganization and cleaning up Oleg Oshmyan <chortos@inbox.lv> parents: 136 diff changeset	31 'OutputOnlyTestCase')
136 ed4035661b85 Added a C implementation of the unix module (called _unix) Oleg Oshmyan <chortos@inbox.lv> parents: 134 diff changeset	32
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	33
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	34
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	35 # Helper context managers
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	36
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	37 class CopyDeleting(object):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	38 __slots__ = 'case', 'file', 'name'
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	39
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	40 def __init__(self, case, file, name):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	41 self.case = case
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	42 self.file = file
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	43 self.name = name
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	44
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	45 def __enter__(self):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	46 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	47 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	48 self.file.copy(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	49 except:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	50 try:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	51 self.__exit__(None, None, None)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	52 except:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	53 pass
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	54 raise
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	55
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	56 def __exit__(self, exc_type, exc_val, exc_tb):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	57 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	58 self.case.files_to_delete.append(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	59
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	60
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	61 class Copying(object):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	62 __slots__ = 'file', 'name'
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	63
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	64 def __init__(self, file, name):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	65 self.file = file
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	66 self.name = name
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	67
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	68 def __enter__(self):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	69 if self.name:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	70 self.file.copy(self.name)
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	71
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	72 def __exit__(self, exc_type, exc_val, exc_tb):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	73 pass
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	74
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	75
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	76
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	77 # Test case types
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	78
f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	79 class TestCase(object):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	80 __slots__ = ('problem', 'id', 'isdummy', 'infile', 'outfile', 'points',
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	81 'process', 'time_started', 'time_stopped',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	82 'realinname', 'realoutname', 'maxcputime', 'maxwalltime',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	83 'maxmemory', 'has_called_back', 'files_to_delete',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	84 'cpu_time_limit_string', 'wall_time_limit_string',
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	85 'time_limit_string')
106 aa0378ea1f93 Replaced constant properties with regular attributes Oleg Oshmyan <chortos@inbox.lv> parents: 105 diff changeset	86 has_ansfile = has_iofiles = False
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	87 needs_realinname = True
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	88
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	89 if ABCMeta:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	90 __metaclass__ = ABCMeta
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	91
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	92 def __init__(case, prob, id, isdummy, points):
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	93 case.problem = prob
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	94 case.id = id
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	95 case.isdummy = isdummy
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	96 case.points = points
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	97 case.maxcputime = case.problem.config.maxcputime
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	98 case.maxwalltime = case.problem.config.maxwalltime
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	99 case.maxmemory = case.problem.config.maxmemory
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	100 if case.maxcputime:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	101 case.cpu_time_limit_string = '/%.3f' % case.maxcputime
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	102 else:
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	103 case.cpu_time_limit_string = ''
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	104 if case.maxwalltime:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	105 case.wall_time_limit_string = '/%.3f' % case.maxwalltime
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	106 else:
06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	107 case.wall_time_limit_string = ''
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	108 if not isdummy:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	109 if case.needs_realinname:
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	110 case.realinname = case.problem.config.testcaseinname
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	111 case.realoutname = case.problem.config.testcaseoutname
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	112 else:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	113 if case.needs_realinname:
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	114 case.realinname = case.problem.config.dummyinname
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	115 case.realoutname = case.problem.config.dummyoutname
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	116
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	117 @abstractmethod
217 ce1285728952 Fixed the signature of @abstractmethod TestCase.test Oleg Oshmyan <chortos@inbox.lv> parents: 208 diff changeset	118 def test(case, callback):
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	119 raise NotImplementedError
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	120
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	121 def __call__(case, callback):
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	122 case.has_called_back = False
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	123 case.files_to_delete = []
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	124 case.time_limit_string = case.wall_time_limit_string
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	125 try:
218 65b5c9390010 With _unix, Escape presses now cancel test data unarchiving Oleg Oshmyan <chortos@inbox.lv> parents: 217 diff changeset	126 install_escape_handler()
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	127 return case.test(callback)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	128 finally:
218 65b5c9390010 With _unix, Escape presses now cancel test data unarchiving Oleg Oshmyan <chortos@inbox.lv> parents: 217 diff changeset	129 remove_escape_handler()
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	130 now = clock()
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	131 if getattr(case, 'time_started', None) is None:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	132 case.time_started = case.time_stopped = now
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	133 elif getattr(case, 'time_stopped', None) is None:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	134 case.time_stopped = now
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	135 if not case.has_called_back:
f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	136 callback()
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	137 case.cleanup()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	138
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	139 def cleanup(case):
136 ed4035661b85 Added a C implementation of the unix module (called _unix) Oleg Oshmyan <chortos@inbox.lv> parents: 134 diff changeset	140 # Note that native extensions clean up on their own
ed4035661b85 Added a C implementation of the unix module (called _unix) Oleg Oshmyan <chortos@inbox.lv> parents: 134 diff changeset	141 # and never let this condition be satisfied
82 06356af50bf9 Finished testcases reorganization and CPU time limit implementation Oleg Oshmyan <chortos@inbox.lv> parents: 81 diff changeset	142 if getattr(case, 'process', None) and case.process.returncode is None:
134 e84f33a60a5c Moved process killing logic into platform-specific modules Oleg Oshmyan <chortos@inbox.lv> parents: 130 diff changeset	143 kill(case.process)
e84f33a60a5c Moved process killing logic into platform-specific modules Oleg Oshmyan <chortos@inbox.lv> parents: 130 diff changeset	144 for name in case.files_to_delete:
e84f33a60a5c Moved process killing logic into platform-specific modules Oleg Oshmyan <chortos@inbox.lv> parents: 130 diff changeset	145 try:
e84f33a60a5c Moved process killing logic into platform-specific modules Oleg Oshmyan <chortos@inbox.lv> parents: 130 diff changeset	146 os.remove(name)
e84f33a60a5c Moved process killing logic into platform-specific modules Oleg Oshmyan <chortos@inbox.lv> parents: 130 diff changeset	147 except OSError:
e84f33a60a5c Moved process killing logic into platform-specific modules Oleg Oshmyan <chortos@inbox.lv> parents: 130 diff changeset	148 # It can't be helped
e84f33a60a5c Moved process killing logic into platform-specific modules Oleg Oshmyan <chortos@inbox.lv> parents: 130 diff changeset	149 pass
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	150
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	151 def open_infile(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	152 try:
208 ede78fbd509a Revamped files.File Oleg Oshmyan <chortos@inbox.lv> parents: 206 diff changeset	153 case.infile = files.File.from_virtual_path('/'.join((case.problem.name, case.realinname.replace('$', case.id))), False, 'test data')
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	154 except IOError:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	155 e = sys.exc_info()[1]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	156 raise CannotReadInputFile(e)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	157
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	158 def open_outfile(case):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	159 try:
208 ede78fbd509a Revamped files.File Oleg Oshmyan <chortos@inbox.lv> parents: 206 diff changeset	160 case.outfile = files.File.from_virtual_path('/'.join((case.problem.name, case.realoutname.replace('$', case.id))), False, 'test data')
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	161 except IOError:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	162 e = sys.exc_info()[1]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	163 raise CannotReadAnswerFile(e)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	164
16 f2279b7602d3 Initial 2.00 commit Oleg Oshmyan <chortos@inbox.lv> parents: diff changeset	165
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	166 class SkippedTestCase(TestCase):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	167 __slots__ = ()
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	168
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	169 def test(case, callback):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	170 raise TestCaseSkipped
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	171
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	172
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	173 class ValidatedTestCase(TestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	174 __slots__ = 'validator'
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	175
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	176 def __init__(case, *args):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	177 TestCase.__init__(case, *args)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	178 if not case.problem.config.tester:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	179 case.validator = None
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	180 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	181 case.validator = case.problem.config.tester
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	182
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	183 def validate(case, output):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	184 if not case.validator:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	185 # Compare the output with the reference output
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	186 buffer = refbuffer = crlfhalf = refcrlfhalf = ''.encode()
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	187 crlf = '\r\n'.encode('ascii')
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	188 case.open_outfile()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	189 with case.outfile.open() as refoutput:
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	190 while True:
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	191 data = output.read(4096 - len(buffer))
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	192 refdata = refoutput.read(4096 - len(refbuffer))
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	193 if not case.problem.config.binary:
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	194 data, refdata = crlfhalf + data, refcrlfhalf + refdata
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	195 size, refsize = len(data), len(refdata)
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	196 if data and data != crlfhalf and data[-1] == crlf[0]:
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	197 size -= 1
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	198 crlfhalf = data[-1:]
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	199 else:
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	200 crlfhalf = ''.encode()
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	201 if refdata and refdata != refcrlfhalf and refdata[-1] == crlf[0]:
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	202 refsize -= 1
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	203 refcrlfhalf = refdata[-1:]
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	204 else:
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	205 refcrlfhalf = ''.encode()
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	206 data = data[:size].replace(crlf, crlf[1:])
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	207 data = data.replace(crlf[:1], crlf[1:])
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	208 refdata = refdata[:refsize].replace(crlf, crlf[1:])
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	209 refdata = refdata.replace(crlf[:1], crlf[1:])
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	210 buffer += data
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	211 refbuffer += refdata
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	212 if not (buffer or refbuffer or crlfhalf or refcrlfhalf):
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	213 break
180 760d38ee86d6 Fixed hanging in the built-in output validator when output lengths differ Oleg Oshmyan <chortos@inbox.lv> parents: 176 diff changeset	214 elif not buffer and not crlfhalf or not refbuffer and not refcrlfhalf:
760d38ee86d6 Fixed hanging in the built-in output validator when output lengths differ Oleg Oshmyan <chortos@inbox.lv> parents: 176 diff changeset	215 raise WrongAnswer
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	216 size = min(len(buffer), len(refbuffer))
e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	217 if buffer[:size] != refbuffer[:size]:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	218 raise WrongAnswer
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	219 buffer, refbuffer = buffer[size:], refbuffer[size:]
24 c23d81f4a1a3 Score returned by TestCase.__call__() is now normalized to 0..1 Oleg Oshmyan <chortos@inbox.lv> parents: 23 diff changeset	220 return 1
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	221 elif callable(case.validator):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	222 return case.validator(output)
204 00c80bba7f13 Removed some long-unnoticed trailing whitespace Oleg Oshmyan <chortos@inbox.lv> parents: 200 diff changeset	223 else:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	224 # Call the validator program
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	225 output.close()
23 c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	226 if case.problem.config.ansname:
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	227 case.open_outfile()
c1f52b5d80d6 Compatibility and bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 22 diff changeset	228 case.outfile.copy(case.problem.config.ansname)
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	229 try:
232 f94f9724c543 Switched from str to bytes for external validator output Oleg Oshmyan <chortos@inbox.lv> parents: 218 diff changeset	230 case.process = Popen(case.validator, stdin=devnull, stdout=PIPE, stderr=STDOUT, bufsize=-1)
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	231 except OSError:
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	232 raise CannotStartValidator(sys.exc_info()[1])
108 218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	233 with signal_ignorer:
218b8c28549c Fixed a crash due to SIGCHLD interrupting validator output pipe reads Oleg Oshmyan <chortos@inbox.lv> parents: 106 diff changeset	234 comment = case.process.communicate()[0].strip()
232 f94f9724c543 Switched from str to bytes for external validator output Oleg Oshmyan <chortos@inbox.lv> parents: 218 diff changeset	235 match = re.match(r'(?i)(?:ok\|(?:(?:in)?correct\|wrong)(?:(?:\s\|_)answer)?)(?:$\|\s+\|[.,!:]+\s)'.encode('ascii'), comment)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	236 if match:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	237 comment = comment[match.end():]
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	238 if not case.problem.config.maxexitcode:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	239 if case.process.returncode:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	240 raise WrongAnswer(comment)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	241 else:
205 166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	242 return 1, True, comment
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	243 else:
205 166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	244 if case.problem.config.okexitcodeflag:
166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	245 correct = bool(case.process.returncode & case.problem.config.okexitcodeflag)
166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	246 case.process.returncode &= ~case.problem.config.okexitcodeflag
166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	247 else:
166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	248 correct = case.process.returncode >= case.problem.config.maxexitcode
166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	249 if not correct and not case.process.returncode:
166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	250 raise WrongAnswer(comment)
166a23999bf7 Added confvar okexitcodemask; changed the validator protocol Oleg Oshmyan <chortos@inbox.lv> parents: 204 diff changeset	251 return case.process.returncode / case.problem.config.maxexitcode, correct, comment
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	252
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	253
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	254 class BatchTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	255 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	256
90 1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	257 @property
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	258 def has_iofiles(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	259 return (not case.problem.config.stdio or
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	260 case.validator and not callable(case.validator))
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	261
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	262 @property
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	263 def has_ansfile(case):
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	264 return case.validator and not callable(case.validator)
1fb319ec33af Skimming mode added (-k/--skim option) Oleg Oshmyan <chortos@inbox.lv> parents: 89 diff changeset	265
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	266 def test(case, callback):
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	267 case.open_infile()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	268 if case.problem.config.stdio:
54 1914ae9cfdce Bug fixes Oleg Oshmyan <chortos@inbox.lv> parents: 50 diff changeset	269 if options.erase and not case.validator or not case.problem.config.inname:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	270 # TODO: re-use the same file name if possible
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	271 # FIXME: 2.5 lacks the delete parameter
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	272 with tempfile.NamedTemporaryFile(delete=False) as f:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	273 inputdatafname = f.name
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	274 contextmgr = CopyDeleting(case, case.infile, inputdatafname)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	275 else:
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	276 inputdatafname = case.problem.config.inname
25 b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	277 contextmgr = Copying(case.infile, inputdatafname)
b500e117080e Bug fixes and overhead reduction Oleg Oshmyan <chortos@inbox.lv> parents: 24 diff changeset	278 with contextmgr:
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	279 with tempfile.TemporaryFile('w+b') if options.erase and (not case.validator or callable(case.validator)) else open(case.problem.config.outname, 'w+b') as outfile:
163 8198aa2ed20d Some broken output validators on Win32 no longer get access-denied errors Oleg Oshmyan <chortos@inbox.lv> parents: 146 diff changeset	280 with open(inputdatafname) as infile:
130 652028850ea4 Removed useless arguments to call() from testcases.BatchTestCase.test() Oleg Oshmyan <chortos@inbox.lv> parents: 128 diff changeset	281 call(case.problem.config.path, case=case, stdin=infile, stdout=outfile, stderr=devnull)
200 fa81289ee407 force_zero_exitcode is now a problem-specific configuration variable Oleg Oshmyan <chortos@inbox.lv> parents: 180 diff changeset	282 if case.problem.config.force_zero_exitcode and case.process.returncode or case.process.returncode < 0:
163 8198aa2ed20d Some broken output validators on Win32 no longer get access-denied errors Oleg Oshmyan <chortos@inbox.lv> parents: 146 diff changeset	283 raise NonZeroExitCode(case.process.returncode)
8198aa2ed20d Some broken output validators on Win32 no longer get access-denied errors Oleg Oshmyan <chortos@inbox.lv> parents: 146 diff changeset	284 case.has_called_back = True
8198aa2ed20d Some broken output validators on Win32 no longer get access-denied errors Oleg Oshmyan <chortos@inbox.lv> parents: 146 diff changeset	285 callback()
8198aa2ed20d Some broken output validators on Win32 no longer get access-denied errors Oleg Oshmyan <chortos@inbox.lv> parents: 146 diff changeset	286 outfile.seek(0)
8198aa2ed20d Some broken output validators on Win32 no longer get access-denied errors Oleg Oshmyan <chortos@inbox.lv> parents: 146 diff changeset	287 return case.validate(outfile)
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	288 else:
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	289 case.infile.copy(case.problem.config.inname)
176 88e1e6786f67 Fixed crashing on stdio=False with _unix Oleg Oshmyan <chortos@inbox.lv> parents: 174 diff changeset	290 call(case.problem.config.path, case=case, stdin=devnull, stdout=devnull, stderr=devnull)
200 fa81289ee407 force_zero_exitcode is now a problem-specific configuration variable Oleg Oshmyan <chortos@inbox.lv> parents: 180 diff changeset	291 if case.problem.config.force_zero_exitcode and case.process.returncode or case.process.returncode < 0:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	292 raise NonZeroExitCode(case.process.returncode)
103 4e6f231f055f Fixed race condition resulting in calling back twice from TestCase.test Oleg Oshmyan <chortos@inbox.lv> parents: 99 diff changeset	293 case.has_called_back = True
22 f07b7a431ea6 Further 2.00 work Oleg Oshmyan <chortos@inbox.lv> parents: 21 diff changeset	294 callback()
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	295 try:
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	296 output = open(case.problem.config.outname, 'rb')
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	297 except IOError:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	298 raise CannotReadOutputFile(sys.exc_info()[1])
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	299 with output as output:
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	300 return case.validate(output)
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	301
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	302
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	303 # This is the only test case type not executing any programs to be tested
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	304 class OutputOnlyTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	305 __slots__ = ()
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	306 needs_realinname = False
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	307
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	308 def cleanup(case):
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	309 pass
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	310
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	311 def test(case, callback):
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	312 case.time_stopped = case.time_started = 0
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	313 case.has_called_back = True
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	314 callback()
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	315 try:
174 e0b2fbd7ebe0 Improved built-in output validator; added conf. var. binary Oleg Oshmyan <chortos@inbox.lv> parents: 163 diff changeset	316 output = open(case.problem.config.outname.replace('$', case.id), 'rb')
105 9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	317 except IOError:
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	318 raise CannotReadOutputFile(sys.exc_info()[1])
9f922b11c98a Absent output files no longer crash Upreckon Oleg Oshmyan <chortos@inbox.lv> parents: 104 diff changeset	319 with output as output:
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	320 return case.validate(output)
8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	321
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	322
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	323 class BestOutputTestCase(ValidatedTestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	324 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	325
104 8f46e84922f9 Output-only problems are now supported Oleg Oshmyan <chortos@inbox.lv> parents: 103 diff changeset	326
21 ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	327 # This is the only test case type executing two programs simultaneously
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	328 class ReactiveTestCase(TestCase):
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	329 __slots__ = ()
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	330 # The basic idea is to launch the program to be tested and the grader
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	331 # and to pipe their standard I/O from and to each other,
ec6f1a132109 A pretty usable version Oleg Oshmyan <chortos@inbox.lv> parents: 16 diff changeset	332 # and then to capture the grader's exit code and use it
26 5bbb68833868 Output text improvements Oleg Oshmyan <chortos@inbox.lv> parents: 25 diff changeset	333 # like the exit code of an output validator is used.

Mercurial > ~astiob > upreckon > hgweb

annotate upreckon/testcases.py @ 232:f94f9724c543