comparison upreckon/testcases.py @ 205:166a23999bf7

Added confvar okexitcodemask; changed the validator protocol Callable validators now return three-tuples (number granted, bool correct, str comment) instead of two-tuples (number granted, str comment). They are still allowed to return single numbers. Callable validators must now explicitly raise upreckon.exceptions.WrongAnswer if they want the verdict to be Wrong Answer rather than Partly Correct. okexitcodemask specifies a bitmask ANDed with the exit code of the external validator to get a boolean flag showing whether the answer is to be marked as 'OK' rather than 'partly correct'. The bits covered by the bitmask are reset to zeroes before devising the number of points granted from the resulting number.
author Oleg Oshmyan <chortos@inbox.lv>
date Wed, 17 Aug 2011 20:44:54 +0300
parents 00c80bba7f13
children 4edb6ef5a676
comparison
equal deleted inserted replaced
204:00c80bba7f13 205:166a23999bf7
233 comment = comment[match.end():] 233 comment = comment[match.end():]
234 if not case.problem.config.maxexitcode: 234 if not case.problem.config.maxexitcode:
235 if case.process.returncode: 235 if case.process.returncode:
236 raise WrongAnswer(comment) 236 raise WrongAnswer(comment)
237 else: 237 else:
238 return 1, comment 238 return 1, True, comment
239 else: 239 else:
240 return case.process.returncode / case.problem.config.maxexitcode, comment 240 if case.problem.config.okexitcodeflag:
241 correct = bool(case.process.returncode & case.problem.config.okexitcodeflag)
242 case.process.returncode &= ~case.problem.config.okexitcodeflag
243 else:
244 correct = case.process.returncode >= case.problem.config.maxexitcode
245 if not correct and not case.process.returncode:
246 raise WrongAnswer(comment)
247 return case.process.returncode / case.problem.config.maxexitcode, correct, comment
241 248
242 249
243 class BatchTestCase(ValidatedTestCase): 250 class BatchTestCase(ValidatedTestCase):
244 __slots__ = () 251 __slots__ = ()
245 252