Return to the home page of Upreckon

Upreckon Bug and Wish Tracker

Since the Upreckon 2.00 release is shaping to be an almost complete rewrite and I want it to be devoid of any peculiar deficiencies test.py 1.x currently has, I have decided to create this page to keep track of changes that I want to implement, am implementing or have implemented, as well as changes users want to see and bugs that all of us want to get fixed.

If you want to suggest another feature or report a bug (which you should do whenever you encounter a bug), please create a new issue on Bitbucket (which is preferred) or directly contact me using one of the means of communication I have listed on my homepage (email is included).

Upreckon bugs, features and feature suggestions
Short descriptionStatus
CPU time should be able to be measured instead of wall-clock timeimplemented in 2.001
memory usage of the program being tested should be able to be limitedimplemented in 2.001
output validators should be able to be given as Python callablesimplemented in 2.00
points should be able to be awarded for whole test groups rather than just for individual test casesimplemented in 2.00
run-time overhead should be significantly reducedimplemented in 2.00
signal names, when available, should be shown in addition to their numbersimplemented in 2.00
single-test-case groups should print totals if at least one group spans multiple test casesimplemented in 2.00
test.py should have a serious nameimplemented in 2.00 (Upreckon)
testing within a test group should be stopped as soon as one test case failsimplemented in 2.00 (--skim)
the ansname configuration variable should always be optionalimplemented in 2.00
the output of output validators should be cleaned from text like ‘OK’, ‘correct’ and ‘wrong’implemented in 2.00
the program being tested should be able to be manually killed by pressing Escapeimplemented in 2.00
Upreckon should be fully compatible with Python 3implemented in 2.00
Win32 errors caused by the program being tested should be reported in hexadecimalimplemented in 2.00
Windows Error Reporting should be suppressedimplemented in 2.00
absent output files should not crash the whole Upreckonimplemented in 2.00.1
CPU time should be natively limited on platforms with posix_spawnimplemented in 2.01 (in C only)
output-only problems should be supportedimplemented in 2.01
time-sensitive and simply very platform-dependent code should be re-implemented in C+POSIXimplemented in 2.01
Upreckon should be installable using distutilsimplemented in 2.01
test case identifier sets should be able to be defined by regular expressionsimplemented in 2.03
command-line test case identifier overrides should work with match='re'implemented in 2.03.1
pressing Escape should cancel test input unarchivingimplemented _unix-only in 2.04
time-sensitive and simply very platform-dependent code should be re-implemented in C+Win32being implemented for 2.0x
Upreckon should be fully compatible with Python 2.5being implemented for 2.0x
Upreckon should follow PEP 8being implemented for 2.0x
all exceptions at least somewhat expected should show a friendly messageset to be implemented in 2.0x
configuration for multiple problems should be able to be given in the same fileset to be implemented in 2.0x
configuration variable types should be prescribed and validatedset to be implemented in 2.0x
different time limits should be able to be given for different test casesset to be implemented in 2.0x
problem.load_testcases and problem.Problem.test should communicate using a sensible protocolset to be implemented in 2.0x
test and configuration data should be always searched in the same locations and in the same orderset to be implemented in 2.0x
test case identifier sets should be able to be defined by glob wildcardsset to be implemented in 2.0x
Upreckon should be fully compatible with IronPythonset to be implemented in 2.0x
Upreckon should be fully compatible with Jythonset to be implemented in 2.0x
Upreckon should be fully compatible with PyPyset to be implemented in 2.0x
Upreckon should be in PyPIset to be implemented in 2.0x
Upreckon should be well-documentedset to be implemented in 2.0x
Upreckon should follow PEP 257set to be implemented in 2.0x
output validators should be able to be supplied in archiveshoped to be implemented in 2.0x
partial score should be allowed to be given to slow but correct solutionshoped to be implemented in 2.0x
tape archives should be allowed to contain testconf.py files with non-LF line endingshoped to be implemented in 2.0x
differences in whitespace should be able to be ignored when validating outputtaken into consideration for 2.0x
custom pre-testing actions should be supportedhoped to be implemented in 2.x
interactive (reactive) problems should be supportedhoped to be implemented in 2.x
output of programs using standard I/O should be piped to output validatorshoped to be implemented in 2.x
directories named taskname in the current working directory should not be checked for test and configuration datataken into consideration for 2.x
the first test case should be run twice if reasonable with only the second run valued (think caching/whatever)taken into consideration for 2.x
Web judges should be supported (just uploading and proxying the results)taken into consideration for 2.x
the front-end (reading and parsing the command line and configuration files) should be easily swappablehoped to be implemented in 3.00
programs that append to the output file instead of overwriting it should be granted Wrong Answerrejected (impossibly objective)

1 Support varies from platform to platform. See the wiki on Bitbucket for more details.