Update documentation about configuring a contest.

Author: giomasce

docs/Configuring a contest.rst

@@ -15,40 +15,43 @@ Contest administrators can limit the ability of users to submit submissions and
 
 - ``max_submission_number`` / ``max_user_test_number``
 
-  These set, respectively, the maximum number of submissions/user_tests that will be accepted for a certain user. Any attempt to send in additional submissions/user_tests after that limit has been reached will fail.
+  These set, respectively, the maximum number of submissions or user tests that will be accepted for a certain user. Any attempt to send in additional submissions or user tests after that limit has been reached will fail.
 
 - ``min_submission_interval`` / ``min_user_test_interval``
 
-  These set, respectively, the minimum amount of time the user is required to wait after a submission/user_test has been submitted before he's allowed to send in new ones. Any attempt to submit a submission/user_test before this timeout has expired will fail.
+  These set, respectively, the minimum amount of time the user is required to wait after a submission or user test has been submitted before they are allowed to send in new ones. Any attempt to submit a submission or user test before this timeout has expired will fail.
 
-The limits can be set both on individual tasks and on the whole contest. For a submission to be accepted it has to verify the conditions on both the task *and* the contest. This means that a submission/user_test will be accepted only if the number of submissions/user_tests received so far for that task is strictly less that the task's "max_number" *and* the number of submissions/user_tests received so far for the whole contest (i.e. in all tasks) is strictly less than the contest's "max_number". The same holds for "min_interval" too: a submission/user_test will be accepted only if the time passed since the last submission/user_test for that task is greater than the task's "min_interval" *and* the time passed since the last submission/user_test received for the whole contest (i.e. in any of the tasks) is greater than the contest's "min_interval".
+The limits can be set both for individual tasks and for the whole contest. A submission or user test is accepted if it verifies the conditions on both the task *and* the contest. This means that a submission or user test will be accepted if the number of submissions or user tests received so far for that task is strictly less that the task's maximum number *and* the number of submissions or user tests received so far for the whole contest (i.e. in all tasks) is strictly less than the contest's maximum number. The same holds for the minimum interval too: a submission or user test will be accepted if the time passed since the last submission or user test for that task is greater than the task's minimum interval *and* the time passed since the last submission or user test received for the whole contest (i.e. in any of the tasks) is greater than the contest's minimum interval.
 
 Each of these fields can be left unset to prevent the corresponding limitation from being enforced.
 
 
 .. _configuringacontest_tokens:
 
-Tokens
-======
+Feedback and tokens
+===================
 
-Tokens are a concept introduced to provide contestants with limited access to the detailed results of their submissions on the private testcases.
+Each testcase can be marked as public or private. During the contest, contestants can see the result of their submissions on the public testcases (the content of the input and output data themselves remain hidden, though). Tokens are a concept introduced to provide contestants with limited access to the detailed results of their submissions on the private testcases as well.
 
-For every submission sent in for evaluation, a contestant is always able to see if it succesfully compiled. He/she is also able to see its scores on the public testcases of the task, if any. All information about the other so-called private testcases is kept hidden. Yet, a contestant can choose to use one of its tokens to "unlock" a certain submission of his/her choice. After he/she does this, detailed results are available for all testcases, as if they were all public. A token, once used, is consumed and lost forever. Contestants have a set of available tokens at their disposal, where the ones they use are picked from. These sets are managed by CMS according to rules defined by the contest administrators, as explained later in this section. Note that the public/private results are arbitrarily provided by the :doc:`score types <Score types>` and are *usually* based on public/private testcases, but they could actually use other criteria.
+For every submission sent in for evaluation, a contestant is always able to see if it succesfully compiled. They are also able to see its scores on the public testcases of the task, if any. All information about the other so-called private testcases is kept hidden. Yet, a contestant can choose to use one of its tokens to "unlock" a certain submission of their choice. After they do so, detailed results are available for all testcases, as if they were all public. A token, once used, is consumed and lost forever. Contestants have a set of available tokens at their disposal, where the ones they use are picked from. These sets are managed by CMS according to rules defined by the contest administrators, as explained later in this section. For all official :doc:`score types <Score types>`, the public score is the score on public testcases, whereas the detailed score is the score on all testcases. This is not necessarily true for custom score types, as they can implement arbitrary logics to compute those values.
 
 Tokens also affect the score computation. That is, all "tokened" submissions will be considered, together with the last submitted one, when computing the score for a task. See also :ref:`configuringacontest_score-rounding`.
 
-There are two types of tokens: contest-tokens and task-tokens. When a contestant uses a token to unlock a submission he/she is in reality using two tokens, one of each type, and therefore needs to have both available. As the names suggest, contest-tokens are bound to the contest while task-tokens are bound to a specific task. That means that there is just one set of contest-tokens but there can be many sets of task-tokens (precisely one for every task). These sets are controlled independently by rules defined either on the contest or on the task.
+There are two types of tokens: contest-tokens and task-tokens. When a contestant uses a token to unlock a submission, they are really using one token of each type, and therefore needs to have both available. As the names suggest, contest-tokens are bound to the contest while task-tokens are bound to a specific task. That means that there is just one set of contest-tokens but there can be many sets of task-tokens (precisely one for every task). These sets are controlled independently by rules defined either on the contest or on the task.
 
 A token set can be disabled (i.e. there will never be tokens available for use), infinite (i.e. there will always be tokens available for use) or finite. This setting is controlled by the ``token_mode`` parameter.
 
 If the token set is finite it can be effectively represented by a non-negative integer counter: its cardinality. When the contest starts (or when the user starts its per-user time-frame, see :ref:`configuringacontest_usaco-like-contests`) the set will be filled with ``token_gen_initial`` tokens (i.e. the counter is set to ``token_gen_initial``). If the set is not empty (i.e. the counter is not zero) the user can use a token. After that, the token is discarded (i.e. the counter is decremented by one). New tokens can be generated during the contest: ``token_gen_number`` new tokens will be given to the user after every ``token_gen_interval`` minutes from the start (note that ``token_gen_number`` can be zero, thus disabling token generation). If ``token_gen_max`` is set, the set cannot contain more than ``token_gen_max`` tokens (i.e. the counter is capped at that value). Generation will continue but will be ineffective until the contestant uses a token. Unset ``token_gen_max`` to disable this limit.
 
-The use of tokens can be limited with ``token_max_number`` and ``token_min_interval``: users cannot use more that ``token_max_number`` tokens in total (this parameter can be unset), and they have to wait at least ``token_min_interval`` seconds after they used a token before they can use another one (this parameter can be zero). Note that these have no effect in case of infinite tokens.
+The use of tokens can be limited with ``token_max_number`` and ``token_min_interval``: users cannot use more that ``token_max_number`` tokens in total (this parameter can be unset), and they have to wait at least ``token_min_interval`` seconds after they used a token before they can use another one (this parameter can be zero). These have no effect in case of infinite tokens.
 
-Having a finite set of both contest- and task-tokens can be very confusing, for the contestants as well as for the contest administrators. Therefore it's common to limit just one type of tokens, setting the other type to be infinite, in order to make the general token availability depend only on the availability of that type (e.g. if you just want to enforce a contest-wide limit on tokens set the contest-token set to be finite and set all task-token sets to be infinite). CWS is aware of this "implementation details" and when one type is infinite it just shows information about the other type, calling it simply "token" (i.e. removing the "contest-" or "task-" prefix).
+Having a finite set of both contest- and task-tokens can be very confusing, for the contestants as well as for the contest administrators. Therefore it is common to limit just one type of tokens, setting the other type to be infinite, in order to make the general token availability depend only on the availability of that type (e.g. if you just want to enforce a contest-wide limit on tokens set the contest-token set to be finite and set all task-token sets to be infinite). CWS is aware of this "implementation details" and when one type is infinite it just shows information about the other type, calling it simply "token" (i.e. removing the "contest-" or "task-" prefix).
 
 Note that "token sets" are "intangible": they're just a counter shown to the user, computed dynamically every time. Yet, once a token is used, a Token object will be created, stored in the database and associated with the submission it was used on.
 
+.. note::
+   The full-feedback mode introduced in IOI 2013 has not been ported upstream yet (see :gh_issue:`246`). Note that although disabling tokens and making all testcases public would give full feedback, the final scores would be computed differently: the one of the latest submission would be used instead of the maximum among all submissions. To achieve the correct scoring behavior, get in touch with the developers or check `the ML archives <http://www.freelists.org/post/contestms/applying-tokens-automatically,1>`_.
+
 Changing token rules during a contest may lead to inconsistencies. Do so at your own risk!
 
 
@@ -89,31 +92,35 @@ When CWS needs to show a timestamp to the user it first tries to show it accordi
 User login
 ==========
 
-Users log into CWS using a username and a password. These have to be specified, respectively, in the ``username`` and ``password`` fields (in cleartext!). These credentials need to be inserted (i.e. there's no way to have an automatic login, a "guest" session, etc.) and, if they match, the login (usually) succeeds. The user needs to login again if he/she doesn't navigate the site for ``cookie_duration`` seconds (specified in the :file:`cms.conf` file).
+Users log into CWS using a username and a password. These have to be specified, respectively, in the ``username`` and ``password`` fields (in cleartext!). These credentials need to be inserted (i.e. there's no way to have an automatic login, a "guest" session, etc.) and, if they match, the login (usually) succeeds. The user needs to login again if they do not navigate the site for ``cookie_duration`` seconds (specified in the :file:`cms.conf` file).
 
-In fact, there are other reasons that can cause the login to fail. If the ``ip_lock`` option (in :file:`cms.conf`) is set to ``true`` then the login will fail if the IP address that attempted it doesn't match the address or subnet in the ``ip`` field of the specified user. If ``ip`` is not set then this check will be skipped, even if ``ip_lock`` is ``true``. Note that if a reverse-proxy (like nginx) is in use then it is necessary to set ``is_proxy_used`` (in :file:`cms.conf`) to ``true``.
+In fact, there are other reasons that can cause the login to fail. If the ``ip_lock`` option (in :file:`cms.conf`) is set to ``true`` then the login will fail if the IP address that attempted it doesn't match the address or subnet in the ``ip`` field of the specified user. If ``ip`` is not set then this check is skipped, even if ``ip_lock`` is ``true``. Note that if a reverse-proxy (like nginx) is in use then it is necessary to set ``is_proxy_used`` (in :file:`cms.conf`) to ``true`` and configure the proxy in order to properly pass the ``X-Forwarded-For``-style headers (see :ref:`running-cms_recommended-setup`).
 
-The login can also fail if ``block_hidden_users`` (in :file:`cms.conf`) is ``true`` and the user one wants to login as has the ``hidden`` field set.
+The login can also fail if ``block_hidden_users`` (in :file:`cms.conf`) is ``true`` and the user trying to login as has the ``hidden`` field set.
 
 
 .. _configuringacontest_usaco-like-contests:
 
 USACO-like contests
 ===================
 
-The most peculiar trait of the `USACO <http://usaco.org/>`_ contests is that the contests themselves are many days long but each user is only able to compete for a few hours after their first login (after that he/she is not able to send any more submissions). This can be done in CMS too, using the ``per_user_time`` field of contests. If it is unset the contest will behave "normally", that is all users will be able to submit solutions from the contest's ``start`` until the contest's ``stop``. If, instead, ``per_user_time`` is set to a positive integer value then an use will only have a limited amount of time. In particular, after he/she logs in, he/she will be presented with an interface similar to the pre-contest one, with one addition: a "start" button. Clicking on this button starts the time-frame in which the user can compete (i.e. read statements, download attachments, submit solutions, use tokens, send user_tests, etc.). This time-frame ends after ``per_user_time`` seconds or when the contest ``stop`` time is reached, whichever comes first. After that the interface will be identical to the post-contest one: the user won't be able to do anything. See :gh_issue:`61`.
+One trait of the `USACO <http://usaco.org/>`_ contests is that the contests themselves are many days long but each user is only able to compete for a few hours after their first login (after that they are not able to send any more submissions). This can be done in CMS too, using the ``per_user_time`` field of contests. If it is unset the contest will behave "normally", that is all users will be able to submit solutions from the contest's beginning until the contest's end. If, instead, ``per_user_time`` is set to a positive integer value, then a user will only have a limited amount of time. In particular, after they log in, they will be presented with an interface similar to the pre-contest one, with one additional "start" button. Clicking on this button starts the time frame in which the user can compete (i.e. read statements, download attachments, submit solutions, use tokens, send user tests, etc.). This time frame ends after ``per_user_time`` seconds or when the contest ``stop`` time is reached, whichever comes first. After that the interface will be identical to the post-contest one: the user won't be able to do anything. See :gh_issue:`61`.
 
-The time at which the user clicks the "start" button is recorded in the ``starting_time`` field of the user. You can change that to shift the user's time-frame (but we suggest to use ``extra_time`` for that, explained in :ref:`configuringacontest_extra-time`) or unset it to make the user able to start its time-frame again. Do so at your own risk!
+The time at which the user clicks the "start" button is recorded in the ``starting_time`` field of the user. You can change that to shift the user's time frame (but we suggest to use ``extra_time`` for that, explained in :ref:`configuringacontest_extra-time`) or unset it to make the user able to start its time frame again. Do so at your own risk!
 
 
 .. _configuringacontest_extra-time:
 
-Extra time
-==========
+Extra time and delay time
+=========================
+
+Contest administrators may want to give some users a short additional amount of time in which they can compete to compensate for an incident (e.g. a hardware failure) that made them unable to compete for a while during the "intended" time frame. That's what the ``extra_time`` field of the users is for. The time frame in which the user is allowed to compete is expanded by its ``extra_time``, even if this would lead the user to be able to submit after the end of the contest.
+
+During extra time the user will continue to receive newly generated tokens. If you don't want them to have more tokens that other contestants, set the ``token_max_number`` parameter described above to the number of tokens you expect a user to have at their disposal during the whole contest (if it doesn't already have a value less than or equal to this). See also :gh_issue:`29`.
 
-Contest administrators may want to give some users a short additional amount of time in which they can compete to compensate for an incident (e.g. a hardware failure) that made them unable to compete for a while during the "intended" time-frame. That's what the ``extra_time`` field of the users is for. The time-frame in which the user is allowed to submit solutions is expanded by its ``extra_time``, even if this would lead the user to be able to submit after the end of the contest.
+Contest administrators can also alter the competition time of a contestant setting ``delay_time``, which has the effect of translating the competition time window for that contestant of the specified numer of seconds in the future. Thus, while setting ``extra_time`` *adds* some times at the end of the contest, setting ``delay_time`` *moves* the whole time window. As for ``extra_time``, setting ``delay_time`` may extend the contestant time window beyond the end of the contest itself.
 
-Note that in its extra time the user will continue to receive newly generated tokens. If you don't want him/her to have more tokens that other contestants set the ``token_max_number`` parameter described above to the number of tokens you expect a user to have at his/her disposal during the whole contest (if it doesn't already have a value less than or equal to this). See also :gh_issue:`29`.
+Both options have to be set to a non negative number. They can be used together, producing both their effects. Please read :doc:`Detailed timing configuration` for a more in-depth discussion of their exact effect.
 
 Note also that submissions sent during the extra time will continue to be considered when computing the score, even if the ``extra_time`` field of the user is later reset to zero (for example in case the user loses the appeal): you need to completely delete them from the database.
 

docs/Running CMS.rst

@@ -97,6 +97,8 @@ The flag ``-a`` informs ResourceService that it has to start all other services,
 Note that it is your duty to keep CMS's configuration synchronized among the machines.
 
 
+.. _running-cms_recommended-setup:
+
 Recommended setup
 =================