This site is from a past semester! The current version will be here when the new semester starts.

tP: Practical ExamtP: Supervision


tP: Grading

Note that project grading is not competitive (not bell curved). CS2103T projects will be assessed separately from CS2103 projects. Given below is the marking scheme.

Total: 50 marks ( 35 individual marks + 10 team marks)

See the sections below for details of how we assess each aspect.

1. Project Grading: Product Design [ 5 marks]

Evaluates:

  • how well your features fit together to form a cohesive product
    (not how many features or how big/novel/interesting/difficult the features are)
  • how well it matches the target user

Evaluated by:

  • the teaching team (based on product demo and user guide)
  • peers from other teams (based on peer testing and user guide)

In addition, feature flaws reported in the PE will be considered when grading this aspect.

These are considered feature flaws:
The feature does not solve the stated problem of the intended user i.e., the feature is 'incomplete'
Hard-to-test features
Features that don't fit well with the product
Features that are not optimized enough for fast-typists or target users
Violations of given project constraints

2. Project Grading: Implementation [ 15 marks]

2A. Code quality

Evaluates: the quality of the parts of the code you claim as written by you

Evaluation method: manual inspection by tutors + automated-analysis by a script

Criteria:

  • At least some evidence of these (see here for more info)

    • logging
    • exceptions
    • assertions
    • defensive coding

  • No coding standard violations e.g. all boolean variables/methods sounds like booleans. Checkstyle can prevent only some coding standard violations; others need to be checked manually.

  • SLAP is applied at a reasonable level. Long methods or deeply-nested code are symptoms of low-SLAP.

  • No noticeable code duplications i.e. if there multiple blocks of code that vary only in minor ways, try to extract out similarities into one place, especially in test code.

  • Evidence of applying code quality guidelines covered in the module.

2B. Effort

Evaluates: how much value you contributed to the product

Method:

  • Step 1: Evaluate the effort for the entire project. This is evaluated by peers who tested your product, and tutors.

  • Step 2: Evaluate how much of that effort can be attributed to you. This is evaluated by team members, and tutors.

  • Baseline: If your team received a value higher than 10 in step 1 and the team agrees that you did roughly an equal share of implementation work, you should receive full marks for effort.

3. Project Grading: QA [ 10 marks]

3A. Developer Testing:

Evaluates: How well you tested your own feature

Based on:

  1. functionality bugs in your work found by others during the Practical Exam (PE)
  2. your test code (note our expectations for automated testing)

These are considered functionality bugs:
Behavior differs from the User Guide
A legitimate user behavior is not handled e.g. incorrect commands, extra parameters
Behavior is not specified and differs from normal expectations e.g. error message does not match the error

3B. System/Acceptance Testing:

Evaluates: How well you can system-test/acceptance-test a product

Based on: bugs you found in the PE. In addition to functionality bugs, you get credit for reporting documentation bugs and feature flaws.

Grading bugs found in the PE
  • Of Developer Testing component, based on the bugs found in your code3A and System/Acceptance Testing component, based on the bugs found in others' code3B above, the one you do better will be given a 70% weight and the other a 30% weight so that your total score is driven by your strengths rather than weaknesses.
  • Bugs rejected by the dev team, if the rejection is approved by the teaching team, will not affect marks of the tester or the developer.
  • The penalty/credit for a bug varies based on the severity of the bug: severity.High > severity.Medium > severity.Low > severity.VeryLow
  • The three types (i.e., type.FunctionalityBug, type.DocumentationBug, type.FeatureFlaw) are counted for three different grade components. The penalty/credit can vary based on the bug type. Given that you are not told which type has a bigger impact on the grade, always choose the most suitable type for a bug rather than try to choose a type that benefits your grade.
  • The penalty for a bug is divided equally among assignees.
  • Developers are not penalized for duplicate bug reports they received but the testers earn credit for duplicate bug reports they submitted as long as the duplicates are not submitted by the same tester.
  • i.e., the same bug reported by many testersObvious bugs earn less credit for the tester and slightly higher penalty for the developer.
  • If the team you tested has a low bug count i.e., total bugs found by all testers is low, we will fall back on other means (e.g., performance in PE dry run) to calculate your marks for system/acceptance testing.
  • Your marks for developer testing depends on the bug density rather than total bug count. Here's an example:
    • n bugs found in your feature; it is a big feature consisting of lot of code → 4/5 marks
    • n bugs found in your feature; it is a small feature with a small amount of code → 1/5 marks
  • You don't need to find all bugs in the product to get full marks. For example, finding half of the bugs of that product or 4 bugs, whichever the lower, could earn you full marks.
  • Excessive incorrect downgrading/rejecting/marking as duplicatesduplicate-flagging, if deemed an attempt to game the system, will be penalized.

4. Project Grading: Documentation [ 10 marks]

Evaluates: your contribution to project documents

Method: Evaluated in two steps.

  • Step 1: Evaluate the whole UG and DG. This is evaluated by peers who tested your product, and tutors.

  • Step 2: Evaluate how much of that effort can be attributed to you. This is evaluated by team members, and tutors.

  • In addition, UG and DG bugs you received in the PE will be considered for grading this component.

These are considered UG bugs (if they hinder the reader):

Use of visuals

  • Not enough visuals e.g., screenshots/diagrams
  • The visuals are not well integrated to the explanation
  • The visuals are unnecessarily repetitive e.g., same visual repeated with minor changes

Use of examples:

  • Not enough or too many examples e.g., sample inputs/outputs

Explanations:

  • The target user for the product and/or the value proposition is not specified clearly.
  • The explanation is too brief or unnecessarily long.
  • The information is hard to understand for the target audience. e.g., using terms the reader might not know

Neatness/correctness:

  • looks messy
  • not well-formatted
  • broken links, other inaccuracies, typos, etc.
  • hard to read/understand
  • unnecessary repetitions (i.e., hard to see what's similar and what's different)

These are considered DG bugs (if they hinder the reader):

Those given as possible UG bugs ...

Architecture:

  • Symbols used are not intuitive
  • Indiscriminate use of double-headed arrows
  • e.g., the sequence diagram showing interactions between main componentsarchitecture-level diagrams contain lower-level details
  • Description given are not sufficiently high-level

UML diagrams:

  • Notation incorrect or not compliant with the notation covered in the module.
  • Some other type of diagram used when a UML diagram would have worked just as well.
  • The diagram used is not suitable for the purpose it is used.
  • The diagram is too complicated.

Code snippets:

  • Excessive use of code e.g., a large chunk of code is cited when a smaller extract would have sufficed.

Problems in User Stories. Examples:

  • Incorrect format
  • All three parts are not present
  • The three parts do not match with each other
  • Important user stories missing

Problems in Use Cases. Examples:

  • Important use cases missing (a use case is important if it involves a user interaction that is worthy of documenting e.g., it has multiple extensions -- this is not the same as the feature being important)
  • Formatting/notational errors
  • Incorrect step numbering
  • Unnecessary UI details mentioned
  • Missing/unnecessary steps
  • Missing extensions

Problems in NFRs. Examples:

  • Not really a Non-Functional Requirement
  • Not scoped clearly (i.e., hard to decide when it has been met)
  • Not reasonably achievable
  • Highly relevant NFRs missing

Problems in Glossary. Examples:

  • Unnecessary terms included
  • Important terms missing

5. Project Grading: Project Management [ 5 + 5 = 10 marks]

5A. Process:

Evaluates: How well you did in project management related aspects of the project, as an individual and as a team

Based on: tutor/grading-script observations of project milestones and GitHub data

Grading criteria:

  • Project done iteratively and incrementally (opposite: doing most of the work in one big burst)

  • Milestones reached on time (i.e., the midnight before of the tutorial) (to get a good grade for this aspect, achieve at least 75% of the recommended milestone progress).

  • Good use of GitHub milestones mechanism.

  • Good use of GitHub releases mechanism.

  • Good version control, based on the repo.

  • Reasonable attempt to use the forking workflow at least for the early part of the project.

  • Good task definition, assignment and tracking, based on the issue tracker.

  • Good use of buffers (opposite: everything at the last minute).

5B. Team-tasks:

Evaluates: How much you contributed to team-tasks

Based on: peer evaluations, tutor observations

Grading criteria: Do these to earn full marks.

  • Do close to an equal share of the team tasks (you can earn bonus marks by doing more than an equal share).
  • Have your code merged in at least four of weeks 7, 8, 9, 10, 11, 12

tP: Practical ExamtP: Supervision