Difference between revisions of "Judging/Rubric"

m
 
Line 127: Line 127:
 
<tr><td>4</td>
 
<tr><td>4</td>
 
<td>Project</td>
 
<td>Project</td>
−
<td>How much did the team accomplish?</td>
+
<td>How much did the team accomplish (addressed a real world problem, produced functional BioBricks, carried out Human Practices, created a wiki, presentation, poster, etc.)?</td>
 
</tr>
 
</tr>
 
<tr><td>5</td>
 
<tr><td>5</td>
Line 135: Line 135:
 
<tr><td>6</td>
 
<tr><td>6</td>
 
<td>Project</td>
 
<td>Project</td>
−
<td>How well are engineering principles used?</td>
+
<td>How well were engineering principles (for example: modularity, protoyping, debugging, standardized measurements, etc.) used?</td>
 
</tr>
 
</tr>
 
<tr><td>7</td>
 
<tr><td>7</td>
Line 155: Line 155:
 
<tr><td>10</td>
 
<tr><td>10</td>
 
<td>Track Specific - Standard Tracks</td>
 
<td>Track Specific - Standard Tracks</td>
−
<td>Are the parts functions and behaviors well-documented in the Registry?</td>
+
<td>Are the parts well documented in the Registry?</td>
 
</tr>
 
</tr>
 
<tr><td> </td>
 
<tr><td> </td>
Line 167: Line 167:
 
<tr><td>10</td>
 
<tr><td>10</td>
 
<td>Track Specific - Special Tracks</td>
 
<td>Track Specific - Special Tracks</td>
−
<td>Are the project components (hardware, software, art &amp; design, etc) thoroughly documented on their wiki?</td>
+
<td>Are the project components (hardware, software, art & design, etc.) thoroughly documented on their wiki?</td>
 
</tr>
 
</tr>
 
<tr><td> </td>
 
<tr><td> </td>
Line 191: Line 191:
 
<tr><td>3</td>
 
<tr><td>3</td>
 
<td>Wiki</td>
 
<td>Wiki</td>
−
<td>Does the team provide convincing evidence to support their conclusions?</td>
+
<td>Does the team clearly document their project and support their results with convincing evidence? </td>
 
</tr>
 
</tr>
 
<tr><td>4</td>
 
<tr><td>4</td>
 
<td>Wiki</td>
 
<td>Wiki</td>
−
<td>How well does the team describe what they did and what was done by others?</td>
+
<td>How well does the team describe what they did and what was done by others on the Attributions page?</td>
 
</tr>
 
</tr>
 
<tr><td>5</td>
 
<tr><td>5</td>
Line 207: Line 207:
 
<tr><td>1</td>
 
<tr><td>1</td>
 
<td>Presentation</td>
 
<td>Presentation</td>
−
<td>Did the presentation flow well?</td>
+
<td>Was the presentation thorough, clear, and easy to understand?</td>
 
</tr>
 
</tr>
 
<tr><td>2</td>
 
<tr><td>2</td>
 
<td>Presentation</td>
 
<td>Presentation</td>
−
<td>How professional is the graphic design in terms of layout and composition?</td>
+
<td>How visually appealing was the presentation?</td>
 
</tr>
 
</tr>
 
<tr><td>3</td>
 
<tr><td>3</td>
Line 231: Line 231:
 
<tr><td>2</td>
 
<tr><td>2</td>
 
<td>Poster</td>
 
<td>Poster</td>
−
<td>How professional is the graphic design in terms of layout and composition?</td>
+
<td>How well is the project described on their poster?</td>
 
</tr>
 
</tr>
 
<tr><td>3</td>
 
<tr><td>3</td>
 
<td>Poster</td>
 
<td>Poster</td>
−
<td>Did you find the poster appealing?</td>
+
<td>Did you find the poster visually appealing?</td>
 
</tr>
 
</tr>
 
<tr><td>4</td>
 
<tr><td>4</td>
Line 247: Line 247:
 
<tr><td>1</td>
 
<tr><td>1</td>
 
<td>Integrated Human Practices</td>
 
<td>Integrated Human Practices</td>
−
<td>Was their work integrated into their project?</td>
+
<td>Was their Human Practices work integrated into their project?</td>
 
</tr>
 
</tr>
 
<tr><td>2</td>
 
<tr><td>2</td>
Line 267: Line 267:
 
<tr><td>1</td>
 
<tr><td>1</td>
 
<td>Education &amp; Public Engagement</td>
 
<td>Education &amp; Public Engagement</td>
−
<td>Did their work establish a dialogue?</td>
+
<td>How well did their work promote mutual learning and engagement?</td>
 
</tr>
 
</tr>
 
<tr><td>2</td>
 
<tr><td>2</td>
Line 287: Line 287:
 
<tr><td>1</td>
 
<tr><td>1</td>
 
<td>Model</td>
 
<td>Model</td>
−
<td>How impressive is the mathematical modeling?</td>
+
<td>How impressive is the modeling?</td>
 
</tr>
 
</tr>
 
<tr><td>2</td>
 
<tr><td>2</td>
 
<td>Model</td>
 
<td>Model</td>
−
<td>Did the model help the team understand their part or device?</td>
+
<td>Did the model help the team understand a part, device, or system?</td>
 
</tr>
 
</tr>
 
<tr><td>3</td>
 
<tr><td>3</td>
 
<td>Model</td>
 
<td>Model</td>
−
<td>Did the team use measurements of the device to develop the model?</td>
+
<td>Did the team use measurements of a part, device, or system to develop the model?</td>
 
</tr>
 
</tr>
 
<tr><td>4</td>
 
<tr><td>4</td>
Line 315: Line 315:
 
<tr><td>3</td>
 
<tr><td>3</td>
 
<td>Measurement</td>
 
<td>Measurement</td>
−
<td>Are there web-based support materials?</td>
+
<td>Is it useful to other projects?</td>
 
</tr>
 
</tr>
 
<tr><td>4</td>
 
<tr><td>4</td>
 
<td>Measurement</td>
 
<td>Measurement</td>
−
<td>Is it useful to other projects?</td>
+
<td>Did the team appropriately use controls to validate the measurement process and calibrate units?</td>
−
</tr>
+
−
<tr><td>5</td>
+
−
<td>Measurement</td>
+
−
<td>Was a standard reference sample included?</td>
+
 
</tr>
 
</tr>
 
<tr><td> </td>
 
<tr><td> </td>
Line 354: Line 350:
 
</tr>
 
</tr>
 
<tr><td>1</td>
 
<tr><td>1</td>
−
<td>Applied Design</td>
+
<td>Product Design</td>
−
<td>How well did the project address potential applications and implications of synthetic biology?</td>
+
<td>Does the team's synthetic biology product address a real-world need?</td>
 
</tr>
 
</tr>
 
<tr><td>2</td>
 
<tr><td>2</td>
−
<td>Applied Design</td>
+
<td>Product Design</td>
−
<td>How creative, original, and compelling was the project?</td>
+
<td>Does their product successfuly incorporate synthetic biology into its design?</td>
 
</tr>
 
</tr>
 
<tr><td>3</td>
 
<tr><td>3</td>
−
<td>Applied Design</td>
+
<td>Product Design</td>
−
<td>How impressive was the project installation in the art &amp; design exhibition space?</td>
+
<td>How impressive was the demonstration (at the Giant Jamboree or through video) and documentation of their product?</td>
 
</tr>
 
</tr>
 
<tr><td>4</td>
 
<tr><td>4</td>
−
<td>Applied Design</td>
+
<td>Product Design</td>
−
<td>How well did the team engage in collaboration with people outside of their primary fields?</td>
+
<td>How well did the team engage with potential users and/or experts and incorporate feedback into the product design?</td>
 +
</tr>
 +
<tr><td>5</td>
 +
<td>Product Design</td>
 +
<td>Has the team thoughtfully considered the positive and negative implications of their product?</td>
 
</tr>
 
</tr>
 
<tr><td> </td>
 
<tr><td> </td>
Line 383: Line 383:
 
<tr><td>3</td>
 
<tr><td>3</td>
 
<td>Software Tool</td>
 
<td>Software Tool</td>
−
<td>Did the team use non-trivial algorithms or designs?</td>
+
<td>Is it useful to other projects?</td>
 
</tr>
 
</tr>
 
<tr><td>4</td>
 
<tr><td>4</td>
 
<td>Software Tool</td>
 
<td>Software Tool</td>
−
<td>How easily can others embed this software in new workflows?</td>
+
<td>Does the team demonstrate that their software can be embedded in new workflows?</td>
 
</tr>
 
</tr>
 
<tr><td>5</td>
 
<tr><td>5</td>
Line 419: Line 419:
 
<tr><td>1</td>
 
<tr><td>1</td>
 
<td>Plant Synthetic Biology</td>
 
<td>Plant Synthetic Biology</td>
−
<td>How impressive was the use of a plant chassis?</td>
+
<td>How successful was the team in engineering a plant or algal cell?</td>
 
</tr>
 
</tr>
 
<tr><td>2</td>
 
<tr><td>2</td>
 
<td>Plant Synthetic Biology</td>
 
<td>Plant Synthetic Biology</td>
−
<td>How impressive was the collection of parts made for the plant chassis?</td>
+
<td>Does their work address a need or problem in plant synthetic biology?</td>
 
</tr>
 
</tr>
 
<tr><td>3</td>
 
<tr><td>3</td>
Line 452: Line 452:
 
<td>New Basic Part</td>
 
<td>New Basic Part</td>
 
<td>Is it useful to the community?</td>
 
<td>Is it useful to the community?</td>
 +
</tr>
 +
<tr><td>5</td>
 +
<td>New Basic Part</td>
 +
<td>How well characterized (experimentally measured) is this Basic Part when tested in a device?</td>
 
</tr>
 
</tr>
 
<tr><td> </td>
 
<tr><td> </td>
Line 472: Line 476:
 
<td>New Composite Part</td>
 
<td>New Composite Part</td>
 
<td>Is it useful to the community?</td>
 
<td>Is it useful to the community?</td>
 +
</tr>
 +
<tr><td>5</td>
 +
<td>New Composite Part</td>
 +
<td>How well characterized (experimentally measured) is this Composite Part?</td>
 
</tr>
 
</tr>
 
<tr><td> </td>
 
<tr><td> </td>
Line 483: Line 491:
 
<tr><td>2</td>
 
<tr><td>2</td>
 
<td>Part Collection</td>
 
<td>Part Collection</td>
−
<td>How does the documentation compare to BBa_K747000 and BBa_K525710?</td>
+
<td>How does the documentation compare to the BBa_K747000-095 collection?</td>
 
</tr>
 
</tr>
 
<tr><td>3</td>
 
<tr><td>3</td>
 
<td>Part Collection</td>
 
<td>Part Collection</td>
−
<td>Did the team submit an internally complete collection allowing it to be used without any further manipulation or parts from outside Registry?</td>
+
<td>Did the team submit a complete collection allowing it to be used without any further manipulation or parts from outside of the Registry?</td>
 
</tr>
 
</tr>
 
<tr><td>4</td>
 
<tr><td>4</td>
Line 495: Line 503:
 
<tr><td>5</td>
 
<tr><td>5</td>
 
<td>Part Collection</td>
 
<td>Part Collection</td>
−
<td>Did the team create excellent documentation to allow future use of this collection?</td>
+
<td>Is it useful to the community?</td>
 
</tr>
 
</tr>
 
<tr><td> </td>
 
<tr><td> </td>

Latest revision as of 19:02, 28 June 2018

Loading...

iGEM Judging Rubric

Judging is a complex task and can seem mysterious to iGEMers at times. We're aiming to help teams understand how they are evaluated and provide more information ahead of time. While the individual decisions judges make about teams must remain confidential until after the Jamboree, the systems they use do not.

The main mechanism through which iGEM teams are evaluated is called the rubric. The rubric is composed of three main sections:

  1. Medals Section
  2. Project Section
  3. Special Awards Section

Each section is called a category. Within each category, there are 2 - 8 questions that we call aspects (shown below). Each aspect has 6 language choices that covers a range of how the judge evaluating should feel about the quality of the work. Unlike the aspects, these language choices will not be shown. We want iGEMers to know how they are being evaluated, but we don't want to "teach to the test." The language choices correspond to roughly:

  1. Amazing!
  2. Great
  3. Good
  4. Present
  5. Bad
  6. Absent

Each section of the rubric has a separate function and correlates with different awards. The Medals section obviously refers to a team's work convincing the judges they have achieved specific medal criteria. These criteria can be found on the Medals Page and won't be reiterated here.

The Project section is composed of two sub-sections: the main project category and the track-specific category. The main project category has eight aspects while the track-specific category only has two. Combined, these ten aspects determine the scores for the teams who will win their tracks and will also determine the finalist teams. This category is arguably the most important part of the evaluation for an iGEM team.

The final section of the judging rubric determines special awards. Each award has its own category in the rubric with either four or five aspects. This part of the evaluation integrates with the Pages for Awards system. To be eligible for an award, teams need to complete the corresponding page on the wiki and fill out a 150 word description on the judging form (click on your team name, and the link to your judging form is at the top of your team information page) .

This rubric is the result of more than four years of development, hundreds of hours of discussion, dozens and dozens of meetings, and thousands of emails between some of the most experienced advisers in iGEM. We are continuously improving and tweaking the rubric, but the system we have is extremely effective at selecting for the winning teams that best represent the values of iGEM.

Number Category Aspects
1 Project How impressive is this project?
2 Project How creative is the team's project?
3 Project Did the project work?
4 Project How much did the team accomplish (addressed a real world problem, produced functional BioBricks, carried out Human Practices, created a wiki, presentation, poster, etc.)?
5 Project Is the project likely to have an impact?
6 Project How well were engineering principles (for example: modularity, protoyping, debugging, standardized measurements, etc.) used?
7 Project How thoughtful and thorough was the team's consideration of human practices?
8 Project How much of the work did the team do themselves and how much was done by others?
9 Track Specific - Standard Tracks Did the team design a project based on synthetic biology and standard parts?
10 Track Specific - Standard Tracks Are the parts well documented in the Registry?
9 Track Specific - Special Tracks Did the team design a project based on synthetic biology?
10 Track Specific - Special Tracks Are the project components (hardware, software, art & design, etc.) thoroughly documented on their wiki?
Special Prizes
1 Wiki Do I understand what the team accomplished?
2 Wiki Is the wiki attractive and easy to navigate?
3 Wiki Does the team clearly document their project and support their results with convincing evidence?
4 Wiki How well does the team describe what they did and what was done by others on the Attributions page?
5 Wiki Will the wiki be a compelling record of the team's project for future teams?
1 Presentation Was the presentation thorough, clear, and easy to understand?
2 Presentation How visually appealing was the presentation?
3 Presentation Did you find the presentation engaging?
4 Presentation How competent were the team members at answering questions?
1 Poster Did the poster flow well?
2 Poster How well is the project described on their poster?
3 Poster Did you find the poster visually appealing?
4 Poster How competent were the team members at answering questions?
1 Integrated Human Practices Was their Human Practices work integrated into their project?
2 Integrated Human Practices Does it serve as an inspiring example to others?
3 Integrated Human Practices Is it documented in a way that others can build upon?
4 Integrated Human Practices Was it thoughtfully implemented? (did they explain the context, rationale, prior work)
1 Education & Public Engagement How well did their work promote mutual learning and engagement?
2 Education & Public Engagement Does it serve as an inspiring example to others?
3 Education & Public Engagement Is it documented in a way that others can build upon?
4 Education & Public Engagement Was it thoughtfully implemented? (did they explain the context, rationale, prior work)
1 Model How impressive is the modeling?
2 Model Did the model help the team understand a part, device, or system?
3 Model Did the team use measurements of a part, device, or system to develop the model?
4 Model Does the modeling approach provide a good example for others?
1 Measurement Is the measurement potentially repeatable?
2 Measurement Is the protocol well described?
3 Measurement Is it useful to other projects?
4 Measurement Did the team appropriately use controls to validate the measurement process and calibrate units?
1 Entrepreneurship Customer Discovery - Has the team interviewed a representative number of potential customers for the technology and clearly communicated what they learned?
2 Entrepreneurship Based on their interviews, does the team have a clear hypothesis describing their customers' needs?
3 Entrepreneurship Does the team present a convincing case that their product meets the customers' needs?
4 Entrepreneurship Has the team demonstrated a minimum viable (MVP) product? And does the team have customers to commit (LOI, etc.) to purchasing it / using it?
5 Entrepreneurship Does the team have a viable and understood business model/value proposition to take their company to market?
1 Product Design Does the team's synthetic biology product address a real-world need?
2 Product Design Does their product successfuly incorporate synthetic biology into its design?
3 Product Design How impressive was the demonstration (at the Giant Jamboree or through video) and documentation of their product?
4 Product Design How well did the team engage with potential users and/or experts and incorporate feedback into the product design?
5 Product Design Has the team thoughtfully considered the positive and negative implications of their product?
1 Software Tool How well is the software using and supporting existing synthetic biology standards and platforms?
2 Software Tool Was this software validated by experimental work?
3 Software Tool Is it useful to other projects?
4 Software Tool Does the team demonstrate that their software can be embedded in new workflows?
5 Software Tool How user-friendly is the software?
1 Hardware Does the hardware address a need or problem in synthetic biology?
2 Hardware Did the team conduct user testing and learn from user feedback?
3 Hardware Did the team demonstrate utility and functionality in their hardware proof of concept?
4 Hardware Is the documentation of the hardware system sufficient to enable reproduction by other teams?
1 Plant Synthetic Biology How successful was the team in engineering a plant or algal cell?
2 Plant Synthetic Biology Does their work address a need or problem in plant synthetic biology?
3 Plant Synthetic Biology How well did the team use the special attributes of the plant chassis?
4 Plant Synthetic Biology Are the parts/tools/protocols for plants made during this project useful to other teams?
1 New Basic Part How does the documentation compare to BBa_K863006 and BBa_K863001?
2 New Basic Part How new/innovative is it?
3 New Basic Part Did the team show the part works as expected?
4 New Basic Part Is it useful to the community?
5 New Basic Part How well characterized (experimentally measured) is this Basic Part when tested in a device?
1 New Composite Part How does the documentation compare to BBa_K404122 and BBa_K863005?
2 New Composite Part How new/innovative is it?
3 New Composite Part Did the team show the part works as expected?
4 New Composite Part Is it useful to the community?
5 New Composite Part How well characterized (experimentally measured) is this Composite Part?
1 Part Collection Is this collection a coherent group of parts meant to be used as a collection, or just a list of all the parts the team made?
2 Part Collection How does the documentation compare to the BBa_K747000-095 collection?
3 Part Collection Did the team submit a complete collection allowing it to be used without any further manipulation or parts from outside of the Registry?
4 Part Collection Did the team finish building a functional system using this collection?
5 Part Collection Is it useful to the community?