Home Login
Documentation / Call for papers / Evaluation Process

Evaluation Process

Set up a structured process to review and rate submissions with your team.

Overview

The evaluation process helps you:

Setting Up Evaluation

Add Evaluators

  1. Go to Team or Users
  2. Click "+ Add User"
  3. Select "Evaluator" role
  4. Enter email address
  5. Send invitation

Evaluator Permissions

Evaluators can:

Evaluators cannot:

Assigning Submissions

Manual Assignment

  1. Go to Submissions
  2. Open a submission
  3. Click "Assign Evaluator"
  4. Select evaluator(s)
  5. Save

Bulk Assignment

  1. Select multiple submissions
  2. Click "Batch Actions"
  3. Choose "Assign Evaluators"
  4. Select evaluators
  5. Apply

Auto-Assignment

If available:

  1. Go to CFP Settings > Evaluation
  2. Enable "Auto-assign"
  3. Set distribution rules
  4. Evaluators receive equal workload

Evaluation Workflow

Evaluator Process

  1. Login - Access evaluator dashboard
  2. View assignments - See assigned submissions
  3. Read submission - Review content
  4. Rate - Provide scores
  5. Comment - Add feedback
  6. Submit - Save evaluation

Tracking Progress

Monitor evaluation status:

Status Meaning
Not Started No evaluations yet
In Progress Some evaluations complete
Complete All assigned reviews done

Rating System

Scoring

Typical rating scale:

Score Meaning
5 Excellent - Must include
4 Very Good - Strong candidate
3 Good - Consider if space
2 Fair - Below average
1 Poor - Do not accept

Rating Criteria

Suggested criteria:

Criterion Description
Topic Relevance Fits event theme
Content Quality Well-structured proposal
Speaker Experience Qualified presenter
Audience Interest Appeals to attendees
Originality Fresh perspective

Feedback Guidelines

For Evaluators

When reviewing, consider:

Constructive Feedback

Write feedback that:

Good feedback example:

"Strong technical topic with clear learning objectives. The abstract could benefit from more detail about the hands-on portion. Speaker's background in Kubernetes is a great fit."

Viewing Evaluations

As an Organizer

  1. Go to Submissions
  2. Open a submission
  3. View "Evaluations" section
  4. See all ratings and comments

Evaluation Summary

For each submission:

Comparison View

Compare submissions:

  1. Go to Submissions > Analytics
  2. View ranking by score
  3. Filter by track/category

Making Decisions

Using Evaluation Data

After evaluations:

  1. Sort by score - Highest rated first
  2. Review distribution - Ensure balance
  3. Consider factors - Diversity, topics, levels
  4. Make selections - Accept top submissions

Handling Edge Cases

Tied scores:

Split opinions:

Evaluation Reports

Generate Report

  1. Go to CFP > Reports
  2. Select "Evaluation Report"
  3. Choose format (Excel/PDF)
  4. Download

Report Contents

Best Practices

For Organizers

For Evaluators

Timeline

Phase Duration
Assign evaluators 1-2 days
Evaluation period 2-3 weeks
Review results 3-5 days
Final decisions 2-3 days

Blind Reviews

For unbiased evaluation:

  1. Go to CFP Settings > Evaluation
  2. Enable "Blind Review"
  3. Speaker names hidden from evaluators

Previous: Managing Submissions Next: Speaker Communication