Reviewing candidates

The review page

From the expert sidebar, navigate to Vetting → Applications. The queue lists every candidate application currently in a commit or reveal phase for any guild you're a member of. Each row shows the candidate, the job they applied to, the current phase, and a deadline countdown.

Clicking into an application opens the review page. It has three sections:

1. Candidate profile. Name, headline, LinkedIn, portfolio, resume. Read this first — it's the quickest way to build a prior.
2. Application responses. Every answer the candidate gave to the guild's application questions, structured by section.
3. Your review. The rubric form, sticky on the right side of the page on desktop. This is where you score.

Working the rubric

Every guild defines its own rubric. An Engineering rubric might have three criteria — Systems Thinking, Code Quality, Communication — each scored 0 to 100 with a short description of what each band means. A Design guild might have different criteria including Visual Quality and Crypto Fundamentals.

The rubric always appears in the right-hand panel with a slider or radio selector per criterion. An overall score is computed from the per-criterion scores. Every criterion also has an optional comment field — use it when your score on that criterion is meaningfully above or below your overall.

Three practical tips:

• Score against the band descriptions, not against the last candidate you saw. Relative grading creates drift.
• Use the whole range. If you only ever score between 60 and 80, you're compressing the signal and making consensus harder to calculate.
• Don't half-score. If you're genuinely split between two bands, write a comment explaining what tipped you.

Writing comments

Comments are the most under-used leverage point in reviewing. They're shown to other experts after reveal phase and surface in the candidate feedback aggregation that the hiring company receives. Good comments change votes and change hires.

What makes a good comment:

• Specific to the application. Reference actual answers or portfolio work. Generic praise and generic criticism both add no value.
• Focused on evidence. "The systems design answer conflates throughput and latency" is actionable. "Weak technical skills" is not.
• Short. Two to four sentences. Long comments rarely get read end-to-end.

What to avoid: hiring decisions, personal attacks, anything that couldn't be read aloud in a guild meeting without embarrassment.

Common mistakes

• Anchoring on the first signal. If the candidate's LinkedIn is impressive, it's easy to score their actual answers generously. Fight this by filling in your rubric scores based on the application answers themselves before you let background polish colour your read.
• Scoring on intent, not evidence. "They probably would do well at this" is a prediction. The rubric asks about what's actually in the application.
• Staying in the middle to avoid risk. Central-tendency bias is a thing. Clustering every score at 60 seems safe but actually gets you flagged as misaligned when consensus lands at 30 or 90.
• Forgetting to comment on extreme scores. If you give a 20 or a 95, write a comment. Anonymised outlier scores with no justification are the single largest source of post-reveal disputes.

Conflicts of interest

The platform doesn't currently enforce conflict checks automatically. It's on you to abstain from reviews where you have a material relationship with the candidate — current or former coworker, close personal friend, direct report, and so on.

To abstain, don't click into the review. There's no penalty for skipping reviews due to conflicts; there's a significant reputation penalty if another expert notices and reports you after the fact.