Scorecards

Scorecards score every entity against rules you define across four categories: documentation, manifest quality, ownership, and security. Each entity gets a letter grade (A+ to F) that updates automatically as metadata changes.

How Scorecards Work

Define categories and rules - What standards matter to your organization
Automated evaluation - Shoehorn scores entities against rules automatically
Grade assignment - Each entity receives a letter grade (A+ to F)
Track improvement - Monitor scores over time

Default Categories

Shoehorn ships with four default scoring categories:

Category	Weight	Description
Documentation	25%	README, descriptions, runbooks, changelogs
Manifest	35%	Service metadata completeness
Ownership	20%	Team assignment and relations
Security	20%	Security tooling and vulnerability status

Default Rules

Documentation (25 points)

Rule	Points	Severity	What It Checks
`readme-exists`	10	Required	Entity has a README file
`description-exists`	5	Required	Entity has a description
`runbooks-exist`	5	Recommended	Runbooks are defined
`changelog-exists`	5	Recommended	Changelog path is set

Manifest (35 points)

Rule	Points	Severity	What It Checks
`service-name`	5	Required	Service name is defined
`service-type`	5	Required	Service type is set
`service-tier`	5	Recommended	Service tier is assigned
`lifecycle-set`	5	Required	Lifecycle stage is set
`has-tags`	5	Recommended	At least one tag exists
`has-links`	5	Recommended	At least one link exists
`has-interfaces`	5	Recommended	API interfaces are defined

Ownership (20 points)

Rule	Points	Severity	What It Checks
`has-owner`	10	Required	An owner team is assigned
`team-topic`	5	Recommended	Repository has team topic
`has-relations`	5	Recommended	Relations are defined

Security (20 points)

Rule	Points	Severity	What It Checks
`dependabot-enabled`	5	Recommended	Dependabot is active on repo
`codeql-enabled`	5	Recommended	CodeQL scanning is enabled
`secret-scanning`	5	Recommended	Secret scanning is enabled
`no-critical-vulns`	5	Required	No critical/high vulnerabilities

Grading Scale

Grade	Score Range
A+	97-100
A	93-96
A-	90-92
B+	87-89
B	83-86
B-	80-82
C+	77-79
C	73-76
C-	70-72
D+	67-69
D	63-66
D-	60-62
F	0-59

Check Types

Rules use different check types to evaluate entities:

Check Type	Description	Example Config
`exists`	Checks if something exists	`target: "README.md"`
`manifest_field`	Checks a manifest field value	`field: "service.name"`
`pattern`	Pattern matching	`pattern: "^[a-z0-9-]+$"`
`external`	External data check	`source: "repository_topics"`
`percentage`	Percentage-based threshold	`source: "coverage", value: 80`

Viewing Scorecards

Entity Scorecard

Navigate to any entity and click the Scorecard tab to see:

Overall score and grade
Category breakdown with individual rule results
Pass/fail status for each rule
Recommendations for improvement

Governance Dashboard

The governance dashboard at Governance > Dashboard shows:

Organization-wide average score and grade
Score distribution across all entities
Top entities needing improvement

Custom Rules

Creating a Rule

curl -X POST https://shoehorn.example.com/api/v1/scorecard/rules \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "id": "has-oncall",
    "name": "On-Call Rotation",
    "description": "Service must have an on-call rotation defined",
    "category_id": "ownership",
    "check_type": "manifest_field",
    "config": {
      "field": "extras",
      "expected": "oncall"
    },
    "points": 10,
    "severity": "recommended"
  }'

Creating a Category

curl -X POST https://shoehorn.example.com/api/v1/scorecard/categories \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "reliability",
    "display_name": "Reliability",
    "weight": 15,
    "description": "Service reliability and SLA standards"
  }'

Rule Overrides

Teams can override specific rules for their entities with justification:

curl -X POST https://shoehorn.example.com/api/v1/entities/<id>/scorecard/overrides \
  -H "Authorization: Bearer <token>" \
  -H "Content-Type: application/json" \
  -d '{
    "rule_id": "dependabot-enabled",
    "disabled": true,
    "reason": "Using Snyk instead of Dependabot",
    "expires_at": "2027-06-30"
  }'

Overrides can:

Disable a rule (with required justification)
Change points (custom point value)
Set expiration (auto-revert after date)

Scorecard in Manifests

You can define scorecard configuration within your manifest for team-level customization:

schemaVersion: 1
kind: TeamScorecard
metadata:
  team: payments-team
  description: "Custom scorecard for payments team"

extends: "global"  # Inherit from global scorecard

categories:
  reliability:
    weight: 20
    rules:
      - id: has-slo
        name: "SLO Defined"
        description: "Service has an SLO target"
        checkType: manifest_field
        config:
          field: "annotations.slo-target"
        points: 10
        severity: required