Measuring and Evaluating Affordances

Core Concepts

The Measurement Gap

You have spent the preceding modules shaping affordances in APIs, code structure, and collaborative practices. A natural question follows: how do you know if it worked? The honest answer is that there is no single, validated instrument for measuring affordances. Empirical research confirms that across information systems and engineering contexts, researchers have independently developed diverse techniques, tools, and methods that proliferate across disciplines without convergence. The transfer of affordance concepts from ecological psychology into software engineering has produced fragmentation in how affordances are operationalized and assessed.

This is not an excuse for avoiding evaluation. It is a constraint that shapes how you evaluate. Rather than scoring affordance quality with a single metric, you assemble evidence from multiple methods calibrated to what you are trying to understand.

The Three Affordance Dimensions You Are Evaluating

Effective evaluation depends on being precise about which dimension of affordance you are targeting. The literature consistently distinguishes three that require different methods:

• Intended affordances — what you designed and believe is possible. Assessed by expert inspection of the design itself.
• Perceived affordances — what engineers believe they can do when encountering the system. Accessed through behavioral observation, think-aloud protocols, and phenomenological interviews.
• Actualized affordances — what engineers actually do, as evidenced by behavioral records, log analysis, and usage patterns.

Different dimensions require different methodologies: there is no instrument that measures all three simultaneously, and conflating them produces misleading conclusions.

Affordances Are Multi-Actor

The same codebase, API, or architectural pattern will be actualized differently by different engineers depending on their capabilities, goals, and organizational context. Affordance actualization is inherently multi-actor and context-dependent: multiple valid actualizations coexist simultaneously for the same technology. This has a direct consequence for evaluation — measuring what one group of engineers does tells you something real, but not the whole story. An affordance that seniors actualize fluidly may remain entirely invisible to new joiners.

Organizational Sensemaking Shapes Perception

Organizational sensemaking processes directly shape which affordances engineers perceive as available. Affordances do not announce themselves. They emerge through collective interpretation — in code reviews, architectural discussions, onboarding conversations, and documentation practices. Organizations with structured sensemaking rituals enable teams to discover affordances that would otherwise remain hidden. Without deliberate sensemaking practices, affordance perception stays implicit and unevenly distributed across the team.

Affordances Bundle Across Levels

Affordances operate at individual, team, and organizational levels of analysis. An API that affords confident use by individual engineers may or may not enable team-level affordances for coordinated change. Those team affordances may or may not translate into organizational capability. Evaluation that only looks at individual usage misses whether the system is producing collective outcomes. When you assess affordance quality, ask: does individual facility with this system translate into effective team action? Does team practice translate into organizational capability?

---

Key Principles

1. Match method to dimension. Heuristic evaluation reveals gaps in intended affordances. Behavioral observation captures actualized affordances. Phenomenological interviews access perceived affordances. Using the wrong method for the dimension you care about produces noise, not insight.

2. Qualitative evidence is not a consolation prize. The dominance of qualitative methods in affordance research is not a sign of methodological weakness — it reflects the relational, context-sensitive nature of affordances. Rich qualitative data from interviews, observations, and artifact analysis produces actionable findings that aggregate quantitative metrics miss.

3. Measure variation, not just central tendency. Because the same affordance will be actualized differently by different actors, evaluation that only reports average behavior conceals the most informative signal — divergence between what senior engineers perceive and what new engineers perceive, or between what the team intends and what actually gets used.

4. Design for appropriability, evaluate for surprise. Contemporary design practice increasingly argues for building systems with "appropriability" — structural flexibility that enables engineers to discover and propagate new affordances through use. If evaluation only checks whether engineers are using the system as intended, it will consistently miss emergent use patterns that are valuable precisely because they were unanticipated.

5. Embed evaluation in existing rituals. Affordance assessment should not be a one-off audit. Architectural review cycles, developer experience retrospectives, and onboarding feedback loops are natural homes for affordance evidence gathering. Sensemaking practices that are structured and recurring enable teams to perceive affordances collectively that individuals would miss alone.

---

Step-by-Step Procedure: Conducting an Affordance Audit

This procedure describes a lightweight affordance audit for a codebase or API surface. It is designed to run within existing engineering cycles, not as a standalone research project.

Phase 1: Scope and Dimension Selection

1. Identify what you are evaluating. Choose a bounded surface: a specific module, a public API, a set of architectural patterns, or an onboarding workflow. Broad scope diffuses effort; narrow scope produces actionable findings.

2. Decide which dimension is most urgent. Are you concerned that engineers do not notice certain capabilities (perceived affordances)? Or that they notice them but do not use them effectively (actualization)? Or that the design itself contains conceptual mismatches (intended affordances)?

3. Select your actor groups. Who uses this surface? New joiners? Senior engineers? Engineers from adjacent teams? Product engineers versus platform engineers? Variation across groups is often where the most important gaps appear.

Phase 2: Heuristic Evaluation (Intended Affordances)

4. Assemble a small group of expert evaluators — 2–4 engineers who know the system well but can adopt a critical distance. Reviewers from adjacent teams work well.

5. Apply affordance-specific criteria. For each component of the surface under review, ask: Does this clearly signal what actions it supports? Does its structure match the mental models of its intended users? Are its constraints perceivable before use? Affordance-based evaluation frameworks examine the match between designed affordances and the actual capabilities of the user population.

   Decision point: If evaluators disagree substantially about what a component affords, this is itself a finding — the affordance is ambiguous, not just absent.

6. Document identified gaps as candidate affordance problems: affordances that are invisible, misleading, or absent given the intended actor group.

Phase 3: Behavioral Observation (Actualized Affordances)

7. Recruit engineers across your actor groups. Aim for variety in experience level and role. Even 3–5 participants surfaces meaningful patterns.

8. Define representative tasks. Ask participants to perform realistic work with the surface under evaluation — navigating to understand a module boundary, implementing a new feature using an API, debugging a constraint violation.

9. Observe and record. Note hesitation patterns, erroneous attempts, workarounds, and abandonment. Key indicators of affordance problems include: pausing before interaction, attempting to use elements in ways they do not support, and wide variation in task completion time across participants of similar experience.

   Decision point: If experienced engineers actualize affordances that novices do not perceive at all, you have a discoverability problem — not a capability problem. These require different interventions.

10. Optionally use think-aloud protocol. Ask participants to narrate their reasoning during the task. This gives you access to the perceived affordance layer without requiring separate interviews.

Phase 4: Micro-Phenomenological Interview (Perceived Affordances)

11. Review observations and identify critical moments. Look for instances of hesitation, unexpected discovery, adaptation, or confusion in recorded sessions.

12. Conduct brief focused interviews anchored to specific moments: "When you paused here — what were you looking for? What did you expect would happen?" This method combines behavioral data with subjective phenomenological accounts, capturing how affordances were perceived in situated action. Pure behavioral observation cannot directly access perception; interviews alone cannot capture what was actually present in the environment.

13. Probe for valence and intentionality. Did the engineer notice the affordance but avoid using it? Why? Was it unfamiliar, risky, or simply not relevant to their immediate goal? A five-dimensional framework — perceptibility, valence, compositionality, normativity, and intentionality — provides a vocabulary for categorizing what you find.

Phase 5: Multi-Actor Mapping and Synthesis

14. Map findings by actor group. Create a simple matrix: affordances on one axis, actor groups on the other. Mark which groups perceived, actualized, or missed each affordance. This is the multi-actor view — it surfaces divergence that aggregate analysis conceals.

15. Identify bundling failures. Affordances must aggregate from individual to team and organizational levels to produce collective outcomes. Look for affordances that individuals use well but that do not appear to translate into team-level capability — these often indicate missing coordination affordances (naming conventions, shared patterns, documentation practices).

16. Apply the Affordance Structure Matrix as a documentation tool. The Affordance Structure Matrix (ASM) systematically captures user requirements alongside affordances, documenting how affordances are form-dependent and whether they support or hinder user goals. Use it to record your findings in a format that can be shared and iterated on across review cycles.

17. Prioritize by gap type. Distinguish: (a) affordances that no actor perceives — likely a signifier problem in the design; (b) affordances that some actors perceive but do not actualize — likely a valence or normativity problem; (c) affordances that actors actualize but not at team level — likely a sensemaking or coordination problem.

Phase 6: Iteration and Feedback Loop

18. Propose targeted interventions for the highest-priority gaps. Interventions differ by gap type: renaming or restructuring addresses signifier problems; documentation and examples address valence; architectural review ceremonies and onboarding activities address sensemaking and coordination.

19. Schedule a re-evaluation cadence. Affordance actualization involves phases of perception, enactment, and outcomes — changes to design take time to propagate into behavior. Plan for a minimum of one architectural cycle before assessing whether an intervention worked.

20. Remain open to appropriated use. Systems designed for appropriability reward engineers who discover and propagate unanticipated affordances. When evaluation surfaces an unexpected pattern of use that is effective, document and amplify it — do not suppress it in favor of intended design.

---

Active Exercise

Building Your Affordance Audit Matrix

Choose a surface you own or work with regularly — a module API, a set of architectural conventions, a developer toolchain interface.

Step 1. List 5–8 affordances you believe the surface offers (intended affordances). These might include things like: "the naming convention reveals module ownership," "error types are discriminable enough to route to the right handler," or "the test helper makes it obvious how to set up state."

Step 2. For each intended affordance, identify at least two actor groups who use the surface — for example, engineers who contributed to this module versus engineers who encounter it from outside; senior engineers versus recent joiners.

Step 3. For each affordance × actor group cell, write down your honest current assessment: is this affordance perceived? actualized? unknown? This is the baseline matrix.

Step 4. Identify the cell with the widest expected gap — the affordance you suspect is invisible or underused by a specific group. Design one lightweight observation or interview that would test your hypothesis. Who would you recruit? What task would you ask them to perform? What would confirm or disconfirm your assumption?

Step 5. Review your matrix for bundling failures. Are there affordances that individuals use well but that do not appear to produce team-level coordination or shared practice? What sensemaking ritual is missing that would enable collective discovery?

Write up your matrix and the observation plan. Bring it to your next architectural review or developer experience sync as a working document.

---

Stretch Challenge

You have spent eight modules building a vocabulary and set of practices for engineering affordances. Now turn the lens on your own organization.

The challenge:

Map the affordance evaluation gap in your current engineering environment. Not a gap in a specific system — a gap in your organization's capacity to evaluate affordances at all.

Work through the following:

1. What evidence currently exists about how engineers perceive and actualize the systems they work with? Audit your existing mechanisms: Are there developer experience surveys? Are onboarding retrospectives conducted and analyzed? Do architectural reviews include any form of behavioral observation, or are they purely design-review exercises?

2. Where is the multi-actor view absent? Identify one system or practice where evaluation, if it exists at all, only reflects the perspective of the engineers who built it. What would a credible multi-actor affordance map look like for that system?

3. What is the organizational sensemaking gap? IT affordances emerge through organizational sensemaking — collective interpretation in rituals, conversations, and documentation. Identify one affordance in your codebase that you believe remains unrealized because there is no structured occasion for engineers to collectively discover it. What ritual would change that?

4. Design a minimal evaluation infrastructure. Propose three changes — no more — to existing engineering processes that would, over the next two quarters, produce ongoing affordance evidence. The changes should be small enough to survive without a champion, and should connect to rituals that already exist.

Present your findings as a memo. The memo should be honest about current gaps and precise about what would be different, and for whom, if the proposed changes were adopted.