Dynamic Evidence Collection Ecosystem for Assessment Integrity and Authentic Competence
Organizations: NAPS, Melbourne, Australia · NAPS, Sydney, Australia · MIT, Melbourne, Australia
Abstract
Generative Artificial Intelligence (GenAI) can produce high-quality essays, code, and design artefacts, challenging the validity of conventional assessments that rely on single-point submissions and product-only grading. This paper proposes a design framework called "Dynamic Evidence Collection Ecosystem" that shifts assessment toward continuous, authentic, multi-source evidence of student learning over time. The framework collects process evidence through iterative artefacts, design logs, activity rounds, self-reflection, and peer collaboration, supported by an AI-enabled layer for learning analytics, formative feedback, and transparency. The approach is grounded in recent assessment-redesign scholarship in AI-rich contexts and aligned with contemporary views of authenticity in assessment. This paper builds on the hypothesis that academic integrity is strengthened when it is treated as an assessment design rather than as an AI detection problem. The tools have limitations and risks of use that carry academic penalties. This paper presents an implementation scenario to support institutional adoption.
Figures & tables
| Evidence | Purpose | Applications | Examples | Measured Skill(s) |
|---|---|---|---|---|
| Iterative design artefacts | Captures the evolution of design thinking, documenting iterative refinement, informed decision-making, and structured problem-solving | Product development, software engineering, and complex problem-solving | Computer-Aided Design (CAD) models; network designs; circuit schematics; technical drawings | Complex systems design; Iterative solution development and refinement; Decision making; Design thinking; Problem solving |
| Version controlled records | Provides a transparent and traceable account of the development process: individual contributions, decision-making, and changes | Computer science, software engineering, and data science | Code commits; version histories; debugging logs; feature iterations | Proficiency in engineering tools; individual contribution; systematic development practices |
| Simulation & modelling outputs | Demonstrates analytical reasoning through structured experimentation, evaluation, and validation of solutions | Engineering, physics, networking, and AI/ML domains | Simulation results; parameter tuning records; performance evaluations | Problem analysis; solution testing and evaluation; analytical thinking; modelling proficiency |
| Experiment logs | Captures scientific reasoning through documentation of experimental design, observations, and iterative investigation | Physics, chemistry, biology, and engineering laboratory contexts | Experimental setups; recorded observations; measurement data; troubleshooting records | Scientific inquiry; methodical experimentation; experimental proficiency; data analysis skills |
| Design decision logs | Demonstrates engineering judgement through structured documentation of decisions, trade-offs, and rationale | Systems engineering, network design, and AI system development | Architectural decisions; algorithm selection; trade-off analyses | Decision-making; justification of technical trade-offs; engineering judgement; critical thinking |
| Prototype development | Demonstrates the application of theoretical knowledge through the design, construction, and iterative refinement of practical solutions | Robotics, product design, and software engineering contexts | Hardware prototypes; software prototypes; staged system implementations | Implementation proficiency; practical engineering skills; knowledge application; |
| Component (Evidence Type) | Description (Timeline) | Weight |
|---|---|---|
| Three (3) iterations (Data-driven) | Versions (v1, v2, v3) with change notes and reasoning for the changes (Initial, Mid and Final) | 40% |
| Two (2) critiques (Human-driven) | Two reflections/peer-reviews that support iterations/changes (Mid and Post development) | 30% |
| One (1) defence (Human-driven) | Oral explanation, viva for defence that supports iterations and critique (Post development) | 30% |