NIST only participates in the February and August reviews.
The Applied Systems team at the Center for AI Standards and Innovation leverages multidisciplinary methodologies to assess, evaluate, and measure AI systems in application and real-world settings to accelerate trustworthy innovation, support the adoption of reliable AI, and protect national security. Our work has wide breadth and reach, including deliverables such as best-practice guidelines, tooling to support evaluations, innovative AI measurement, research papers for publication, workshops and convenings.
Proposals are welcome within the scope of measurement and methods for AI evaluations in application, including but not limited to the research areas enumerated here. For example, to develop and advance metrology and methodologies for evaluating AI systems, including executing evaluations in real settings (e.g., AI evaluation for critical infrastructure, field studies in high risk sectors, longitudinal continuous monitoring of AI post-adoption, etc).
A successful applicant would have the opportunity to lead research workstreams, write papers, and engage with AI and measurement science experts from across CAISI and NIST. The artificial intelligence interest group at NIST consists of over 100 scientists who interact regularly via meetings, seminars and informal gatherings. Furthermore, CAISI often collaborates on research with external experts via Cooperative Research and Development Agreements and the AI Consortium. The selected candidate will have the opportunity to contribute to impactful tooling, guidance, research, and best practices that bolster state-of-the-art AI evaluation practices in the US government and the wider field of practitioners.
Artificial Intelligence; Machine Learning; TEVV; AI Evaluation; Metrology; Psychometrics; Field Studies; Uplift Studies
level
Open to Postdoctoral applicants