Component Mismatches Are a Critical Bottleneck to Fielding AI-Enabled Systems in the Public Sector

by   Grace A. Lewis, et al.

The use of machine learning or artificial intelligence (ML/AI) holds substantial potential toward improving many functions and needs of the public sector. In practice however, integrating ML/AI components into public sector applications is severely limited not only by the fragility of these components and their algorithms, but also because of mismatches between components of ML-enabled systems. For example, if an ML model is trained on data that is different from data in the operational environment, field performance of the ML component will be dramatically reduced. Separate from software engineering considerations, the expertise needed to field an ML/AI component within a system frequently comes from outside software engineering. As a result, assumptions and even descriptive language used by practitioners from these different disciplines can exacerbate other challenges to integrating ML/AI components into larger systems. We are investigating classes of mismatches in ML/AI systems integration, to identify the implicit assumptions made by practitioners in different fields (data scientists, software engineers, operations staff) and find ways to communicate the appropriate information explicitly. We will discuss a few categories of mismatch, and provide examples from each class. To enable ML/AI components to be fielded in a meaningful way, we will need to understand the mismatches that exist and develop practices to mitigate the impacts of these mismatches.


page 2

page 4


What is Software Quality for AI Engineers? Towards a Thinning of the Fog

It is often overseen that AI-enabled systems are also software systems a...

AI in Software Engineering: A Survey on Project Management Applications

Artificial Intelligence (AI) refers to the intelligence demonstrated by ...

Towards Productizing AI/ML Models: An Industry Perspective from Data Scientists

The transition from AI/ML models to production-ready AI-based systems is...

Teaching Software Engineering for AI-Enabled Systems

Software engineers have significant expertise to offer when building int...

Providing Assurance and Scrutability on Shared Data and Machine Learning Models with Verifiable Credentials

Adopting shared data resources requires scientists to place trust in the...

PerfDetectiveAI – Performance Gap Analysis and Recommendation in Software Applications

PerfDetectiveAI, a conceptual framework for performance gap analysis and...

Feature Interactions on Steroids: On the Composition of ML Models

The lack of specifications is a key difference between traditional softw...

Please sign up or login with your details

Forgot password? Click here to reset