Understanding the Limits of Artificial Intelligence for Warfighters

Research Questions

  1. How does the current RSP failure analysis approach perform in a retrospective analysis against historical data?
  2. How can AI help inform the failure analysis process, and what are its limitations?
  3. What other potential improvements can augment the existing approach?

The U.S. Air Force (USAF) deploys flying units with readiness spares packages (RSPs) to try to ensure that the units are stocked with enough parts to be self-sufficient for 30 days. Predicting which parts are likely to fail—and, therefore, which parts should be included in the RSPs—is important because overstocking can be expensive and understocking can threaten mission readiness. This report presents a discussion of whether and when artificial intelligence (AI) methods could be used to improve parts failure analysis, which currently uses a model that assumes a probability distribution. To do this, several machine-learning models were developed and tested on historical data to compare their performance with the optimization and prediction software currently employed by the USAF, using A-10C aircraft data as a test case.

This report is the third in a five-volume series addressing how AI could be employed to assist warfighters in four distinct areas: cybersecurity, predictive maintenance, wargames, and mission planning. This report is aimed primarily at those with an interest in predictive maintenance, RSPs, and AI applications more generally.

Key Findings

  • AI can improve failure analysis for RSPs on a case-by-case basis. The current probability based prediction process is a poor predictor of the performance of many parts. The AI model not only made better predictions but also, as a result, was much more cost effective. Updating the current prediction process with data can achieve a performance level that is quite close to the AI model used in this research.
  • It is necessary to establish a complex and labor-intensive data operations pipeline to USAF maintenance databases before any large-scale AI implementation can occur. Historical data are essential to train and test AI models, but pulling this data from the relevant systems is a complex, manual process that involves scripting, drop-down lists, and nested menus. Moreover, considerable data cleaning is also needed. Given this situation, leveraging AI is practical only as a proof-of-concept model.
  • AI cannot alleviate the scarcity of wartime data. It is unclear whether RSPs developed using peacetime data will be adequate for wartime operations. Moreover, one of the main limitations of AI for this application is its inability to estimate truly rare events, which might be more likely during wartime operations. As a result, different approaches to modeling AI could be required to deal with these changing circumstances. However, regular retraining and updating, which is possible with an AI model, can ensure the adaptability of these models during wartime.

Recommendations

  • Air Force Materiel Command (AFMC) should work with USAF Logistics to build a data operations pipeline to conduct retrospective analysis of aircraft maintenance and RSP efficiency. Aircraft maintenance programs and databases function effectively for the purposes for which they were designed, but they clearly were not designed for retrospective analysis or to train AI models. Unless the data can be properly conditioned and pulled for this analysis, none of the following recommendations can be implemented.
  • AFMC should experiment with AI to improve failure analysis for RSPs. Extend the proof-of-concept models to all aircraft. This will likely have to be done on a part-by-part, platform-by-platform basis. Automated or partially automated data extraction will likely be necessary if AI is relied on to conduct these analyses. For RSP parts with hard-to-predict rare failures, the AI cost function can be modified to prefer overpredictions or rely on overpredictions via a Poisson distribution or the problem can be modeled as survival analysis (predict time to failure).
  • AFMC should limit AI to failure analysis within the RSP process. The ASM software tackles a large and complex operations research problem of selecting which parts to send from which depot to which base. Current AI capabilities are data hungry and better suited to solving narrowly scoped problems. Splitting parts failures across multiple depots and bases will fragment the data too much for algorithms to learn anything useful.

Table of Contents

  • Chapter One

    Introduction

  • Chapter Two

    Aircraft Sustainability Model Failure Analysis Calculations

  • Chapter Three

    Assessing the Potential for Artificial Intelligence

  • Chapter Four

    Challenges in Collecting and Preparing Maintenance Data for Artificial Intelligence

  • Chapter Five

    Summary of Findings and Recommendations

  • Appendix A

    In-Depth Methodology for Deriving Parts Failure Data from LIMS-EV and Other Supporting Information

  • Appendix B

    Exploratory Analysis Using Days to Failure as a Predictor of Failure

Research conducted by

This research was prepared for the Department of the Air Force and conducted within the Force Modernization and Employment Program of RAND Project AIR FORCE.

This report is part of the RAND Corporation Research report series. RAND reports present research findings and objective analysis that address the challenges facing the public and private sectors. All RAND reports undergo rigorous peer review to ensure high standards for research quality and objectivity.

This document and trademark(s) contained herein are protected by law. This representation of RAND intellectual property is provided for noncommercial use only. Unauthorized posting of this publication online is prohibited; linking directly to this product page is encouraged. Permission is required from RAND to reproduce, or reuse in another form, any of its research documents for commercial purposes. For information on reprint and reuse permissions, please visit www.rand.org/pubs/permissions.

The RAND Corporation is a nonprofit institution that helps improve policy and decisionmaking through research and analysis. RAND’s publications do not necessarily reflect the opinions of its research clients and sponsors.

Download

Download eBook for Free

Full Document

Format File Size Notes
PDF file 1.4 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

Research Summary

Format File Size Notes
PDF file 0.1 MB

Use Adobe Acrobat Reader version 10 or higher for the best experience.

© 版权声明
THE END
觉得文章有用,可以点个赞
点赞0 分享