Introduction
From the time of their arrival, information systems held the promise of simulating human intelligence. The term Artificial Intelligence was coined in a conference by John McCarthy in 1958 to denote such simulation in information systems. Since then, there were efforts among academia and the industry to realise this promise. Some of these efforts succeeded in creating domain-specific AI solutions for playing games, giving expert advice, recognising speech and speakers, identifying faces and objects etc. The performance was proving to be satisfactory, but many of these solutions did not scale well when the sizes of the input domain were sought to be expanded. Two developments in recent years have changed the scenario. One is that large annotated data sets have become available to train AI software. The second is that the computing capability of modern information systems is closing the gap with that of the raw processing capability of human brains. As a consequence, large scale AI applications seem to be entirely possible today, both for civilian and military purposes.
Knowledge & AI
The central factor in AI solutions is the knowledge they use to produce the output. This knowledge has to be explicit and structured in a way the software can enhance it and also apply it to process input to produce output. The challenge is to find and codify this knowledge. Usually, the help of experts in a domain is sought to address this challenge. This approach has its limitations; one the externalised knowledge of experts may not be complete, and second, in many scenarios, such externalisation may not be feasible. An alternate approach is to teach or train AI software with labelled data that is available in plenty today. Depending upon the method of training used, knowledge is extracted from the data in different forms of representations. The scope of the labelled data is expected to decide the completeness of the extracted knowledge.
Knowledge extraction from data
Two broad training methods are followed to extract knowledge out of data. One is to make explicit rules, models, or decision trees. For example, to recognise speakers, the training process takes samples of their speeches and creates what are called Gaussian mixture models (GMMs). Once a number of models are created and a model-base is built up, the software can process any test speech, create the required model, match it with the model-base and fetch the closest match. If the closest match is not close enough, the test speech may be deemed unrecognisable. More frequently, labelled data is used to create a decision tree to help in classifying new input data. A simple algorithm to create a decision tree is the ID3 (Iterative Dichotomizer Version 3). Figure 1 shows pictorially how a table of labelled data for deciding to play golf is transformed into a decision tree [4]. Once the tree is available, using independent input values, one navigates the tree from the root, and finds the dependent label or classifying tag at the leaf node. The power of explicit knowledge structures such as the decision trees is that the solution is practically a white box and an explanation for the final decision can be worked out if required. The champions of explainable AI advocate this method for its transparency.
The second method of extracting knowledge from labelled data is to train a neural network using the deep learning algorithm. It is based on the developments in modelling the human brain using neural networks. One advantage of this method over the previous one is that this can tolerate noise in the labelled data to a certain extent. Another is the relative speed with which it can produce its decisions. In terms of results achieved so far, the deep learning algorithm has been hugely successful, especially in the domain of computer vision. Figure 2 gives an example of a deep neural network classifying objects in a busy street as humans, vehicles, traffic lights etc. in real-time [7]. This network is a fundamental enabler of self-driving cars. With the increasing availability of labelled data and high-performance computing servers, this method is expected to do even better in computer vision and tackle other domains such as language processing too. However, a major issue with this method is the knowledge extracted from the data is distributed in the connection weights of the neural networks and is not easy to comprehend or interpret. In a sense, this knowledge is a black box and the deep learning solution cannot explain the logic of arriving at its decisions in a way humans can understand.
Notwithstanding the pros and cons of these two methods, they are both used for crafting AI solutions today. They are powering the adoption of AI over the Internet. Defence forces all over the world have been enamoured of the potential of AI-based autonomous and unmanned systems for modern warfare. AI can help identify military targets and launch autonomous missiles towards them. A new type of missile can even loiter in a chosen area so as to find its target when the target comes out of its hiding place. Unmanned ground vehicles are already part of many defence forces. So are unmanned surveillance and combat aerial drones. These systems have varying degrees of autonomy depending on the military function assigned to them. The least autonomous have humans in the loop either deciding or verifying the decision of the machine. The next level has humans on the loop who keep a watch and intervene only when required. The third and most autonomous level has humans out of the loop and has machines that are goal-oriented. These machines keep trying to achieve the set goals.
AI For Defence
The case for AI in defence had never been more attractive. AI-enabled defence systems can, of course, save precious human lives for own country. They can give the speed advantage and help in seeing and responding faster. The old chestnut of ‘know your enemy before he knows you’ requires perception at scale. With AI, one can build a common operational picture, try out the effectiveness of tactics and strategies, and identify weaknesses of the adversary. On a serious note, an equivalent of a nuclear weapon is possible too. According to some AI experts, a million unmanned combat aerial vehicles in swarm mode can be a weapon of mass destruction (WMD).
A serious concern is the complexity of these systems and the possibility of errors in them. These errors can cause fratricide and loss of own forces. They can also be exploited by the adversary in a cyber-warfare scenario. Finally, when the defence systems are produced using globally sourced components, supply chain infection and hidden Trojans cannot be ruled out. Testing and certification may not be good enough. These are conventional concerns of AI systems being information systems. A concern specific to AI systems is what if the input data for an adaptive AI system is manipulated to change the nature of its functioning.
Securing The AI Part
A holistic solution for secure AI has to address both the conventional concerns and the ones specific to the adaptive nature of AI systems. Controls to address conventional concerns assume that the functionality of a system to be protected is a given. Thus, integrity management control tries to ensure functional integrity during operational usage. In this regard, an adaptive system poses a challenge. The instantaneous functionality of an adaptive system is defined partly by the input. As complete verification of the input is out of the question, the resultant functionality cannot be assumed to be always correct. In fact, during actual operation the input may have noise and errors; it may be deliberately engineered by an adversary to distort the resultant function.
A simple solution to tackle this challenge is to separate the adaptive or learning phase from the usage phase. Thus, learning happens in controlled conditions much as how the development of the system happens. During this exercise, input data may be carefully selected to create the knowledge required for later usage on the field. Either explicit or implicit knowledge extraction methods can be used in this phase. Functional integrity can be ensured later during usage without any problem. The drawback of this approach is that the system is denied the advantages of learning on the field. One approach to address this drawback is to ensure that the training data has certain completeness about the domain of interest. In that happy scenario, learning on the field would be superfluous. If such completeness cannot be ensured, the system may be programmed to refer to its human master to deal with field inputs outside the scope of training data.
AI systems that can learn on the field securely are not impossible. However, the discriminative knowledge to ignore the irrelevant or dubious data is a higher form of knowledge that will remain the preserve of humankind, at least for some time into the future. The simple reason is that we do not yet know how to externalise and make this knowledge explicit. Nor do we know how to identify and source the training data needed to create such discriminative knowledge.
Initiatives From The DRDO
Recognising the importance of AI for Indian Defence, the DRDO established the Centre for Artificial Intelligence and Robotics (CAIR) in Bengaluru in the 1980s. Since its inception, CAIR has expanded its role to apply AI to Command-Control and Cyber Security systems. A number of its aero and armament cluster labs have been focussing on the development of autonomous air and ground vehicles since the 1990s. The Aeronautical Development Establishment (ADE) in Bengaluru and the Research and Development Establishment (Engineers) (R&D (E)) in Pune are two leading examples from these two clusters. Very recently, the DRDO has set up two young scientist labs, namely DYSL-AI and DYSL-AT, to work on AI and drones respectively. Through a streamlined mechanism of transfer of technology (ToT), the DRDO has been facilitating our industry to take its AI technologies to its customers. Likewise, through its recent initiative called Technology Development Fund (TDF), the DRDO tries to help start-ups and small industries working on AI among other areas.
Conclusion
Notwithstanding the challenges in tapping the full potential of AI for defence, the global race to realise this potential is on. Our country has built a base over the last few decades in setting up several labs dedicated to AI and autonomous systems. Through the efforts of these labs, the capability is been seeded in our industry which is beginning to sprout. Based on this platform, however inadequate it may be, the DRDO and the country have to launch a massive programme to enable our defence forces with AI. While doing this, the safety and security of this AI in our defence has to be kept among the top objectives.
Dr G. Athithan, DRDO Chair, CAIR
