Abstract of game playing in ai

Forza Motorsport 5 [Turn 10 Studios, ] and its successors gather data about how players drive, that is then processed using Machine Learning techniques. A similar goal was attained in the Killer Instinct fighting game [Iron Galaxy Studios, ] using case-based reasoning. For other uses of Machine learning techniques in games, the survey by Nguyen et al. Multi-agent systems have been suggested as powerful solutions to intelligent NPCs Dignum et al.

However, real-time synchronisation of many agents acting autonomously, for instance in battlefield games such as the Call of Duty series [Infinity Ward, from ], easily produce performance problems. Several advanced techniques for optimisation such as flow field and congestion concepts Pentheny , , context steering Fray , and even robotics-inspired Velocity Obstacle techniques Guy and Karamouzas have been applied.

The blockbuster game Grand Theft Auto V [Rockstar North, ] uses multi-agent based architectures for the simulation of subsystems, as do most of the real-time modern strategy games, like those based on the Clausewitz Engine from Paradox Development Studios. Advances in Natural Language Processing NLP have opened up new opportunities to support natural dialogues with NPCs, either companions or enemies, and to support interactive storytelling Yannakakis and Togelius In the multiplayer online battle arena League of Legends [Riot Games] NLP trained models have been used to recognise and remove toxic behaviour from the player chat channels Maher Generally, these applications of AI are proprietary solutions bound and tuned to a particular game and not accessible and reusable by other parties.

Also, their application in serious games has been quite limited. Given the diversity of software platforms, programming languages, browsers and operating systems, favourable conditions for the reuse of software by game developers should be accommodated by a shared architectural framework. The main starting points for the architecture include: 1 Extendibility The architecture should be robust over extending the set of components with new software components, 2 Addressing platform and hardware dependencies Direct access to the operating system should be avoided; a conservative approach as to avoid browser version issues as much as possible , 3 Portability across game engines and programming languages, 4 Avoiding dependencies on external software frameworks and libraries such as jQuery or MooTools for JavaScript , 5 Neutrality with respect to different software design methodologies the development process , 6 Neutrality with respect to game genre, design and style avoiding direct access to the interface; components just provide smart functionality under the hood , and 7 Truly lightweight easy to use in different operational contexts.

In close collaboration with game industry representatives, a component-based design framework Bachmann et al. Client-side plug-in characteristics of software components are created by relying on well-established coding practices and software patterns that procure abstraction, viz. Remote communications with server-side components are covered by web services. The architecture has been extensively tested and validated in connection with a wide diversity of development tools, target platforms and programming languages that are being used in practice Van der Vegt et al.

The recently launched gamecomponents. Notably, the portal offerings are fully platform independent, while — in contrast - existing game portals are either driven by commercial game platform vendors e. Moreover, existing portals focus mostly on media assets e.

Also, the gamecomponents. Nevertheless, leisure games could also benefit from the technologies exposed on our portal. Figure 1 shows a screenshot of a software catalogue page, revealing taxonomy-based filtering, keyword search and a results section displaying available game components. Software developers can describe and submit their contribution through a component-authoring widget that provides a stepwise guidance through the submission process, allowing for entering the software or software references e.

Github , its metadata a specific game component metadata schema was designed for this Georgiev et al. In addition, it offers an interactive stakeholder map, a set of tools for taxonomy management, training course creation and eCommerce management, and it uses open-ID user management, offers social API for the exchange with social networks e. Slideshare, Mendeley and incudes a rating system based on scores by end-users. The portal is available at gamecomponents.

An initial set of about 40 game components are exposed on the portal. The components exposed at the portal cover a wide range of AI-based functionalities that are relevant for serious game development, including personalisation, game difficulty balancing, assessment, player analytics, competence modelling, social gamification, language technologies and affective computing, among other topics.

All components are open source and free of charge. Additional conditions to promote the adoption and reuse of the software have been met: 1 successful integration has been demonstrated with various game development environments e.

Unity, Xamarin, Cocos, Mono , 2 the integration in games is easy, 3 all components have been used and tested in real games with real end-users to provide empirical evidence of practicability; 4 the components have been enriched with ample documentation, tutorials, demos, research articles and evaluation results; 5 they use the highly flexible Apache 2.

To further the viability and sustainability of the marketplace portal and to attain critical mass of relevant game software, third-party providers — either game research projects or IT-oriented companies - are expressly being invited to post their game software, whether or not compliant with the component-based design framework, onto the portal.

This section presents a selection of reusable game AI components that have been made available at the gamecomponents. Their relevance for serious games is readily explained by the pedagogical frame of teaching, which assumes a teaching agent cf. PEM and, when needed, engages in a supportive dialogue with the player cf. NLP to provide guidance or feedback. So far, however, the capturing of physiological signals has been problematic since it requires hardware sensors that are often too obtrusive and unpractical for continuous application.

Natural language processing is the field of AI focused on the understanding, interpretation and manipulation of human language by computers. It allows the computer to assess any textual messages or documents sent by the player, and thereby it allows to respond to these automatically in a meaningful way. So far, NLP has scarcely been used in games. The following NLP services will be presented:.

Game AI for NPCs has a longstanding history, in particular focused on navigation and other low levels of control Yannakakis Recent research, however, has been focusing on a variety of high level NPC behaviours that should effect more flexible, believable, knowledgeable, human-like, and intelligent behaviours, including realistic bodily motion, NPC emotion modelling, and compliance with socio-cultural conventions.

This reflects a more holistic perspective on the NPC capable of flexible responses, as opposed to fully scripted applications. The following NPC components will be presented:.

Artificial Emotional Intelligence AEI , which is also known as emotion recognition or emotion detection, is a technology that extracts human emotions from displayed behavioural or physiological features Schuller and Schuller Human facial expressions have demonstrated to produce the most informative data for computer awareness of emotions Sebe , outperforming approaches that make use of either speech and vocal intonations, physiological signals, body gesture and pose, text, or combinations of two or more of these approaches Bahreini et al.

So far, the use of emotion recognition functionality has not been a feasible option well within reach of game developers, because of the complexity of the implementation involved, limited accuracy, problems with facial hair and glasses, specific requirements with respect to lighting conditions, extensive post-processing and some more Pantic et al.

The real-time facial emotion recognition component, created by the Open University of the Netherlands, solves many of these problems Bahreini et al. Emotions are a significant influential factor in the process of learning, as they affect memory and action Pekrun Any classroom teacher would take into account the emotional states of learners during the lessons.

Finally, emotion recognition can be used to collect emotion data during play testing. This software component uses artificial emotional intelligence to unobtrusively cover unbiased facial expressions of emotion from any image, either from a still, a video file, a video stream or a webcam.

The technology uses a combination of fuzzy logic rules and machine learning. Alternative machine learning approaches, such as neural networks, Bayesian networks, and decision trees are less practical for real-time operation as they require extensive processing, while offering weaker performance.

A usage example of the real-time facial emotion recognition component would be the Jobquest game, which offers a job application interview training Gutu et al. During the job interview, the players should control their manifest emotions and never display anger, fear or disgust. Emotion recognition has also been used for communication training in the Communication Advisor game Bahreini et al. This game places the players in a variety of real life situations to which they have to respond via a natural dialogue.

Feedback to players is based on their facial expressions. The real-time facial emotion detection is a client side software component that is to be integrated in the game engine. It returns a string value representing the seven basic emotion classes, which can be used for further processing in the game. It can also process a single image file, or a recorded video file.

Also, presence of multiple players in one shot can be accommodated as it can detect multiple faces and interpret their emotions at the same time. It can easily be integrated in many game engines, including, for instance Unity3D. Game difficulty balancing is deemed an essential mechanism to preserve player engagement, improve player motivation, and improve the overall gameplay experience. Dynamic game difficulty balancing avoids both frustration of the player when tasks are too complex and boredom when tasks are too easy.

This also holds for teams: difficulty balancing is commonly applied in online multiplayer games such as the first-person shooter series Halo and multiplayer online battle arena games like League of Legends to ensure that opposing teams are evenly matched in terms of skills Claypool et al. While the adaptation mechanism is often mistaken and confused with a simple if-then-else structure or a level closure, it should incorporate a sophisticated self-adjusting optimisation algorithm that frequently reiterates both task difficulty and skills mastery.

The implementation and testing of such algorithm is anything but straightforward. The adaptation and assessment component created by the Open University of the Netherlands offers a fully automated, self-adjusting balancing algorithm that exposes superior reliability and stability.

It comes as an easy to use software component that can be readily integrated in various game engines. The engaging capabilities of games are to be largely attributed to the process of game balancing. By avoiding frustration and boredom and offering doable challenges a well-balanced serious game enhances and preserves learner motivation, which is a principal determinant of learning.

In accordance with the Zone of Proximal Development theory Vygotsky , a real-time adaptation of the game difficulty enables a smoother learning experience. The AI algorithm controls difficulty so that the player is challenged to improve a skill or acquire new knowledge without facing overly difficult tasks beyond player skill level. This means that the learning process becomes highly efficient: progression is optimised, while no time is wasted on tasks that do not contribute to learning.

This also holds for one of the widely-known examples of such balancing algorithms, namely TrueSkill Herbrich et al. However, TrueSkill was designed specifically to assess and match players in large-scale commercial online games.

Another example would be the Computerized Adaptive Practice system CAP , which was specifically developed to assess player skill in a serious game Klinkenberg et al. In turn, the methods from IRT enable CAP to adapt the game difficulty based on the player skill using the previously estimated ratings.

The Adaptation and Assessment component presented here adds several theoretical and practical improvements to CAP. Multiple selection criteria can mitigate the selection bias, and fuzzy logic allows to combine these criteria into a single selection rule.

As a result, the improved algorithm is more robust and accurate than CAP especially during the calibration period when true skill and difficulty ratings are not well approximated Nyamsuren et al. Publicly available data from a Math Garden game collected from over Dutch schools featuring 87, unique players were reused to validate the improved performance of the algorithm of the adaptation and assessment component. The TwoA component has also been used in an entrepreneurial skills training game Hatch at Hull College.

From a practical perspective, this adaptation and assessment component offers an open-source, highly portable, and easy-to-use implementation of the AI algorithm. As a reusable component compliant with the RAGE architecture, it can be easily integrated with the most modern game engines Van der Vegt et al. The component hides all the complexities of the algorithm behind a simple interface. Apart from the management of player and game data, its operation requires only two method calls from the game to the component.

This means that behavioural data e. In practice, however, the application of stealth assessment in serious games is a complex and time-consuming process Moore and Shute Therefore, its uptake has been below par as yet. The generic tool provided by the Open University of the Netherlands removes many of the practical barriers for applying stealth assessment, as it has largely automated the many data processing steps that so far need to be handled manually Georgiadis et al.

Games are expressly suited for the acquisition of highly contextualised, tacit knowledge and action-bound skills, which are notably hard to capture in formal tests and exams. Cases in point would be social skills, communication skills, group moderation skills, but also competencies such as persistence, creativity, self-efficacy, teamwork and the wider collection of twenty-first century skills, all of which are deemed essential for successful future careers and presupposing a strong link with concrete action Dede Given this tacit knowledge dimension, the assessments should not be administered solely as separate oral or written assignments, but instead should be directly based on the activities displayed.

Stealth assessment provides an attractive alternative to the existing de-contextualised assessment methods by linking the assessment directly to the practical use of knowledge and skills in relevant situations. Moreover, these situations should entail scenarios that require the application of various competencies at the same time. This is exactly what serious games are capable of providing. ECD is a conceptual assessment framework that can be used to express the statistical relationships between competency constructs, in-game observables, and in-game tasks.

As for the machine learning algorithms, originally Bayesian Networks were used Shute although alternative solutions have also been examined Decision Trees, Support Vector Machines, and Deep Learning Sabourin et al. The new, generic application for stealth-assessment presented here allows the user to 1 define and configure ECDs, 2 import numerical data from log files deriving from any serious game, and 3 declare desirable machine learning optimisations e.

Thereby the need for specific machine learning expertise is minimised as the tools cover machine learning functions automatically. So far, stealth assessment has been proven to be robust for assessing several competencies in serious games, such as qualitative physics Shute et al. The application has been rigorously tested and validated with a large volume of simulation data sets including different competencies, in particular with respect to stability, accuracy and robustness under conditions of normality violation.

As a next step practical validation with authentic game data is anticipated. The stealth assessment component is currently available as a console application.

It was coded in C using the. NET framework and it functions as a stand-alone client-side console application. It includes various data reformatting procedures. It makes use of ML libraries from the Accord.

NET framework. On top of the console application a graphical user interface is being developed, including a wizard that supports the workflow and assists the user e. The ReaderBench framework Dascalu et al.

Sentiment analysis also referred to as opinion mining consists of the automated extraction of subjective information related to human feelings and opinions from natural language texts Liu In the context of serious games, sentiment analysis can be used, for instance, in dialogues, commonly available either in multi-player communication or in discussions with a virtual character.

The arising insights about how people feel and interact during these interactions can then be fed back to the game for further usage in the game scenario, or can be provided as feedback for the game development team. Alternatively, sentiments can be extracted from written free text or spoken assignments in the game, such as reports, pitches or answers to open-ended questions.

The LSTM networks are probably the most used type of text encoder for the majority of tasks involving text comprehension. A practical example of using sentiment analysis in a serious game is the Jobquest game, referred to above. Users are requested to prepare and optimise their Curriculum Vitae CV in French language, in view of a specific job opening. The textual content of the uploaded CV is then analysed with ReaderBench services, which returns specific feedback, including sentiment valence scores, indicators of emotions, textual complexity factors and general statistics related to visual or contents quality Gutu et al.

A French corpus consisting of a collection of articles published by the Le Monde newsarticle was used to train the system. The sentiment analysis service provided by ReaderBench can be accessed as a remote web service through a dedicated endpoint exposed within the ReaderBench API. The service is open and does not require authentication.

In terms of semantic models, developers can either use pre-trained corpora, or they can train a custom model for their specific scenarios. The sentiment analysis service currently supports multiple languages, namely English, French, and Dutch. In serious games writing assignments and open-ended questions are scarce, because of the intensive manual effort needed for assessing the learner productions.

Writing assignments, however, accommodate deeper knowledge processing since they require explicit consideration of learned concepts, principles and their relationships, reflection about the significance and appraisal of the experiences, and the creative synthesis of argumentation Westera et al. In addition, writing assignments would provide an excellent diagnostic tool for detailed assessment of learning progress. That is why schools and universities often require students to write reports or theses as proofs of mastery.

Also, most professions require excellent writing skills, for instance in journalism, health, education, marketing, business consultancy and many other areas. Now that automated processing is becoming available, writing assignments need no longer be omitted in serious games.

The very method of essay scoring can also be used to inform the game development team about the complexity of instructional texts and other textual learning materials exposed in the game, which allows their adjustment for a better fit with the player characteristics and needs.

For various languages a separate NLP pipeline model was created, using language specific dictionaries, stop words elimination, word lemmatisation, and part-of-speech tagging. In addition, the WordNet lexical ontology was used to identify lexical chains Budanitsky and Hirst A dedicated essay scoring model is then used by feeding a training set of example essays and their assigned scores to the system.

ReaderBench services provide various textual complexity indices such as Dascalu et al. After training the essay scoring model, the testing and validation of additional student essays can be performed.

Accuracy scores strongly depend on the text volumes and number of example documents used for training. An example of essay scoring in French language involves the classification of documents from primary school manuals into five complexity classes Dascalu et al.

In this video game, master students adopt the role of an environmental policy consultancy charged with the investigation of authentic environmental problem cases.

As part of the game scenario, students have to summarise and explain their findings, obtained from a variety of legal documents, calculations and simulated stakeholder interviews cf. Teachers are supposed to manually assess these reports and return the outcomes to the students in the game. In practice, the manual assessment of many reports generates an unacceptably high teacher workload. In this game, the ReaderBench essay scoring software has demonstrated to offer an excellent replacement, offering high accuracy and considerable workload reduction.

The essay scoring service provided by ReaderBench can be accessed in a similar manner as the sentiment analysis service. Developers can either use pre-trained corpora and textual complexity models, or they can create their tailored models specific for their learning requirements.

The essay scoring service provides a wide range of textual complexity indices, freely available for English, French, Dutch, Spanish, Romanian, and Italian languages. It is a collection of open-source tools that help researchers, game developers and roboticists to incorporate a computational model of emotion and decision-making in their projects. In particular, it enables developers to easily create Role Play Characters.

These are socially intelligent characters with detailed AI modules that makes them autonomous regarding social interactions. The added value of socially intelligent characters in a serious game is twofold. First, game characters that expose believable emotional responses give the illusion of interactions with real human participants, which deepens the learning experiences. This is especially relevant for games that aim to address social skills and communication skills.

In recent years, these skills have been re-established as crucial generic skills for meeting the demands of the digital age: the so-called twenty-first century skills Dede Another most promising application area is the therapy and training of people with special social needs, for example, children with autism, who can use games with artificial social characters as a safe environment to mitigate the anxiety associated with social interactions Bernardini et al.

For this, it follows a character-centred approach rather than a plot-centred approach. The authoring is focused on defining general profiles a set of rules of how characters should respond emotionally in their games across different scenarios and contexts. Reasoner components are used in conjunction to augment the capabilities of the decision making and emotional responses of each agent in different socio-cultural contexts. An example is a reasoner addressing social importance; it allows to create groups of agents that would act and feel according to different cultural values Mascarenhas et al.

A second reasoner, which is named CiF-CK, is based on a model that describes different social exchanges and its consequences within a social environment Guimaraes et al. The toolkit is modular allowing other types of reasoners to be easily added to the system. In the Space Modules Inc. Customers with a variety of starting moods and emotional dispositions get in touch with the helpdesk about problems they are experiencing.

The player has to manage diverse situations and has to decide how best to respond. The FAtiMA Toolkit is used to model the decisions and emotional reactions of the diverse virtual customers, the outcomes of which can be used to change their on-screen appearances Fig. A similar application is the Sports Team Manager game, which is about composing and managing the best performing sailing team. The player first interviews the various virtual characters to identify their skills and personalities and then must communicate with the team, deciding which members are placed into each position per race and resolve conflict situations as they arise.

Other usage examples are in a Virtual Reality experience designed as police interrogation exercise, and in robotics: controlling the decisions of two social robots playing the card game Sueca with two human players, while exposing group-based emotions having each robot appraising both its own actions and the actions of its partner.

To facilitate its integration with game engines it works as a C library. Although any text editor of choice can be used for authoring, each component included in the role play character comes with a dedicated editor, providing a graphical user interface, syntax error detection and the capability to edit the complex intertwined data structures needed for covering the characters emotions, autobiographical memory, and appraisal rules, among other things.

The Behavior Mark-up Language BML Realizer created by Utrecht University defines and controls the on-screen representation of virtual characters, in particular their non-verbal behaviours: facial expressions, body movements, gestures, and gaze, respectively. The importance of non-verbal behaviours either from avatars or non-playing virtual characters should not be underestimated. For inducing intense, realistic game experiences the challenge is not only to make virtual characters just look like humans but also to make them behave like humans.

The behaviours should provide an illusion of realism, firstly by demonstrating responsiveness to the actions of players and other agents in the game, secondly by taking into account the context of operation, and thirdly, by securing that the behaviours are meaningful and interpretable.

In other words, the displayed behaviours should reflect the inner state of the artificial character Thiebaux et al. Thus, virtual characters should be equipped with properties such as personality, emotions and expressive non-verbal behaviours in order to engage the users to the game.

As many serious games rely on experiential learning, which means they aim to provide intense and meaningful learning experiences and allow the active participation of players in contexts that in many cases mimic professional practice, a large degree of realism or authenticity is indicated Westera et al.

Moreover, the realism supports the acquisition of tacit, implicit knowledge bound to the experiences and helps to promote successful transfer to the real world situations. In this respect, the believability of virtual characters is evident, either as personas in realistic game scenarios for instance in a job interview training or as virtual tutors that guide students during their game sessions.

There are two main approaches for modelling non-verbal behaviours and animations: rule-based procedural and machine learning Beck et al. Rule-based approaches are based on findings from social sciences and biomechanics. These rules are typically obtained through empirical analysis of human behaviour. The disadvantage of such methods is that they might not capture the full complexity of the motion trajectories.

However, they provide greater level of control, while keeping the realism at a sufficient level. Machine learning approaches automate this process and find regularities and dependencies between factors using statistics, and they learn from a larger amount of data to cover various cases. However, obtaining good annotated data is problematic.

Moreover, these data typically apply to the specific conditions of the context where they were collected, but do not necessarily generalise well. Therefore, a rule-based procedural approach was chosen for the realisation of non-verbal behaviours, providing maximum control in various application contexts. The rule-based coding approach of the BML Realizer allows to efficiently define a controlled set of the non-verbal behaviours, while avoiding the laborious job of separately coding the animations of all non-verbal behavioural attributes.

Each behaviour is divided into six animation phases bounded with seven synchronisation points: start, ready, stroke-start, stroke, stroke-end, relax, and end, respectively. Synchrony is achieved by assigning the sync-point of one behaviour to the sync-point of another. The behaviour planner that produces the BML also gets information back from the behaviour realizers about the success and failure of the behaviour requests.

The Virtual Human Controller has been successfully used in various applications. An example would be the Job-Quest game, which is a full-3D application interview training game. Also, it has been used for controlling the Virtual Receptionist character at the entrance of the computer science building at Utrecht University.

The BML realizer can be used in the Unity 3D game engine and allows to define speech, gaze and gesture animation for a conversational character. The animation pipeline includes the following steps: 1 Importing a 3D character that supports animation from an.

Beyond these functionalities, we have added Google speech recognition and chatting functionalities using AIML Pandorabots. The BML realizer has been successfully integrated with the Communicate! The LipSync Generator produces lip-synchronised speech animation. This is an important element of believable NPCs and contributes significantly to the illusion of realism and to accommodating more natural human-computer dialogues. Start on. Show related SlideShares at end. WordPress Shortcode. Share Email.

Top clipped slide. Download Now Download Download to read offline. Game Playing in Artificial Intelligence Apr. CV-French Translator1. Certificate of Appreciation for Excellence. Artificial Intelligence in games.

Practical AI in Games. Related Books Free with a 30 day trial from Scribd. Related Audiobooks Free with a 30 day trial from Scribd. Elizabeth Howell. Game Playing in Artificial Intelligence 1. Why are games relevant to AI? Usually, there is not enough time to work out the perfect move.

Hearts, Bridge, etc. If it loses, it does the opposite. It coordinates attacks, calls for backup, and retreats when hurt. Friendly AI can even be given orders by the player. Individual unit scripts are added in to some games, but in the over all picture, the AI still simply pours units at an enemy until it is defeated. So, we need another search procedures that improve — Generate procedure so that only good moves are generated. Test procedure so that the best move can be explored first.

The most common search technique in game playing is Minimax search procedure. It is depth-first depth-limited search procedure. It is used for games like chess and tic-tac-toe. Skip to content. Change Language. Related Articles. Table of Contents.