Reaching High Interactive Levels with Situated Agents

Abstract

The paper presents a coordinated process through which the situated agents are capable of working jointly using knowledge about their skills to interact with other similar entities. A good coordination is reached because each situated agent is able to represent its suitability rates to perform any action in an effective way. Besides using such rates, an agent can coordinate its actions with other agents. In this sense, each situated agent tries to select and perform only the actions with the highest execution rates. Some experiments in a real robotic soccer test-bed are designed to put in practice the proposed coordinated approach. Final remarks conclude the effectiveness and advantages of the presented work in order to increase the performance of a team composed by situated agents when they must solve complex tasks in a dynamical, competitive and unpredictable scenario.

Resumen

Este artículo presenta un proceso de coordinación a través del cual, los agentes situados son capaces de trabajar conjuntamente utilizando conocimiento acerca de sus capacidades para interactuar con otras entidades similares. Una buena coordinación se puede lograr debido a que cada agente situado es capaz de representar, de manera efectiva, sus rangos de efectividad para ejecutar cualquier acción. Además, usando la información puede coordinar su trabajo con otros agentes. En este sentido, cada agente situado trata de seleccionar y ejecutar únicamente las acciones para las que tiene un alto nivel de efectividad. Algunos experimentos en el escenario de futbol robótico se desarrollaron para poner en práctica el método de coordinación aquí propuesto. Los comentarios finales concluyen la efectividad y ventajas del trabajo presente para incrementar el rendimiento de un equipo de agentes situados cuando ellos deben solucionar tareas complejas en escenarios dinámicos, competitivos e impredecibles.

Introduction

Simply stated, decision-making structure is a process where agents should be able to select a specific action out of multiple alternatives. Such process occurs continuously in daily life. Human beings, for instance, usually must take decisions about what cloths to wear, what food to eat, etc. Likewise, an agent is defined as anything that is situated in a specific environment and acts based on its observation, interpretation and knowledge about the situation of such environment, to select a particular action. Therefore, to take decisions, agents must obtain knowledge that allows them to be aware of what actions can or cannot perform. Agents who should work jointly must be able to converse with other agents in order to communicate information related to the actions that each can do. Methods for cooperative multi-agent decisions are therefore, in most cases, intensive software applications and highly sophisticated algorithms that use advanced design technologies. In particular, some research trends have led to managing complex and cooperative problems using agents. A situated agent is presented as a physical and mobile entity capable of flexible and autonomous actions in dynamic, unpredictable and typically cooperative environments (Luck et al., 2005). In this sense, several results to solve coordinated actions have been obtained using agent technology (Bogdan et al., 2008; Zhang and Parker, 2010; Barret et al., 2010). In particular, relevant knowledge is not appropriately reflected and communicated by the agents. These deficiencies do not allow agents to make reliable collective decisions when these are requested (Ibarra, 2008). Obviously, lack of the appropriate reasoning on these knowledge results in a lower cooperative performance among agents, where a proper managing of such knowledge is quite relevant to achieve sure and trustworthy commitments. Achieving cooperative agents is still a challenge, because many issues must be considered in order to develop a working cooperative team, such as task allocation, conflict of interest, communication, etc. (Fang and Parker, 2007; Parker, 2008; Jeong et al., 2006; Benson et al., 2007). In such a case, cooperative agent systems often work in dynamic and unpredictable scenarios, for thus, the members of a team-agent must respond robustly and adaptively to unexpected environmental changes, such as unsuitable communication among them or modification in the agent system configuration due to failures of performance. In particular, cooperative systems of situated agents are characterized by distributed control of heterogeneous agents. A number of interesting reasons exist to support the cooperation among these agents. In fact, the objective of the cooperation is to maintain maximum utilization of multi-agent resources while ensuring job performance at the highest productive level.

How to reach interaction

High interactive levels are desired due to the fact that the solution to a more complex problem could be more successfully performed if multiples autonomous and intelligent entities can work together with the same global objective in mind. To reach interaction, first, an agent must be able to differentiate from each other in order to perceive its situation. Such information allows agents to know what action they can do by estimating their operatively rates from all the possible alternatives.

Let us define a situated agent (SA_j as an intelligent entity with a physical representation and through which the system can perform physical changes on the environment. These situated agents are embodied by considering the knowledge involved in their capability to execute an action within their knowledge base. Let us suppose that a SA_j is part of a cooperative group of situated agents (G_SA). A group of situated agents must generally involve more than one situated agent for the fulfillment of a particular task.

where G_SA = {SA₁, SA₂, SA₃, …, SA_m}

According to this, an agent uses the information provided by three (3) parameters, called decision axes, where each axis provides situated agents with knowledge related to its capability to execute any determined action with a particular kind of information in a determined time (t) inside a specific spatial region, called scene (S). Axis 1, environmental conditions (EC), is composed by information related to the state of the environment directly involved in the performance of a particular action. Axis 2, physical knowledge (PK), is represented by the specification, the structure and other relevant details related to the agents' physical skills and characteristics. Finally, axis 3, trust value (TV), is related to the capability of an agent to communicate, to interact and other relevant details to entrust in other agents. In this light, the situated agent's knowledge base KB(SA_j) is therefore based on the combination of the three axes above (EC, PK and TV) directly implicated in the execution of any action, such as it is described by (1).

In particular the situated agent's knowledge base for the execution of a specific role (R_β) in a given time (t), in a determined scene (S_α), is given by (2).

In particular, this paper assumes that each situated agent is capable to evaluate its range of aptitude to perform any action. Such evaluation is performed by a match that includes two important aspects used to calculate their suitability rate to all the proposed action. These aspects are:

• •

  The capacity of a situated agent taking into account the information provided by the three axes.

• •

  The influence degree that each axis has over the execution of any determined action.

In particular, the influence degree γ refers to the relevance that decision axes have in the selection of a determined action in a particular scene. Such influence aims to calculate critically the suitability of a situated agent to execute any role R_β from all the roles R in a successful and reliable way. In this sense, such influence degree γ is represented by (3).

where, γECRβγPKRβγTVRβ∈0,1

and

γECRβ is the relevance of the environmental conditions,

γPKRβ is the relevance of the physical knowledge and

γTVRβ is the relevance of the trust value.

In particular the influence degree for the development of any specific role in any determined scene is given by (4).

Then, the suitability rate ξ of any situated agent is obtained by a match between the requirement of the roles γRβtsα and the capability of the situated agents KBSAj,RβtSα. Let us suppose that a situated agent SA_j is capable of executing a role R_β with a suitability rate ξ in a time t, within a scene S_α as it is described in (5).

To the end, this match ensures that any situated agent is capable of knowing which roles it can do in a suitable and reliable way. This fact guarantees an increase in the performance of the team-agent because almost all the time each role is performed by a trustworthy situated agent.

Some experiments and results

Experiments are devoted to prove the efficiency of the proposed approach. Hence, some features have been fixed to do an implementation in the real robotic soccer test-bed. Particularly, each axis is implemented using different information of the field, as it is described.

Environmental conditions (axis 1), considered here as a parameter of proximity P, are related to the real distance between the current location of a situated agent and the current location of the proposed role, and it is provided by (6).

Where d max means the maximal distance of a situated agent with the proposed roles in a scene S_α as it is described in (7)

Physical knowledge (axis 2) refers to the cognitive ability of each situated agent to estimate the knowledge related to the capabilities of its body involved in the execution of a proposed action, represented as a capability of introspection I. Introspection parameter is calculated implementing feed-forward back-propagation neural networks.

Trust value (axis 3), here called trust T, refers to the social relationship among agents taking into account both the amount of “good” actions which are actions executed in a suitable way and the amount of “bad” actions which are actions that are executed in a negative way. So that, the trust measure of a situated agent is provided by (9).

Once the implementation of each axis has been introduced, an agent is capable to define how the relevance of the set of decision axes can influence in the calculus of the situated agents' suitability rates. We have designed a classification of a binary combination of the three axes (i.e., [0, 0, 0] where 0 means that the parameter is disable and [1, 1, 1] where 1 means that the parameter is used). From this, we have obtained eight study cases. In particular, each study case denotes the behavior of each one of the team that we have used in the empirical experiments. It means that each team-agent uses one of the binary combinations to allow agents self-calculate their suitability rates. In particular, Figure 1 shows the general scheme of the robot soccer test-bed used in this experimental phase.

Figure 1.

General scheme of the Robot Soccer Test-bed used in the experiments

These experiments are related to eight different teams of agents which use each one of the eight possible influence degree. These teams compete among themselves in ten (10) episodes of only one game with a predefined period of five (5) minutes. In particular, the overall expected team's performance is related to reaching the greater number of achieved points and goals. For example, Figure 2 illustrates the obtained results in one of the episodes of the experiments. Here, the overall expected performance by each team is measured using the total score of points to obtain a goal average (G_AVE). The goal average is calculated by the difference between the obtained goals (G+) and the received goals (G−).

Figure 2.

Results of the first episode: a) Scores obtained in each game, b) Ranking of the teams

In addition, the Figure 3 illustrates the teams' performance taking into account the successful number of points obtained based on the won and tied games along all the episodes. The performance does not improve significantly somewhere beyond the 6 episodes of the experiment. The number of trials (episodes) to initially confirm the systems performance was fixed in 10.

Figure 3.

Performance comparison between the agents-teams

As a conclusion, these statistical results confirm clearly the idea that situated agents can make sure decisions, and carry out a better performance when they use knowledge related to their situation to perform any proposed action. Due to the difficult, in the development of these experiments, the number of games is small but not less important for the aims of this research.

So, the exposed results confirm clearly the difference between the system performances where it uses coordinated task problem-solving strategy. In particular, there is an improvement rate of around an 63% in this scenario between the best team (using the selection of parameters [1, 1, 1]) and the worst team (using the parameters [0, 0, 0]).

Final remarks

In this paper it is possible to argue how situated agents that use information about their rates of actions are capable to reach better decisions out of all their possible alternatives. In this light, the decision performance (successful decisions) of the situated agents is better when they are able to estimate their situation (i.e., their knowledge base) related to the execution of any proposed actions, than when the agents do not use it. Successful decisions in the experiments are related to the ratio between the number of successful tasks performed by the situated agent and the total number of actions performed by the same agent. According to this, the situated agents increase significantly the number of successful actions if they are able to base its decisions on their situation considering the three proposed parameters over the environment.

The data from the experiments discloses that the implementation of the three parameters of the decision axes combined within the agents' decision-making structure, produces best performance in all the experiments. But more importantly, the system performance (successful performance) is significantly better when the agents increase the information (i.e., when the agents use grater amount of knowledge) involved in their decision-making to perform any action. However, the remaining cases show interesting results but not an optimal strategy for the present domains at all. This fact illustrates that the choice of a strategy that includes knowledge in the agents' decision-making structure is far from trivial. In this case, the obtained results are significant and show the need for further investigation about the agents' situation and its effect in the performance of complex problems in dynamic and cooperative environments.

This is a complicated process because the number of roles grows exponentially. Therefore, each agent is able to perceive and interpret the information involved in the proposed actions to generate its knowledge base. In this sense; redundancy in the tasks execution is then avoided. Finally, this new and effective approach contributes to enhance multi-agent efficiency and performance in dynamic and cooperative environments because the agents are capable of knowing if they can perform the proposed action.

Acknowledgments

This work is supported by the Facultad de Ingenieria “Arturo Narro Siller” at the Universidad Autonoma de Tamaulipas.