PLoS ONE

PLoS One. 14(6): e0217004

When the goal is to generate a series of activities: A self-organized simulated robot arm

Tim Koglin, Data curation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing¹, http://orcid.org/0000-0003-4887-7859Bulcsú Sándor, Conceptualization, Investigation, Methodology, Software, Supervision, Visualization, Writing – review & editing²*, Claudius Gros, Conceptualization, Methodology, Supervision, Writing – original draft, Writing – review & editing¹

Dante R. Chialvo, Editor

1. Institute for Theoretical Physics, Goethe University Frankfurt, Frankfurt am Main, Germany

2. Department of Physics, Babeș-Bolyai University, Cluj-Napoca, Romania

Consejo Nacional de Investigaciones Cientificas y Tecnicas, ARGENTINA

Competing Interests: The authors have declared that no competing interests exist.

* E-mail: bulcsu.sandor@phys.ubbcluj.ro

© 2019 Koglin et al

DOI: 10.1371/journal.pone.0217004

Published online: 19 June 2019

Abstract

Behavior is characterized by sequences of goal oriented conducts, such as food uptake, socializing and resting. Classically, one would define for each task a corresponding satisfaction level, with the agent engaging, at a given time, in the activity having the lowest satisfaction level. Alternatively, one may consider that the agent follows the overarching objective to generate sequences of distinct activities. To achieve a balanced distribution of activities would then be the primary goal, and not to master a specific task. In this setting the agent would show two types of behaviors, task-oriented and task-searching phases, with the latter interseeding the former. We study the emergence of autonomous task switching for the case of a simulated robot arm. Grasping one of several moving objects corresponds in this setting to a specific activity. Overall, the arm should follow a given object temporarily and then move away, in order to search for a new target and reengage. We show that this behavior can be generated robustly when modeling the arm as an adaptive dynamical system. The dissipation function is in this approach time dependent. The arm is in a dissipative state when searching for a nearby object, dissipating energy on approach. Once close, the dissipation function starts to increase, with the eventual sign change implying that the arm will take up energy and wander off. The resulting explorative state ends when the dissipation function becomes again negative and the arm selects a new target. We believe that our approach may be generalized to generate self-organized sequences of activities in general.

Introduction

Besides their industrial and practical applications, real and simulated robots are used increasingly to study the principles underlying embodied cognition [1] and locomotion [2], together with the self organization of critical sensorimotor states [3] and motor primitives [4]. Simulated robots may be considered in addition as proxies for cognitive and information processing agents [5].

It is well known that gaits and other regular muscle contractions, like breathing [6], are induced in many cases by central pattern generators [7, 8], even though it is currently controversial whether this is the case for biped locomotion [9], viz for human walking. Abstracting from animal models, one may ask conversely to which extent compliant locomotion may be generated via self-organizing principles [10], that is in the absence of top-down control in the form of a central pattern generator. One talks in this context of ‘embodiment’ [11], when part of the computation generating locomotion is carried out by the elasto-mechanical properties of the constituting body [12]. For quadruped robots with legs that are independently controlled by single non-linear phase oscillators [13], it has been shown that the limb-specific sensorimotor feedback derived form pressure sensors leads to self-organized interlimb communications, with emerging gaits that correspond to walking, trotting and galloping [14].

Self-organizing principles may be implemented within the sensorimotor loop [10], which is comprised of environment, body, actuator and sensory readings, with the latter being restricted in the pure case to propiosensation, viz to the internal state of the robot. The attractors self-stabilizing in the sensorimotor loop may then give rise to complex patterns of regular and of chaotic motion primitives [15], which can be selected in a second step using ‘kick control’ [16]. From a general perspective, kick control is an instance of a higher-level control mechanism exploiting the reduction in control complexity provided by morphologically computing robots [17, 18]. These approaches are hence different from other works where closed-loop policies are applied on the top of open-loop gait cycles [19, 20]. Alternatively, sequential switching between self-organizing behaviors in the combined phase space of the controller, body and environment can also be generated via self-exploration of the attractor landscape using an adaptive repelling potential [21].

Motor primitives and their generating guidelines are part of the basic constituents of a cognitive system [22]. Here we investigate whether self-organizing principles may be used also on a higher level. As a background we consider a setting where an agent has to follow a certain number of goals successively, with a typical example being that of an animal needing to forage, to watch out for predators, to rest and to socialize [23]. The agent is hence confronted with tasks that can be tackled only sequentially, a problem that may be cast into the framework of multi objective optimization [24], an approach which is however not taken in the present study. We examine instead to which extend a self-organized dynamical system may solve the time allocation problem implicitly.

As a basic protocol we consider an agent having to solve a series of indistinguishable tasks, with the agent being given by a simulated two-dimensional robot arm, as depicted in Fig 1. Within the reach of the arm there are a number of slowly moving objects the end actuator needs to reach and follow. Upon success, the self-organized dynamics of the arm should become ‘bored’ of the object, move away and search for a new one. We consider this protocol as a proxy for an agent showing a non-trivial sequence of behaviors generated not by top-down commands, but that emerges from underlying self-organizing principles.

Fig 1View larger versionFig 1. The simulated robot arm.The two angles α and β are actuated, with (5) governing the evolution of α. An equivalent dynamical system is in place for β. The arm has the task to catch one of the slowly moving objects mi, to follow it for a while, with r ≈ mi, and to switch autonomously to a distinct object.

Materials and methods

The simulated robot arm sketched in Fig 1 has two degrees of freedom, the angles α and β, with the position r = (r₁, r₂) of the end effector, the hand, being given by

[Image: pone.0217004.e001.jpg]r1=l1cos(α)-l2cos(β-α)(1)

[Image: pone.0217004.e002.jpg]r2=l1sin(α)+l2sin(β-α),(2)

where l₁ and l₂ are the respective arm lengths. We define a generalized potential U as

[Image: pone.0217004.e003.jpg]U=Um∏iT2(Ri),Ri=(r-mi)2,(3)

where Ri is the Euclidean distance between the position mi of the ith target object and r = r(α, β). In (3) we used a squashing function T,

[Image: pone.0217004.e004.jpg]T(z)=κztanh(z/sz),∂T∂θ=κzsz(1-T2κz2)∂z∂θ,(4)

which is characterized by a maximal value κz and a scale sz. We use T(z) throughout this study for the renormalization of several dynamical quantities, with the purpose to avoid exceedingly large forces or velocities. For the case of the distance we select a maximum value κR → 1, such that we have T(Ri) = tanh(Ri/sR), as entering (3). Um is then the maximal value for the potential U = U(α, β).

Robot arm dynamics

The dynamics of the angle α is controlled by

[Image: pone.0217004.e005.jpg]α˙=T(vα),v˙α=f(U)T(vα)-∇αU(α,β),(5)

where the objective function U(α, β) has the form of a mechanical potential, with ∇α denoting the gradient with respect to α. Equivalent equations govern the time evolution of β. Eq (5) corresponds to a mechanical system with a potential U and a dissipation function f(U), for which the velocity vα has been renormalized by T(z).

Mechanical systems with dissipation functions f(U) depending exclusively on the potential U, as in (5), can be considered on a general level as versatile prototype dynamical systems which exhibit, beside other, complex bifurcation cascades [25]. Several forms may be selected for the dissipation function f(U), as proposed further below. The system is adaptive [26], dispersing and taking up energy respectively for f < 0 and f > 0.

In the dissipative stage, when f(U) < 0, the arm will follow a damped trajectory towards the next minimum of the potential U = U(R), that is towards the next object mi.

For a dynamical dissipation function f(U), that is for a f = f(U) which depends functionally but not necessarily explicitly on the potential U, one can achieve that the state r ≈ mi becomes progressively unstable, such that the arm eventually moves away from the object upon taking up energy after f(U) becomes positive.

The mechanical potential in (5) treats all targets mi on an equal footing, the setup studied here.

Dissipation function dynamics

The generic principle for selecting the dissipation function f(U) is that the system needs to be dissipative when far away from all objects mi, with the configuration r ≈ mi becoming unstable once a specific target has been reached and followed for a certain time. Distinct ways to implement this principle are conceivable, here we study three possibilities.

Exponentially damped (ED). One may presume that the dissipation should become small far away from the objects, viz for large potentials U, as expressed by the ansatz

[Image: pone.0217004.e006.jpg]f(U)=f0exp(-μU),τff˙0=Et-U.(6)

The prefactor f₀ changes sign when the potential U stays below the reference energy Et for a period comparable to τf, viz when the end effector remains close to an object. Once f₀ turns positive, the arm will start to move away from the current object mi.

Trailing potential (TP). In this setup the dissipation function is explicitly time dependent, with the evolution equation being determined by the trailing potential UT = UT(t),

[Image: pone.0217004.e007.jpg]τff˙=Et-UT,τTU˙T=U-UT,(7)

where the integration time scales are regulated by τf and τT. The system is dissipative when UT is large, taking up energy once it falls below the reference energy Et.

Adapting threshold (AT). One postulates that f(U) becomes positive when the potential U falls below a time dependent threshold Uθ = Uθ(t):

[Image: pone.0217004.e008.jpg]f(U)=f0(Uθ-U)exp(-μU),τθU˙θ=Et-U,(8)

where Et is a reference energy. The overall scale for f(U) is regulated by f₀, with τθ determining the time needed for starting to take up energy, after the target has been reached dissipatively.

Further below we will present comparative results for the above three types of dissipation function dynamics, with in-detail investigations of robustness and other dynamical properties concentrating on ED.

Moving objects

For the dynamics of the moving objects, the robot arm has to grab, we used two closely related algorithms.

Polar representation of the velocity (M-PV). In the first case the absolute velocity |vi| of an object mi is drawn from an uniform distribution in [0, a], with the angle φi being drawn from [0, 2π].

Cartesian representation of the velocity (M-CV). In the second approach the Cartesian xy-components of vi are drawn independently from an uniform distribution in [−b, b].

The resulting velocity vi is applied in both cases for a time span ti which is drawn uniformly from [0, tmax]. The diffusion of the object is restricted in addition to a circular area of radius rarea, reflecting at the boundary. We generally selected rarea to coincide with the reach of the robot arm. For the other parameters we took a = b = 0.001 and tmax = 10.

As the simulation results for M-PV and M-CV are very similar, we show in the following the ones for M-PV.

Parameters

The overall length L = l₁ + l₂ of the arm is set to L = 2, with the lengths of the two segments being identical, l₁ = l₂ = 1. The parameters for the squashing function (4) for the distance are κR = 1 and [Image: pone.0217004.e009.jpg]sR=3/nL/2. For n = 3 moving objects we have hence sR = L/2 = 1.

For the maximum of the potential Um and for the reference energy Et we used Um = 17 and Et = 0.05Um, respectively, with all other parameters being listed in Table 1. For the simulation a time step of dt = 0.01 has been used.

See full tableTable 1. Simulation parameters.The parameters κv and sv entering the renormalization of the velocity of the mechanical system (5) have been adapted slightly for the three different dissipation function dynamics, ED, TP and AT. Listed are furthermore all parameters entering the respective defining Eqs (6), (7) and (8). Note that μ is given in units of 1/Um.

Results

For the parameters given in Table 1 we find transients in which the arm tends to stay close to a target it has approached. The flow in phase space is laminar when the arm is close to a target, accelerating however considerably once the dissipation function f(U) turns positive, compare (5) together with (6), (7) and (8). For a first understanding we present in Fig 2 the probability ρ(Ri) to observe the distance Ri between the end effector and a given target i, see (3). With all n = 3 targets being equivalent, one has ρ(Ri) = ρ(Rj), for all i, j ∈ [1, n].

Fig 2View larger versionFig 2. Distance statistics.The probability distribution ρ(Ri) for he distance Ri between the end effector and a selected object i, as averaged over time. The targets are indistinguishable, which implies that ρ(Ri) = ρ(Rj) for all i, j ∈ [1, n], where n = 3 is the number of moving objects. Shown are the results for three different dissipation functions dynamics, ED (top, click for animation or see S1 Video), TP (middle, click for animation or see S2 Video), and AT (bottom, click for animation or see S3 Video), as defined respectively by (6), (7) and (8). The parameters are listed in Table 1.

Following vs. explorative phase

The distribution of the distance Ri presented in Fig 2 shows that the motion of the arm can be subdivided into a phase of small Ri and a phase of medium to large distances of all sizes, modulo fine details. That this is the case for three different types of dissipation function dynamics proves that the underlying generating principles is both robust and versatile. For the three variants considered here, (6), (7) and (8), the arm will start to take up energy whenever it did hover for a certain time close to a target, dissipating on the other side energy when far away.

The evolution of key variables as a function of simulation time is presented in Fig 3. Shown are, for the ED dissipation function dynamics, the velocities vα, vβ and varm, of the actuators and respectively of the arm, together with the evolution of the dissipation function f, of the potential U, and of the distances Ri between the hand of the arm and the individual objects.

Fig 3View larger versionFig 3. Time series for three moving objects.As a function of simulation time t, the evolution of key variables for the ED dissipation-function dynamics, compare (6). (top) The angular velocities vα and vβ. (second from top) The modulus |varm| of velocity varm of the end effector. (second from bottom) The dissipation function f and the potential U, see (3), with the shading indicating that the criterion (9) is fulfilled. The separation of time scales characterizing the dynamics of f, for which a fast drop to negative values is followed by a slow recovery, drives the distinction between irregular searching phases and the laminar flow observed when the end-effector is close to a specific target. (bottom) The distances Ri to the n = 3 moving objects.

One can distinguish in Fig 3 laminar ‘following phases’ and highly irregular ‘explorative phases’. Particularly evident is the driving role of the dissipation function, which remains negative for most of the smooth following phase. Visible is also a certain time lag between the crossing of f from negative to positive values, which results from the time the system needs to take up enough energy for the angular velocities vα and vβ, and the potential U to become visible.

Robustness with respect to parameter changes

For a criterion that determines whether the end effector follows a given target we use

[Image: pone.0217004.e010.jpg]U<Et,f(U)<0,|varm|<vtarmax,(9)

which demands that the potential U is small with respect to the threshold energy Et and that the system is momentarily dissipative, viz that the dissipation function f(U) is negative. The last term in (9) rules out coincidental crossings at high velocities, which occur when magnitude of the velocity varm of the end effector is larger than the maximal velocity [Image: pone.0217004.e011.jpg]vtarmax of the targets. With the dynamics of the targets being generated, as described, [Image: pone.0217004.e012.jpg]vtarmax is known. For practical applications it would be in any case sufficient to use an empirical estimate for [Image: pone.0217004.e013.jpg]vtarmax.

Using the criterion (9), one can define a probability Pclose that measures the relative fraction of time the arm follows a target, with following and the exploration being the two dominant states of the system, as evident from Fig 3.

In Fig 4 we present for the ED dissipation function dynamics the numerical result for Pclose. Starting from the reference set of parameters Um = 17, sR = L/2 = 1, κv = 2.8 and sv = 1, compare also Table 1, the parameters have been modified one by one and the probability for the arm to follow a target evaluated. Also included in Fig 4 is the probability Pnew, namely that two targets approached successively differ.

The probability Pclose for the arm to be in the following phase increases monotonically with the strength Um of the potential, an intuitive result. Pnew decreases conversely, with the reason being that a larger Um makes it more difficult to escape the local potential well.

Increasing the characteristic length sR for the distance between the arm and a target, which enters the squashing function (4), decreases Pclose dramatically. This is because the local potential wells attracting the end actuator to a target in first place tend to disappear for large sR. Pnew increases on the other side.

The squashing parameters κv and sv for the velocity of the actuators can be changed considerable without affecting either Pclose or Pnew, implying that the system is robust with respect to both κv and sv.

Fig 4View larger versionFig 4. Parameter sweep.For the ED dissipation function dynamics, the probability Pclose for the arm to be close to one of the n = 3 targets (red circles), as defined by (9), and Pnew, which measures the chance that two targets approached one after the another are different (green triangles). With respect to the reference values Um = 17, sR = L/2 = 1, κv = 2.8 and sv = 1, the values of the parameters have been changed individually.

The data shown in Fig 4 describes the influence of global parameters. In Fig 5 we present for completeness the effect of changing the parameters Et, μ and τf of the ED dissipation function dynamics, see (6). We find the generating principle to be robust, viz that the dependency of Pclose and Pnew on Et, μ and τf is moderate.

Fig 5View larger versionFig 5. Robustness of the dissipation function dynamics.For the ED dissipation function dynamics, the probability Pclose for the arm to be close to a target (red circles), as defined by (9), and Pnew, which measures the chance that two targets approached one after the another are different (green triangles). With respect to the reference values Et = 0.05Um = 0.85, μ = 25/Um = 1.47 τf = 1.2, the values of the parameters have been changed individually for n = 3. Also included are the values of Pclose and Pnew upon changing the number n of targets. Here [Image: pone.0217004.e014]sR=3/nL/2.

Also included in Fig 5 are the values of Pclose and Pnew obtained upon changing the number n of targets. One observes that the relative fraction of time Pclose the arm spends close to a target remains flat. For n = 1 the probability to change targets vanishes, as it must, becoming on the other side substantial for large numbers of targets n.

The here presented sequential task-switching behavior, generated by the prototype dynamical system (5) does not rely on the particular choice of the generalized dissipation function dynamics. As demonstrated by Fig 2, similar distance distributions ρ(Ri) may result from very different dissipation function implementations. This is also reflected by the fraction of time spent with following and the probability of switching targets, Pclose = 0.44/0.69/0.44 and Pnew = 0.17/0.07/0.14, when comparing the dissipation functions ED/TP/AT see Eqs (6), (7) and (8) respectively, for the parameters given in Table 1.

Robustness with respect to target properties

It is clear that the arm would not be able to follow a target if the maximal velocity [Image: pone.0217004.e015.jpg]vtarmax is too large. We find, however, that the here proposed generating principle works for a substantial range of [Image: pone.0217004.e016.jpg]vtarmax. For the ED dissipation function dynamics we present in Fig 6 the time series of the dissipation function and of the potential both for the case of [Image: pone.0217004.e017.jpg]vtarmax=0.1, as used hitherto, and for [Image: pone.0217004.e018.jpg]vtarmax=0.5. We find that only details of the overall dynamics change. This holds also when increasing the number of moving objects from n = 3 to n = 8.

Fig 6View larger versionFig 6. Variable object characteristics.As a function of simulation time t, the evolution of the dissipation function f (red) and of the potential U (blue) for the ED dissipation-function dynamics. The shaded regions indicate that the criterion (9) for the arm to be in the following phase is fulfilled. (top) For n = 3 objects for which the maximal velocity is 0.5, viz five times larger than in Fig 3 (click for animation or see S4 Video). (middle) For n = 8 objects with a maximal velocity 0.1 (click for animation or see S5 Video). (bottom) For n = 8 objects with a maximal velocity 0.5 (click for animation or see S6 Video).

A single non-moving target

From the dynamical system perspective it is of interest to investigate the case of a single stationary target. With noise being absent, the system is deterministic.

Fixpoints. In case of a purely dissipative dynamics, with f(U) = f₀ < 0, the system disposes of two stable fixpoints, defined by vanshing angular velocities vα, vβ → 0, that correspond to a right- and respectively to a left bend.

Limit cycle attractors. With the dynamical dissipation function ED, it is evident that the robot arm settles into a limit cycle in which the destabilized fixpoints are revisited, see Fig 7. There exist, hence, multiple symmetry related limit cycles even for a single resting target (only one of them is shown).

Fig 7View larger versionFig 7. A single non-moving object.As a function of simulation time t, the evolution of key variables for the ED dissipation-function dynamics, as for Fig 3, but here for a single non-moving object located at (3/8, 3/8)L. The system is fully deterministic, with the robot arm settling into a limit cycle. The criterion (9) for the arm to be close to the object is not applicable, as [Image: pone.0217004.e019]vtarmax=0.

Therefore, in the presence of multiple fixed targets, several different activity sequences may be generated, even for the same starting position r(0) of the arm, viz for different initial conditions of the internal variables.

Discussion

Action switching in embodied agents may be guided by fitness considerations, f.i. when the task is to collect a series of different food sources [27]. Typically, the action selected at a given time will be then the one with the most pressing need. We have followed here a different approach, examining an overarching generation principle and not the generation of action sequences driven by an utility optimization that is local in time.

The stationarity principle

The question how to decide in which action to engage has been termed the motivational problem [28]. The utility of many activities, like foraging, socializing and resting, that are regularly repeated, address distinct needs, which implies that they cannot be lumped together into an overarching utility function. In terms of multi-objective optimization [24] the agent must dedicate time to a range of activities, with the constraint that the resulting distribution of utilities remains within a given range. This constraint may be expressed as a stationarity principle, namely that the statistical properties of the time series of activities should become stationary for extended time spans.

The result presented here for the self-organized robot arm can be viewed as an implementation of the stationarity principle. With the dynamics being irregular, viz chaotic, in the explorative phase, the exact sequence of objects followed is not pre-determined. The long term statistics, such as the distance distribution presented in Fig 2, is however stationary.

The stationarity principle is a guiding principle that can be used in various settings. Statistical learning, e.g. of receptive fields [29], is characterized by statistically stationary sensory inputs, with learning continuing until the statistics of the output activity becomes also stationary [30]. It has been shown, that one can use the Fisher information of the neural firing rate to encode the stationarity principle [31] and that one obtains Hebbian learning when minimizing the Fisher information, viz when the stationarity condition is enforced.

Transient-state dynamics

With the agent being formulated in term of a mechanical system, see Eq (5), one can abstract from the behavioral level and describe the robot arm within dynamical system theory [26]. The striking alternation of dynamical states, as visible in Fig 3, can be interpreted in this context as an example of transient-state dynamics [32]. The following phase corresponds on a dynamical level to a transient attractor that becomes unstable on an extended time scale, namely when the dissipation function turns positive.

The here discussed mechanism, the coupling of an attracting state to a slow variable, is the core route for generating transient-state dynamics in general [33], with the flow being laminar during the transient dynamics, and irregular during the transition periods. We note that transient-state dynamics may be viewed as a form of metastability, which may arise either from the brain dynamics as such [34], or from sensorimotor couplings in response to tasks demanding behavioral flexibility [35].

Distinguishable vs. non-distinguishable targets

It would be possible to introduce a bias bi = bi(t) that allows to differentiate between distinct objects. In this case one would work with the generalized Euclidean distance

[Image: pone.0217004.e020.jpg]Ri→Ri2+bi2.(10)

instead of (3), for which the bias bi encodes the depth of the potential, and with this indirectly also the relative importance of the respective object. For an appropriate evolution equation for bi(t), the respective target would become repelling once the end effector of the robot has reached it. Two routes on how the dynamical system (5) induces an autonomously generated sequence of behaviors are hence possible.

Distinguishable targets. One works with a constant dissipation function, f(U) → f₀, with every object being characterized by a time-dependent attribute, namely bi = bi(t).

Indistinguishable targets. When all bi ≡ 0 there is no variable distinguishing the individual objects. The sequence of behaviors is then a consequence of dynamical instabilities resulting from the dynamics of the dissipation function.

In this study we concentrate on the second case as the basic generative mechanism, noting that the resulting residence times, viz when r ≈ mi, could be fine-tuned in a second step by allowing the bi to be weakly time dependent. This protocol is left for future studies.

Conclusion

One of the biggest challenges in the design of controllers for autonomous agents is the combination of different goal oriented behaviors into a series of self-organized activities [36]. Here, we investigated how such a higher order controller may be constructed within a dynamical systems framework, by adapting a recently introduced versatile prototype system [25] to the problem of an object-following arm. By introducing a model with a dynamically changing generalized dissipation function we provide a proof of concept demonstration of how target following can be turned into a sequential task switching behavior in terms of transient-state dynamics [32].

Within this framework the goal oriented activities are represented by a target-following behavior of a simulated arm, while the switching dynamics between targets corresponds to an explorative phase upon getting bored of the respective task.

Such a self-organized behavior can be generated both at the level of motion primitives, in case of robotic locomotion [10], and on the level of action selection [27], as demonstrated here. The resulting behavior is robust within a wide range of parameters, as it does not require precise fine tuning, which simplifies the selection of an adequate parameter set with, e.g., machine learning techniques. Being based on self-organized attractors in the overarching phase space of agent and environment, the sensorimotor loop, our approach is resistant to external noise, retaining at the same time the flexibility to adapt to the environment or to interact with other agents [15].

The proposed framework can be generalized to produce series of activities with a well-defined order or a given multi-modal probability distribution by modulating the Euclidean distance as a function of the actual importance of the respective task – a research direction left for future studies.

Supplementary Material

Open In Web Browser

Articles from PLoS ONE are provided here courtesy of Public Library of Science

PMC Copyright Notice

The articles available from the PMC site are protected by copyright, even though access is free. Copyright is held by the respective authors or publishers who provide these articles to PMC. Users of PMC are responsible for complying with the terms and conditions defined by the copyright holder.

Users should assume that standard copyright protection applies to articles in PMC, unless an article contains an explicit license statement that gives a user additional reuse or redistribution rights. PMC does not allow automated/bulk downloading of articles that have standard copyright protection.

See the copyright notice on the PMC site, https://www.ncbi.nlm.nih.gov/pmc/about/copyright/, for further details and specific exceptions.

References

Martius G, Der R, Ay N. Information driven self-organization of complex robotic behaviors. PloS one. 2013;8(5):e63400 10.1371/journal.pone.0063400 23723979

Ijspeert AJ. Biorobotics: Using robots to emulate and investigate agile locomotion. science. 2014;346(6206):196–203. 10.1126/science.1254486 25301621

Aguilera M, Barandiaran XE, Bedia MG, Seron F. Self-organized criticality, plasticity and sensorimotor coupling. Explorations with a neurorobotic model in a behavioural preference task. PloS one. 2015;10(2):e0117465 10.1371/journal.pone.0117465 25706744

Tani J, Ito M. Self-organization of behavioral primitives as multiple attractor dynamics: A robot experiment. IEEE Transactions on Systems, Man, and Cybernetics-Part A: Systems and Humans. 2003;33(4):481–488. 10.1109/TSMCA.2003.809171

Beer RD, Williams PL. Information processing and dynamics in minimally cognitive agents. Cognitive science. 2015;39(1):1–38. 10.1111/cogs.12142 25039535

Arshavsky Y, Deliagina T, Orlovsky G. Central Pattern Generators: Mechanisms of Operation and Their Role in Controlling Automatic Movements. Neuroscience and Behavioral Physiology. 2016;46(6):696–718. 10.1007/s11055-016-0299-5

Marder E, Bucher D. Central pattern generators and the control of rhythmic movements. Current biology. 2001;11(23):R986–R996. 10.1016/S0960-9822(01)00581-4 11728329

Ijspeert AJ. Central pattern generators for locomotion control in animals and robots: A review. Neural Networks. 2008;21(4):642–653. 10.1016/j.neunet.2008.03.014 18555958

Minassian K, Hofstoetter US, Dzeladini F, Guertin PA, Ijspeert A. The human central pattern generator for locomotion: Does it exist and contribute to walking? The Neuroscientist. 2017;23(6):649–663. 10.1177/1073858417699790 28351197

Sándor B, Jahn T, Martin L, Gros C. The sensorimotor loop as a dynamical system: how regular motion primitives may emerge from self-organized limit cycles. Frontiers in Robotics and AI. 2015;2:31.

Pfeifer R, Lungarella M, Iida F. Self-organization, embodiment, and biologically inspired robotics. science. 2007;318(5853):1088–1093. 10.1126/science.1145803 18006736

Aguilar J, Zhang T, Qian F, Kingsbury M, McInroe B, Mazouchova N, et al A review on locomotion robophysics: the study of movement at the intersection of robotics, soft matter and dynamical systems. Reports on Progress in Physics. 2016;79(11):110001 10.1088/0034-4885/79/11/110001 27652614

Owaki D, Kano T, Nagasawa K, Tero A, Ishiguro A. Simple robot suggests physical interlimb communication is essential for quadruped walking. Journal of The Royal Society Interface. 2013;10(78):20120669 10.1098/rsif.2012.0669

Owaki D, Ishiguro A. A quadruped robot exhibiting spontaneous gait transitions from walking to trotting to galloping. Scientific reports. 2017;7(1):277 10.1038/s41598-017-00348-9 28325917

Martin L, Sándor B, Gros C. Closed-loop robots driven by short-term synaptic plasticity: Emergent explorative vs. limit-cycle locomotion. Frontiers in neurorobotics. 2016;10:12 10.3389/fnbot.2016.00012 27803661

Sándor B, Nowak M, Koglin T, Martin L, Gros C. Kick control: using the attracting states arising within the sensorimotor loop of self-organized robots as motor primitives. Frontiers in neurorobotics. 2018;12 10.3389/fnbot.2018.00040 30050427

Müller VC, Hoffmann M. What is morphological computation? On how the body contributes to cognition and control. Artificial life. 2017;23(1):1–24. 10.1162/ARTL_a_00219 28140632

Ghazi-Zahedi K, Langer C, Ay N. Morphological computation: Synergy of body and brain. Entropy. 2017;19(9):456 10.3390/e19090456

Spröwitz A, Tuleu A, Vespignani M, Ajallooeian M, Badri E, Ijspeert AJ. Towards dynamic trot gait locomotion: Design, control, and experiments with Cheetah-cub, a compliant quadruped robot. The International Journal of Robotics Research. 2013;32(8):932–950. 10.1177/0278364913489205

Travers M, Ansari A, Choset H. A dynamical systems approach to obstacle navigation for a series-elastic hexapod robot. In: Decision and Control (CDC), 2016 IEEE 55th Conference on. IEEE; 2016. p. 5152–5157.

Pinneri C, Martius G. Systematic self-exploration of behaviors for robots in a dynamical systems framework. In: The 2018 Conference on Artificial Life. Cambridge, MA: MIT Press; 2018. p. 319–326.

Gros C. Cognition and emotion: perspectives of a closing gap. Cognitive Computation. 2010;2(2):78–85. 10.1007/s12559-010-9034-7

Sibly R, McFarland D. On the fitness of behavior sequences. The American Naturalist. 1976;110(974):601–617. 10.1086/283093

Deb K. Multi-objective optimization In: Search methodologies. Springer; 2014 p. 403–449.

Sándor B, Gros C. A versatile class of prototype dynamical systems for complex bifurcation cascades of limit cycles. Scientific reports. 2015;5:12316 10.1038/srep12316 26198731

Gros C. Complex and adaptive dynamical systems: A primer. Springer; 2015.

Agmon E, Beer RD. The evolution and analysis of action switching in embodied agents. Adaptive Behavior. 2014;22(1):3–20. 10.1177/1059712313511649

Gros C. Emotional Control–Conditio Sine Qua Non for Advanced Artificial Intelligences? Philosophy and Theory of Artificial Intelligence. 2012;5:187 10.1007/978-3-642-31674-6_14

Brito CS, Gerstner W. Nonlinear Hebbian learning as a unifying principle in receptive field formation. PLoS computational biology. 2016;12(9):e1005070 10.1371/journal.pcbi.1005070 27690349

Echeveste R, Eckmann S, Gros C. The fisher information as a neural guiding principle for independent component analysis. Entropy. 2015;17(6):3838–3856. 10.3390/e17063838

Echeveste R, Gros C. Generating functionals for computational intelligence: the Fisher information as an objective function for self-limiting Hebbian learning rules. Frontiers in Robotics and AI. 2014;1:1 10.3389/frobt.2014.00001

Gros C. Neural networks with transient state dynamics. New Journal of Physics. 2007;9(4):109 10.1088/1367-2630/9/4/109

Gros C. Cognitive computation with autonomously active neural networks: an emerging field. Cognitive Computation. 2009;1(1):77–90. 10.1007/s12559-008-9000-9

Kelso JS. Multistability and metastability: understanding dynamic coordination in the brain. Phil Trans R Soc B. 2012;367(1591):906–918. 10.1098/rstb.2011.0351 22371613

Aguilera M, Bedia MG, Barandiaran XE. Extended Neural Metastability in an Embodied Model of Sensorimotor Coupling. Frontiers in systems neuroscience. 2016;10:76 10.3389/fnsys.2016.00076 27721746

Bekey GA. Autonomous robots: from biological inspiration to implementation and control. MIT Press; 2005.

[Back]

Fig 1. The simulated robot arm.

The two angles α and β are actuated, with (5) governing the evolution of α. An equivalent dynamical system is in place for β. The arm has the task to catch one of the slowly moving objects mi, to follow it for a while, with r ≈ mi, and to switch autonomously to a distinct object.

[Back]

[Back]

Table 1. Simulation parameters.

The parameters κv and sv entering the renormalization of the velocity of the mechanical system (5) have been adapted slightly for the three different dissipation function dynamics, ED, TP and AT. Listed are furthermore all parameters entering the respective defining Eqs (6), (7) and (8). Note that μ is given in units of 1/Um.

| κ v | s v | μU m | τ f | τ T | τ θ | f ₀
ED | 2.8 | 1 | 25 | 1.2 | • | • | •
TP | 4.3 | 3 | • | 6.0 | 4.0 | • | •
AT | 4.0 | 2 | 34 | • | • | 1 | 0.5

DOI: 10.1371/journal.pone.0217004.t001

[Back]

[Back]

Fig 2. Distance statistics.

The probability distribution ρ(Ri) for he distance Ri between the end effector and a selected object i, as averaged over time. The targets are indistinguishable, which implies that ρ(Ri) = ρ(Rj) for all i, j ∈ [1, n], where n = 3 is the number of moving objects. Shown are the results for three different dissipation functions dynamics, ED (top, click for animation or see S1 Video), TP (middle, click for animation or see S2 Video), and AT (bottom, click for animation or see S3 Video), as defined respectively by (6), (7) and (8). The parameters are listed in Table 1.

[Back]

[Back]

Fig 3. Time series for three moving objects.

As a function of simulation time t, the evolution of key variables for the ED dissipation-function dynamics, compare (6). (top) The angular velocities vα and vβ. (second from top) The modulus |varm| of velocity varm of the end effector. (second from bottom) The dissipation function f and the potential U, see (3), with the shading indicating that the criterion (9) is fulfilled. The separation of time scales characterizing the dynamics of f, for which a fast drop to negative values is followed by a slow recovery, drives the distinction between irregular searching phases and the laminar flow observed when the end-effector is close to a specific target. (bottom) The distances Ri to the n = 3 moving objects.

[Back]

[Back]

Fig 4. Parameter sweep.

For the ED dissipation function dynamics, the probability Pclose for the arm to be close to one of the n = 3 targets (red circles), as defined by (9), and Pnew, which measures the chance that two targets approached one after the another are different (green triangles). With respect to the reference values Um = 17, sR = L/2 = 1, κv = 2.8 and sv = 1, the values of the parameters have been changed individually.

[Back]

[Back]

Fig 5. Robustness of the dissipation function dynamics.

For the ED dissipation function dynamics, the probability Pclose for the arm to be close to a target (red circles), as defined by (9), and Pnew, which measures the chance that two targets approached one after the another are different (green triangles). With respect to the reference values Et = 0.05Um = 0.85, μ = 25/Um = 1.47 τf = 1.2, the values of the parameters have been changed individually for n = 3. Also included are the values of Pclose and Pnew upon changing the number n of targets. Here [Image: pone.0217004.e014]sR=3/nL/2.

[Back]

[Back]

Fig 6. Variable object characteristics.

As a function of simulation time t, the evolution of the dissipation function f (red) and of the potential U (blue) for the ED dissipation-function dynamics. The shaded regions indicate that the criterion (9) for the arm to be in the following phase is fulfilled. (top) For n = 3 objects for which the maximal velocity is 0.5, viz five times larger than in Fig 3 (click for animation or see S4 Video). (middle) For n = 8 objects with a maximal velocity 0.1 (click for animation or see S5 Video). (bottom) For n = 8 objects with a maximal velocity 0.5 (click for animation or see S6 Video).

[Back]

[Back]

Fig 7. A single non-moving object.

As a function of simulation time t, the evolution of key variables for the ED dissipation-function dynamics, as for Fig 3, but here for a single non-moving object located at (3/8, 3/8)L. The system is fully deterministic, with the robot arm settling into a limit cycle. The criterion (9) for the arm to be close to the object is not applicable, as [Image: pone.0217004.e019]vtarmax=0.

[Back]