ARTICLE IN PRESS 1 57 2 58 3 4 Neural Networks xx (2003) 1–6 www.elsevier.com/locate/neunet 59 60 5 61 6 62 7 63 Controlling chaos in a chaotic neural network 8 9 10 11 12 13 14 Guoguang Hea,*,1, Zhitong Caoa, Ping Zhua, Hisakazu Ogurab a Department of Physics, Zhejiang University, Hangzhou 310028, People’s Republic of China Department of Human and Artificial Intelligent Systems, Faculty of Engineering, Fukui University, 3-9-1 Bunkyo, Fukui 910-8507, Japan b Received 18 May 2001; accepted 31 January 2003 15 16 19 20 21 22 23 24 25 26 27 28 65 66 67 68 69 70 71 72 17 18 64 73 74 Abstract The chaotic neural network constructed with chaotic neuron shows the associative memory function, but its memory searching process cannot be stabilized in a stored state because of the chaotic motion of the network. In this paper, a pinning control method focused on the chaotic neural network is proposed. The computer simulation proves that the chaos in the chaotic neural network can be controlled with this method and the states of the network can converge in one of its stored patterns if the control strength and the pinning density are chosen suitable. It is found that in general the threshold of the control strength of a controlled network is smaller at higher pinned density and the chaos of the chaotic neural network can be controlled more easily if the pinning control is added to the variant neurons between the initial pattern and the target pattern. q 2003 Elsevier Science Ltd. All rights reserved. Keywords: Chaotic dynamic; Chaotic neural network; Controlling chaos; Pinning control method 75 76 77 78 79 80 81 82 83 84 29 85 30 86 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 1. Introduction Recently the chaotic neural network constructed with chaotic neurons has received much attention because of its complex dynamics and its potential application in the associative dynamics, optimization and information processing, etc. (Adachi & Aihara, 1997; Aihara, Takabe, & Toyoda, 1990; Tokuda, Nagashima, & Aihara, 1997). Besides, the electrophysiological experiments of animals have proved that chaos dynamics exists in real neurons and neural networks play an important role in neuron activity (Degn, Holden, & Olsen, 1987; Freeman, 1987; Tsuda, 1991). It is believed that the investigation of the dynamics characters of chaotic neural networks is helpful to an understanding of the memory rules of the brain. In Adachi and Aihara (1997), the non-periodic associative dynamics of the chaotic neural network was studied in detail. The network can retrieve the stored patterns, but they appear nonperiodically since the network is in chaos. So it is * Corresponding author. Tel./fax: þ 86-571-8884-1465. E-mail address: [email protected], [email protected] (G. He). 1 Address until October 2003: Ogura Laboratory, Department of Human and Artificial Intelligent Systems, Faculty of Engineering, Fukui University, 3-9-1, Bunkyo, Fukui 910-8507, Japan. difficult to distinguish the stored patterns from each other in the chaotic neural network (Kobori, Ikoda, & Nakayama, 1996), which limits the applications of the associative memory function of the network. If one wants to realize the information processing in the associative dynamics system, the chaotic orbit of the network should be controlled in a certain expected way and the network should be stabilized in a stored pattern. The problem of controlling the chaos of the chaotic neural network therefore is put forward. Since the initial work on chaotic control was made by Ott, Grebogi, and Yorke (1990) (OGY), much progress has been achieved in this field. Several control methods were put forward, such as chaos synchronization (Pecora & Carroll, 1990), OPF (Hunt, 1991) and delayed controlling feedback (Pyragas, 1992). But all the above methods only suited to the low dimension chaotic systems. For a high dimension spatial-temporal chaotic system, Hu proposed a control method, i.e. the chaotic pinning control method, which has been used to successfully control the chaos of a coupled map lattice systems in one dimension (Hu & Qu, 1994). The chaotic neural network shows the more complex spatial-temporal chaotic dynamics compared to the coupled map lattice system in which each lattice site of coupled map lattice systems is only coupled to its nearest 0893-6080/03/$ - see front matter q 2003 Elsevier Science Ltd. All rights reserved. doi:10.1016/S0893-6080(03)00055-8 NN 1731—29/3/2003—12:48—SHYLAJA—65591— MODEL 5 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 ARTICLE IN PRESS 2 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 G. He et al. / Neural Networks xx (2003) 1–6 ones. Besides the neurons in the chaotic neural networks are connected to each other in whole spatial, the delay feedback of the networks is more complicated than the coupled map lattice system. We applied Hu’s pinning control method to the chaotic neural network and found that the method was not succeeded in controlling the chaos of the chaotic neural network. Several other attempts have been made to control the chaos of the network, but the target of the control are spatially uniform or periodic states which are far from some states useful for information processing (Adachi, 1995; Mizutani, Sato, Uchiyama, & Sonehara, 1995). In this work, we propose a pinning method focused on the chaotic neural network. The feedback control in our pinning method is imposed on the internal state of neurons while the feedback control in Hu’s method is put to the output of the system. The computer simulation proved that the chaos of the chaotic neural network could be controlled by the pinning control method and the output of the network could be stabilized in a stored pattern. In Section 2, the chaotic neural network used as a controlling object in the work is described. Our pinning control method and its mathematical model are proposed in Section 3. In Section 4 are given the computer simulation results and the discussion about two control parameters, pinning distance and strength of control. Conclusion and discussion are in Section 5. 141 142 143 2. Chaotic neural network model 144 145 146 147 148 149 150 The chaotic neural network model used in the paper is constructed with chaotic neurons by considering the spatiotemporal summation of both external inputs and feedback inputs from other chaotic neurons (Adachi & Aihara, 1997; Aihara et al., 1990). Fig. 1 shows the structure of the neural network. The dynamics of the ith chaotic neuron in 151 152 153 154 the chaotic neural network is described as follows: 2 M t N t X X X X ked Aj ðt 2 dÞ þ wij kfd xj ðt 2 dÞ xi ðt þ 1Þ ¼f 4 vij j¼1 2a j¼1 d¼0 # M X vij Aj ðtÞ þ ke ji ðtÞ ð2Þ j¼1 hi ðt þ 1Þ ¼ N X wij xj ðtÞ þ kf hi ðtÞ ð3Þ ui ; Qi ð1 2 kr Þ ð4Þ xi ðt þ 1Þ ¼ f {ji ðt þ 1Þ þ hi ðt þ 1Þ þ zi ðt þ 1Þ} ð5Þ where: ji ðt þ 1Þ; hi ðt þ 1Þ and zi ðt þ 1Þ are internal state terms for external inputs, feedback inputs from the constituent neurons in network and refractoriness, respectively. The chaotic neural network is constructed with 100 chaotic neurons in this paper. As the external input term Aj is taken as a temporally constant in the chaotic neural network (Adachi & Aihara, 1997), ji ðtÞ can be included in the threshold. If we define a new threshold ai ; Eq. (5) can be simplified as the following form with only two internal states hi and zi : 100 X wij xj ðtÞ ð6Þ ð7Þ 162 163 164 165 166 wij ¼ 167 Fig. 1. Chaotic neural network model. NN 1731—29/3/2003—12:48—SHYLAJA—65591— MODEL 5 1 n n X p¼1 178 179 180 181 182 183 184 185 186 187 188 189 190 192 193 195 196 ð2xpi 2 1Þð2xpj 2 1Þ 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 ð8Þ 215 216 We take the output function of the neuron f ðxÞ as Sigmoid function with the steepness parameter 1; i.e. f ðyÞ ¼ 1={1 þ expð2y=1Þ}; refractoriness function as gðxÞ ¼ x: The feedback weights are defined according to the following symmetric auto-associative matrix of n binary patterns: 161 177 197 zi ðt þ 1Þ ¼ kr zi ðtÞ 2 ag{xi ðtÞ} þ ai 160 176 194 zi ðt þ 1Þ ¼ 2ag{xi ðtÞ} þ kr zi ðtÞ 2 ui j¼1 159 174 191 j¼1 158 171 172 175 where Aj is the ith external stimulation, f ð·Þ and gð·Þ are the output function and the refractory function of the neuron, respectively, a is the refractory scaling parameter, Qi is the threshold of the ith neuron, vij and wij are synaptic weights to the ith constituent neuron from the jth external input and from the jth constituent neuron, respectively. A neuron does not receive a feedback from itself, i.e. wii ¼ 0: ke ; kf and kr are the decay parameters for the external inputs, the feedback inputs, and the refractoriness, respectively. We can transform Eq. (1) into the following reduced and simultaneous forms: ji ðt þ 1Þ ¼ 170 173 ð1Þ d¼0 hi ðt þ 1Þ ¼ kf hi ðtÞ þ 157 d¼0 krd g{xi ðt 2 dÞ} 2 Qi xi ðt þ 1Þ ¼ f {hi ðt þ 1Þ þ zi ðt þ 1Þ} 155 156 168 t X 169 217 218 219 220 221 222 ð9Þ 223 224 ARTICLE IN PRESS G. He et al. / Neural Networks xx (2003) 1–6 3 225 281 226 282 227 228 283 284 229 285 230 Fig. 3. Sketch map of control methods. (a) Our control method, (b) Hu’s control method. Fig. 2. Four stored patterns. 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 where xpi is the ith component of the pth binary pattern. In this way, the binary patterns can be stored as basal memory patterns. We use a picture composed of 10 £ 10 matrix to show the stored patterns of the neural network constructed with 100 neurons. A neuron with its output xi equal to 1 is represented by a block ‘B’ while a neuron with its output xi equal to 0 is denoted by a dot ‘·’. Four stored patterns employed in this paper are shown in Fig. 2. When the decay parameters of the network are set to certain values, the network generates nonperiodic sequential patterns including the stored ones as its output sequence. The states of the network are usually described by the Hamming distance between the output pattern of the network and one of the four stored patterns, which is defined by Hp ¼ 100 X lxi 2 xpi l i¼1 for the pth stored pattern. When the network retrieves the pth stored pattern or its reverse one exactly, the Hamming distance will be 0 or 100. 255 257 3. Pinning control method focused on the chaotic neural network 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 287 288 254 256 286 Each neuron in the chaotic neural network defined by Eqs. (6) – (8) is coupled with others and evolve in nonlinear way as time. If a part of the neurons in the network are controlled, the controlling effect can be spread to the entire neural network as time elapses because of its space coupling and delay feedback. That is the main idea of the pinning control. The dynamics of our chaotic neural network is different from those of Hu’s coupled map lattice system (Hu & Qu, 1994). First, any neuron is coupled to all other neurons in the network while each site is only coupled to its nearest ones in the coupled map lattice system, and then the input of each neuron in our network is related to not only its own history but also the history of other neurons while only last outputs are feedback in Hu’s model. Our simulation proved that the pinning control method used in the coupled map lattice system was not suitable for our network. We propose a pinning control method focused on the chaotic neural network. The chaotic neural network with pinning control is described in the following way: xi ðt þ 1Þ ¼ f {hi ðt þ 1Þ þ zi ðt þ 1Þ} ð10Þ hi ðt þ 1Þ ¼ kf hi ðtÞ þ 100 X ( wij xj ðtÞ þ j¼1 100=I X ) dðj 2 Il 2 1ÞKuj ðtÞ l¼0 ð11Þ uj ðtÞ ¼ xj ðtÞ 2 x~ j ð12Þ zi ðt þ 1Þ ¼ kr zi ðtÞ 2 ag{xi ðtÞ} þ ai ð13Þ 289 290 291 292 293 294 295 296 297 298 where dð·Þ is a pulse function which is used to distinguish the pinned neurons from the unpinned ones. The jth neuron is pinned if j 2 Il–1 ¼ 0: uj is controlling signal which is constructed by comparing the last output and the expected output ð~xj Þ for each pinned neuron. I is pinning distance, a distance between two nearest pinned neurons, K is a control strength. The delay feedback is determined by the feedback input of the internal state and the refractoriness in the chaotic neural network. We therefore believe that the control aim can be reached more easily by changing the internal state than by adding control to its output. As shown in Fig. 3, the feedback control in our pinning method is imposed on the internal state of neurons while the feedback control in Hu’s method is put to the output of the system (Hu & Qu, 1994). The role of the feedback control on the network is carried out through the output function of the neurons, which is nonlinear in the chaotic neural network. Though we take a linear feedback control signal in our control method, its function is non-linear. In Hu’s pinning method, a non-linear feedback control was considered. 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 4. Computer simulation Our computer simulation is made according to Eqs. (10) – (13). We take a ¼ 10:0; kr ¼ 0:95; kf ¼ 0:20 and ai ¼ 2:0 (i ¼ 1,2,…,100) in our simulation. The network without control signal will be in chaos under the above parameters and the stored patterns in Fig. 2 are retrieved nonperiodically. The initial state of the network is given in Fig. 4. The fourth stored pattern (shown in Fig. 2d) is chosen as our expected output pattern, i.e. control target. The Hamming distance between the initial state and the expected output pattern is 8. Without the pinning control, its Hamming distances between the output of the network and the controlling target at different times are displayed in Fig. 5. In order to determine whether the network is in chaos, we NN 1731—29/3/2003—12:48—SHYLAJA—65591— MODEL 5 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 ARTICLE IN PRESS 4 G. He et al. / Neural Networks xx (2003) 1–6 as the control target. Fortunately, it was found that the outputs of the controlled network converge in the fourth stored pattern. Carrying out further simulations by taking different initial states and control targets, we found that the output of the controlled network always converge in the reverse of the target with the pinning control method given in Eqs. (10) –(13). If the state of every neuron of a target pattern, x~ i ; is replaced by 1 2 x~ i ; the target pattern will become its reverse. Obviously, the network will be stabilized in the target patterns if one replaces the controlling signals of Eq. 12 by the following equation: 337 338 339 340 341 342 343 344 345 346 347 348 349 Fig. 4. Initial state. 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 calculated the Lyapunov spectrum (Shimada & Nagashima, 1979), the largest Lyapunov exponent lm ¼ 0:000280: Obviously, the system is in chaos. The detail duscussion about chaotic dynamics of the neural network can be found in Adachi and Aihara (1997). When the pinning control is added to the network, its Hamming distances at different times are shown in Fig. 6. In the simulation, we take the pinning distance I ¼ 2 and the control strength K ¼ 20:0: The control signals are added to the system from 12,000 simulation step to 16,000 simulation step. In Fig. 6, one can find that when the pinning control is added to the system the network is stabilized to the reverse of the control target whose Hamming distance from the target is 100. The output of the stabilized network is given in Fig. 7. The chaos control of the network is realized, but the stable output of the controlled network is the reverse of the stored patterns. Our control aim is to let the network be stabilized in one of the stored patterns. We performed a simulation by taking the reverse of the fourth stored pattern uj ðtÞ ¼ xj ðtÞ 2 ð1 2 x~ j Þ ð14Þ In following simulation, we will replace Eq. (12) by Eq. (14) and the network will be stabilized in our expect patterns. From the simulation of I ¼ 2; we found that the chaos of the network is controlled only when K is taken to be larger than a threshold. To find out the correlation among the pinning distance, the control strength and the control effect, we make simulation under the different pinning distances and control strengths. For a fixed pinning distance, the control target can be reached only when the control strength is larger than a threshold or is called minimal control strength. If the control strength is smaller than the threshold, the system is still in chaos. The thresholds of the control strength for the different pinning distances are listed in case A of Table 1. The data are obtained by using the state given in Fig. 4 as the initial state and the fourth stored pattern as the target pattern. It is observed from Table 1 that threshold of the control strength is non-linear to the pinning distance. The threshold of the control strength is the smallest when the pinning distance is 3. By comparison with the expected output, the variations in the initial pattern exist in neurons 6, 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 371 372 427 428 373 429 374 430 375 431 376 432 377 433 378 434 379 380 435 436 381 437 382 438 383 439 384 440 385 441 386 442 387 388 443 444 389 445 390 446 391 392 447 Fig. 5. The Hamming distance between the orbit and the expected output pattern without the pinning control. NN 1731—29/3/2003—12:48—SHYLAJA—65591— MODEL 5 448 ARTICLE IN PRESS G. He et al. / Neural Networks xx (2003) 1–6 5 449 505 450 506 451 452 507 508 453 509 454 510 455 511 456 512 457 513 458 514 459 460 515 516 461 517 462 518 463 519 464 520 465 521 466 522 467 468 523 524 469 Fig. 6. The Hamming distance between the orbit and the expected output pattern under the pinning control added from 12,000 step to 16,000 step. 525 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 526 22, 23, 46, 52, 62, 72, 89. When the pinning distance I is 3, there are three variant neurons pinned, which are neurons 22, 46 and 52, according to j ¼ nI þ 1: For I ¼ 2; there are two variant neurons pinned while there is only one variant neuron pinned for I ¼ 4 and I ¼ 5; and no variant neuron pinned for I ¼ 6: For I ¼ 3; more variant neurons are pinned than any other case. Besides, we found that the threshold of the control strength is independent on when the pinning is added. Based on the above results, we guess that the threshold of the control strength may be related to the pinned sites. The chaos of the chaotic neural network can be controlled more easily if more variant neurons are pinned. To demonstrate our conjecture, we calculated the thresholds of the control strength of the controlled networks with the same pinned density and the different pinned sites. Beside the neurons pinned according to j ¼ nI þ 1; we added the pinning control to other three variant neurons for case B and other three un-variant neurons for case C. The way of pinning these neurons is 491 492 the same as described in Eqs. (11) and (14). Their thresholds of the control strength are also listed in Table 1. The initial state and target pattern in both case B and C are the same as in case A. The pinned density is same in both case B and case C, but the pinned sites are different in both cases. In case B, more variant neurons are pinned. As we expected, the threshold of the control strength of case B is smaller than that of case C at the same pinning density except I ¼ 3 in which the threshold values for both cases are the same. The results of Table 1 are obtained by taking the state shown in Fig. 4 as the initial state. In Table 2, we listed the thresholds of the control strength in different initial states. In our simulation, the nearest stored pattern of an initial state is chosen as the control target. From Tables 1 and 2, one can find that in general the threshold value of the control strength is dependent on both the pinning density and the pinning site. The threshold of the control strength of a controlled network is smaller at higher pinned density. At the same pinned density, the control aim of the chaotic neural network can be 493 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 494 550 Table 1 The correlation between pinning distance and control strength 495 496 Pinning distance I 497 498 Minimal control strength K min 499 500 501 502 503 504 527 Fig. 7. Output of the stabilized network. A B C 551 2 3 4 5 6 20 16 19 18 17 17 48 30 40 80 39 61 59 35 59 Note: case A: the control is described by Eqs. (11) and (14); case B: besides the neurons pinned as in case A, other three variant neurons are pinned. Case C: besides the neurons pinned as in case A, other three un-variant neurons are pinned. NN 1731—29/3/2003—12:48—SHYLAJA—65591— MODEL 5 552 553 554 555 556 557 558 559 560 ARTICLE IN PRESS 6 561 562 G. He et al. / Neural Networks xx (2003) 1–6 Table 2 The correlation between pinning distance and control strength under different initial states and pinning site 563 564 Pinning distance I 565 Minimal control strength K min Initial state 566 567 Initial state 568 569 570 Initial state 571 572 617 618 2 3 4 5 6 619 620 A B C 12 11 12 34 30 34 39 27 30 46 46 33 117 78 117 621 A B C 14 14 14 22 24 24 78 47 39 78 59 78 NC NC NC 624 A B C 14 14 14 34 34 34 30 27 30 77 58 77 NC NC NC 573 574 577 578 579 580 Acknowledgements 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 627 628 630 632 634 635 636 637 638 5. Conclusion and discussion References In this paper, a pinning control method focused on the chaos control of a chaotic neural network has been proposed. Our computer simulation proves that the chaos in the chaotic neural network can be controlled with our pinning control method. The chaotic neural network can be stabilized in a stored pattern if the control strength is larger than its threshold and the pinning density is not too small. Our study on the correlation among the threshold value of the control strength, the pinning density and the pinning site demonstrates that the threshold of the control strength is dependent on both the pinning density and the pinning site. In general, the threshold of the control strength of a controlled network is smaller at higher pinning density and the chaos control of the chaotic neural network can be reached more easily if more variant neurons are pinned. But some exceptions exist in Tables 1 and 2, which indicates a more complex correlation among the threshold of the control strength, the pinning density and the pinning site. This work is the first step of our study on the chaos control of the chaotic neural network. Since the stored pattern as the control target is chosen previously, the control method cannot be applied to information processing directly. Our next work will be focused on finding the method to search the control targets automatically and realizing effective information processing using the chaotic neural network. Adachi, M. (1995). Controlling a simple chaotic neural network using response to perturbation. Proceedings of NOLTA’95, 989 –992. Adachi, M., & Aihara, K. (1997). Associative dynamics in a chaotic neural network. Neural Networks, 10, 83 –98. Aihara, K., Takabe, T., & Toyoda, M. (1990). Chaotic neural networks. Physical Letters A, 144, 333 –340. Degn, H., Holden, A. V., & Olsen, L. F. (Eds.), (1987). Chaos in biological systems. New York: Plenum Press. Freeman, W. J. (1987). Simulation of chaotic EEG patterns with a dynamic model of the olfactory system. Biological Cybernetics, 56, 139–150. Hu, G., & Qu, Z. (1994). Controlling spatiotemporal chaos in coupled map lattice systems. Physical Review Letters, 72, 68–77. Hunt, E. R. (1991). Stabling high-period orbits in a chaotic system: the diode resonator. Physical Review Letters, 67, 1953– 1955. Kobori, E. R., Ikoda, K., & Nakayama, K. (1996). Model of dynamic associative memory. IEEE International Conference on Neural Networks Conference Proceedings, 2, 804–809. Mizutani, S., Sato, T., Uchiyama, T., & Sonehara, N. (1995). Controlling chaos in neural networks. Proceedings of ICNN, Perth, 6, 3038– 3043. Ott, E., Grebogi, C., & Yorke, J. A. (1990). Controlling chaos. Physical Review Letters, 64, 1196–1199. Pecora, L. M., & Carroll, T. L. (1990). Synchronization in chaotic systems. Physical Review Letters, 64, 821 –824. Pyragas, K. (1992). Continuous control of chaos by self-controlling feedback. Physical Letters A, 170, 421–428. Shimada, I., & Nagashima, T. (1979). A numerical approach to ergodic problem of dissipative dynamical systems. Progress of Theoretical Physics, 61, 1605–1616. Tokuda, I., Nagashima, T., & Aihara, K. (1997). Global bifurcation structure of chaotic neural networks and its application to traveling salesman problem. Neural Networks, 10, 1673–1690. Tsuda, I. (1991). Chaotic itinerancy as a dynamical basis of hermeneutics in brain and mind. World Futures, 32, 167–184. 584 586 626 633 The authors appreciated the helpful discussion with Dr. J. Kuroiwa. This work is supported by the National Natural Science Foundation of China (Grant No. 50077019) 582 585 625 631 reached more easily if more variant neurons are pinned. Besides, one can observe that the chaos of the network cannot be controlled if the pinning density is too small. 581 583 623 629 Note: case A, case B and case C are as the same as described in Table 1. NC means that the network cannot be controlled. 575 576 622 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 613 669 614 670 615 671 616 672 NN 1731—29/3/2003—12:48—SHYLAJA—65591— MODEL 5
© Copyright 2024 Paperzz