javascript - playing 40.000 unit army for game -

solved!! (look @ last edit)

i want make army fight of 20.000 vs 20.000 units on canvas. so, every unit data is:

{  'id' => 17854, 'x' => 1488,  'y' => 1269,  'team' => 'red',  'health' => 10, 'target' => [1486, 1271] } 

and see fight in real time (25 frames per second). if generate 1 frame json , save on file, 2.5mb size (40k such units data).

1 second(25 frames) = 62.5 mb file size. , fight last 30 minutes, should use 112gb. bad. if make file binary data, should take 27 times less place, or 4gb 30 mins. that's still bad. 2 hrs movie takes 700mb.

need save many fights on server. real players should make armies , fight each other, need fights saved. every fight unique, because every damage of every unit random 0~10 , after unit kills enemy, regenerates 1 health. i'm doing calculations php , saving files on server. 40.000 units on 1 screen, can visible @ once, want that. now, units single 2x2 pixel cubes, red , blue teams, easy load javascript.

but, generate file php need 1 hour. that's problem. because every frame need iterate these 40.000 units (for updating x/y, searching nearby enemies or friends, doing damage , returning target or killed enemy coordinates), iterate on again unset killed units, , before putting file, need iterate on , remove unused data used calculations. , finish 30 mins fight need repeat 45000 times. also, every minute there less , less units. point somehow make file generating in less minute, need logical way, if exist.

questions:
1) best way save files on server , make files less on size? (so far use binary data , compress zip)
2) fastest way calculate fights? (so far compile c++)

// edited. here whole game code, :)

this main action:

class simulator {     private $units;     private $places = [];     private $oldplaces = [];      public function initiategame() {         $this->createunits();         $this->startmoving();     }      private function createunits() {         foreach(range(0, 150) $column) { // exact army formation nice, 150x140=21000 units             foreach (range(0, 140) $row) {                 $this->setunits($column, $row);             }         }         $this->oldplaces = $this->places;     }      private function setunits($column, $row) {         $beginning_of_team_a = 6; //starting point on canvas unit nice on screen         $unit_size_and_free_place = 6; //unit size= 3x3 (look js), , free place between each unit 3 pixels.         $beginning_of_team_b = 1100; // next side enemy army starts appearing         $x_a = $beginning_of_team_a + $column * $unit_size_and_free_place; // team         $y = $beginning_of_team_a + $row * $unit_size_and_free_place; // same both teams         $unita = new unit($x_a, $y, 1); // 1 team (it goes +1 pixel every frame)         $this->units[] = $unita;         $x_b = $beginning_of_team_b + $column * $unit_size_and_free_place;  // team b         $unitb = new unit($x_b, $y, -1); // -1 team b (it goes -1 pixel every frame)         $this->units[] = $unitb;         $this->places[$x_a.','.$y] = 1; // way tracking units, , calculating next move         $this->places[$x_b.','.$y] = -2;     }      private function startmoving() {         set_time_limit(30000); // default after 1 minute throws exception         foreach(range(0, 400) $frame) { //giving 400 frames 400/40=10 seconds of action             $this->places = [];             foreach($this->units $unit) {                 $returned = $unit->move($this->oldplaces); //giving whole units list every unit forward                 $this->places[$returned[0]] = $returned[1]; // returns (next x/y position string ['1514,148'] ) = ( id int [15] )             }             file_put_contents('assets/games/'.$frame.'.json', json_encode($this->units)); // writing file  every frame , uses ~2mb             $this->oldplaces = $this->places; //resetting old positions         }     }  } 

this unit:

class unit {     public $x = 0;     public $y = 0;     public $team = 1;     public $stopped;      public function __construct($x, $y, $team, $stopped = false) {         $this->x = $x;         $this->y = $y;         $this->team = $team;         $this->stopped = $stopped;     }      public function move($places) {         $this->checkforward($places);         return [$this->x.','.$this->y, $this->team];     }      private function checkforward($places) {         $forward = $this->x + $this->team; // todo: find out formula replace 4 ifs         $forward1 = $this->x + $this->team*2;         $forward2 = $this->x + $this->team*3;         $forward3 = $this->x + $this->team*4;         if(isset($places[$forward.','.$this->y])) {             $this->stopped = true;         } else if (isset($places[$forward1.','.$this->y])) {             $this->stopped = true;         } else if (isset($places[$forward2.','.$this->y])) {             $this->stopped = true;         } else if (isset($places[$forward3.','.$this->y])) {             $this->stopped = true;         } else {             $this->stopped = false;         }          if($this->stopped == false) { // move forward not stopped             $this->x = $this->x + $this->team;         }     } } 

this js:

var app = angular.module('app', []);  app.controller('game', function($scope, $http, $interval) {     var canvas  = document.getelementbyid("game"),         context = canvas.getcontext("2d");     var frame = -2;     $scope.attacking = false;     var units = [];      function start_animation_loop() {         $scope.promise = $interval(function() {             if($scope.attacking == true) {                 frame ++;                 if(frame >= 0) {                     downloadfile();                     animate();                 }             }         }, 40 );     }      function downloadfile() {         $http.get('assets/games/'+frame+'.json').success(function(response) {             units = response;         });     }      function animate() {         clear_canvas();         draw();     }      function clear_canvas() {         context.clearrect(0, 0, 1800, 912);     }      function draw() {         for(var a=0; a<units.length; a++) {             context.beginpath();             context.fillrect(units[a]['x'], units[a]['y'], 3, 3);             if(units[a]['team'] == 1) {                 context.fillstyle = 'red';             } else {                 context.fillstyle = 'blue';             }         }     }     start_animation_loop();  }); 

solved! thx colleague in work! gave me brilliant idea!

to result needed need every next battle generate random number (0~10000) , place single mysql database. in addition put there formations, units, starting strength, health , else.
, calculations javascript:
1 constant number (given backend) make formula reproduce same army fight -> every unit move forwards , stop @ same time anyways no matter calculation. uniqueness random damage every unit gives , process after. , damage "x/y position somehow compared constant number" , whatever single damage, random every unit because in different map positions, damage 0~10. same constant number, units same damage after calculation , move same @ every replay, die , same damage @ every replay. hardest work on javascript - make calculations using constant number.
random number can any. if first fight generate random number "17", , next battle generate random number "19666516546", not mean battle number "17" less damage - "random" damage 0~15 every unit, replays same formations, unit numbers, starting position , random generated number same -> no more need save files! , can add various spec effects, add defences, evasions, , fit in 2 mysql rows - every team :) cool!!

id can implicit in storage medium. sure, means have save gaps, can compress said gaps.

'x' => 1488,  'y' => 1269,  

depending on canvas size, can compressed. if canvas 1e6 x 1e6 (a million million), there 1e12 locations, fits in ~40 bits.

'team' => 'red',  

with 2 sides, 1 bit.

'health' => 10, 

the vast majority of units have low health. can units health < 15 stored in 4 bits. if bits set, have units health elsewhere (with id->health table).

'target' => [1486, 1271] 

we store independent target each unit, doesn't match how ui works. select pile of units, , tell them go somewhere, no? ~40 bits location, ~24 bits reference count, 8 bytes per target.

if give each side limit of ~65k targets, 16 bits.

16+4+1+40 = 61 bits. means have 3 more bits play pack them 64 bit per unit.

at 64 bits per unit, 160k per side. plus half-a-meg of target data, can handled dynamically.

plus health overflow table (that maps id health), should close empty. if sometimes, can set before-after id maps maintain consistent history (say, when half units dead, compression pass before-after id mapping).

if id need not consistent, can compress units down no longer sparse.

the target trick -- target unit id if less 40,000, , if above value waypoint. reduces waypoints ~15k ones. (i assuming target set ui, select chunk of units, order them go somewhere).

you'd want iterate on packed data, skipping "dead" units (bitmask on health), unpacking them usable structure, evaluating action, , writing them back. double-buffering option (where read 1 buffer, , write another), has modest cost. because units fixed size, can lookup other units fast (and unpack them), helps if targets other unit ids, , things doing damage.

single-buffering makes things easier, because things simultaneous damage tricky. mean lower-id units act first -- can fix flipping coin each turn determine if iterate forward or backwards (and stick 1 side in low-ids, other in high-ids), or make initiative check @ start of combat.