The World is BIG.

Shards: An In-Depth Look

As many are probably aware, world shards have been announced, and the launch date is near. Testing has been ongoing on the PTR for over a week now as players have been honing in on problems and other potential issues related to the launch of world shards. With average tick times at over five seconds, the faster shard will provide a welcome reprieve, and major players have already announced plans to abandon the current world in favor of the new shard.

“No Shards”

Of course, many players remember a time when not having world shards was a major selling point of Screeps. Only recently did references to “no shards” get removed from the website. While not using the term “shard”, the original Indiegogo campaign and a later campaign to open-source the engine and launch the game on steam both make references to a “single world” which, while not explicitly saying that the world did not have sharding, might imply it to some extent, though the new shards will still be connected. Before judging too harshly for failing to stick to the claims here, it’s important to consider the steps that have led to this point. Over past months, with Screeps experiencing tick times over five seconds per tick, players have been complaining about performance and the Screeps Team was forced to look into solutions to reduce tick times.

To fully understand the scope of the problem, one must understand a bit about what’s happening behind the scenes. Screeps can be described as a traditional star architecture- a MongoDB server connects to twelve Node.js servers, each of which handle a part of the load from player scripts. These scripts run within Node VMs, isolated from each other. Resets occur when the server a player’s code is on clears its cache, forcing data to be re-cached. Players are regularly shifted between servers as load is balanced between them. Unfortunately, the database server still has a maximum input/output speed, and this ends up bottle-necking the entire process. I spoke to Dissi, the Screeps Community Manager, and he said that on a monthly basis, Screeps had just under 2300 different players updating their code. A large number of those are active memberships- players taking up considerably more than the 10 cpu awarded with the purchase of Screeps on Steam. The world, once relatively small, now measures two hundred rooms across and contains the better part of forty thousand rooms.

The World is BIG. Really Big. You just won’t believe how vastly, hugely, mind-bogglingly big it is.

 

What this all means is that Screeps has reached the point where simply adding more servers to handle player code won’t reduce tick times. Since then, a number of solutions have been attempted.

  • In May, Screeps updated to Node 7, but was forced to roll back due to performance degradation in the branch.
  • Also in May, Screeps own Marcel Laverdet managed to solve the “hard reset” problem that was causing CPU spikes for players an increasing server load
  • In June, Screeps updated to Node 8.1.3, but was forced to roll back due to stability issues.
  • Also in June, Marcel Laverdet wrote an isolated-vm module to fully isolate player scripts from each other, improving performance
  • Also in June, Screeps experimentally tested using Apache Cassandra instead of MongoDB, but no benefit was achieved
  • Sharding object types, such as creeps, structures, etc. as opposed to rooms and worlds was attempted, but no benefit was achieved

All of this goes to show that considerable efforts have been made to reduce tick times, but success has been wanting. This isn’t a failure of execution on the part of the Screeps Team, but a measure of the difficulty and complexity of the problem of running thousands of players code simultaneously within a truly massive game world, and doing so efficiently and quickly. Therefore, as of the third of August, Screeps new shard, shard1, will be going live. The new shard will have different terrain compared to the current world, which is being identified as shard0.

A New Shard – Mechanics

The new shard, while running on a separate server cluster, will still be connected to the previous world by portals. Sixty days after the launch of shard1, walls around portals in highway crossroad rooms will fall, allowing current players access to the new shard, but allowing new players to establish themselves. Players will be able to split their CPU between shards if they so wish, allowing cross-shard colonies, though Artem has stated that this will need to be configured manually by the player as code-based control would undermine the isolation that would allow shards in the first place. Markets will be specific to shards as well, but credits won’t, meaning players with good inter-shard and market code code may be able to make a profit moving goods between shards. CPU and GCL will be split between shards, with players being able to set a distribution of CPU and GCL per shard once per day.

Concerns

Due to concerns about players being able to use other shards as a sort of ‘rapid transit’- using faster tick rates in one shard to move more quickly across the map than players in another, any creep using an inter-shard portal loses a significant number of ticks to live- Claimers lose 200 ticks, all other creeps lose 800. There have been some concerns that these balance changes may be preemptive- CLAIM creeps are already range-limited as it is; inter-shard CLAIM creeps will be limited to a mere 300 ticks.

Players have also raised concerns about the lack of an in-game API to control GCL and CPU between shards. This means players favoring the pure-code approach, such as the TooAngel community, may find themselves locked into a specific shard and unable to use the new feature. Others have raised concerns that, with only a minority of players using shard portals, players without the ability to shift CPU to a different shard and without a set minimum CPU-per-shard, would be unable to counterattack and defend their rooms against inter-shard attackers. Beyond that, some fear that containing players too much within shards could fracture the community.

Overall, shards are a major change to Screeps, but one many players may not be seeing a great deal of aside from faster ticks. With possible changes and updates to shards still possibilities after launch, it’s impossible to say what the end result will look like.

For ScreepsWorld, this is Knightshade.

EDIT 2017-8-2: A correction- Artem has confirmed that a code-based API will not be available on launch, but will be developed and released during the two month period following the release of shards.

Share this post

Share on facebook
Share on google
Share on twitter
Share on linkedin
Share on pinterest
Share on print
Share on email