CoolJWB
Veteran Member
Cool guys don't look back at explosions.
Posts: 734
| Likes: 330
|
Post by CoolJWB on Jun 20, 2020 2:14:31 GMT
I've poked around in the configuration section of the Aikar's timings and I noticed that there are a few options we can change that would increase stability and performance. To start, Aikar's GC flags could be used to get server stability and reliability (decrease large garbage collection spikes) with the cost of slight CPU usage. I have tried this myself on a memory limited test server where I could see a clear performance increase. Another way to increase server performance would be to customize the bukkit, spigot, and paper yml files to the needs of the server. You can read more about what every modification does here and here. The first is information directly from Paper docs and the second one is a user-submitted guide for all configuration files (I looked into these but had in mind the TotalFreedom needs). I have already spent a few hours on this and here is the result (feel free to make contradictions to each option but please explain why): Bukkit.yml Modified lines: # Limits spawning of entities. spawn-limits: monsters: 50 animals: 8 water-animals: 3 ambient: 1
# How often the server tries to spawn mobs. ticks-per: animal-spawns: 400 monster-spawns: 4 water-spawns: 4 ambient-spawns: 4
Spigot.yml Modified lines: # Simply groups items and exp. merge-radius: item: 4.0 exp: 6.0
# Lowers the range that entities can "see". entity-activation-range: animals: 16 monsters: 32 raiders: 48 misc: 16 water: 16
tick-inactive-villagers: false # Does not keep villagers active when not in range of players.
Paper.yml Modified lines: page-max: 100 # As a page can contain 256 characters there should be plenty of space for whoever wants to write large books. use-faster-eigencraft-redstone: true # If I remember correctly we used PandaWire but as it did not work correctly it was removed, this is heavily tested as it has been used by plenty of other servers. non-player-arrow-despawn-rate: 20 # Makes sure to clear away arrows after they have landed. creative-arrow-despawn-rate: 20 prevent-tnt-from-moving-in-water: true # Disables tnt movement in water (as tnt is disabled we can disable tnt movement in water to minimize resources). grass-spread-tick-rate: 4 # Slower grass growth to spread out resource needs over time. mob-spawner-tick-rate: 2 # Makes spawners do fewer ticks and thus fewer spawns. max-auto-save-chunks-per-tick: 12 # Spread out chunk saving overtime to minimize lag spikes. prevent-moving-into-unloaded-chunks: true # If the server lags players shouldn't be able to request more chunks to load as it will only make the lag worse. optimize-explosions: true # Better explosion calculations to minimize the lag when huge amounts of TNT is lighted. max-entity-collisions: 4 # Fewer entity collisions means less code to run for entities (especially great when huge amounts of entities are spawned). remove-corrupt-tile-entities: true # Papers built-in corrupt tile entity removal system, should be useful if something goes wrong. disable-chest-cat-detection: true # Minor modification that disables useless checks for cats sitting on chests.
# Unloads entities when no players are close to them. despawn-ranges: soft: 24 hard: 64
|
|
|
Post by Telesphoreo on Jun 20, 2020 3:09:00 GMT
already done on smp can confirm this helps
|
|
|
Post by Polaris Seltzeris on Jun 20, 2020 3:24:22 GMT
Also, a big issue is just the horrible scalability of Minecraft especially when you have a server like this where anyone can use build tools to make the server lag out. We don't throw nearly enough server power as we really need and unfortunately that's just how Minecraft's architecture is built. I think Wild has mentioned this before in that we should strive for some sort of redundantly efficient server setup which can handle everything we need, because you know how things are otherwise.
|
|
|
Post by Telesphoreo on Jun 20, 2020 4:01:35 GMT
i personally think that at this point, it's stupid to prevent people from crashing the server. the goal should be to get the server back online as quickly as possible when the server crashes. not even because of stability, just because of an op finding some obscure NMS exploit that can't be patched. not sure how that can be done to be perfectly honest
|
|
|
Post by Polaris Seltzeris on Jun 20, 2020 4:10:19 GMT
i personally think that at this point, it's stupid to prevent people from crashing the server. the goal should be to get the server back online as quickly as possible when the server crashes. not even because of stability, just because of an op finding some obscure NMS exploit that can't be patched. not sure how that can be done to be perfectly honest Well also I think it's way too easy for this server to crash, we need to consider options such as using a cluster of daemons to prevent any unscheduled downtime. Throwing server power at this and using multiple servers is basically the only way other than random software optimizations to actually solve the problem. If we went with the clustered idea, we could have certain things only handled on certain daemons, for instance anything related to worlds would go on a particular server. If we did that, not only would we be way fucking more resilient, but we'd also be able to have a system to keep the server as a whole running normally even if one daemon were to hang. But it should be stupidly easy to "get the server back online as quickly as possible," that should be entirely automated anyways.
|
|
|
Post by DragonSlayer2189 on Jun 20, 2020 5:18:06 GMT
disable-explosion-knockback: true this is my only problem with it, i love the tnt knockback because then you can use things like explosive arrows and a crossbow to make basicly a rocket jump thing, and it is so fucking cool and i would hate for this to be disabled, other than that, it looks pretty good
|
|
|
Post by Telesphoreo on Jun 20, 2020 5:53:37 GMT
i personally think that at this point, it's stupid to prevent people from crashing the server. the goal should be to get the server back online as quickly as possible when the server crashes. not even because of stability, just because of an op finding some obscure NMS exploit that can't be patched. not sure how that can be done to be perfectly honest Well also I think it's way too easy for this server to crash, we need to consider options such as using a cluster of daemons to prevent any unscheduled downtime. Throwing server power at this and using multiple servers is basically the only way other than random software optimizations to actually solve the problem. If we went with the clustered idea, we could have certain things only handled on certain daemons, for instance anything related to worlds would go on a particular server. If we did that, not only would we be way fucking more resilient, but we'd also be able to have a system to keep the server as a whole running normally even if one daemon were to hang. But it should be stupidly easy to "get the server back online as quickly as possible," that should be entirely automated anyways. Depends on the exploit. The thing is I'm not too caught up on what the exploits nowadays are. Crash chunks would hopefully be easy to solve, as it's 100% possible to load/unload worlds and generate new ones on the fly. The problem with this is that if there were crash chunks, it would be easy to teleport everyone to a fresh world and wipe the old one, but then you're nuking the builds. I know there are some exploits where the server just straight up hangs. Not sure how to combat this. The one idea that I have is when the exploits generate the stacktrace in console, I'm wondering if it's possible to read these logs and fix itself on the fly. If a person keeps reconnecting and gets disconnected with the error, it should purge their player data and send them to spawn. Not sure how possible this is to do. I know reading the console is possible, as there's a plugin called QuietCord for bungee servers to run that basically suppresses any messages that spam the console when the server pings itself
|
|
StevenNL2000
Forum Admin
Posts: 6,415
| Likes: 6,936
IGN: StevenNL2000
Timezone: UTC+01:00
Member is Staff. Need immediate assistance? Send a PM
|
Post by StevenNL2000 on Jun 20, 2020 7:27:20 GMT
The one idea that I have is when the exploits generate the stacktrace in console, I'm wondering if it's possible to read these logs and fix itself on the fly. A stupid idea I once had was for the server to check what was causing the error and simply delete it from the world. However, it would have to be a jar modification because you can't get the objects involved in an exception from the Exception object.
|
|
|
Post by Telesphoreo on Jun 20, 2020 7:34:09 GMT
The one idea that I have is when the exploits generate the stacktrace in console, I'm wondering if it's possible to read these logs and fix itself on the fly. A stupid idea I once had was for the server to check what was causing the error and simply delete it from the world. However, it would have to be a jar modification because you can't get the objects involved in an exception from the Exception object. Couldn't it get some other information? Like are the coordinates there? WorldEdit has a regenerate command. Maybe it would be possible to automate regenerating crash chunks based on the coordinates. You can get coordinates from where the player was last online before they crashed and use that to regenerate the chunk. Not sure how easy or practical this is. Really wish Minecraft was more scalable. Take discord for example. If one of their servers goes down during a voice call, you're automatically transferred to a different one. Problem is that when those servers crash, it's because something internally went wrong. It would be so easy to just join and take down TF like dominoes. crash server 1, get connected to server 2, troll crashes that, you get the idea. i think that a lot of companies with redundant servers aren't expecting all of them to crash within a few minutes of each other, and when that occasionally happens (discord) the whole thing has an outage
|
|
StevenNL2000
Forum Admin
Posts: 6,415
| Likes: 6,936
IGN: StevenNL2000
Timezone: UTC+01:00
Member is Staff. Need immediate assistance? Send a PM
|
Post by StevenNL2000 on Jun 20, 2020 8:14:01 GMT
A stupid idea I once had was for the server to check what was causing the error and simply delete it from the world. However, it would have to be a jar modification because you can't get the objects involved in an exception from the Exception object. Couldn't it get some other information? Like are the coordinates there? WorldEdit has a regenerate command. Maybe it would be possible to automate regenerating crash chunks based on the coordinates. You can get coordinates from where the player was last online before they crashed and use that to regenerate the chunk. Not sure how easy or practical this is. I was talking about deleting specific blocks and entities: it's easier with whole chunks, but you'll end up obliterating builds. It might be possible to have ProtocolLib monitor packets and derive what object in the world caused the issue from that. I've previously suggested using ProtocolLib as a global fix for all exploits that use NBT data overflow because you can simply cancel the oversized packets.
|
|
|
Post by Polaris Seltzeris on Jun 20, 2020 8:23:19 GMT
Couldn't it get some other information? Like are the coordinates there? WorldEdit has a regenerate command. Maybe it would be possible to automate regenerating crash chunks based on the coordinates. You can get coordinates from where the player was last online before they crashed and use that to regenerate the chunk. Not sure how easy or practical this is. I was talking about deleting specific blocks and entities: it's easier with whole chunks, but you'll end up obliterating builds. It might be possible to have ProtocolLib monitor packets and derive what object in the world caused the issue from that. I've previously suggested using ProtocolLib as a global fix for all exploits that use NBT data overflow because you can simply cancel the oversized packets. More of a hackfix than an actual fix. It's delegating the solution to the problem to a middleman instead of the root of the problem, which is that the server accepts oversized packets and distributes them to clients, and so the solution would be to modify the server software. Depending on how much a crash chunk affects the server, I'm wondering if it's possible from the server itself to monitor the activity of chunks in terms of what operations are being performed and be able to cancel the cause of problems before they occur.
|
|
StevenNL2000
Forum Admin
Posts: 6,415
| Likes: 6,936
IGN: StevenNL2000
Timezone: UTC+01:00
Member is Staff. Need immediate assistance? Send a PM
|
Post by StevenNL2000 on Jun 20, 2020 10:03:19 GMT
I was talking about deleting specific blocks and entities: it's easier with whole chunks, but you'll end up obliterating builds. It might be possible to have ProtocolLib monitor packets and derive what object in the world caused the issue from that. I've previously suggested using ProtocolLib as a global fix for all exploits that use NBT data overflow because you can simply cancel the oversized packets. More of a hackfix than an actual fix. It's delegating the solution to the problem to a middleman instead of the root of the problem, which is that the server accepts oversized packets and distributes them to clients, and so the solution would be to modify the server software. Depending on how much a crash chunk affects the server, I'm wondering if it's possible from the server itself to monitor the activity of chunks in terms of what operations are being performed and be able to cancel the cause of problems before they occur. Modifying the server is itself a hack. I strongly believe maintaining what is essentially our own fork of the server implementation creates way more technical debt than using an established API.
|
|
Wild1145
Club 4000 Member
Inactive Player & Inactive Senior Admin
Posts: 10,414
| Likes: 9,680
|
Post by Wild1145 on Jun 20, 2020 12:00:48 GMT
One thing we did with the re-launch of CJFreedom which we later abandoned because fuck advertising was having it as a 4 node bungee freedom server. That way we ran 2 servers with WorldEdit and 2 without, players could join servers with their friends, that way if a server crashed they'd be returned to the hub or if they were kicked they could instantly re-join another server with their friends.
The big issue with Minecraft is it's historically been shit at multi-threading, so a lot of the plugins and such are also not designed to utilise multiple processors, and as a result once a process starts to hang everything else goes to shit. It's hard to do HA with servers like this just because of their very nature, and to be honest without throwing a lot of money at running a custom server entirely it would be fairly difficult to do.
|
|
|
Post by Polaris Seltzeris on Jun 20, 2020 19:24:54 GMT
More of a hackfix than an actual fix. It's delegating the solution to the problem to a middleman instead of the root of the problem, which is that the server accepts oversized packets and distributes them to clients, and so the solution would be to modify the server software. Depending on how much a crash chunk affects the server, I'm wondering if it's possible from the server itself to monitor the activity of chunks in terms of what operations are being performed and be able to cancel the cause of problems before they occur. Modifying the server is itself a hack. I strongly believe maintaining what is essentially our own fork of the server implementation creates way more technical debt than using an established API. If this was years ago and Bukkit was still the upstream server software, I'd agree with you, but I think the circumstances are a bit different today because of how god awful the server actually is. It's an established API but one which is pretty much a zombie of its former self and the maintainers which cared about not hacking things up are long gone. The fundamental issue is that the server itself is not scalable and you shouldn't need to use a plugin to stop crash chunks and packet overflows, it's an internal server problem that hasn't been addressed by actual maintainers and using a plugin to go around those issues is still more of a hacky workaround than addressing the problem.
|
|
Wild1145
Club 4000 Member
Inactive Player & Inactive Senior Admin
Posts: 10,414
| Likes: 9,680
|
Post by Wild1145 on Jun 20, 2020 20:21:33 GMT
Modifying the server is itself a hack. I strongly believe maintaining what is essentially our own fork of the server implementation creates way more technical debt than using an established API. If this was years ago and Bukkit was still the upstream server software, I'd agree with you, but I think the circumstances are a bit different today because of how god awful the server actually is. It's an established API but one which is pretty much a zombie of its former self and the maintainers which cared about not hacking things up are long gone. The fundamental issue is that the server itself is not scalable and you shouldn't need to use a plugin to stop crash chunks and packet overflows, it's an internal server problem that hasn't been addressed by actual maintainers and using a plugin to go around those issues is still more of a hacky workaround than addressing the problem. To be honest it'd be interesting looking at re-writing stuff to work with something like Sponge which has an entirely different API and is based back off of the actual Minecraft server rather than forks of forks. Not sure it'd be practical but would certainly be interesting.
|
|