Today we’re excited to release Flare, a diagnostics and profiling plugin for Hytale. While working with our partners and clients to bring new and exciting Hytale experiences, we noticed high RAM usage and performance bottlenecks. We wanted to delve into these performance issues but noticed there’s no good tools available for Hytale yet. That’s where Flare comes in.
We developed Flare as a way to poke and prod at the server. We wanted something akin to Spark for Minecraft, so we made our own. Flare is an open-source plugin that allows you to view information about the server’s TPS, threads, garbage collection, and more. It's MIT license and available on GitHub today, and will be available on CurseForge shortly after approval.
Flare’s Features
Flare allows you to monitor many different aspects of your Hytale server to diagnose performance issues, all available through simple to use commands in-game. Some key features of Flare can be seen below.
Memory & Garbage Collection
With Flare, you’re able to track the heap (what the RAM gauge on Nodepanel is) and detailed statistics from Java’s garbage collector. Garbage collection (GC) acts as a cleanup crew for your server's RAM, specifically the "heap." As your Java server runs, it creates temporary objects in the heap. When these objects are no longer needed, they become "garbage." The GC regularly sweeps the heap, identifies this unused memory, and disposes of it to free up RAM. A good analogy for this is a game like Hytale unloading chunks of the world that are no longer inhabited or visible to any players; these unused chunks are essentially "garbage" that the server unloads to free up system resources. This ensures the server has space for new objects and prevents an eventual out-of-memory crash.
TPS
Much like Minecraft, Hytale’s server performance can be tracked by a TPS metric. Hytale’s default TPS is 30, which means 30 times a second the game is processing new information for the world. Flare allows you to view and monitor your server’s TPS throughout gameplay.
World Metrics
Flare’s tracking monitors various stats about your world, including entities loaded, chunk counts, tick times and state. Flare's robust tracking system provides comprehensive, real-time statistics regarding the operational status of your game world. It monitors essential metrics for optimization and troubleshooting, including the total count of loaded entities, detailed chunk counts for managing world size and memory allocation, and precise server tick times.
Live Profiling System
The live profiling system within Flare is powered by the same underlying technology as Spark, async-profiler. Async-profiler allows us to capture and profile a ton of different statistics about the Java server in an extremely high-frequency (we’re capturing data once every 4ms, which is 8 times per tick). With this we’re able to track exactly what is happening in the server at any point in time. Currently we’re only tracking CPU sampling data from async-profiler but we’re looking to gather more data from this amazing tool in the future.
Alongside the high-frequency data collection from async-profiler we do snapshots of the server’s state. This tracks all the information you can glean from the commands but once every second, allowing you to correlate symptoms.
Future Plans
Much like Spark, we’re planning on creating a web viewer for Flare reports. This will make it much easier for server owners to diagnose issues on their own. Instead of having to dig through technical jargon and lines of JSON, you’ll be able to look at pretty graphs and gauges.
We’d also love to look into utilizing Hytale’s UI system to build in-game dashboards for admins to use with Flare.
If you have any feature requests, or run into any issues while using the plugin, please feel free to open an issue on the GitHub project and we'll be happy to help.
