JVM Garbage Collector Tuning
Automatic Memory Tiers
All Apinizer modules (Worker, Cache, Manager, Integration, Portal) apply automatic JVM memory settings through entrypoint.sh at container startup:
- The container memory limit is detected via cgroup (v1/v2); if there is no limit, the server's total memory is used.
- The memory is classified into tiers (in 1 GB steps for Worker, Manager, Integration, and Portal; 1 / 2 / 4 / 8 GB and above for Cache).
- Based on the selected tier, the heap percentage (
MaxRAMPercentage), direct memory, metaspace, thread stack size, and GC parameters are set.
The selected tier is written to the pod logs. Example (Worker, 4 GB):
Detected container memory: 4096MB
JVM tier: 4GB (heap 65%, direct 512m, metaspace 256m, stack 512k)
jvmOverride not set: tier defaults will override JAVA_OPTS
The JVM_MEMORY_PROFILE (low/medium/high) environment variable and the "automatic ZGC above 1536 MB" behavior from earlier versions have been removed. Setting this variable no longer has any effect; you can delete it from your environment variables.
Worker (Gateway)
Worker uses G1GC, the Java 25 default GC, in every tier.
| Container Memory | Heap | Direct Memory | Metaspace | GC Parameters |
|---|---|---|---|---|
| ≤ 1 GB | 50% | 128m | 128m | G1HeapRegionSize=1m |
| ≤ 2 GB | 60% | 256m | 192m | G1HeapRegionSize=1m |
| ≤ 3 GB | 60% | 384m | 256m | G1HeapRegionSize=2m |
| ≤ 4 GB | 65% | 512m | 256m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 5 GB | 65% | 768m | 320m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 6 GB | 65% | 1024m | 384m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 7 GB | 65% | 1280m | 384m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 8 GB | 65% | 1536m | 384m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| > 8 GB | 65% | ~20% of memory (max 4096m) | 512m | AlwaysPreTouch, MaxGCPauseMillis=100 |
Cache (Hazelcast)
The Cache module uses ZGC at 2 GB and above. Because Hazelcast also uses memory outside the heap, its heap ratios are lower than the Worker's.
| Container Memory | GC | Heap | Direct Memory | Metaspace |
|---|---|---|---|---|
| ≤ 1 GB | G1GC | 50% | 64m | 128m |
| ≤ 2 GB | ZGC | 50% | 128m | 128m |
| ≤ 4 GB | ZGC + AlwaysPreTouch | 55% | 192m | 192m |
| ≤ 8 GB | ZGC + AlwaysPreTouch | 55% | 256m | 256m |
| > 8 GB | ZGC + AlwaysPreTouch | 55% | 384m | 384m |
Manager
Manager uses G1GC in every tier. Metaspace is fixed at 512m in all tiers, because the number of loaded classes does not depend on the pod size.
| Container Memory | Heap | Direct Memory | GC Parameters |
|---|---|---|---|
| ≤ 1 GB | 50% | 128m | G1HeapRegionSize=1m |
| ≤ 2 GB | 55% | 192m | G1HeapRegionSize=1m |
| ≤ 3 GB | 55% | 256m | G1HeapRegionSize=2m |
| ≤ 4 GB | 60% | 384m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 5 GB | 60% | 512m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 6 GB | 60% | 640m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 7 GB | 60% | 768m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| ≤ 8 GB | 60% | 1024m | AlwaysPreTouch, MaxGCPauseMillis=100 |
| > 8 GB | 60% | ~15% of memory | AlwaysPreTouch, MaxGCPauseMillis=100 |
Allocate at least 2 GB for the Manager. In the 1 GB tier, the following warning is written to the pod logs and memory may run out after full warm-up:
WARNING: 1GB is below the Manager minimum; heap + metaspace + JVM overhead exceed the limit under full warm-up. Use >= 2GB.
Integration and Portal
Integration and Portal use the same heap and direct memory tiers and G1GC as the Manager; only metaspace varies by tier (≤ 1 GB: 128m, ≤ 2 GB: 192m, ≤ 4 GB: 256m, ≤ 5 GB: 320m, ≤ 8 GB: 384m, > 8 GB: 512m).
JAVA_OPTS and jvmOverride
The parameters you provide in JAVA_OPTS are passed to the JVM together with the tier values. When the same parameter is given more than once, the last one wins; the jvmOverride environment variable decides which set is written last:
jvmOverride | Behavior |
|---|---|
| Not set (default) | Tier values are written last. Parameters in JAVA_OPTS that the tier also sets (-XX:MaxRAMPercentage, -XX:MaxDirectMemorySize, -XX:MaxMetaspaceSize, -Xss; -XX:MaxGCPauseMillis at 4 GB and above) do not take effect. Parameters the tier does not set (e.g. -Xmx, -Xms) are applied. |
true | JAVA_OPTS is written last; your values take precedence over the tier values. |
If you set a value such as -XX:MaxRAMPercentage=75.0 in JAVA_OPTS without defining jvmOverride=true, that value is not applied; the tier value wins. The JVM tier: ... line in the pod log shows which values are used. Add jvmOverride=true to the environment variables if you want your own values to take effect.
GC Selection
GC selection always belongs to the user and is independent of jvmOverride:
- If no GC is specified in
JAVA_OPTS, the module default is used (G1GC for Worker, Manager, Integration, and Portal; ZGC at ≥ 2 GB for Cache). - If a GC other than G1 (
-XX:+UseZGC,-XX:+UseSerialGC,-XX:+UseParallelGC,-XX:+UseShenandoahGC) is specified inJAVA_OPTS, G1-specific parameters (G1HeapRegionSize) are automatically removed from the tier values. - In the Cache module, if any GC is specified in
JAVA_OPTS, the tier's GC parameters (UseZGC,G1HeapRegionSize,AlwaysPreTouch) are removed.
In the Cache module, where the tier sets the GC explicitly, this cleanup prevents two different GC flags from being passed at once, which would stop the JVM from starting. In the other modules the tier does not set a GC; the cleanup only prevents G1-specific parameters from producing warnings.
Garbage Collector Types
Because Apinizer's base image is Eclipse Temurin 25, the following GC algorithms are supported.
G1GC (Garbage-First)
The Java 25 default GC. It is region-based and runs in parallel and concurrently; it divides the heap into equal regions to make garbage collection time predictable.
- In Apinizer: all tiers of Worker, Manager, Integration, and Portal; the 1 GB tier of Cache
- Advantage: Balanced throughput and low GC pauses
- JVM parameter: No need to specify, since it is the default (
-XX:+UseG1GC)
ZGC (Z Garbage Collector)
A low-latency, scalable GC. GC pauses are typically under 1 ms regardless of heap size.
- In Apinizer: default in the Cache module's 2 GB and higher tiers
- Advantage: Consistent, very low GC pauses; good performance with large heaps
- Disadvantage: Consumes more native memory due to colored pointers and multi-mapping
- JVM parameter:
-XX:+UseZGC
Serial GC
The simplest, single-threaded GC algorithm. It is not selected automatically by any Apinizer tier; it can be selected manually with JAVA_OPTS.
- JVM parameter:
-XX:+UseSerialGC
ZGC Safety Limit
ZGC uses more native memory (off-heap memory) than other GCs. Therefore, in the Worker, Manager, Integration, and Portal modules, if you select ZGC in JAVA_OPTS and set MaxRAMPercentage above 70%, the value is automatically lowered to 70% and the following warning is written to the pod log:
WARNING: MaxRAMPercentage=80% is too high for ZGC. Capping to 70%.
This limit is not applied in the Cache module; Cache's ZGC tiers already use 50–55% heap.
GC Logs
The Worker writes GC events to stdout by default (-Xlog:gc, about one line per collection). These logs are visible with kubectl logs and through your normal log collection pipeline, and let you assess GC pressure after the fact during OOM or slowness investigations.
| Environment Variable | Default | Description |
|---|---|---|
APINIZER_GC_LOG | (not set = enabled) | Set to false to disable Worker GC logs |
An automatic heap dump on OOM is intentionally not enabled: a 4–16 GB dump can exceed the container's ephemeral storage limit and get the pod evicted. To take a heap dump, use the diagnostics endpoints described on the Memory Leaks and OOM Errors page.
Configuration Examples
Automatic Tier (Recommended)
Do not add heap/GC parameters to JAVA_OPTS; only set the container memory limit:
resources:
requests:
memory: "4Gi"
limits:
memory: "4Gi"
# Worker result: G1GC, 65% heap, direct 512m, metaspace 256m
Using Your Own Heap Value
env:
- name: JAVA_OPTS
value: "-XX:MaxRAMPercentage=70.0"
- name: jvmOverride
value: "true"
Without jvmOverride=true, the MaxRAMPercentage value is overridden by the tier value.
Selecting a Different GC
env:
- name: JAVA_OPTS
value: "-XX:+UseZGC"
GC selection does not require jvmOverride. The heap percentage comes from the tier value; do not exceed the 70% limit for ZGC.
MaxRAMPercentage Safe Limits Table
If you set your own heap value with jvmOverride=true, it is recommended not to exceed the following upper limits:
| GC Type | Safe Max Percentage | Reason |
|---|---|---|
| Serial GC | 80% | Minimal native memory overhead |
| G1GC | 75% | Moderate native memory usage (region metadata, remembered sets) |
| ZGC | 70% | High native memory consumption due to colored pointers and multi-mapping |
These limits are general recommendations. Workloads with heavy thread usage, large native buffers, or many file descriptors may require lower values.
Troubleshooting
OOM Kill
Symptom: The pod restarts continuously, and OOMKilled appears in the kubectl describe pod output.
| Cause | Solution |
|---|---|
High MaxRAMPercentage with jvmOverride=true | Remove jvmOverride to return to tier values, or lower the percentage below the table above |
High MaxRAMPercentage with ZGC | Lower it to 70% or below |
| Insufficient container memory | Increase resources.limits.memory (at least 2 GB for the Manager) |
| Native memory leak | See Memory Leaks and OOM Errors |
High GC Pause
Symptom: Response times increase periodically, P99 latency is high.
- Check collection frequency and duration in the Worker GC logs (
kubectl logs). - In 4 GB and higher tiers,
MaxGCPauseMillis=100is applied automatically; for smaller pods, increasing memory to 4 GB enables the pause target.
My JAVA_OPTS Value Is Not Applied
Symptom: The MaxRAMPercentage or MaxDirectMemorySize you set in JAVA_OPTS seems to have no effect.
Cause: When jvmOverride is not set, the tier values have the last word. The pod log shows the line jvmOverride not set: tier defaults will override JAVA_OPTS (in Cache: jvmOverride=false (default): tier defaults will override JAVA_OPTS).
Solution: Add jvmOverride=true to the environment variables.