Trying to understand how NPC spawning works?

[Beagle]

Hello, I’m working on an MP mod that puts a high focus on PvE and an important part of that is controlling how AI ships spawn. While so far I think I have a pretty good grasp on how encounters and npcships and all the rest work, the parameters that are driving me mad are the ones in the system file; working in the Colorado system in my case, li03.ini.

Basically, my expectations going in about the fields density, repop_time, max_battle_size, relief_time and density_restriction all don’t seem to work the way I thought.

Specifically, I thought either repop_time or relief_time would be the amount of time in between spawn waves in seconds, but in practice that just doesn’t seem to be the case. Something as simple as having a zone have a large number of possible spawns but having each one be built up to very slowly over a long period of time seems all but impossible. Instead of being able to restrict the spawns to a slower pacing, the game just seems to shotgun them out quickly to fill up to the max density size as fast as possible.
The only reliable way to keep numbers down in the short-term seems to be to hard-cap the number of potential max spawns in the area to one or two, because if there’s the possibility to spawn more, the game seems to jump at the chance, ignoring repop_time and relief_time as time-based limiters on how often new enemies should spawn.

I’d really appreciate any help or insight from people who’ve tackled this problem before or who understand how to dig under the hood and see what’s going on, because right now these parameters all seem totally vague to me!

What I want is basically:

• The designated area has the potential to spawn up to ten NPCs before it stops spawning NPCs at all
• The game only tries to spawn a set of NPCs at certain timed intervals, so lets say, every sixty seconds it performs a check for a chance to spawn another permutation of enemies from the encounter file.
• The game never tries to spawn NPCs faster than that timed interval, ensuring that players aren’t overwhelmed with too many spawns in a short amount of time, but still allowing the potential for many NPCs to show up eventually if the battle takes a long time.

What is currently occurring:

• The designated area has the potential to spawn up to ten NPCs before it stops spawning NPCs at all
• The game seems to spawn these NPCs whenever it feels like, seeming to mostly ignore what I’ve tried to set as time-restrictions in repop_time and relief_time, and just overwhelming the player with spawns too fast for comfort
• The parameters seem to have some but a vague and misunderstood effect to me, and I don’t really know how to fix what’s going on under the hood

Any knowledge or experience on this would be greatly appreciated! Or even just some way of knowing exactly what the parameters are meant to be doing…

density
repop_time
max_battle_size
relief_time
density_restriction
encounter = <encounter file=“”>, <level number=“”>, <chance to=“” spawn?=“”>All of these parameters seemed intuitive when I first started working with them, but quickly it seems like I don’t know how they really work.

Thanks for any help!</chance></level></encounter>

[IrateRedKite]

To add to this with my findings as well:

• A lot of the information in the ‘Trying to understand encounters’ pdf that’s floating around doesn’t match what we’ve seen on the ground with regards to zone and encounter values.

• I’ve found that [c]density_restriction[/c] is inconsistent unless you override shipclasses in the encounter file using make_class and restrict that value.

• [c]allow_simultaneous_creation = no[/c] in the encounter file results in wildly different behavior (spawns ships up to the allowed cap, then won’t spawn any more more until the last one is destroyed.)

There’s also a ton of behaviors I’ve love to unpick around [c]pop_type[/c] in the zone and [c]arrival[/c] in the encounter file. I suspect that [c]pop_type[/c] may simply be a tag and has no impact on NPC behavior but have not been able to confirm this.

[Cpt_Rei_Fukai]

I haven’t played with it that much, but I found some old seemingly relevant posts:
https://the-starport.net/freelancer/forum/viewtopic.php?post_id=235#forumpost235
https://the-starport.net/freelancer/forum/viewtopic.php?post_id=16881#forumpost16881
https://the-starport.net/freelancer/forum/viewtopic.php?post_id=23760#forumpost23760

I think a lot of the confusion arises from two factors:

• If zones that spawn encounters overlap

• Freelancer spawns encounters, not individual ships! E.g The zone is one ship below max battle size, so Fl spawns an encounter. If this encounter includes five ships, it will spawn five ships and not only one, going over the maximum

[IrateRedKite]

Cpt_Rei_Fukai wrote:
I haven’t played with it that much, but I found some old seemingly relevant posts:
https://the-starport.net/freelancer/forum/viewtopic.php?post_id=235#forumpost235
https://the-starport.net/freelancer/forum/viewtopic.php?post_id=16881#forumpost16881
https://the-starport.net/freelancer/forum/viewtopic.php?post_id=23760#forumpost23760

I think a lot of the confusion arises from two factors:

• If zones that spawn encounters overlap

• Freelancer spawns encounters, not individual ships! E.g The zone is one ship below max battle size, so Fl spawns an encounter. If this encounter includes five ships, it will spawn five ships and not only one, going over the maximum

Thanks, this does sort of highlight the issue with the encounter system though. It’s never really been unpicked! One of these posts suggests that [c]relief_time[/c] is the time before ships start to spawn once the field is cleared, or how quickly ships will despawn after spawning, but testing with a single blank zone and a variety of values dispels this notion pretty quickly! It definitely has some relation to spawn time, but it really isn’t clear what precisely that is. There’s a ton of information out there on this that’s not correct, but is generally accepted as not that many folks have sat down and tested these rigorously.

[c]density[/c] broadly makes sense, but what is the relation to [c]max_battle_size[/c]? Most posts say this is ‘self explanatory’ but it’s not! Surely [c]density[/c] would be the limiting factor here?

[adoxa]

Ruppetthemuppet wrote:
There’s also a ton of behaviors I’ve love to unpick around [c]pop_type[/c] in the zone and [c]arrival[/c] in the encounter file.

[c]pop_type[/c] is simple: it’s not used.
I haven’t looked at how [c]arrival[/c] is used, but I can tell you what it accepts:

all - short for all of them (plus one without a specific name);
object_all - short for all the object_* items;
tradelane;
object_docking_ring;
object_jump_gate;
object_station;
object_capital;
cruise;
buzz.

Prefix with [c]-[/c] to disable (thus [c]all, -tradelane[/c] will enable all, then disable tradelane). Looking at the disassembly it doesn’t seem to be used, but I’d have to run through the debugger to be sure.

[IrateRedKite]

adoxa wrote:
Prefix with [c]-[/c] to disable (thus [c]all, -tradelane[/c] will enable all, then disable tradelane). Looking at the disassembly it doesn’t seem to be used, but I’d have to run through the debugger to be sure.

Do you mean that the [c]-[/c] value isn’t used? I’ve observed some weird behavior with patrol paths where crashes can occur if [c]cruise[/c] isn’t used for arrival. It would be neat to do some testing here (and I intend to!)

It’s great to know that [c]pop_type[/c] is just flat-out unused, I can rule it out as a factor in my testing. I’ll update the wiki on that one!

[adoxa]

Do you mean that the [c]-[/c] value isn’t used?

No, that’s used; I meant [c]arrival[/c] itself, but if you’ve experienced crashes with it not being set, I guess I wasn’t looking for the right thing.

[IrateRedKite]

Wiki updated, planning to do a bit of testing this morning.

Digging through IDA, I’ve come across a few keys that look like they belong in encounters, but I’ve never seen before. A little testing seems to suggest that they are not used (Although they might be read?). Has anyone come across these before?

longevity
feeling_to_group
feeling_to_formation
explicit_group

Interestingly, arrival isn’t always a mandatory key! I ran a few zone encounters that didn’t include it, and NPCs spawned/behaved normally. I assume there’s some default that’s chosen when the value isn’t defined (Probably all?)

[adoxa]

[c]longevity[/c] is read, but haven’t checked if it’s used.

There’s also [c]feeling_to_local[/c], all of which are read, but seemingly ignored - it checks for [c]group_reroll_frequency[/c] instead; [c]feeling_to_formation[/c] is processed (but again, unsure if it’s actually used) and definitely causes the others to be ignored.

[IrateRedKite]

Beagle wrote:
density
repop_time
max_battle_size
relief_time
density_restriction
encounter = <encounter file=“”>, <level number=“”>,<chance to=“” spawn?=“”></chance></level></encounter>

So based on discussions in the #modding channel of the community discord (Many thanks to Skotty and Treewyrm):

[c]repop_time[/c] is basically when pop count is incremented, adding available population to spawn
[c]relief_time[/c] is a guaranteed (of sorts) time for next spawn
[c]max_battle_size[/c] is the ship count for battles. It’s unclear whether or not this battle has to involve the player or not.
[c]density_restriction[/c] isn’t used correctly in vanilla zones and only keys off [c]make_class[/c] values set in the encounter. It also limits by ships, not encounters (unlike density)

UPDATE: I’ve done some fairly extensive testing of these with a stopwatch and a single encounter, and it appears that repop_time and relief_time are not even remotely consistent. Is there a random value added to these before they are used?

ANOTHER UPDATE: More stopwatch testing and some discussion: Is it possible repop_time and relief_time are minimums where the further past the listed time you get, the more likely a spawn is to occur? This fits observed behavior, but seems difficult to verify without more rummaging around in the assembly.

Skotty has also clued me in on [c]ship_by_npc_arch[/c], which rather than grabbing a ship from a class group, lets you swipe a nickname from npcships.ini! The ship in question must have the corresponding d-class for the zone however, otherwise the game will crash!

I’ve also stumbled across the additional behaviors: goto_guide, dock_guide and patrol_guide. presumably these are used when NPCs are docking/moving between waypoints/etc?

[Beagle]

adoxa wrote:
[c]longevity[/c] is read, but haven’t checked if it’s used.

There’s also [c]feeling_to_local[/c], all of which are read, but seemingly ignored - it checks for [c]group_reroll_frequency[/c] instead; [c]feeling_to_formation[/c] is processed (but again, unsure if it’s actually used) and definitely causes the others to be ignored.

Intersting, do you have any idea what group_reroll_frequency does? The name sounds as though it could be useful…

For the discussion at large, just to throw another wrench in the works of these guesses and tests, repop_time seems to be relative to density. That is, if you set repop_time to 1000, and your density is 10, encounters spawn slowly. If you still have repop on 1000, but then you set your density to 1000 as well, encounters spawn much much faster.

I haven’t been able to confirm anything but one guess of mine is that repop_time means something like “Time until the area has been repopulated up to the desired density”… however I haven’t been able to figure stuff out for sure which is why I came here hoping to figure it out.

Thanks for everyone who’s helping work this stuff out, I really just want to be able to spawn enemies at reliable intervals so my players don’t get overwhelmed too fast!! I’m hoping to get down to a real understanding of these parameters at some point…

[adoxa]

Beagle wrote:
Intersting, do you have any idea what group_reroll_frequency does?

As it stands, nothing. It tests for (feeling_to_local or feeling_to_group or feeling_to_formation or explicit_group) and then tests for (group_reroll_frequency or feeling_to_formation), so it can never be matched. Once feeling_to_formation is used, it ignores them all.

It looks like it takes overlapping paths/tradelanes into account, summing all the relief times then dividing by density (of the highest). My test was with Zone_Li01_001_Planet_Li01_01, which for some reason is grouped with Zone_Li01_path_police4_1; the former has the higher density, so that is what is used. It then looks at:

Zone_Li01_path_junkers3_1
Zone_Li01_path_bounty1_1
Zone_Li01_path_junkers3_2
Zone_Li01_path_bounty2_1
Zone_Li01_path_police8_1
Zone_Li01_path_police8_7
Zone_Li01_path_police7_5
Zone_Li01_Tradelane_22
Zone_Li01_path_police2_1
Zone_Li01_path_police3_1

which altogether has a combined relief time of 805, which is then divided by the density. That’s as far as I went…

[Beagle]

Huh, now that’s very interesting - nice testing! I am indeed overlapping multiple area and patrol path encounters, and with the highest density set at 10, it makes sense that the respawn times are getting divided down to be really fast and unexpected to me.

You say it sums all the relief times together, do you mean both repop time and relief time or specifically just relief time? I’m going to have to do some testing of this on my own now and try and go further with understanding it. Thank you very much for putting me on the right track with this.

[adoxa]

Oh, it was the [c]repop_time[/c], sorry. And it’s probably not specific to paths/tradelanes, presumably they were the ones that overlapped; and the reason for the other was simply because I expect I was in both.

[Beagle]

I’ve been testing this a bit this morning, removing all overlapping fields to try and just nail down some consistency on timings out of one field with a single encounter and a single permutation within that encounter whose weight is set to 1.

Initially, and at times, things seem to very much reflect what I’d expect with the new knowledge! For instance, with these settings,

density = 10
repop_time = 300
relief_time = 1200

I was expecting an initial spawn to take about 30 seconds, and then reinforcement spawns during that fight to only come in at 120 second intervals. This is exactly what happened at first! I was super excited that the issue might have been cracked completely.

However after further testing there still seems to be things I’m not grasping. After leaving the fight, nothing else spawned for 5+ minutes after, and then after quick loading my SP save I was testing on I couldn’t get anything else to spawn at all for a similar amount of time before giving up.

Further, in a subsequent test I just lowered repop_time to = 1, expecting an instant initial spawn but then for the 1200 relief time to make those reinforcement intervals still be at 120 seconds. This was not the case, and more enemies spawned constantly, seeming to only be running off the repop time here.

I then tried the opposite - repop time left to 300, but relief time turned down to 1. Results got even less like what I expected here. Initial spawn was quick again, at about 20 seconds, but it then took around three minutes and 40 seconds (if I remember right) to spawn a reinforcement enemy. Trying to see if there would be consistency, I killed the initial enemy then stayed in the fight against the reinforcement for another three minutes and 40 seconds - another enemy spawned around this time, which makes the timing seem consistent!

However at that point, another enemy then spawned with seconds afterwards, and then another right after, and another, for a total of four. They seemed to stop spawning rapid fire at this point, but I exploded about 20 to 30 seconds after so I’m not sure what would’ve happened next.
My guess is that relief_time was not in effect until that moment they started rapid-fire spawning, but why? What made that moment when relief_time was called in to spawn things?

So, while initial results seemed to play by the expected rules, things ended up feeling more random again in the end. I’m not sure if it really is some random element to the game, or if it’s just there’s still more math not being understood here - perhaps things are getting divided or multiplied by other things too? If all these values interact and multiply each other in unknown ways, it would make sense why the end results seem so inscrutable and random to me.

The questions I’m left with after this to test further are:

1: Why is it so hard to get consistent timings out of these parameters? Is there a random element designated somewhere else that we don’t know about, or is it just an intended part of these parameters that they’re averages or min and max boundaries rather than exact, static times?

2: What exactly decides when repop_time is in play vs. when relief_time is? Both seem to have their own effects on when an enemy is spawned, but it’s not as straightforward as I expect. Are the seemingly random results actually because the triggers for relief time and repop time to take effect are different to what I think, and are occurring at certain times and contexts in gameplay that I don’t know about?

3: What role does toughness and max_battle_size play in these questions, if any?

4: Are these parameters tracking their values persistently in between loading saves? Is my testing method of quick loading and flying back into the same area part of why these values seem to sometimes misbehave? Does the game need to be fully shut down and restarted to have a good next testing environment?

I’ll keep on testing these things and trying to nail down a controlled and consistent result, but if you Adoxa or anyone else can help figure out the answer to these as well and the knowledge needed to gain that consistency, I’d really appreciate it. I already feel like I’ve got a bit more control now understanding the division element at play with repop time, so that’s been a huge boon already, thanks so much for that.

[adoxa]

Just so we’re testing the same thing, would you be able to send me your zone/encounter?

[Beagle]

Yeah sure thing, posting my encounter (bmod_area_fc_x_grp_scouts_1) and the zone (Cheyenne Asteroid Field in Colorado) below

Note we made our own custom fighter classes we’re using for ship_by_class so you’ll want to set those back to whatever vanilla fighter class instead

;Permutations of 1 enemies per spawn

;Perm 0, 1 MFs
[EncounterFormation]
ship_by_class = 1, 1, bmod_sc_heavy_fighters
pilot_job = scout_job
make_class = wanderer
formation_by_class = fighters
behavior = wander
arrival = all, -tradelane, -object_jump_gate
allow_simultaneous_creation = yes
zone_creation_distance = 0
times_to_create = infinite

[Creation]
;1-2 Enemy Cap

;Perm 0, 1-2 MFs
permutation = 0, 1

[zone]
nickname = Zone_Li03_Cheyenne_field
ids_name = 261217
pos = 61910, 0, -39659
rotate = 0, 30, 0
shape = ELLIPSOID
size = 17000, 10000, 30000
property_flags = 64
visit = 32
spacedust = asteroiddust
spacedust_maxparticles = 50
ids_info = 65930
sort = 25
toughness = 10
density = 10
repop_time = 300
max_battle_size = 10
pop_type = lootable_field
relief_time = 1200
encounter = bmod_area_fc_x_grp_scouts_1, 3, 1
faction = fc_x_grp, 1
Music = zone_field_asteroid_rock

Let me know if you’d have rather I just send you the files by the way instead of the relevant snippets

[adoxa]

Snippets are fine, thanks. I’ll give that a go later.

As an aside, I see that as post #17 at 17:17.

[adoxa]

First off, [c]toughness[/c] is not used in zones, so one more thing to forget about ([c]sort[/c], too).

Secondly, [c]longevity[/c] is read & tested, but doesn’t seem to have any effect.

After the first encounter is created, every five seconds or so (presumably related to [c]SpacePop, LOWEST[/c] in Freelancer.ini, didn’t track it down any further than that) there’s a [c](density - ships) / repop_time[/c] chance of creating another ship/encounter (didn’t test which). Of course, if [c]ships >= density[/c] nothing is created; [c]ships[/c] is the current number of ships in the zone (or perhaps vicinity). I only tested that in a non-overlapping zone, presumably overlapping zones have the sum I found earlier come into play.

[Beagle]

Huh, so you reckon repop_time isn’t related to time at all, but is only a variable in the chance to spawn? That would explain why stuff seems so random - it’s literally always been a chance every 5 seconds, not a cooldown.

Thanks Adoxa, this is terrific to learn. I’ll try following your formula you’ve uncovered and see if things start doing what I’d expect.

Looking at what I have set up now, I can see that in a certain field in my mod, I have Density 16, Repop Time 420 - so without any enemies present, that would be 16 divided by 420 for 0.038 or a 3.8% chance for anything to spawn, every 5 seconds. Thinking about it, this makes a ton of sense why sometimes I get dry runs that seem to last forever, and other times, I just seem to luck out and get constant spawns.

Excellent investigating, thank you so much for saving my sanity here. Is there any way to adjust how often that chance pops? Make it roll every minute or two instead? EDIT: I see now that I can experiment with the Spacepop value you noted, I might try setting it to BELOW_NORMAL, NORMAL, HIGHEST so on and see what happens if anything.

Being able to adjust when the chance is rolled in any way would help smooth out this randomness a lot - having the chance be rolled every minute or thirty seconds, but a lot higher chance to spawn, for instance, would be a lot more reliable to balance with. Same as being able to adjust that per zone/encounter/whatever for different spawn rates, but that’s icing on the cake.

EDIT: Also, lol at the #17 at 17:17 - I’m same timezone as you so I saw it the same 8-)