Crash Offsets

Is there any possibility to get a kind of callstack of a current happended crash? It would be nice if its logged somewhere in a file… I have tried to debug a crash offset (by using breakpoints) but the routine is way too much called to trigger any crash reason ;(

So actually when the crash happens and having the callstack would be a really great…

Finally ntdll.dll crashes - Not supported parameter or typing error

@Huor try WinAPIOverride32

00091f38 in common.dll

because ship_archetype = <blank>in players file</blank>

is there something wrong with the engbase.dll. I dont see anything in depends and another dll viewer is also empty

i just could figure out the crash offset in windebug - but there is one little question i have. The offset that is given is that the offset from the module load entry or from the start of freelancer.exe? For sure both result in another address ;D
It seems there are no further information about engbase.dll at least all i could retrieve is assambler code - and my understanding of it is heavily lacking ;(

Whats wrong with engbase.dll? - I do not remember, remind please

constantly getting crashes, almost on a 10-20 minute schedule in dalib.dll

dalib.dll v. 1.0.1254.11 at 00004353

What happens usually is the server is running fine and all of a sudden all players experience red lag. As I watch console view I see usually 3 hits in a row, Sat Jul 31 17:18:35 2010
.\HookFunctions.cpp(887) : *** ERROR: Exception in Hook_IServerImpl_TradeResponse (unhandled exception)
and then all internet players get booted. Usually FLServer also crashes generating the dalib.dll error in the event viewer.

Here’s what I’ve done so far.
Replaced EVERYTHING between the monitor that I look at and the utility pole outside.

ISP ran new cable to the house, new modem, new account, I replaced the router and the server machine, reinstalled Windows XP several times (latest install WinXPsp3 with no updates), reinstalled all software, (FL, FLAC) and the problem persists. This is while running the mod or not running the mod.

Also, (I’ll start another thread on this) I’m wondering if it might be server list related. A couple times I’ve noticed that FLServer has lost connection to the list server and reacquired it. On at least one or more occasions, the crash happened at that time.

One more thing…
If I use Crazy’s List Server mod my server will show up on the list and on the server activity graph at flserver.de.
If I use w0dk4’s gls server workaround patch (v1.0), no connection to the list server. I can get on being on the LAN, but nobody outside the LAN can get on.

I’ll have to contact w0dk4 for more info on that.

R

@robocop: That’s due to a memory allocation failure (in CDPMsgList::Add, if that helps anybody). It’s only asking for 12 bytes, so there must be some serious memory leakage going on.

@Huor: Engbase is accessed by virtual functions (retrieved by DACOM_Acquire), which is why there’s nothing exported.

@adoxa
I wondered about memory leakage…
The ONLY thing this machine is running is WinXPsp3 (no updates past sp3), FLServer, and FLAC.

All unnecessary services have been disabled.
This started about a year ago out of the blue. No known cause. I thought it might be hardware related and ran the server on a different machine. Same problems. I replaced the server with a completely different setup again, same results.

It’s independent of the mod and FLAC hasn’t been changed or updated since well before the problems began.

So, is it possible that somebody is doing something malicious via the FL port(s) that might be causing this?

A server reboot usually forestalls the problem for a couple hours before it begins again so you may be on the right track with memory leaks.

What can I do to get rid of this problem? I don’t have much more hair to pull out.

R

@robocop:

I have just realised that I might I have the same problem on a test server. Every 20 minutes or so the server freezes and is restarted. Sometimes a OS reboot seems to fix the problem for a random period of time - sometimes days.

When the server freezes, the flhook console is locked up but the FLServer uptime counter continues to count up.

The server is running in a VM and I thought it might have been related to the VM software. The OS is XP 32bit SP3, a clean install.

When did you install SP3 - a year ago?

The mod definitely works okay on my XP machine. I’m sure it is not the mod. It could be flhook stuff but I am/was running production plugins.

I threw windbg at it and I think it was freezing when it was trying to execute level0? ring code but I really have no idea what I’m talking about.

If anybody wants to debug it for me, remote access is available…

Also, (I’ll start another thread on this) I’m wondering if it might be server list related. A couple times I’ve noticed that FLServer has lost connection to the list server and reacquired it. On at least one or more occasions, the crash happened at that time.

I wondered the same thing but I removed all list server stuff and it didn’t seem to help.

Well…, welcome to the club. I was starting to think everyone was looking at me sideways on this issue…

This has been going on for over a year now. The current install is only a week old. WinXPsp3 with no other updates.

Perhaps I’ll try running WinXPsp2 to see if the problem goes away but I’m pretty sure the problem existed well after upgrading to service pack 3.

Watch your event viewer. Application section should show any FLServer crashes. In there you should see which module is affecting flserver.

I’m not sure if this will help but picking up on adoxa’s comment XP SP3 introduced a memory leak problem as described here: http://support.microsoft.com/kb/959658.

Perhaps this is what is happening?

-Alpha

When the server freezes, the flhook console is locked up but the FLServer uptime counter continues to count up.

The server is running in a VM and I thought it might have been related to the VM software.

1. It was in VirtualBox (not Oracle version yet)

2. On vmware ESXi i never had this problem

3. Simple Vmware Server 2.0 - i have not checked it, ESXi is better

HeIIoween wrote:
Whats wrong with engbase.dll? - I do not remember, remind please

Um yeah there is a crash of FLServer in this module. At least thats what is logged. Therefor i was searching for any entry that could give me some information on that offset - but i didnt found any in a dll viewer. What Adoxa explained pretty well explains it

Nevertheless i got around this problem as i have attached windbg to the running exe and so could find the entry point of engbase.dll. So far so good. But now i dont know if the offset from the crash log is the offset from the module entry point or the offset of the exe base start…

Finally if the engbase functions are virtually accessed it maybe that the Dacom routines are causing the crash?! Once i have found out the address for the breakpoint I could try to figure throughout the callstack what might cause the crash - thatswhy i was asking about that offset stuff ;D

If still anyone might have an idea I really would appreciate it and am grateful in advance.

Try perdr, and look in readme.txt of it

The crash report is for the module, so it’ll be relative to engbase.dll. If you look at the details, it should give you the actual address. Depending on how the crash occurs, WinDbg should actually break there (at least, OllyDbg usually does).

yes it would if it would crash while attached

Well, the hotfix didn’t work and neither did a reinstall with WinXPsp2 only.
Still crashing in dalib.dll at 00004353

I’m willing to kill for a solution…

Also, during the freeze/crashes, procmon shows buffer overflows in csrss.exe and lsass.exe as well as a bunch of dxdiag stuff flooding the monitor.

That help at all?

I might have a fix for this problem: “FLServer locks up every 10-20 minutes, mostly commonly after about 18 minutes. It tends to lock up more often if flhook event socket stuff is happening. I have both DSPM and DAM connected to flhook doing unicode event socket traffic. DirectPlay traffic still works but no game events are passed.”

The lock up is caused by the thread CRemotePhysicsSimulation+0x2660 ceasing to do stuff. I think this is the main FLServer loop. FLhook event processing is called from it and so if this thread stops, the flhook socket stuff will stop too.

Reviewing one of the stack traces I suspect that problem might be related to a QueryPerformanceTimer call. On futher investigation it might be possible for this to return an invalid time potentially causing FLServer to wait a very long time before doing processing. I haven’t actually proven that this is happening but it seems possible particularly on multi-core processors and/or running virtual machines.

Adding /usepmtimer to your boot.ini as described in this article might workaround this problem. See See http://support.microsoft.com/kb/895980 for instructions.

I’ve applied this change and so far, I’m on 1 hour 30 minutes of no crashes. I’ll edit this post if it really does fix it or not.

Still crashing in dalib.dll at 00004353

According to the code, this crash apparently happens when the server starts hosting. It is in the function call CDPServer::GetHostAddresses()

I might recall having this crash on GC ages ago. I believe it was related to a corrupt character file, causing a stack corruption which some how ended up in this function. I’m sorry, I’m not more certain that this.

I’m not sure your first suggestion about the boot.ini applies to us as we don’t use FLHook.

The second interests me but, I know each player character has been ‘cleaned’ by FL PlayerCleaner on more than one occasion, and FLAC is not identifying any attempts to log in with corrupt player chars.

So, short of doing a player wipe, how do you find the corrupt player char(s) if that’s indeed the problem?