CMP to SUR Conversion Tests

Xarian_Prime

Rubs hands together in excitement *

Have i told you lately… that we love you 8-)

… sry… I’ve been watching this closely, sleep well LS & dream of a world without empty space collision detection.

w0dk4

Keep it up LS

adoxa

** Section header
4		ULONG	Surface Type 0-, 1-, 2-, 3-Equip/Ammo/Damage, 4-Ship/Station
```This is actually a count of the tags in each section.

** Surface tag
1 byte Flag?

** Face list header
4 ULONG Cmpnd/HP CRC, also something else, see last entry in manufacturing_platform_lod.sur for an example (0x00002540)
1 byte DirectX vertex type

** Bits section (as in 24 bit bitmap)
4 ULONG Color (RR GG BB xx) - xx is always 00


I've rewritten [SurFileDump](http://adoxa.110mb.com/freelancer/surdump.zip) as a console application, with all this additional information included. Hope it proves helpful, 'cos that's probably as far as I can go…

LancerSolurus

**I have taken a break from messing with it tonight but will look into what you have found listed above. District 9 was a very good movie!

Also, I have my own routines for decompiling surs and will play with comparison routines for the different parts. The face offset stuff appears to be offsets to opposite but matching edges in the faces, as in an index of 128,129 points to the first instance of 129,128 excluding the current index, with a limit of -128 to 128 range.

The bits color number is a bit intriguing, I will see what I can come up with a relevance to a normalized scaler for the xyz radius since you mentioned its used by the radius value. I used to use such a thing back when integer arithmetic was alot faster than floating point arithmetic functions. (I’m showing my age now)

The tags sections count I’m a bit leary of, a single tag does not give a type 1 sur, it is usually a type 4, I will double check though since you pulled it out of the source code (assuming here) unless it is counted as header+face+vertex+bits sections…**

adoxa

By tags, I mean the surf, exts, !fxd and hpid codes. The “surface type” is the count of those. That’s how PhyArch::LoadSurfaces interprets it. The bits radius stuff is in a routine at common.0634a9b0 - [esi+0x14] is the radius and +0x18/19/1a the “rgb”.

StarTrader

Heheheh….

I feel like a small boy sitting in my toy digger watching you guys in those bloody great 50-ton-per-scoop earth movers!

Your work is very much appreciated guys.

LancerSolurus

**Ahh, now that makes sense. I have also seen 2 exts in a row instead of !fxd then exts…

My assembly is a bit rusty, looks like I need to study up on it.**

LancerSolurus

**Making progress, screenshot from HardCMP…

Getting a good collision on the root mesh but not on the hardpoint meshes yet in-game. Small spheres are the hardpoints, the oblong sphere in the center is the root sur.

Additional info found - the sur radius after the inertial damping setting detemines the size of the visible mesh. This will stop ‘popping’ if it encloses the entire mesh.**

LancerSolurus

Forgot to mention this, the vanilla sur files store LOD (Level Of Detail) info in the Bits sections as a tree structure. Just like with material and mesh LOD stuff, it stores decreasing levels of hitbox face counts with each LOD level. The root bit node contains the pointer to the actual sur data. It’s a bit hard to explain without understanding the sur format, I would assume this was another method of speeding up collision detection on far away objects.

StarTrader

Lancer, some questions…

1. as the hull sur is much smaller than the hull and inside it, are the hits registering on the entire hull surface or only on the sur sphere itself?

2. doesn’t there need to be a hull part sur touching them for the Hardpoint surs to work?

3. will the ultimate sur have full-sized sur parts covering the entire hull and booms etc, or will it be similar to your screenshot?

4. will you have an option in the exporter to make a shield bubble for the small ships?

Thanks.

PS - Since this is not an Off-topic thread, can it be moved to the proper place honouring its true status? 8-)

LancerSolurus

**Well, interestingly every sur piece you see is part of the root sur. From studying the vanilla surs it appears that there is a total of 3 surs per sur file. I believe Bejaymac has mentioned this back when I was working on the original cmp to sur converter in 2008 (sorry, never completed that one).

I shrank the Root sur mesh (oblong sphere in center) because it was causing all of the collisions and gun hits. If the hardpoint is inside of it I would be unable to test if it was taking hits on the hardpoint surs.

I will be getting back to work on this tonight and see if it was just a simple typo or something I missed. The way it’s designed now I built all of the info into the root sur according to if it is part of it such as the hardpoints.**

LancerSolurus

**1 - only on the sur sphere

2 - possibly, may be why Im not getting hits on the hardpoint surs

3 - all parts will be covered when Im done, this is just for the root sur only

4 - shield bubble, yes, btw, why would you exclude the larger ships from having a shield bubble?

@ Adoxa - verified, the ‘Sur Type’ is a count

exts + exts = 1
!fxd + exts = 2

So a type 3 can be exts,exts,surf & hpid
also it can be !fxd,exts & surf
Type 2 is normally exts,exts & surf
Type 4 is !fxd,exts,surf & hpid**

StarTrader

Thanks Lancer, clearer now.

Why no shield bubble on large ships? Because it would be bulbous, and would also enclose empty space between booms and other hull extensions which would register hits when they should not.

We want hits to register only on the hull I think.

And the standard FL ships don’t have them.

In some testing I did some time ago, I vaguely recall that the shield bubble registers hits despite the shield_link shield being far smaller.

If you like the idea of having one on large ships, and the programming is not extensive, perhaps you could make it an option and generate a closer-hugging shield bubble for large ships if the modder wants one?

adoxa

I’ve updated SurDump. Now all but three mesh CRCs have been identified and it converts the triangle and bits offset values into indices.

@LS: It is not “exts, exts” - it is one exts for each of two meshes. Have you not looked at the output of the new SurDump?

LancerSolurus

**You are correct, it was a bug in my sur decompiler. I didn’t skip over the tag string output when the type was exts. Now it’s showing the correct number of headers, oops…

No offense, but no I haven’t tried it. I did download it but I avoid command line programs since my version uses a file dialog box to pick the sur I want to decompile. Your utility will be useful for others since I can’t release mine, it’s part of a testbed program that has stuff in it I won’t release since it can royally screw a mod up if you aren’t careful with it…

Additional info found tonight…
The D3D primitive type appears to be a flag, bit 1 denotes last face list
The flag in the face list has bit 8 turned on in the last face list, this is in all of the flags
This one you found, type 5 face list is a relative pointer to the bits entry, that bits entry points back to that face list
Still verifying…
Looks like the bits sections are in reverse order of the face lists
If sibling and face pointers are non-zero then it is the end of a bits tree

Currently I have implemented all of the crc values for every vanilla model in FL, this includes the lod1 stuff which was ripped from the Part node entries

Here is the output from the or_osiris.sur file after the exts fix added…
http://www.digitalbrilliance.org/Files/SUR_or_osiris.log**

LancerSolurus

A note on the file, the tabs are 4 spaces, not 8 like notepad and wordpad uses. The tables will be out of alignment unless you adjust your tabbing size. Visual Studio Express is a free download from MS, that is what the file is formatted to be viewed in…

adoxa

Here’s my dump of or_osiris.sur. No tabs are used. Your face offset is wrong - it’s a 15-bit signed value, with the flag being a single bit. I’ve used “size” for a type 5 face, but it probably should be “bits offset”.

I’m missing two CRCs from bw_freighter.sur (0x0E4ADA66 and 0xE5A5F5DD) and one from bw_fighter_rtc.sur (0xE5036F2F). Happen to know what those are?

LancerSolurus

**Yeah, the bit offset is wrong since I did not do a subtraction on it. Actually it is a 7 bit offset, 128 to 255 is negative, 0 to 127 is positive (bit 8 is the sign). Since I am the only person to see it until the previous post I didn’t bother with converting it to the proper number….

I just checked, all 3 of those entries are not part of the standard cmp node entries in any vanilla ship, I am getting an ‘unknown’ value on those 3. A very interesting discovery. I will run a full decompilation on every cmp & 3db file in FL tommorrow to see if it pops up somewhere else. Also since it could be a naming difference in a Fix, Rev, Pris or Sphere object name, I will check those as well on those specific ships. My decompiler only pulled the node names, not the Cmpnd info…

Good catch, if you want, I can post the string list for every node name if you want it for your sur decompiler, I simply build the crc list as I load them.**

adoxa

No, it really is a 15-bit number:```
mov ecx, [eax] ; retrieve the encompassing dword
shl ecx, 1 ; remove the flag bit
sar ecx, 17 ; strip the low 16 bits, giving the 15-bit signed offset

StarTrader

Erm… sorry to rain on your parade guys…

You haven’t picked up the duplicate sur parts that cover the hull parts and (wings+weapons) parts. These exist in every standard FL ship, and have hashcodes like 00000290, 00000270, 00001b10, 000018d0, 00001e10. The easy ones are the 00001b10, 000018d0 and 00001e10 which are the shield bubble duplicates, and usually these have the “4-digit” numbers as I call them. The “3-digit” ones like 00000270 and 00000290 are wings+weapon surs but have no clear meaning of left/right or function - for example in the bw_elite (stiletto), there are two identical left wing+weapon surs both with the hash 00000270, but of the two identical right wing+weapon surs, one has the hash 00000270 and the other has the hash 00000290 !

In the bw_vheavy_fighter (sabre) there are two wingsets, upper and lower, and these have only 1 wing+weapon sur each - the two upper wing+weapon surs (1 left and 1 right) both have the hash 00000540 ! And the two bottom wing+weapon surs (1 left and 1 right) both have the hash 00000270 !

The 2 bw_freighter hashcodes you haven’t identified cover the front hull of the dromedary and the rear hull, and both have identical duplicates. I am still looking but it will just be a random name someone chose like maybe “dromedary_front_hull” and maybe “dromedary_rear_hull” so they won’t be found easily!!

The odd thing is neither of them match any of the .cmp group names so I don’t see how they can work under the conditions that we have deduced for them to work, but obviously they do work !

Here’s the full listing of the dromedary (bw_freighter) surs as imported by the sur importer, and observations…

M01_Root - covers middle of hull
M02_f00fb9de (HpMount) - the Shield Bubble
M03_HpMine01 - HpMine01
M04_HpCM01 - HpCM01
M05_HpShield01 - HpSHield01
M06_HpThruster01 - HpThruster01
M07_HpWeapon02 - HpWeapon02
M08_HpWeapon03 - HpWeapon03
M09_HpWeapon01 - HpWeapon01
M10_HpTurret04 - HpTurret04
M11_HpTurret01 - HpTurret01
M12_HpTurret05 - HpTurret05
M13_bw_star_eng_lod1 - covers starboard (right) engine
M14_bw_port_eng_lod1 - covers port (left) engine
M15_bw_port_wing_lod1 - covers port (left) wing
M16_HpTurret02 - covers HpTurret02, on top of port wing
M17_bw_star_wing_lod1 - covers starboard (right) wing
M18_HpTurret03 - covers HpTurret03, on top of starboard wing
M19_bw_glass_lod1 - covers cockpit glass
M20_E5A5F5DD - covers front part of hull
M21_0E4ADA66 - covers rear part of hull
M22_baydoor01_lod1 - covers right baydoor
M23_baydoor02_lod1 - covers left baydoor
M24_00001B10 - duplicate of shield bubble, same location
M25_bw_star_eng_lod1 - duplicate of M13 but located near origin. Should be centred on M13.
M26_bw_port_eng_lod1 - duplicate of M14 but centred near origin. Should be centred on M14.
M27_bw_port_wing_lod1 - duplicate of M15 but centred near origin. Should be centred on M15.
M28_HpTurret02 - duplicate of M16 but centred near origin. Should be centred on M16.
M29_00000290 - shape covers port wing plus HpTurret02 but is centred near origin. Should be located on M15+M16.
M30_bw_star_wing_lod1 - duplicate of M17 but centred near origin. Should be centred on M17.
M31_HpTurret03 - duplicate of M18 but centred near origin. Should be centred on M18.
M32_00000290 - shape covers starboard wing plus HpTurret03 but centred near origin. Should be located on M17+M18.
M33_bw_glass_lod1 - duplicate of M19 but centred near origin. Should be centred on M19.
M34_E5A5F5DD - duplicate of M20, same location.
M35_0E4ADA66 - duplicate of m21, same location.
M36_baydoor01_lod1 - duplicate of M22 but centred near origin and rotated 90 degrees clockwise (looking from top)
M37_baydoor02_lod1 - duplicate of M23 but entred newr origin and rotated 270 degrees clockwise (looking from top)