Once more into SUR files (advanced)
-
I have never had any luck with sur files - but recently I have come across a sentence that got me interested. Back in one of the topics, BJ dropped half a sentence - “then you can go change the type to scenery and hex edit all the internal part names to 00 00 00 00”. By doing this I finally got a sur file (partially) working for a huge solar - a terrain 3db. I’ve got coll and hit detection on the terrain and four (or three) out of five hills. That’s not bad. The last hill registers nothing. (I have no clue about the fourth hill, because of issues with object radius and disappearing, I can’t test it)
The key to this was a shrinkwrapped sur. FLModelTool can generate an excellent text dump of a sur file, which comes extremely handy. Using this and the sur importer plugin for ms3d I come across some interesting discoveries:
- shield bubble sur files look like this:
1 component
1 shield bubble
all other components
1 shield bubble - shrink wrapped sur files look like this
all components
shrink wrap
This explains why shield bubble won’t work for objects that require a shrink wrap. But we all know we should never ever export anything with shrinkwrap, unless we are splicing the stuff. So I decided to use the splicer. In the end the sur did not register anything. The reason is probably in the structure of the spliced sur file:
Regular shrinkwrap exports look like this:- id header
- other
- triangle group header
- …
- triangle group header
-…
and the whole thing ends with one single vertex list and bits section.
Sur_spliced shrinkwrap exports look like this:
- id header
- other
- triangle group header
- …
- vertex list, bits section
- hpid header
- id header
- other
- triangle group header
- …
- vertex list, bits section
and so on.
As you can see the main difference between regular shrinkwrap exports and spliced surs is that their structure is totally different, -and- the spliced sur also lacks the last “shrinkwrap” component.
I have had the luck of getting a GOOD shrinkwrap out of sur exporter 1.1 - my terrain file being so simple, and I have had moderate success with collission detection - only the last (two) parts have some kind of a problem, whereas the spliced sur won’t work at all. The sur parts are essentially the same, therefore they are convex.
The problem is: Sur exporter 1.1 exports the sur file correctly, only it makes a bit mess of the shrink wrap in most cases. And it will generate a shrink wrap. I believe most if not all our shrinkwrap problems would be solved if the plugin would somehow be modified to make a shrinkwrap part out of a milkshape group we have made on our own, not go after its problematic shrinkwrap generator. If you take a close look in FLMT, the shrinkwrap group is slightly different to the other parts, it has type 5 and not type 4 - this and perhaps other, if there are any differences should be noted, of course. My question is, does someone have the power to either modify the exporter plugin accordingly, or make an app of some kind that does the proper hex editing steps (i.e. remove last group of a shrinkwrap export, and edit the previous group to gain the properties of a shrinkwrap shield)
- shield bubble sur files look like this:
-
Ello BB,
I’m afraid I was not able to fully understand what you have discovered, sorry for that.
Can you please re-read your information above and explain the unclear parts of it again?
Thanks. Then maybe we can work something out.
-
Heheh, sorry for the mess.
I will try this once again, but it won’t be easy:We will use FLModelTool, and Milkshape with the Sur importer plugin.
Open up a sur file in FLModelTool. Start with the sur file of one of the 3dbs in Data\Solar\Misc.
3dbs differ to cmps in that they do not have any part names (like: xy_PortWing) as far as I’m concerned, however, they are solars, and use shrinkwrap. If you open up a sur like that in FlModelTool you can see that it has a structure like the one I described: all convex shapes come first, then followed by a shrink wrap.
Shield bubble shields are different in that they have two shield bubble parts, and only one of them is at the end of the file.Now, what I have found is (though it is not proven, just empiric observation) this, see for yourself in FLModelTool:
A sur file has an: ID header, followed by some data concerning the WHOLE sur file (like type = 2 which means scenery). This is followed by a triangle group header, attached to that is data about our first convex shape. This is followed by a second triangle group header, with info about the second convex shape (these sections also have a type = 4 section, but this does not mean “ship/station” like in ID header. Instead, type = 4 means it is a simple convex sur part). If you have followed me in FLMT with a 3db solar’s sur open, you will find that the last of these shapes have type = 5. This is the shrinkwrap. Everything is then followed by a vertex list.
A stock shield bubble sur is similar, except that it has two shield bubbles for some reason. Open one in Milkshape - you will see that it is just like that.
Time for some clarification: In a sur file there is 1 ID Header, giving us info about the whole sur. This is followed by several triangle group headers. Triangle groups are your convex sur shapes. Their internal part names decide if two or three or more convex shapes belong to the same part (like a PortWing) or not. For MISC 3db solars these part names are all 00 - they have 1 part built of more convex shapes. Everything is closed by a vertex list on ALL the vertices in the sur.
If you export a shrinkwrap sur using the sur exporter, you will find that it does the mesh correctly (1 shrink wrap at the end) - the problem is if you re-open the thing using the importer you will find that the shrinkwrap is a mess. This is what makes FL crash.
Ok, take a moment’s break. Open up a sur in FLMT, and see how it is built up. ID header, Triangle groups, Vertex list.
Now, open up a sur_spliced sur. (The one the splicer has made) What you will find is that it is NOT a vanilla sur in any way. It will be built up like this:
ID header, 1 Triangle group, Vertex list. ID header, 1 Triangle group, Vertex list; and so on. (Actually, this is why spliced surs are so big in size)
This looks like anything but FL’s native shrinkwrap sur format. In fact, there isn’t a single shrinkwrap anywhere. Plus, this is not 1 sur file, it is literally many sur files almost “copypasted” into 1 file. No criticism, sur_splicer is a very useful tool - but if you take a look, it is a miracle that these surs actually don’t crash, and actually work to an extent.
This is everything
To sum things up: SUR Exporter 1.1 seems to do not only shield bubbles, but even shrink wraps correctly, too. The problem is that it always and always makes its faulty wrap, there is no way of getting it to use a mesh for a shrinkwrap that you made. This might be necessary, however, as sur_spliced surs are nowhere near FL’s native format, and shield bubble just won’t work for some larger objects, or solars. Remember that FL can make a difference between shield bubbles and shrink wraps: the number and location of bubble/wrap triangle groups varies.
-
Right, I’m beginning to understand, but there is a contradiction here…
You say: “To sum things up: SUR Exporter 1.1 seems to do not only shield bubbles, but even shrink wraps correctly, too. The problem is that it always and always makes its faulty wrap”…
So do you mean that Exporter v1.1 does export the shrinkwrap correctly, or do you mean it never exports the shrinkwrap correctly?
And… do I understand correctly that you are happy there is no problem with shield bubble exports from exporter v1.1?
I remember Bejaymac saying and demonstrating that Exporter v1.1 exports bad shrinkwraps, but can’t find his post anywhere.
By the way for everyone’s info - the first shield bubble (0xF00FB9DE - “Unknown_Mesh_f00fb9de” in FL Model Tool) is actually the hash for “HpMount”.
As you have probably seen from my posts on several forums, I thnk I have probably been the one with the most bad luck with making surs for small ships in the past 4 years! In contrast, my spliced surs, which I only use for large ships, are almost a dream to make, most of mine worked first time after I understood the naming and ensured each piece is convex.
Our problem is that our tools are a mismatch and also none are 100% correct, because they were made by different people with different understanding of the problem in hand.
For example, when we import a vanilla .sur with the sur importer, it’s not easy to see that there are more than one sur for wings, e.g. one covering a wing part, one covering the weapon mounted on that wing, and yet another covering both that wing part and the weapon! There are also similar duplicate surs for engines and countermeasure droppers. These duplicates are misplaced to the origin by the importer, so they overlap and are hard to identify. They also have hex code names that appear hard to decipher (thinking about this, I wonder if they are double-hashes like base rooms?-- hmmm…) and our exporter does not allow us to make these because it re-hashes whatever we put in even if it’s a hex value…
The way to make these duplicate sur parts is not hard, they are just convex “covers” that we can now generate easily in MilkShape and in 3DS Max using the convex tools for each modelling package, but we can’t yet name them correctly for the export, and we need to be able to do just this. It may well be fairly straightforward to rename them using a Hex editor after exporting them but I haven’t explored that path yet. And then we need to know why there can be two of these in the same cmp with the same hash code e.g. 00000270 and 00000290 etc.
So I think we need to find the way to decipher the names of these duplicate sur parts, and revamp the sur importer to recognise their actual locations relative to the model origin, so it does not move them to the origin. (Or maybe their offset is 0 relative to the main part but the importer does not detect this link to the main part).
Thanks to p1p3r we have an improved cmp importer that can import hardpoints, and he is working on an improved cmp exporter. If we can get someone to work on an improved sur importer we can then blackmail him/her to work on an improved exporter!
-
Bejay said multiple times that 1.1 exports bad shrinkwraps… though I have read several sur threads now and in every thread pretty much everybody talks about everything here and there - in the end there is almost no coherent info about how surs actually work. To me it seems almost everybody except Bejaymac is just doing a giant trial and error process until something works which is actually very bad.
There is also no sur tutorial that gives an insight into how the actual sur format is theoratically working, its just always about “do step xyz and if you are lucky you’ll have a working sur”.
No wonder we are still in stone age regarding surs.Sorry for the rant.
-
Not arrogant at all, Mknote…
OK, my spliced surs without shield bubble for large ships all seem to work fine.
My interest is in making shield-bubble surs for small ships, which I like to break into several groups - not too many, usually 4-8.
I have tried all of the techniques that others say are 100% many times and I found no guarantee of success, some worked for a while but most do not. I used to think I had success, but I was only testing for a few minutes. When I tried testing the sur for long periods, FL always crashed within 20-30 minutes. I test by giving the ship with its new sur to a faction attacking a heavily-armed base ship or in encounters with their enemies near a base ship, and monitor by hovering above the base in a standard FL ship to rule out interference from my ship or its sur.
I really have tried all ways, I make each sur piece with a single primitive shape (using only either a cylinder with the ends pulled out slightly, or a box, or a sphere). I use only scaling and rotation, I don’t extrude and I use only 1 shape per piece, ensuring as much as possible to maintain convexity. In some I have used the convex tools in 3DSMax and recently in MilkShape, I wrote tutorials of using those for spliced surs but those do work fine, whereas the small shield-bubble ones do not.
I ensure that each sur piece is named the same as an existing ship group, and recheck the hashes after export to be sure the name matches the cmp group (with _lod1)… I have exported in one single group and as multiple groups, and used FLModelTool to set sometimes as ship and sometimes as scenery as advised, and sometimes make all parts as root and sometimes not - and always the same result no matter which way I do it - they usually crash FL, not always immediately on launch - sometimes after several kills of the ship and sometimes up to 20-30 minutes of combat. When I use a standard sur resized for the same ship, there are no crashes for several hours of testing.
So - would you please explain (yet again) your BJ method, the pitfalls and where you correct the possible problems when you create your small-ship multi-group surs with shield bubbles?
Thank you my friend, your advice and patience is much appreciated, mine are both exhausted.
And my PC broke again, adding to my pain!
-
Sorry for the late reply.
To be honest, I don’t think I’ve tested my SURs as extensively as you have; typically, I fly my ship to Rochester and let the Rogues and Xenos pound away at me while I’m invincible and shieldless. I don’t get any crashes, but I’m only out there for 10 minutes tops. Looking through the downloads, I’ve noticed that my mini ship mod is no longer there, so I can send it to you for more rigorous testing, along with the SUR MS3D files to check against yours.
About the SURs themselves: the convexity is good, and there seems to be little wrong with your method. Here are a few reminders as well as tips I’ve picked up through testing.
1.) Are you sure that each component has a single part in MS3D? That is, each component in the CMP (i.e. Root, left_wing, right_wing, etc) has one and only one group in the SUR. When you export the SUR, the values for Group Quantities should all be one.
2.) The “_lod1” is not needed, and in fact may cause the SUR to fail. I know some have said it’s required, but my experience is that it isn’t. What you want to name them is what appears in UTF editor when opening the CMP file minus _lod1.3db.
3.) In my experience (and this is extremely useful, due to hint 1), only the root part of the SUR must be convex. This fortuitous fact allows for much more skin tight SURs. I doubt that this is the cause of any crashes, but it should be helpful.
If you want me to send the mod and SUR files, let me know and I can e-mail them to you for testing. If they are in fact bad, it’ll send me back to the drawing board, and if not, perhaps you’ll be one step closer to making valid SUR files.
MK
-
@mk: What w0dk4’s saying is that while we can usually manage to create SURs, we don’t know why or how they work or not. We found a step-by-step process which mostly works, but we don’t really seem to grasp how it works in the background. Were we to, we could create a far more structured and reliable method.
It’s a bit like how ALEs were before LS created the ALE Editor. You’d have to hex edit random values until something happened, then tweak some more to make it work the way you wanted.
-
Yes you’re right FF. Hopefully Lancer’s exporter/converter will do the trick for us.
Mknote:-
Thanks for your reply.
In reply:-
1. Yes, one shape only per sur part always. Yes when I export as multiple groups it is 1 sur part per group.
2. I think I should clarify here because many modders don’t really understand this (it took me a long while!) and it will be more confusing if I only reply briefly…
When exporting the sur as one single group, all components are combined into one and are hidden in the first component, which is named “root”, so you are absolutely right, the sur part name is lost and does not matter.
However:- if the sur is exported as multiple groups and the _lod is not present at the end of each of the sur part names (i.e. the name is not identical to one of the Part_xxxx Object Names in the cmp file in UTF editor) then the sur part will not be associated to any ship part, and so it will not register any collision or hits. So the name must be identical to the component name as it appears in UTF Editor.
When exporting multi-group surs, the sur exporter does this automatically so the _lod1 must NOT be present, otherwise you get a name xxx_lod1_lod1.
But if you are exporting single sur parts to splice together, the exporter does NOT add the _lod1 to single-group exports, and this is why you need to import the cons_fix.dat that the splicer makes.
And this is why when you make the splicer’s input ini file you need to rename each part to surpartname_lod1.
(I don’t use additional dummy groups unless its absolutely unavoidable, I like to ensure my ships have enough ship groups to have one per sur part. I hate ohmygodzoombay01 and all its relatives!!!)
By making my sur part names surpartname1_lod1.sur, surpartname2_lod1.sur, surpartname3_lod1.sur I remind myself to just remove the .sur from each name, so it becomes:
surpartname1_lod1.sur root
surpartname2_lod1.sur part2_lod1
surpartname3_lod1.sur part3_lod1And, if you add your own _lod1 to the sur part in MilkShape then you don’t need to import the cons_fix.dat, so this saves me a step when I am using the splicer. I have actually modified the MilkShape Convex Tool so that it adds _lod1 instead of _convex to the convex parts it generates.
Note for those who aren’t certain - there is some more confusion here: The name of the sur part must be exactly the same as a Part_ Object Name in the .cmp file, not the Part’s file name. The Object Name is a node in each of the Part_(partname) nodes.
3. I haven’t tried or tested this, does it work for only a single-group sur export or for a multi-group export?
Thanks mknote, yes I would like to test your mod pls.
Many thanks.
-
1.) Good, good.
2.) I didn’t know this, as I’ve never used SUR splicer except to remove the shield bubbles from destroyed component SURs, and I’ve always done multi-part SURs.
3.) Again, I don’t know about single-part SURs, but they work for multi-part SURs when non-root components are concave. The files I e-mailed should show this.
MK
