.obj -> .sur converter
-
New version 1.1 BETA 1.2 (See attachment)
A series of tutorials can be found here, thanks to Skotty:
From the included info text:
Thanks go to:
- Adoxa & Lancer Solurus for reversing most of the .sur format
- Adoxa for his .sur file dumper and MS .sur importer
- Colin Sanby for the MS Sur Exporter source code (I used some of the helper functions)
- Skotty for helping me testing and thus saving much time
Known issues:
- Some parts and values are still not the same as in vanilla .sur files
What the tool does (besides writing the format):
- Filtering double vertices (always) and merging close ones (if selected)
- Filtering every group for connected triangles and making a new group out of every connected ones
- Checking whether every connected triangle group is convex, else splicing them into single triangles (This is done to make every geometry
work. The same trick is used in vanilla .sur files.) - Generating a complex tree for collision detection later in the game (stored in the bit section)
Keep in mind that you are responsible for the quality of the .sur files. Reproducing the geometry with convex elements is most probably better
than letting everything getting spliced…You can help the tool if you put every convex geometry in a single group.
it would be nice if you would mention me in the credits if you use .sur files created with this tool in your mod.
Hope you don’t discover too much bugs
SchmackbolzenChangelog:
v. 1.1 BETA 1.2:
- Fixed listbox for group selection not behaving correctly on Windows XP
- Made width of listbox more convenient for selection
v. 1.1 BETA 1.1:
- Fixed bug which lead to missing triangle groups
- Added basic mass conversion support for non multipart meshes
- Changed automatic center calculation for target crosshair from mean to median value
- Fixed group list not updating when started via “open with” command
v. 1.1 BETA
- Fully support every feature of the .sur format
- New more accurate way of checking whether a shape is convex
- OBJ loader now reads more variations of the format
- Fixed all known cases where the .sur file was not correctly generated
- Some new options like merging close vertices and using only the outer convex hull for collisions
- Model cleanup before and after conversion
It is recommended to reconvert every .sur file you created with the old versions since there most probably were errors!
v. 1.0 BETA 1.3:
- Some tree generation bugfixes and speedup
v. 1.0 BETA 1.2:
- Fixed another bug in tree generation, collision should work more often now
- Deleting existing .sur file now before creating new (seemed to cause strange bugs sometimes)
v. 1.0 BETA 1.1:
- Fixed memory leak
- Fixed bug in tree generation, collision should work more often now
- Started implementing drag & drop / send to / open with… support as suggested by FriendlyFire
From the included “How to use” text:
General info about collision meshes:
- They always should have the least polygon count possible!
- The best is to reconstruct the original geometry using convex elements, although it will work of you don’t do this (but the filesize will get bigger then).
- The more groups per object there are in the .sur file the more calculation power is needed for collisions. Thus splicing rapidly decreases performance, since
the group count increases a lot with every group spliced. How much is up to you to test.
Don’t use more then 16384 vertices. The format does not support it. You can be lucky and the count is reduced in the cleanup process, else the file creation
will stop.
Don’t use non-triangulated geometry. The format does not support it and the conversion won’t start.If you view the converted geometry there is always a convex hull around everything! This is needed for collision detection and you will not collide with it!
Some explanations about the options:
Q: What do you mean by splicing?
A: If a shape is not convex every triangle will be made double sided and put in a separate group, which is convex, because a doubled sided triangle is
convex. This way geometry is preserved, but you get more data (that’s how vanilla does it!).Q: Is merging close vertices good or bad?
A: Depends. In some cases you can find more convex shapes and reduce file size, in other cases it makes things worse. You should try both and even adapt
the merge radius if structures are small. The file which is smallest is usually the better one.Q: What should I put into the hardpoint text box?
A: Enter all hardpoint names (one name per line!) which have no geometry attached like HpCloakxx etc. To prevent crashes you should at least add HpCMxx,
HpThrusterxx and HpMinexx either there or with geometry attached via groups (use the table on the right then). If no geometry is attached you can’t hit it!
If there is no such hardpoint on the model then of course it is not needed in the .sur file.Q: What center value shall I use if I have assigned groups to a group via the table on the right?
A: If a group has other groups assigned then you also enter the center for this one. All attached groups will be moved so that the center of this one is on the
origin.Q: What happens if I don’t select a group in the table on the right?
A: Everything which is not selected will be assigned to root.If you discover bugs please post them here.
You can convert any geometry you like, e.g. using a poly reducer and then convert it. The disadvantage is that you will get many groups inside the .sur file.
Roadmap (sorta):
V1.0: Get basic collision working as flawlessly as possible
V1.1: Include multipart support, some sort of intuitive gui for that
V1.2: Some optional stuff which might be helpful, real hardpoint support, maybe read them from .cmp files. Support every feature of the .sur format.
V1.3: Optional stuff which makes generation easier, things which came up during development and testing etc. -
Because from what I get this is not a .sur generator, but rather a converter. Apparently, it can also generate a sur from non-convex geometry (aka the raw geometry from a .cmp) but that doesnt seem to be recommended.
So I guess it makes sense to pick the .obj format, because it is an open format supported by a lot of modeling programs.Actually, I think I should also stop developing the 3ds max exporter and make it a standalone .obj -> .cmp converter (would also get rid of the 3ds max versions compatability problems).
-
Ok, sort of get that. An example of issues i’ve had with .obj format is they won’t work for me in MS. I have to load an obj format model into 3dsmax, convert it to 3ds, import that into MS for things to work. Pain in the arse and unique to me i think but even so it’s an awkward workaround.
So from what you’re saying i would have to have a sur to start with? But that’s in sur format so what does obj have to do with anything apart from being a modelling format?
I’m sitting here with this tool and absolutely no idea how to use it. The options are totally confusing to a novice, for example.
From the program,
Option 1 - If maximum distance of connected triangles (spelt wrongly in utility) to convex hull is smaller than (insert number), use convex hull, else splice connected vertices
This means what to somebody picking this up for the first time? How would i know what number to enter?
Option 2 - If difference of connected triangles to convex hull is too great, use convex hull instead of splicing
Meaning?
Option 3 - Append hardpoints to HPid section (one name per line)
An example for those that know nothing about this would be nice.
I’m not knocking this program as i’m sure it does what it’s supposed to but as a new user i’m still trying to workout exactly it’s purpose. It assumes the user knows all the above points but that’s not the case i can assure you. Most people want simply to click on a cmp file, press generate sur and get back to a bit of coding. After looking at this program i still don’t know from the available options what it’s actually supposed to do.
Also does this mean i’d have to convert every model i have to obj format? That isn’t going to happen anytime soon. We all use cmp format, we need a cmp to sur builder, It really is that simple.
-
Hi Schmackbolzen: Thanks for your great work, appreciated.
But I’m afraid I agree with Gibbon - converting many models to .obj is not something I am prepared to do.
This method was not expected, and we didn’t have a chance to tell you this before you put in so much effort.
1. Think of the other utilities that need extra steps, they aren’t used as often as they should be, nor by so many modders. It will be a shame to continue with this project when it will not be used.
2. As you say “this is not what is in the vanilla files”, we have incompatibility - and I lose interest. I don’t want another “nearly OK” tool.
If you need to automatically convert from internal MilkShape memory (model loaded) through .obj and then to .sur as an “internal” step without having the modeller do it, then that is OK.
But I (we all) want a fully compatible, one-step tool, with the selectable options I listed earlier. I have defined the specification pretty well for everyone over the last year.
Hardpoints must be read from the MilkShape memory, modellers should not be expected to re-write them into the exporter/converter by hand.
Thanks if you decide to understand this and make it the way we want it.
For me the ONLY criteria is that the generated sur file for any standard vanilla FL ship, say for example the Stiletto (bw_fighter), MUST be exactly the same as the original vanilla sur file, with all the same components. And it has to work, of course.
If you decide not to, then I am sorry, and thank you for trying, but I won’t use it very much (if at all), because I can already do this with existing MS tools and 3DSMax tools, and it is a pain now. Your utility will not make anything easier if you make it in the way you say.
So many people criticised me for such a long time, and some even insulted me by accusing me of deliberately making bad surs for all this time, until I finally isolated and proved with help from adoxa that it was indeed caused by missing parts, just like Bejaymac used to tell us time and time again.
And so all the surs we have made with these current sub-standard methods and tools are bad.
So the criteria can only be a complete sur identical to the original for vanilla ships.
Once we have this, surs for custom ships will automatically be in the same standard format too.
Thanks for understanding my point of view.
-
w0dk4 wrote:
Actually, I think I should also stop developing the 3ds max exporter and make it a standalone .obj -> .cmp converter (would also get rid of the 3ds max versions compatability problems).
W0dk4 old friend - I was so excited about your exporter until you wrote this. If you do then I will lose interest in your cmp exporter for the same reasons as above.
obj is a simple format, basically lists of vertices in plain text format. So yes it is easy but is also limited.
-
i’d prefer obj converters. you can export those files from any program, no losses at all (wavefront might be limited, max is not, just export obj and mat and youre ready to go). moreover, it makes us independant from the limits of ms3d and from the issues we know from the sur exporters and cmp exporter. no matter what program we use now, we can just save everything up as obj and can convert those to mesh libraries and surs as we wish, faster, better. export to obj is no step too much, you can do it with the modeler program you use.
your quote, Schmackbolzen, seems to tell us, that your converter does surs in the same way vanilla surs are done, too. though this is yet WIP it is already more than we ever had and probably became exactly what we have been waiting for. thank you.
only thing is indeed, what gibbon said, a documentation on options would be helpful.
-
Gisteron:
Which part of this did you not understand?
“- The tree of the bit section still is not the same as in vanilla files. That means depending on the geometry you can fly through if you try hard enough.”
What are you going to do when my ships do fly through it, accuse me of trying too hard to break the tool?
And “You can help the tool if you put every convex geometry in a single group.” defeats the objective entirely.
I will repeat once more: “the criteria for this tool is the creation of a sur for a vanilla FL ship that is identical to the original sur”.
Nothing less will do because the new surs will then not be compatible. All components of the vanilla sur must be present.
You are undermining the construction of what will be a good tool.
If you are happy with this as it is, and think you are already creating good surs, then you already have the tools you need, so be happy, take a back and quiet seat, and let us get the tool we need.
We do not yet have the tool we need.
-
w0dk4 wrote:
Sounds awesome! Now waiting for some real-world user reportsI can tell you that the surs of his converter are working great.
After yesterdays tests I told Schmackbolzen that his surs are nearly perfect, but you can go trough them if you find a right place/try hard enough.
However, I got some of his surs since some weeks and they work great. So my real-world user report gives a good mark for Schmackbolzens work until here
Some comment to StarTraders post:
The converter works best with already prepared convex meshes, thats what Schmackbolzen wrote and thats not the problem! Just use the convex tool of Milkshape and you won’t have any trouble with that part.
If I understood right you got some problems that his converter uses obj files. The fact is, that these files are supported by every 3d model program and converting a cmp to obj is not the problem, isn’t it?
Oh and, if you are not happy with “[…] put every convex geometry in a single group.” I will tell you that this is normal for a lot of other games, for example the Source Engine.
-
gives ST another carrot
i didn’t say its perfect or exactly what we need. but it is more than we ever had and is not the final version. i just don’t start with seeking for the imperfections, i am just a bit enthusiastic and looking forward for this tool to become the very one we need. it probably, even likely will be, don’t you think so? -
To clarify some things:
Convex meshes for physics engines are absolutely normal! Skotty knows this e.g. from valves source engine.
Every vanilla .sur is made out of convex meshes!
The converter just makes sure the geometry is as the vanilla physics engine needs it.
You can use non convex geometry, try it out. I even tested 3000 polygons, it does work!
I know there is no documentation for the options for know, because it is not easy explain it, since I would have to explain the algorithm and mathematics behind it.
I try to make it simple:
First you should know that for every connected triangles a convex hull is being created.
For the number: Imagine a vertex lies behind this convex hull and you enter the maximum distance allowed from the vertex to the corresponding hull polygon.
For the checkbox: There are cases where no corresponding convex hull polygon is found. In this cases the maximum distance is infinity. That’s why this option exists.
If you are satisfied with the explanation I’ll include it in the documentation.
Concerning the .obj format: Normally you get only trouble with texture coordinates and materials. Both are not needed. The .obj parser is from my own 3D engine and tested against over 10 exporters. It should read nearly every file there is now.
If there is an algorithm which can convert normal geometry to geometry consisting only out of convex elements I would try to implement it. But as far as I know not even the modeling programs are capable of that.
Are there still things which are not clear now?
-
well, for the options explanations, it starts to be somewhat clear. could you prolly explain it like what the different options would result or should result in in the end. maybe some examples and a link to your coming post in the documentation.
-
The result is either you get the convex hull of the connected triangles or every triangle will be made double sided and put in a separate group (I called it splicing), which is convex, because a doubled sided triangle is convex. This way geometry is preserved, but you get more data (that’s how vanilla does it!).
It would be best if you import the converted sur with Adoxas MS sur importer and examine the results. This should help you to understand.
I will develop this tool further depending on your experience. So try to understand it first, I even might change the gui. I don’t know how you guys are thinking so you’ll have to help me with that
-
I updated the main post with some explanations…
-
Good lord is it so hard to convert to OBJ? It’s definitely the way that makes the most sense since you should normally always keep a copy of your original models in some form other than CMP (3ds, obj, ms3d, max, etc.) for later editing, right? With OBJ you get a simple format that works with all tools (no recompiling twenty versions of a 3ds Max plugin because each SDK is incompatible with the previous one) and that isn’t limited by the often poorly documented and/or fleshed out APIs.
I think Schmack went the right way by not making this dependent on MS3D or Max. If a Max plugin was what you wanted, I don’t exactly get the problem. Max -> OBJ -> SUR. Oh gosh, an additional step!
-
FriendlyFire wrote:
Good lord is it so hard to convert to OBJ? It’s definitely the way that makes the most sense since you should normally always keep a copy of your original models in some form other than CMP (3ds, obj, ms3d, max, etc.) for later editing, right? With OBJ you get a simple format that works with all tools (no recompiling twenty versions of a 3ds Max plugin because each SDK is incompatible with the previous one) and that isn’t limited by the often poorly documented and/or fleshed out APIs.I think Schmack went the right way by not making this dependent on MS3D or Max. If a Max plugin was what you wanted, I don’t exactly get the problem. Max -> OBJ -> SUR. Oh gosh, an additional step!
I agree with this completely. Thank you Schmack for working on this tool for us. It will be invaluable in the modding process.
-
Thanks guys!
That’s one reason, the other is, that you should not use the model geometry for collision, but rather a simplified one. So it doesn’t make sense to use the geometry of a .cmp file anyway. You should at least run a poly reducer over it. And the more better way would be to make it like vanilla and rebuild the geometry with convex elements. For the lazy I implemented the splicing, for which I count myself in. In our mod I will try to use spliced geometry, we will have to test whether this results in too much server / client load or not.