CMP to SUR Conversion Tests

I have attached both surs. They seem to work pretty good, although the data structure is not yet optimal. For example, milkshape tends to clone every vertex used by a different group when exporting to .obj and I don’t filter that yet.

I now have extended my .obj->.sur converter to work without convex hull meshes.
I appended a demonstration for the industrial_dock.cmp of the aldebaran modular station pack II. You will see that it is 1:1 geometry and it does work in the game. The big box around it is just the shrink wrapped hull, which I don’t automatically generate yet and so I chose a simple box.

Does that mean you figured out how to do vanilla-like surs?
All the geometry in one group without having to make XXX convex-hull parts and then splice them together?

Yes, that is correct. I can convert any geometry you like to a .sur file

Thats awesome! The real breakthrough regarding .surs IMHO

Keep it up

So - what is exactly the difference between your converter and the sur builder?

Well the .sur builder tries to build one convex hull around each mesh group, whereas my converter just converts the whole geometry without any changes. Meaning that you have to do optimizing of the geometry yourself (well it will work without, too. But then there is more calculation for collision needed). The advantage is, that you get exact fitting .sur files of any geometry you can come up with.

The easiest thing to do is just a polygon reduction so that you get a geometry which is close enough to the original, but has as few polygons as possible. Nearly all modeling tools can do that.

If you have a real complex mesh and want to simplify collision calculations you can just build your own collision mesh out of simple boxes and spheres and convert it.

and is it, that this mesh will be compatible with any visual geometry? cuz we heard something of “sur to model attachment” and stuff like that. although the flmodeltool made every vanilla sur compatible with any custom mesh. does it work somehow similar in this point?

That depends on how the .sur file is generated. You can attach certain groups in the .sur to corresponding groups/nodes in a .cmp file. The ones I create for now are all attached to the root node, which every .cmp file should have. But it is possible to change all that.

w0dk4 wrote:
Thats awesome! The real breakthrough regarding .surs IMHO

Keep it up

Thanks, I will

would be nice if the sur parts attach to the model part they are built of. this way we wouldn’t need to worry about e.g. animated parts.

Schmack: You are confusing me - not surprising, I’m easily confused…!

No offence meant, I am only keen to understand…

Do I understand correctly, that your generator makes close-fitting surs tagged Root?

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

(By the way some of my bad surs worked in game under constant destruction for more than 20 minutes before crashing the game, so testing needs to be at least 30 minutes of NPCs using the sur getting destroyed by other NPCs).

Second - why are you making your own, since there is the already sur-builder almost completed? This is re-inventing the wheel?

General: - I found this info on collision solids, maybe we can learn from it?..
http://www.panda3d.org/wiki/index.php/Collision_Solids

I understand that process, but we don’t know how or what exactly FL is testing. Do we?

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Arent vanilla surs concave, too?

The format has been mostly reverse-engineered, but Lancer Solurus’ utility is not open sourced and he’s encountered a block and legitimately is prioritizing his game over finishing the SUR application.

Therefore the only way for Schmack to create a SUR generator is to write his own.

And yes, w0d is right; vanilla SURs are concave in many places.

StarTrader wrote:

Do I understand correctly, that your generator makes close-fitting surs tagged Root?

Yes they are close fitting, depending on the geometry you convert. For now everything is tagged as root, but that can be changed.

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Convex geometries are better in general for collision detection speedup. Freelancer does some silly trick to construct concave geometries, which I just also do. I tested a model with over 3000 polygons and it works without problems or notable performance reduction.

(By the way some of my bad surs worked in game under constant destruction for more than 20 minutes before crashing the game, so testing needs to be at least 30 minutes of NPCs using the sur getting destroyed by other NPCs).

Since our mod has an integrated modupdater I used the opportunity to upload the first .sur file I attached here for testing. No one has had a crash so far.
Although the tree is not created correctly yet, I had no crashes, but collision detection failures on some complex geometry depending from which direction you are nearing. This all fits perfectly in my theory. I will have a look at the .sur files the sure file generator creates to check whether it makes errors.

Second - why are you making your own, since there is the already sur-builder almost completed? This is re-inventing the wheel?

Friendlyfire answered that quiet well.

General: - I found this info on collision solids, maybe we can learn from it?…
http://www.panda3d.org/wiki/index.php/Collision_Solids

Hard to say, since there are many ways to solve collision equations.

I understand that process, but we don’t know how or what exactly FL is testing. Do we?

Not exactly, but since my theory seems to be correct I got a pretty good idea what it does.

w0dk4 wrote:

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Arent vanilla surs concave, too?

No, I’ve not seen even one. All individual sur parts that I have checked are convex.

They may look concave when they are in place next to each other but they are not individually concave.

Can you point me to one? It could be that it has a dfferent layout.

I have made great landing bay surs for big dockable ships using convex primitives for the floor, walls and ceiling, and sometimes for the forward bulkhead, all assembled together, and declared them in concave.ini and they work 100%, all the time. Single-part convex surs didn’t ever work for me.

CzW and Mirkha’s hole-in-base-station sur using the Boolean modifiers had no collision detection in certain places in it, so it is not conclusive that concave are ok…

We have been making convex sur parts for a long time, since concave ones caused crashes.

This could still be because the “Type 5” sur parts are missing.

Schmack: I am looking forward to using your generator.

The link I posted above takes us to see what shapes are used in collision detection and for what purpose, there are references to using spheres, tubes (rounded-ended cylinders), it seems only the sphere is 100% in reliable testing, and the “tube” is second. Other polygons allow slip-throughs.

But with spheres I have not tested to see if the collision is on the sphere or on the hull when the sphere is bigger than the model. Logically it would test on the sphere so it’s not useable for large ships again. We know the deformed convex “tube” or squashed sphere works very well. Boxes and others are unreliable, and this is what we are seeing with boxes and non-convex shapes - as well as the more definite crash!

Anway, back to this subject…

Great work all of you guys, and…

Schmack, we hope to have a fully-working “on-the-button” multi-cmp-group converter with optional “shield-bubble” and all Type 5 sur parts from you.

I hope this is also what you have in mind?

Take your time, no rush - 3-4 days is fine! ROFL!

Oh - one more big (bloody important) question in the light of LS’s bombshell revelation and sudden disappearance from this scene…

Will you be allowing us to have and use the generator / converter when it is finished? ?)

We don’t need the source if you don’t want to release it, just a working executable for intel PC that works on single-core cpus please!

Last idea for today…

Would it be a better idea to start a new thread instead of continuing with this one as it is for sur-builder primarily?

Thanks.

CzW and Mirkha’s hole-in-base-station sur using the Boolean modifiers had no collision detection in certain places in it, so it is not conclusive that concave are ok…

exactly, sometimes that works sometimes not …
and i never figured out why

I vote for maintaining a single thread. The most important thing being developed here is information about the SUR format.

Whether the final product(s) is/are exporters or other tools, there does not appear to be any single source of reliable information on what makes a “good” SUR.

Let’s keep the testing and reporting in one place until we get it figured out.

StarTrader wrote:

w0dk4 wrote:

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Arent vanilla surs concave, too?

No, I’ve not seen even one. All individual sur parts that I have checked are convex.

They may look concave when they are in place next to each other but they are not individually concave.

Can you point me to one? It could be that it has a dfferent layout.

I guess that depends on the definition about concave/convex. Every triangle needs to have an opposite counterpart. If that is missing, you have to split it up and create one. That’s how Freelancer and my converter does it. That’s the easy part. Look at solar\misc\space_industrial.sur and you will see what I mean.

This could still be because the “Type 5” sur parts are missing.

That’s the reason for your crashes and strange behaviour. It is the root of the tree. If it’s missing Freelancer makes wrong decisions on collision calculations (for shield .sur files I have to have a look yet how it’s done there).

Schmack: I am looking forward to using your generator.

The link I posted above takes us to see what shapes are used in collision detection and for what purpose, there are references to using spheres, tubes (rounded-ended cylinders), it seems only the sphere is 100% in reliable testing, and the “tube” is second. Other polygons allow slip-throughs.

But with spheres I have not tested to see if the collision is on the sphere or on the hull when the sphere is bigger than the model. Logically it would test on the sphere so it’s not useable for large ships again. We know the deformed convex “tube” or squashed sphere works very well. Boxes and others are unreliable, and this is what we are seeing with boxes and non-convex shapes - as well as the more definite crash!

Freelancer also does precalculations for collision detection. If not, you would just fly through any object, which is small enough depending on your speed. Thus, theoretically the geometry should not matter. Freelancer does not know simple geometries like spheres and boxes anyway, only special triangle meshes.

Great work all of you guys, and…

Schmack, we hope to have a fully-working “on-the-button” multi-cmp-group converter with optional “shield-bubble” and all Type 5 sur parts from you.

I hope this is also what you have in mind?

Take your time, no rush - 3-4 days is fine! ROFL!

Oh - one more big (bloody important) question in the light of LS’s bombshell revelation and sudden disappearance from this scene…

Will you be allowing us to have and use the generator / converter when it is finished? ?)

We don’t need the source if you don’t want to release it, just a working executable for intel PC that works on single-core cpus please!

I could even compile it for linux, since I am using freepascal + lazarus (yes, I am pascal fan ;)). It will run on any machine, since calculations are not that complex and I don’t waste memory like a certain other tool does (just kidding).

I am planning to release it, first without any code, later maybe with. Please understand that I don’t like the idea of unpolished software, so if it hast certain bugs which to my opinion are not acceptable, I won’t release it until I fixed everything.

The goal is indeed a complete .sur converter, but that will take time, since I will have to create some sort of group selection dialog, where you can say which group should become what and all the other options there are. And I still have to fix the tree sorting and a glitch while merging the root node with the rest of the tree.

I have some comments/questions please…

Schmackbolzen wrote:

I guess that depends on the definition about concave/convex.

Well one online definition of a concave shape is “A shape that curves or bends inward”.

The mathematical definition is probably much more scientific than this, but here is mine specific to low-poly models made of faces (triangles)… in a concave shape, a plane extrapolated on one or more faces (i.e. the plane made by pulling out the triangle to infinity in both directions) will cut into another face on the shape.

In a convex shape, no face plane will cut into another face on that shape.

There can’t be any misunderstanding of this, surely?

Schmackbolzen wrote:

Every triangle needs to have an opposite counterpart. If that is missing, you have to split it up and create one. That’s how Freelancer and my converter does it. That’s the easy part. Look at solar\misc\space_industrial.sur and you will see what I mean.

I’m not sure what you mean by “triangle”. Do you mean a face? The model you mention does have many sur parts made of only 2 triangles, they are interpreted as Type 2 and hashcode “00000000” and I’m not sure of the significance of these, it seems odd, just planes made of two faces?

Or do you mean a mesh made of triangles (faces)? And what do you mean by opposite counterpart? In the sur file? Yes, we know this, that every sur part must have a counterpart cmp group to be attached to in order to function. Is this what you mean?

If you mean triangle = face, in the sur, then I do not understand why each face must have a counterpart.

Schmackbolzen wrote:

That’s the reason for your crashes and strange behaviour. It is the root of the tree. If it’s missing Freelancer makes wrong decisions on collision calculations (for shield .sur files I have to have a look yet how it’s done there).

I thought the root sur part normally corresponds to the largest hull part in the cmp. What do you mean exactly?

But nobody has made surs with Type 5 meshes and many surs do work fine, including many of mine, for ships that should have Type 5 surs.

Schmackbolzen wrote:

Thus, theoretically the geometry should not matter. Freelancer does not know simple geometries like spheres and boxes anyway, only special triangle meshes.

Well I disagree, most of the methods described for other games collision detection do test for spherical first, in order to speed up the operation, and then if the collision box is not a sphere the triangle poly checks are done - it’s described many times.

Should FL be different? Can anyone confirm the collision detection flow/methods in FL?

It is obvious that some of FL’s collision checks fail and others are fine, depending on the shape used to construct the sur part. As I have found, spheres are always fine as sur parts, cylinders are fine only when the ends are pulled out slightly, otherwise they fail sometimes, and boxes are unreliable too.

Deformed spheres (i.e. start with a sphere and scale it in any or all axes but still ensuring it remains convex) are also always fine for my surs.

For dock walls, instead of long tall boxes I found it better to use hexagonal cylinders and ensure the side corners are slightly pulled out, still resembling long box sections but slightly convex.

But then the dock floors and ceilings that I have made of boxes are all fine all the time. So because the box is unreliable this is why it is so difficult to guess the requirements.

Schmackbolzen wrote:

The goal is indeed a complete .sur converter, but that will take time, since I will have to create some sort of group selection dialog, where you can say which group should become what and all the other options there are. And I still have to fix the tree sorting and a glitch while merging the root node with the rest of the tree.

Should the user have so much control in deciding groups? If the converter will make a sur part for each cmp group this is perfect, or do you mean something else? Some groups will be detail parts or parts of the same component but with a different texture, and not needed for a sur, this will be hard to detect I think.

What do you mean by root node, and what is the tree please, I am guessing you mean like the UTF files, but I am not familiar with this term for surs?

You did mention the Type 5 parts as the root of the tree? I don’t understand that either?

And finally -

Adoxa tried to explain that these Type 5 surs don’t need to be over the cmp groups they protect, but this does not make sense yet, how can they be used for collision/hit detection if they are at the origin or have no offset from the origin so the checker can work out collisions/ hits? Can you explain that better for me?

Thanks.

A picture is worth a thousand words…

If only I had a picture

I think we have a few misunderstandings and maybe even bad formulations by me (since I am German). Plus, I have the abstract programmers view, which could make things even more complicated.

Let me try to clarify some things:

I use the word triangle, since Freelancer does not know other polygons than that concerning the .sur files (that might apply to .cmp etc. files as well). You can also call it a face. It’s the same.

Concerning the convex/concave stuff: Yes, it is of course defined and there is no other interpretation. What I meant is, that because of the behavior of Freelancer, you concluded, that concave is “bad”. Well, the reason why it might have caused problems is, that Freelancer needs a triangle counterpart which does not exist in every geometry. Thus, convex works.

Concerning the type 5 sur parts: This defines the bounds of the object. If you use adoxas sur importer he called it “root_bounds” or something like that. You don’t need it if you only have one part (thus, the tree is only the root node).

I don’t know how Freelancer handles collision detection exactly. From my point of view right now, there might be simple geometry involved for tree navigation. But not to the point where it actually collides. Who knows if we will ever know

The way I will design the GUI for the converter will be a work in progress, depending on how I think it’s easiest to use. I can change it later anyway if something works not the way I wanted it.

I don’t think it makes sense to explain my current understanding of how Freelancer handles the sur format, until I got it working without any errors. There is still the possibility, that’s all a coincidence that it works (even if it does look really good right now). If my theory proves to be correct, you can expect a documentation from me (and probably even a picture :P).

I hope that cleared things up a little