CMP to SUR Conversion Tests

Yes, that is correct. I can convert any geometry you like to a .sur file

Thats awesome! The real breakthrough regarding .surs IMHO

Keep it up

So - what is exactly the difference between your converter and the sur builder?

Well the .sur builder tries to build one convex hull around each mesh group, whereas my converter just converts the whole geometry without any changes. Meaning that you have to do optimizing of the geometry yourself (well it will work without, too. But then there is more calculation for collision needed). The advantage is, that you get exact fitting .sur files of any geometry you can come up with.

The easiest thing to do is just a polygon reduction so that you get a geometry which is close enough to the original, but has as few polygons as possible. Nearly all modeling tools can do that.

If you have a real complex mesh and want to simplify collision calculations you can just build your own collision mesh out of simple boxes and spheres and convert it.

and is it, that this mesh will be compatible with any visual geometry? cuz we heard something of “sur to model attachment” and stuff like that. although the flmodeltool made every vanilla sur compatible with any custom mesh. does it work somehow similar in this point?

That depends on how the .sur file is generated. You can attach certain groups in the .sur to corresponding groups/nodes in a .cmp file. The ones I create for now are all attached to the root node, which every .cmp file should have. But it is possible to change all that.

w0dk4 wrote:
Thats awesome! The real breakthrough regarding .surs IMHO

Keep it up

Thanks, I will

would be nice if the sur parts attach to the model part they are built of. this way we wouldn’t need to worry about e.g. animated parts.

Schmack: You are confusing me - not surprising, I’m easily confused…!

No offence meant, I am only keen to understand…

Do I understand correctly, that your generator makes close-fitting surs tagged Root?

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

(By the way some of my bad surs worked in game under constant destruction for more than 20 minutes before crashing the game, so testing needs to be at least 30 minutes of NPCs using the sur getting destroyed by other NPCs).

Second - why are you making your own, since there is the already sur-builder almost completed? This is re-inventing the wheel?

General: - I found this info on collision solids, maybe we can learn from it?..
http://www.panda3d.org/wiki/index.php/Collision_Solids

I understand that process, but we don’t know how or what exactly FL is testing. Do we?

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Arent vanilla surs concave, too?

The format has been mostly reverse-engineered, but Lancer Solurus’ utility is not open sourced and he’s encountered a block and legitimately is prioritizing his game over finishing the SUR application.

Therefore the only way for Schmack to create a SUR generator is to write his own.

And yes, w0d is right; vanilla SURs are concave in many places.

StarTrader wrote:

Do I understand correctly, that your generator makes close-fitting surs tagged Root?

Yes they are close fitting, depending on the geometry you convert. For now everything is tagged as root, but that can be changed.

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Convex geometries are better in general for collision detection speedup. Freelancer does some silly trick to construct concave geometries, which I just also do. I tested a model with over 3000 polygons and it works without problems or notable performance reduction.

(By the way some of my bad surs worked in game under constant destruction for more than 20 minutes before crashing the game, so testing needs to be at least 30 minutes of NPCs using the sur getting destroyed by other NPCs).

Since our mod has an integrated modupdater I used the opportunity to upload the first .sur file I attached here for testing. No one has had a crash so far.
Although the tree is not created correctly yet, I had no crashes, but collision detection failures on some complex geometry depending from which direction you are nearing. This all fits perfectly in my theory. I will have a look at the .sur files the sure file generator creates to check whether it makes errors.

Second - why are you making your own, since there is the already sur-builder almost completed? This is re-inventing the wheel?

Friendlyfire answered that quiet well.

General: - I found this info on collision solids, maybe we can learn from it?…
http://www.panda3d.org/wiki/index.php/Collision_Solids

Hard to say, since there are many ways to solve collision equations.

I understand that process, but we don’t know how or what exactly FL is testing. Do we?

Not exactly, but since my theory seems to be correct I got a pretty good idea what it does.

w0dk4 wrote:

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Arent vanilla surs concave, too?

No, I’ve not seen even one. All individual sur parts that I have checked are convex.

They may look concave when they are in place next to each other but they are not individually concave.

Can you point me to one? It could be that it has a dfferent layout.

I have made great landing bay surs for big dockable ships using convex primitives for the floor, walls and ceiling, and sometimes for the forward bulkhead, all assembled together, and declared them in concave.ini and they work 100%, all the time. Single-part convex surs didn’t ever work for me.

CzW and Mirkha’s hole-in-base-station sur using the Boolean modifiers had no collision detection in certain places in it, so it is not conclusive that concave are ok…

We have been making convex sur parts for a long time, since concave ones caused crashes.

This could still be because the “Type 5” sur parts are missing.

Schmack: I am looking forward to using your generator.

The link I posted above takes us to see what shapes are used in collision detection and for what purpose, there are references to using spheres, tubes (rounded-ended cylinders), it seems only the sphere is 100% in reliable testing, and the “tube” is second. Other polygons allow slip-throughs.

But with spheres I have not tested to see if the collision is on the sphere or on the hull when the sphere is bigger than the model. Logically it would test on the sphere so it’s not useable for large ships again. We know the deformed convex “tube” or squashed sphere works very well. Boxes and others are unreliable, and this is what we are seeing with boxes and non-convex shapes - as well as the more definite crash!

Anway, back to this subject…

Great work all of you guys, and…

Schmack, we hope to have a fully-working “on-the-button” multi-cmp-group converter with optional “shield-bubble” and all Type 5 sur parts from you.

I hope this is also what you have in mind?

Take your time, no rush - 3-4 days is fine! ROFL!

Oh - one more big (bloody important) question in the light of LS’s bombshell revelation and sudden disappearance from this scene…

Will you be allowing us to have and use the generator / converter when it is finished? ?)

We don’t need the source if you don’t want to release it, just a working executable for intel PC that works on single-core cpus please!

Last idea for today…

Would it be a better idea to start a new thread instead of continuing with this one as it is for sur-builder primarily?

Thanks.

CzW and Mirkha’s hole-in-base-station sur using the Boolean modifiers had no collision detection in certain places in it, so it is not conclusive that concave are ok…

exactly, sometimes that works sometimes not …
and i never figured out why

I vote for maintaining a single thread. The most important thing being developed here is information about the SUR format.

Whether the final product(s) is/are exporters or other tools, there does not appear to be any single source of reliable information on what makes a “good” SUR.

Let’s keep the testing and reporting in one place until we get it figured out.

StarTrader wrote:

w0dk4 wrote:

The first query is - they may not work correctly in the game, since your converter will generate concave surs if the model is concave, this is what we are trying to avoid, no?

Arent vanilla surs concave, too?

No, I’ve not seen even one. All individual sur parts that I have checked are convex.

They may look concave when they are in place next to each other but they are not individually concave.

Can you point me to one? It could be that it has a dfferent layout.

I guess that depends on the definition about concave/convex. Every triangle needs to have an opposite counterpart. If that is missing, you have to split it up and create one. That’s how Freelancer and my converter does it. That’s the easy part. Look at solar\misc\space_industrial.sur and you will see what I mean.

This could still be because the “Type 5” sur parts are missing.

That’s the reason for your crashes and strange behaviour. It is the root of the tree. If it’s missing Freelancer makes wrong decisions on collision calculations (for shield .sur files I have to have a look yet how it’s done there).

Schmack: I am looking forward to using your generator.

The link I posted above takes us to see what shapes are used in collision detection and for what purpose, there are references to using spheres, tubes (rounded-ended cylinders), it seems only the sphere is 100% in reliable testing, and the “tube” is second. Other polygons allow slip-throughs.

But with spheres I have not tested to see if the collision is on the sphere or on the hull when the sphere is bigger than the model. Logically it would test on the sphere so it’s not useable for large ships again. We know the deformed convex “tube” or squashed sphere works very well. Boxes and others are unreliable, and this is what we are seeing with boxes and non-convex shapes - as well as the more definite crash!

Freelancer also does precalculations for collision detection. If not, you would just fly through any object, which is small enough depending on your speed. Thus, theoretically the geometry should not matter. Freelancer does not know simple geometries like spheres and boxes anyway, only special triangle meshes.

Great work all of you guys, and…

Schmack, we hope to have a fully-working “on-the-button” multi-cmp-group converter with optional “shield-bubble” and all Type 5 sur parts from you.

I hope this is also what you have in mind?

Take your time, no rush - 3-4 days is fine! ROFL!

Oh - one more big (bloody important) question in the light of LS’s bombshell revelation and sudden disappearance from this scene…

Will you be allowing us to have and use the generator / converter when it is finished? ?)

We don’t need the source if you don’t want to release it, just a working executable for intel PC that works on single-core cpus please!

I could even compile it for linux, since I am using freepascal + lazarus (yes, I am pascal fan ;)). It will run on any machine, since calculations are not that complex and I don’t waste memory like a certain other tool does (just kidding).

I am planning to release it, first without any code, later maybe with. Please understand that I don’t like the idea of unpolished software, so if it hast certain bugs which to my opinion are not acceptable, I won’t release it until I fixed everything.

The goal is indeed a complete .sur converter, but that will take time, since I will have to create some sort of group selection dialog, where you can say which group should become what and all the other options there are. And I still have to fix the tree sorting and a glitch while merging the root node with the rest of the tree.

I have some comments/questions please…

Schmackbolzen wrote:

I guess that depends on the definition about concave/convex.

Well one online definition of a concave shape is “A shape that curves or bends inward”.

The mathematical definition is probably much more scientific than this, but here is mine specific to low-poly models made of faces (triangles)… in a concave shape, a plane extrapolated on one or more faces (i.e. the plane made by pulling out the triangle to infinity in both directions) will cut into another face on the shape.

In a convex shape, no face plane will cut into another face on that shape.

There can’t be any misunderstanding of this, surely?

Schmackbolzen wrote:

Every triangle needs to have an opposite counterpart. If that is missing, you have to split it up and create one. That’s how Freelancer and my converter does it. That’s the easy part. Look at solar\misc\space_industrial.sur and you will see what I mean.

I’m not sure what you mean by “triangle”. Do you mean a face? The model you mention does have many sur parts made of only 2 triangles, they are interpreted as Type 2 and hashcode “00000000” and I’m not sure of the significance of these, it seems odd, just planes made of two faces?

Or do you mean a mesh made of triangles (faces)? And what do you mean by opposite counterpart? In the sur file? Yes, we know this, that every sur part must have a counterpart cmp group to be attached to in order to function. Is this what you mean?

If you mean triangle = face, in the sur, then I do not understand why each face must have a counterpart.

Schmackbolzen wrote:

That’s the reason for your crashes and strange behaviour. It is the root of the tree. If it’s missing Freelancer makes wrong decisions on collision calculations (for shield .sur files I have to have a look yet how it’s done there).

I thought the root sur part normally corresponds to the largest hull part in the cmp. What do you mean exactly?

But nobody has made surs with Type 5 meshes and many surs do work fine, including many of mine, for ships that should have Type 5 surs.

Schmackbolzen wrote:

Thus, theoretically the geometry should not matter. Freelancer does not know simple geometries like spheres and boxes anyway, only special triangle meshes.

Well I disagree, most of the methods described for other games collision detection do test for spherical first, in order to speed up the operation, and then if the collision box is not a sphere the triangle poly checks are done - it’s described many times.

Should FL be different? Can anyone confirm the collision detection flow/methods in FL?

It is obvious that some of FL’s collision checks fail and others are fine, depending on the shape used to construct the sur part. As I have found, spheres are always fine as sur parts, cylinders are fine only when the ends are pulled out slightly, otherwise they fail sometimes, and boxes are unreliable too.

Deformed spheres (i.e. start with a sphere and scale it in any or all axes but still ensuring it remains convex) are also always fine for my surs.

For dock walls, instead of long tall boxes I found it better to use hexagonal cylinders and ensure the side corners are slightly pulled out, still resembling long box sections but slightly convex.

But then the dock floors and ceilings that I have made of boxes are all fine all the time. So because the box is unreliable this is why it is so difficult to guess the requirements.

Schmackbolzen wrote:

The goal is indeed a complete .sur converter, but that will take time, since I will have to create some sort of group selection dialog, where you can say which group should become what and all the other options there are. And I still have to fix the tree sorting and a glitch while merging the root node with the rest of the tree.

Should the user have so much control in deciding groups? If the converter will make a sur part for each cmp group this is perfect, or do you mean something else? Some groups will be detail parts or parts of the same component but with a different texture, and not needed for a sur, this will be hard to detect I think.

What do you mean by root node, and what is the tree please, I am guessing you mean like the UTF files, but I am not familiar with this term for surs?

You did mention the Type 5 parts as the root of the tree? I don’t understand that either?

And finally -

Adoxa tried to explain that these Type 5 surs don’t need to be over the cmp groups they protect, but this does not make sense yet, how can they be used for collision/hit detection if they are at the origin or have no offset from the origin so the checker can work out collisions/ hits? Can you explain that better for me?

Thanks.

A picture is worth a thousand words…

If only I had a picture

I think we have a few misunderstandings and maybe even bad formulations by me (since I am German). Plus, I have the abstract programmers view, which could make things even more complicated.

Let me try to clarify some things:

I use the word triangle, since Freelancer does not know other polygons than that concerning the .sur files (that might apply to .cmp etc. files as well). You can also call it a face. It’s the same.

Concerning the convex/concave stuff: Yes, it is of course defined and there is no other interpretation. What I meant is, that because of the behavior of Freelancer, you concluded, that concave is “bad”. Well, the reason why it might have caused problems is, that Freelancer needs a triangle counterpart which does not exist in every geometry. Thus, convex works.

Concerning the type 5 sur parts: This defines the bounds of the object. If you use adoxas sur importer he called it “root_bounds” or something like that. You don’t need it if you only have one part (thus, the tree is only the root node).

I don’t know how Freelancer handles collision detection exactly. From my point of view right now, there might be simple geometry involved for tree navigation. But not to the point where it actually collides. Who knows if we will ever know

The way I will design the GUI for the converter will be a work in progress, depending on how I think it’s easiest to use. I can change it later anyway if something works not the way I wanted it.

I don’t think it makes sense to explain my current understanding of how Freelancer handles the sur format, until I got it working without any errors. There is still the possibility, that’s all a coincidence that it works (even if it does look really good right now). If my theory proves to be correct, you can expect a documentation from me (and probably even a picture :P).

I hope that cleared things up a little

Heheh, thanks for your patience, Schmackbolzen.

A small digressionary break first…

I am actually embarassed because your fluency in English is amazing, I am very proud of all of you guys whose first language is not English. It is hard to understand the slight differences in meaning or double-meanings of the same word that the English language has, that only natives will know automatically - you others have to learn them and that is hard, I know.

It is because the English language is limited. Greek has far more words that precisely describe their meaning, so there is little misunderstanding, and probably German and French are similarly rich languages? For example “sagani” and “tigani” are different types of large metal pan (the “sagani” has two metal semi-circular handles at opposite sides of the pan, like ears, so it can be put in the oven, and “tigani” is a normal frying pan with one long straight handle or sometimes having a long handle and a small supporting “ear-like” handle at the opposite side) and “saganaki” and “tiganaki” are the corresponding little pans of each type! And “souvla” is a large skewer, and “souvlaki” is a little skewer - so we know if we are having a large oven-baked dish or a large kebab, or a small oven-baked dish or a small kebab! Easy, huh!? And then the sexual double-meanings are fun - “angouri” is a cucumber, and “angouraki” is a little cucumber - most males have one of these! So don’t order “angouraki” with your salad in Greece, ask for “angouri”! lol ( but be careful, in Cyprus it’s the opposite!! )

I have often admitted that Brits are the world’s worst linguists - there were plenty of language opportunities at school - French as the main second language, then German and Spanish were all available to us - as well as both of the dead languages Latin and Ancient Greek that only Scholars would use. Ancient Greek is nothing like the Greek that I know!

What caused the problem in my time was that instead of trying to develop our language abilities at school, the others just laughed at us and derided us when we made mistakes in class, so we spoke less and less instead of more and more. To make matters worse, although I studied and very much enjoyed French for seven years, to GCE ‘A’ level ( General Certificate of Education Advanced level, the highest school certificate ) and my grammatical knowledge was very good, top of the class for several of those years, my teachers were Englishmen and after learning from them I could not speak with a correct emphasis or tone, nor understand when I was spoken to by a French person!

OK, enough of a side-track…

The only misunderstanding I have now is still this one… “Freelancer needs a triangle counterpart which does not exist in every geometry.”

I don’t understand what you mean by a triangle “counterpart” - to me this means a twin, or an associated face in another shape? And so I don’t understand what counterpart a face would need? It could mean either the next face attached to the first one, or an opposite face at the opposite side of the same shape, or it could mean a matching face in the .cmp - but none of these make sense to me so you must mean something else?

Thanks again for clarifying and for your patience, I am really keen to understand in case I can assist in any way, or learn what is causing the problems that I have made with so many surs, even fully convex ones.

One other point that I should remind us of here - Bejaymac stated that if the sur parts are exported in a single group, then only the root must be convex, and the other sur parts can be concave. If there is only one sur part then that part must be convex of course. But if the sur parts are exported in two or more groups, then each one must be convex.

I haven’t really got my head around that (means it doesn’t make sense to me) even after all this time. Because if the sur has several parts and it is exported as one group containing all parts, then all of those will probably make a concave shape when seen as a whole.

My method is to be sure that the cmp is in several groups, and I do not make more than one part per sur group, so I have ensured that each sur part is convex, and that each sur part also has a group of the same name in the cmp.

I don’t like to add dummy groups, so I simplify my sur parts and break up the ship into as many sensible groups as possible to have enough cmp groups for all of the sur parts while staying within the 18 groups ( of course this can be difficult for large ships but that has been overcome by our superstars! ).

Then I either export the sur parts as one part per group, or export the parts individually and then sur splice them. Sur splicing has been the most successful way for me until the sur builder now.

But the sur builder has problems geneating surs for some of my ships, and also I have not had the time to test the sur builder surs fully ( long time testing with NPCs vs NPCs using the sur ) - as I have said before, starting with vanilla surs I had clean overnight runs, then testing my surs one at a time, it has taken over 20 minutes to crash sometimes, but they have crashed. And so I test for a longer time.

Do you see now why I go to such long explanations as this one? It’s because the English language is limited and needs adjectives, similes, metaphors and also examples, all together, in order to clearly define one’s true meaning!

StarTrader wrote:
I have often admitted that Brits are the world’s worst linguists

Agreed… it seems that Britain is the only place on Earth where they do not teach English in the schools yet.

Even Google Translate cannot parse: You can’t have a proper knees-up without a Joanna.