Attention all programmers! Help needed please!
-
What encoding is used for the files?
Which files would I have to parse for this?
shiparch.ini - which links to the cmps/mat files? Is that all?
Should it be any clash between any cmp and mat (ie mat file for one ship and another share same whatever it is?) or what? -
**To reduce the workload I would copy over your ship and solar folder into a new folder. Then run the CRC tool on those files. It will go through both cmp and mat files and get all of the names and the hash number.
Only problem I can see is if it compiles the data into a tree and simply ignores duplicates. Thats very easy to do with a tree structure.**
-
Well its going to be a slog doing it that way Lancer. I’ve been doing one by one using FL Model Cloner and UTF Edit, takes several minutes to check each pair and several more if I need to change them.
Chips - I think the .mat file is fairly easy, it opens cleanly in UTF Edit and the material count and material names are easily seen. But the materials are not obvious in the .CMP file in UTF Edit, I’ll do some chasing and update as soon as possible.
What we need to do is (hope I’m not teaching grandpa to suck eggs here!):
Open a .cmp file, scan for material names. When first is found, add 1 to material counter. Check the material name in the materials table (see below) if it has already been found before in another .cmp file, if yes then output a message “DATA\SHIPS(shipfolder.cmp filename): Duplicate Material (material hash) is also used in (DATA\SHIPS(shipfolder\previous .cmp filename1)(, DATA\SHIPS(shipfolder\previous .cmp filename2),etc.”
Then: Check if the material exists in this .cmp file (because some modders put them in here instead of in a separate .mat file). If found, then store the material name in a materials table along with the .cmp file name, and look for the next material name and repeat. If not found in the .cmp file, then open the .mat file for the same ship and look for the same material. If found, store the name of the material along with the .cmp file name and look for the next material. If not found, output an error message “DATA\SHIPS(shipfolder.cmp filename): Material (material hash) not found” and go on to next material. If end of .cmp file then look for surplus materials in the .mat file. If no more, compare material count with material count in the .mat file, or if more exist then delete them and correct the material count in the .mat file. Open next .cmp file, zero material counter, and repeat.
I’ll post back on the .cmp and .mat file material locations if I can make them out….
-
**Don’t forget the mesh references the material as a CRC value in the VMeshData node. This has to be correct as well otherwise you will get a white mesh. This data is stored immediately after the header, for example…
; DB Software - CMP Editor Script [VMeshData] MeshType = 1 SurfType = 4 ;MeshCnt = 14 ;IdxECnt = 16110 ; Indices = 5370 FVF = 274 ; DirectX FVF XYZ|NORMAL|TEX1 ;VtxCnt = 4015 ; ************* Mesh List ; Format MeshN = Material Unsigned CRC, Start Vertex, Referenced Vertices, Referenced Indices ; Format Mesh = Material Nickname, Start Vertex, Referenced Vertices, Referenced Indices MeshN = 3814968014, 0, 700, 2691 ; 0, Index = 0 <-- 3814968014 aka '8_topmap2_glow' in my Sovereign.mat file (material section) MeshN = 438861915, 0, 272, 1047 ; 1, Index = 897 MeshN = 156482488, 0, 292, 1188 ; 2, Index = 1246 MeshN = 4116083112, 0, 695, 3036 ; 3, Index = 1642 MeshN = 75767902, 0, 562, 2082 ; 4, Index = 2654 MeshN = 197653055, 0, 951, 4062 ; 5, Index = 3348 MeshN = 3949694050, 0, 79, 306 ; 6, Index = 4702 MeshN = 4200320777, 0, 55, 144 ; 7, Index = 4804 MeshN = 536327212, 0, 83, 324 ; 8, Index = 4852 MeshN = 350155985, 0, 31, 72 ; 9, Index = 4960 MeshN = 450581546, 0, 79, 306 ; 10, Index = 4984 MeshN = 4264919517, 0, 51, 156 ; 11, Index = 5086 MeshN = 157538445, 0, 100, 540 ; 12, Index = 5138 MeshN = 4145232669, 0, 51, 156 ; 13, Index = 5318 ; ************* Face Index List ; Format Index = Vertex 1, Vertex 2, Vertex 3 Index = 0, 1, 2 ; 0 Index = 3, 0, 2 ; 1 ```** -
Good on ya, Lancer!
Chips I’m having problems trying to analyze the .cmp file for you, I can’t see the offsets/pointers to the location of the mesh where the material name is held (in hex, reverse-byte order), hope Lancer or someone can do this for us?
FL Model Cloner can do this already if you have access to the source? It lists the hashcodes of all materials in the .cmp file, and when you identify the .mat file it hashes the plain material names and compares them with the hashcodes in the .cmp for a match.
I’m not sure if I can be of much help here but I will try:-
The .mat file seems fairly simple to my untrained eye:
Bytes 0-3: UTF Header
Bytes 4-7: Dunno, 0101 0000 - Editor/Version?
Bytes 8-B: Pointer to start of unidentified block whose bytes 17-14 are the pointer to the end of the file (address of last byte of the file+1). Block seems to consist of chunks of either 44 or 88 bytes, with a sequence of some kind.
Bytes C-F: Pointer? Dunno.
Bytes 18-21: Pointer to the start of the node name table.
(Node names are only given once, for example Dc and Oc which are the names of several nodes, but the name is only listed once so there has to be a pointer somewhere to the name list for each node in order.)Bytes 24-27: Pointer to node values table. In standard FL .mat file this actually points to a delimiter FFFF FFFF before first node’s value. In UTF-Editor saved .mat file the pointer points to the first node’s value. Value contents and data lengths dependent on node type.
Sorry I can’t help more.
-
Cheers, I will read through this shortly as January is a very busy month for me (probably spill into Feb as well) and so I cannot promise swift progress, or even when I’ll start!
My responses may be a little slow at times, other times I may respond quickly. However, if you don’t hear anything for a fortnight, then don’t be afraid to give me a poke. I don’t bite, and at worst will just say “too busy, come back later”
-
Regarding CRCTool, I’ve just put up a new version on my site (is Anton still around?). It includes quite a few more names from UTF files (hardpoints and damage points, stripped extension and timestamp). That has the disadvantage of duplicated codes, but with the advantage of showing where each name is used. I’ve used the trial version of C++Builder, so you have less than four weeks to make feature requests and bug reports…
-
Erm…
“Application failed to start because CC32100MT.dll was not found”
-
Improved CRCTool some more. Now recognises ini values containing spaces and greatly improves search speed (although the first search is still slow).
Now, to bring it back to topic. Various materials are already duplicated (e.g.: engine_grill01), so what distinguishes “good” duplicates from “bad”?
