PyFL - Python Freelancer Toolkit Framework
-
This is a work in progress and should be considered a ‘alpha’ release. Its probably best not create any massive tools with it, but still worth checking out if your a python coder.
I’ve also refactored and cleaned up alot of the code in the last week, so may have unintentionally introduced unexpected bugs in some functions (but I dont think so)–---------------------------------------------------------------------------------------
PyFL - The Python Freelancer Toolkit Framework
Author: Fenris_Wolf - YSPStudios, TekagisTreasure mod**** Please note this is NOT a stand alone tool. ****
It is designed to be a base framework for building tools, or to be used in a interactive python shell.
PyFL reads and parses ALL Freelancer ini files, and can manipulate the files writing changes back into them. It provides brief summaries of the contents (number of ships, guns, systems etc), and detailed summaries of your universe
(which factions own bases in each system, who owns other objects, and who has a npc presence)Its uses are pretty much unlimited when working in a interactive shell, or if you can write custom tool functions.
A few examples of the things I’ve used it for:
- Extracting all non-vanilla files from a mod into a new directory.
- Extracting files that have changed from a mods previous version into a patch.
- Finding all unreferenced unused junk files left over from previous mod versions.
- Resorting the contents of files: putting our 3.4mb mbases.ini, universe.ini and markets all in proper order.
- Nuking specific npc factions from the files.
- Cleaning out mbases.ini removing all npcs execpt vendors, and making sure vendors
have the same faction as the base. - Adding the ids names stored in the dlls as comments to files on the ids_name lines.
- Generating CRC lists of the mod files.
- Generating full navmaps for inserting in player files.
- Generating a .png image file of the universe navmap showing all jumps and systems
colored by local_faction (requires graphvis to be installed). - Reverse hash lookup.
- Error checking.
- Creation of a ‘mod aware’ FLHook bot (using Hooks tcp sockets) with a skype interface
and in game hook commands (including a ship garage command allowing for ships to be
stored and shared by characters on a account). - Embedding python directly into a custom FLHook build allowing for quick generation of new features and plugins, with full support of all of PyFL’s many features. (Currently a work in progress, but functional)
Some knowledge of python coding, you dont need to be a expert, basic knowledge is enough to make use of PyFL, but more advanced knowledge is required to write custom tool functions.
EDIT: old attached file removed, this project is now hosted on github
-
I should point out the parsing rule files were written many years ago for another program I wrote, and probably need tweaking and more modding experience that I possess. In some cases it may generate ‘false positives’ when error checking. The beauty of the system though is you can tweak the rule files to fit, eliminating most if not all false positives.
For those curious about how fast it parses ALL the ini files, heres my last run results on a win7 64bit, 2.2ghz dual core (it uses a single thread for parsing, so only 1 core) 6gb ram:
2840 files read (745947 lines), 95443 Sections, 617896 Keys and 1166083 Arguments Parsed (434 Errors)
8363 files referenced, 30471 times:
.anm = 11,
.ale = 612,
.3db = 1104,
.mat = 159,
.dfm = 204,
.txm = 102,
.wav = 1665,
.cmp = 611,
.ini = 2701,
.thn = 1112,
.sph = 81,700 Bases in 161 Systems, 97 Factions, 109 Ships 679 Weapons, 16 Mines, 39 Shields, 240 Powerplants, 338 Commodities
Total Time: 33.5420000553 secondsCross Reference Results: 486535 items, 125 errors, 3.18000006676 seconds
-
–---------------------------------------------------------------------------------------
KNOWN ISSUES AND BUGS:-
Freelancer.ini has fx\explosions.ini listed as both 'explosions = ’ and 'debris = '. The parser doesnt like this as it only reads a file once, and under a single group. Therefore, ensure ‘explosions = fx\explosions.ini’ comes first in freelancer.ini
-
Room:Room_Info:scene lines are not properly parsed due to only allowing one -o argument per rule line. The only effect is cityscape and planetscape files generate a error on some of these lines if a argument appears after the thn script. The rest of the line is fully parsed but the final argument is dropped.
-
Various ‘color’ keys in the ini’s support both byte (0 to 255) and float (0.0 to 1.0) values. The parser doesnt agree with this, and may generate errors. Tweaking your rule files will fix this if your only uses 1 style (byte or float, but not mixed)
-
Many various rule files need tweaking for better argument matching, currently they may generate false postive errors. The cross reference validation (-m|–match) rules expecially need tweaks as not all of them have been inputted yet.
-
When using DataFile.set() or IniFile.set(), if the value is a list (declared multiline in the rules), the new value list MUST be the same length as the old value list or a NotImplementedError exception is raised.
-
-m|–match rule option generates errors on robots without heads (space_costume = lines in system files)
-
-r regex rule option does not support spaces.
-
BINI compressed files not yet supported.
-
UTF files not yet supported.
-
Biniary hexediting on the exe files not yet supported.
-
Writing ids names and infocards to resource dlls not yet supported (reading is fully functional).
-
-
First few sample tools compiled into .exe format built using PyFL. No need to install python or any of the other requirements for these, just edit the config.ini and run
ErrorChecker - Freelancer ini file error checker
Performs in depth error checking on Freelancer’s ini files, ensuring all arguments are the correct type, and all cross referenced ini’s are correct Uses PyFL’s unique customisable parsing rule system.
ModExtractor - A reverse FLMM
Extracts all non-vanilla files from a installed mod to a separate directory. Also checks the previously extracted mod and copies any new or changed files and copies those to a patch directory.
The idea is to simplify the modding process, as files can be worked on ‘in place’ without having to remember whats changed, or working in a external directory and having to activate/copy your mod into your FL directory each time for testing. -
This sounds interesting (although I am not a Python fan).
Could you also write a collision checker of the generated CRCs? This can cause ugly bugs, since error only occurs when the affected names are loaded. Also duplicate group names in .cmp files etc can cause weird errors (within one cmp and also across all).
Maybe your error checker can scan for that, that would really be helpful. -
Not a bad idea I’ll definitely have to include that when I finish off the utf module (reading is currently supported but not fully tested).
Once thats finished off should be easy enough to write a function to cover that.
TBH though the whole error checking features are more of a side effect, the ini manipulation is the true goal, as we plan to have it embedded and responding to events on the server, changing base affiliations, system encounters and similar stuff, then automatically pushing out updates
