Since it’s been around 9 months since I finished writing the code for this whole project, and the people I originally wrote it for left the Discovery development team just after it was finished so they never got started using it, I think it’s past time to release this project to the community at large.

This thing is essentially a web-based multi-user development environment for adding and updating the text in all the ingame resources; if it goes in the .dlls as a string table or html resource it’s editable here. The goal is not to mirror the functions of FLDev or any other standalone programs that exists, more to augment them and allow for collaborative development from multiple people at the same time, all while not requiring any of them to know anything about the interior workings of the format.

What it provides is:

• wiki-style editing system

• full-text searching like Explorer

• editable permissions for classes of resources if your team is segmented

• very deep inter-linking of resources based on ingame usage

• custom links can be added between resources

• editing interface with bbcode, rich-text, or raw XML, and converters and preview for all

• import and export with FLDev’s .ini format for eventually writing to a .dll (automatically verifying XML the whole way so you’ll never see another malformed card in the bars)

• and more!

For a demo, try out a testing version on my server, using the login info of demo/demo which will give you full read rights but no editing. You’ll have to excuse the self-signed ssl certificate warning, the devs wanted this to be secure and I didn’t feel like paying verisign for something that I can do myself. All of the content there was ripped directly from the Discovery 4.85.3 mod, with the resources from the .dlls and the linking information from parsing the .inis (scripts are included for all of it)

It’s written in php/MySQL with jQuery to make the interface nice, and while most of it is universal to any mod, there are a few things (like the ID class) that are specific to disco, but can be readily modified (changed, removed, or more types added) for any dev team who would like to use it. I’d like to thank mwerte for telling me to make this (and then quitting halfway through), Tazuras specifically for doing a lot of the interface design with me, before he left, Sovereign for testing as well (before leaving), Dusty Lens for some quick testing before leaving the infocard dev team, and really the whole group of them for asking for this but leaving me hanging for a year after I did it for them just to ensure that it would never be used but it would still waste a few months of my time.

If any dev team would like to use this, just let me know and I can put together a zip file with all the scripts, I hesitate to just provide that right now because there must be some bugs left because it was never stress tested.

adoxa, FRC looks neat, I’m sure it will be useful for devs.

For people who don’t deal with the dlls themselves, lets say people who are just the content generation part of a dev team and all just submit resources to the final person who puts them into a dll, I’ve got some things that might make life a bit easier. Because even though many of us can look at raw code and visualize what it will look like, it’s sometimes nice to not have to deal with the low level stuff, and a real preview is always nice.

Here is a set of 3 related functions, all using the XML translator class I posted above, that are an extremely simple copy/paste/click interface for editing cards on the fly and generating valid xml for them. These are by no means intended to be used for major batch jobs, that’s a whole different kettle of fish, but for quick validation I’ve already found them useful myself.

#1 - XML Preview - given raw XML, will validate the syntax and display it in HTML how it will look ingame
#2 - BBCode Editor - using bbcode, will generate XML from the bbcode and show the in-game preview
#3 - Rich-Text Editor - edit rich-text cards, generate XML, and preview. I’ve tested it to work for copy/pasting formatting from a MS Word 2007 document in the current version of Firefox with Windows XP and 7, and it handles paragraphs, color, and sometimes alignment too. Other combinations of OS/Browser/Text Editor may or may not work for importing directly, but it will correctly handle any formatting you give it on the page itself. (IE8/FF/Opera/Safari, Chrome doesn’t have the API last I tested)

I’m not trying to step over your feet here adoxa, but when working with infocard people in the Disco team, they seemed to like these for the ease of use. If someone finds bugs that aren’t already listed, feel free to send me a PM here.

BTW, they are not NUL characters, they’re the high bytes of a single UTF-16 character. It should really look like FEFF 003C 003F … You got away with it these, but there are a few actual Unicode characters used, which you’d miss. FRC takes care of it all.

Right, they are the high bytes, when interpreted as UTF-16. I’m fully aware of what little-endian means here. However, when the string is interpreted as ISO-8859-1 (which some earlier programs do, especially non-freelancer-specific resource viewers), they are NUL. I guess I didn’t make that clear in my first post, they are useful parts of the string when you read it correctly.

There are a total of 5 or 6 characters in the entirety of the vanilla xml resources that use the high bytes, mainly squiggly quotes and accented letters. The PHP scripts I wrote to dump the resources to a SQL database correctly handle real UTF-16LE, but I store it all internally as UTF-8 because it takes half the space for 99.999% of the data, is able to handle every character used in any resource I’ve ever seen, and uses a second byte only when needed.

After quite a bit of reading through the hex of the dll files for my various php-based infocard tools, I’ve found a little issue with nearly all non-vanilla infocards, specifically the part where they have a period at the start of the string, before the opening of the After comparisons to vanilla cards and a bit of research online and talking to people on the Disco dev team, I feel I should share this little tidbit with everyone (especially whoever wrote FL-ids and/or FLDev).

We’ll start by looking at the raw hex code of a vanilla card:
The resource index, language, length, and offset index of the start of the card are all coded in the dll (as the built-in dll tools will do for you), so we’ll just look at the start of the string itself, starting at the indicated offset.

FFFE 3C00 3F00 7800 6D00 6C00 2000 7600
6500 7200 7300 6900 6F00 6E00 3D00 2200
3100 2E00 3000 2200 2000 6500 6E00 6300
6F00 6400 6900 6E00 6700 3D00 2200 5500
5400 4600 2D00 3100 3600 2200 3F00 3E00
```which when read as ISO-8859-1 text looks like: (with the character \x00 "Null" being removed)

> ÿþ

Now, a card from Discovery.dll:

2E00 3C00 3F00 7800 6D00 6C00 2000 7600
6500 7200 7300 6900 6F00 6E00 3D00 2200
3100 2E00 3000 2200 2000 6500 6E00 6300
6F00 6400 6900 6E00 6700 3D00 2200 5500
5400 4600 2D00 3100 3600 2200 3F00 3E00

> .

You can see the difference - the first two bytes are changed from "FFFE" to "2E00", where \xFF and \xFE alone look like junk characters and \x2E is a period. Why was this done?
From talking to the disco infocard team, the very simple answer is "We stripped the junk characters because they don't work when posted to a forum, but if there was nothing there Freelancer ate up the < and all the rest of the infocard was output as a text string and you saw XML everywhere, so we just added something to fix it."

The problem with this is that while it does technically work, that's only because Freelancer itself is somewhat fault tolerant and when it finds invalid data it just assumes the default character encoding. Also, XML strings are not allowed to have text outside the tags, so ".
Now, all this and I have yet to say what the \xFFFE actually is, so here it is:

> A byte order mark (BOM) consists of the character code U+FEFF at the beginning of a data stream, where it can be used as a signature defining the byte order and encoding form, primarily of unmarked plaintext files. Under some higher level protocols, use of a BOM may be mandatory (or prohibited) in the Unicode data stream defined in that protocol.
> (from http://www.unicode.org/faq/utf_bom.html#BOM)

Without going too far into the whole text encoding thing, please see http://en.wikipedia.org/wiki/Byte-order_mark#UTF-16

Basically, Freelancer requires XML text stored in the HTML resource type of DLLs to have a byte order mark at the beginning of the string so it knows the "endian-ness" of the string. When the correct mark is not available (i.e. when a period is substituted), it seems to fail gracefully to expecting UTF-16LE, which is the default for all the vanilla cards.

So, the point of this post: FLDev, FL-ids, and anyone else who has a program that writes dlls for Freelancer, would you please have your tools add the Byte Order Mark to the beginning of the resource instead of just writing UTF-16LE without it? For legacy and compatibility reasons it might help to detect a period before the
-Chris
(Thanks for Sovereign for listening to this over skype and then saying "well, how come nobody else knows about it?" and reminding me to post here.)

After a week or two of work on a web-based infocard development ‘application’ for the Discovery dev team, I have a part of that code that I want to release to the dev community at large, and I hope that it is useful to someone.

A php class that facilitates working with XML infocards:
http://cshake.homeip.net/disco/class.FreelancerXML.phps

Features:

• Input and output XML card strings, user-specified encoding

• Input and output bbcode-style format

• Validates input XML infocards, giving details on any errors

• Outputs valid XHTML that shows exactly what the infocards will look like in-game (when given the blue background)

• Input and output HTML for use in, and generated by, browser-based rich-text editors (compatibility issues with adding new paragraphs or linebreaks in IE8, no support for text justification in IE or Opera, everything else works fine in those, everything works in Firefox 3.6)

• Fully usable in both 32- and 64-bit environments

The BBCode translator works for all possible XML tags, and will produce functionally identical infocard markup when converted back and forth an unlimited number of times.
The supported ‘bbcode’ tags are:```
[b] [i] [u] [align] [font] [color]

The XML writer will use the ‘pretty’ human-readable color=“red”, bold=“true” style TRA tags by default, but will also output the ‘raw’ data/mask/def style tags if desired. In the case of reading in a ‘pretty’ style infocard, the output will be identical (even if converted to bbcode and back). When reading in a ‘raw’ style infocard, the output will be identical if not converted to bbcode, but if it is converted the mask and default values may be slightly different, but will function identically in-game. This comes from me always masking all the color bits when a color is specified, and always masking 0xF8 when a font is specified, while 0x38 (font numbers 0-7) is usually used in the vanilla cards.

The code is (I believe) fully commented and readable. Except for the use of variable-type arrays it should be easy enough to port to C/C++ if the FLDev people want to improve their ‘bbcode’ editor.

Thank you very much to adoxa for the <tra> explanation here!

[edit]6/10/10 - did some fixes, changed the link to the latest version instead of a specific one.[/edit][/color][/font][/align][/u][/i][/b]</tra>

Very nice description, and makes almost complete sense to me.

I do have two questions regarding the color specification:

When you say <number>, is that the numeric equivalent of the hex value used in #RGB, or is it the number that would be the color component of the data=“0xBGRA” field? (I’m assuming the second, but want to verify)

Similarly, when it is ‘0x<hexnumber>’, I’m assuming that’s the same thing that would be in the data field, except ignoring the last (attribute) pair?

Also,

If you choose to use a number for the color, one is subtracted from it, for some reason.

You mean from each color individually, or from the whole number?

I ask this instead of trying it myself because I’m working on an XML to HTML converter that verifies the syntax and shows what it would look like ingame in the browser, and those two are the only remaining unknowns in the script.</hexnumber></number>

adoxa wrote:
http://the-starport.net/freelancer/wiki/index.php/MDB:mbases.ini

Oops, turns out that search box on the top left covers everything but the wiki unless you specifically tell it to. Sorry about that.

So I’m looking through MBases.ini, and there are the two things related to knowledge (know =, knowdb= ), but I’m having trouble figuring out the parameters.

So far I figure that it’s:

know = (offer ids_info), (?), (price?), (required level?)
knowdb = (nickname of object to give info about)

The question is such:
What is the second number (that I’ve marked with a question mark)? It’s always a resource number within NameResources.dll, but the ids_name resource at that index is empty for every instance I’ve checked so far.

The only think I can think of so far is that the second index is what to append to the in-game offer after the price, but since they’re all just a space I haven’t been able to verify. If so, why are there hundreds of essentially empty strings in the dll when they could all just point towards a single index?

Also, is the assumption about the 3rd and 4th parameters right?

Thanks in advance!

Hi all.
I’ve been playing disco for 7 months or so now, and started essentially reverse engineering the ini files and more shortly after.

I’m now a sysop on the wiki there, and have written a large amount of php/sql that in effect does most of what flstat (and more recently, the dll search functions of flidref) does, and then converting it to wikicode output for very quick updates of the wiki. I also wrote some helper scripts for the povray exporter from milkshape and have a nearly 1-click workstream for rendering ships and weapons, which can be seen as all the ship pictures on the wiki right now are mine.

I’ve been talking to Tazuras, mwerte, and Dusty and have somewhat found myself getting into the dev side of disco, and Tazuras told me to come over here as that’s where all the other FL coders are. I’ve got experience with a lot of php/mysql, and can do a good amount of c/c++, but have no experience with GUIs in anything but php/html.

I guess I play sometimes too.