raP File Format - Elite

Caveat

If you are researching the nitty gritty of raP encoding for OFP or ArmA you are in the wrong place. This document is specifically for Elite raPified files on the Xbox. However, if you are not too familiar with this subject, the Introduction in the OFP version will help you best.

Changes for Elite

Two new entry types

/*extern*/ class thing {};
/*extern*/ class thing {};

Enhanced TokenName

name= true _variable (as opposed to name="variable";)

Non contiguous data file

Serious alterations to non-contiguous data stream. The data for class bodies is now located in a flle offset, as opposed to being directly after the class name.

Introduction

raP encoding for Xbox Files

Conventions

asciiz: a zero terminated ascii string.
byte:   obvious.
ushort: unsigned 2 bytes Intel Order (lsb first)
float:  4 bytes. as in not a double
long:   4 bytes
BisShort: 1, or 2 bytes see below

Mandatory PreReading

The Human

BI use c++ class syntax for almost everything connected with text (humanly readable) files. Anything configurable / alterable / makeable / editable by a player (author) involves creating a file or files consisting of class statements. The Mission Editor eg, creates several files for a mission consisting solely of class statements. Everything is defined or declared within classes. EVEN the cfg (configuration files) have implied class statements.

Class syntax can briefly (but accurately) be described as follows:

class thing [: InheritedClass]
{
  body
};

[: InheritedClass] is optional. It is heavily used in the config.cpp's of addons. If specified it must be referenced prior to being used.

The body of a class consists of

{
 #tokens and /or
 #arrays and /or
 #(embedded) classes and /or
 #nothing at all!  class default{};  is perfectly valid eg.
};

Any given file that contains classes, contains nothing else but classes. There aren't, classes, and ummm, err, ummm 'other' things.

The file itself is considered an implied class!

The very first statement of all files is

//class filename {

Everything within that file is embedded within the class filename {....};

The Engine

The class structure is used internally within the engine for all most game data and configuration. The engine holds this data in tokenised form for more efficient processing.

This internal tokenised data can also be held in a file (a savegame eg). But, it is a class structure regardless.

To improve loading speed many files within the ofp tree are pre-tokenised. They were 'converted' to hold the tokenised form of the same class statements.

In fact, the engine does not care which 'type' of file it is dealing with. A mission.sqm can be either. If it hasn't been pre-tokenised (eg contains text-readable class statements) the engine converts it on the fly.

Tokenised files have a magic signature as their first four bytes. "\0raP". The file extension is meaningless in this regard.

In the early days of ofp, when the campaign mission.sqm's came pre-tokenised, it was assumed wrongly, this was some attempt at encryption. They were also called 'binary' files because it was noticed that the tokenised form of a config.cpp was called config.bin (config.rap would have been far more appropriate). In fact, the engine will happily work with a config.cpp that is rap encoded.

Various utilities (tools) arrived on the internet in an attempt to decode them (and conversely, encode them). cpp2bin, bin2cpp, Coc binview are the most well known of these.

They are detailed here as 'raP' files because of their unique signature. This document refers to rapifying and de-rapifying files containing nothing but, class statements.

Xbox

Xbox raP encoding is an extension to Flashpoint PC raP encoding.

Overall Structure

4 byte raP signature (00 r a P)
20 byte Authentication Signature [not on all types of file]
8 Byte Standard Field
4 byte File Offset to #defines
...(Implied class) Body
#defines (optional)

Authentication Signature

Currently unique to Xbox, not known on ArmA.

Following the four byte signature "\0 r a P" is an optional 20 byte signature intended to provide a unique value to this file and to prevent tampering. This value, or 'Authentication signature' is generated from a common 'Authentication Key' plus the content of the file itself. The Authentication key for Xbox Elite is 16 bytes and not listed here.

You can obtain source code for generating signatures elsewhere.

The signature is 'optional' only in the sense that not all types of files have it. A signature appears in .par files for example, but not in the official bis mission.sqms.

8 Byte Standard Field

The following 8 bytes appear after the Authentication Signature (or immediately after the rap signature where an Authentication Signature doesn't exist).

00 00 00 00 08 00 00 00 

This is a constant, throughout all Bis raP encoded files. Purpose unknown.

File Offset to Enumerated listCount

a 4 byte intel long stating the file offset (from beginning of file) to the enumerated list (if any). Also implies location of end of all class entries and bodies. It is non-optional. There may well be, no enumerated list, there will be an ListCount saying so!

CLASSES

The first entry in a rap file is an implied class body. It is implied, because the name of the file, is the class to which this body belongs.

ClassBody

struct ClassBody
{
   Asciiz InheritedClassname; // can be zero
   #CompressedInteger  nEntries;        // can be zero.
       Entry1
       .....
       EntryN
   ulong     AddressofNextClass;
};

There cannot be an inherited classname for the first class body, and nEntries cannot (obviously) be zero.

AddressofNextClass

This offset is used by the engine to more rapidly iterate trhu a collection of items to the Nth item in the class.

This long value refers to the location of the next class at the same indent level or, the next outer level when no more exist at this level. Where no more classes exist at all, it will point to the enumerated list block.

All offsets in raP are relative to the start of file.

CompressedInteger

This is the same construct as found in OFP raP files.

The value is used to declare the number of entries in the class body or nArrays

ClassBody Entries

content consists of zero or more 'entries' Entries have self defining lengths according to their type. There are five types of entry possible. Two of them newly introduced to Elite.

0 (Asciiz-EmbeddedClassName} {4 Byte offset to class body}
1 value = (string, float, integer,variable)
2 array[]={....};
3 extern class label;
4 extern class label;

EntryType 0: Emedded Class

{0}{Asciiz-ClassName} {4 Byte offset to class body}

The offset is the location of the content for this classname. By implication, any class entry is an embedded entry to the class it is in. Thus

class one // -> points to it's body
{
     class two {...};// which contains a class that points to it's body
};

Recall that and the end of and within each class body is a pointer to the next class body at the same level or less.

EntryType 1: Value Eq

  {01} {EqType} {Asciiz name}  {more bytes....}
        Eq Types are as follows
  {01}{00} {Aname} = another (asciiz) name
  {01}{01} {Aname} = xx xx xx xx an intel float 
  {01}{02} {Aname} = xx xx xx xx an intel long
//{01}{03} {AnArray[]}= { .........}; not used
  {01}{04} {Aname}=Asciiz Variable name; // public or private

(01 00) is the method by which an entire exec function can be declared (inside a string)

EQ Type 4: Asciiz Variable Name

This has been newly introduced to Elite (vs OFP).

A variable name (as opposed to a string constant) is as follows

Aname = "fred"; // constant
Aname=_fred;    // public or private variable

EntryType 2: array[]

{02} {Asciiz ArrayName} {nElements}

nElements is a #CompressedInteger declaring how many items make up the array, zero, is a legal value

Array Elements

array elements contain 'Value=' entries

each element of the array consists of a single 'EqType' byte followed by appropriate data for the type. Thus

{00}{Asciiz String constant}
{01}{4 byte float}
{02}{4 byte integer}
{03}{recursive array} 
{04}{Asciiz String variable}

A recursive array recurses into further entries with no name attached.

A recursive array is as follows

AnArray[]= { {1,2} , 3,4};

EntryType 3:ExternClass

{03}{ An Asciiz Classname}

New to Elite. This encoding is used to supply the equivalent in C++ syntax

class Classname;

EntryType 4:ExternClass

{04}{ An Asciiz Classname}

New to Elite. This encoding is used to supply the equivalent in C++ syntax

class Classname;

I am not sure of the difference between types 3 and 4. Encoding either value *appears* to achieve same result.

---

Extern Classes

Unlike OFP, a valuable addition to the Token types has been the definition

/*extern*/ class car;

class BigCar:car {...};
class RedCar:BigCar{...};

The tedious business of listing

class All:Default{};
class Vehicle:All{};
...
class Target:Vehicle{};
class Strategic:Target{};
class car{}:Strategic{};

simply to declate a car !!! has been done away with.

Instead, the new types deal with all things car. (or at least the engine goes looking elsewhere, which it did, anyway, in ofp).

One other immediate, and not immediately apparent, benefit, is you do not need to know anything about how the underlying class (eg CfgVehicles) is constructed.

In fact, the inheritence tree of OFP (for CfgVehicles) is different to Elite, and using the above OFP method of constructon would fail. It would not find, a Target class.

enumerated list

optional

the next four bytes after all class bodie are the enumerated list count., and these defines (if any) continue until end of file.

where no list exist(the very normal case), the count is zero

A list consists of an asciiz name and integer value as follows

(manposnoweapon) {01 00 00 00}

which equates in C to

enum {
manposweapon=1,
manshower=88,
manhungry=12,
....
}

This list is encountered in some few 'official' config.cpp's. It's use by model makers is rare.