P3D File Format - ODOLV4x

From Bohemia Interactive Community
Jump to navigation Jump to search

ODOL v40 File Format

Template:unsupported-doc

Introduction

The general file format of a ArmA ODOL v40 p3d model file is similar to the ODOL v7 format. The major differences are that in ArmA models there is now included (but not always) a model.cfg and the resolutions are ordered in the file in reverse numerical order.

The order of resolutions denoted in the header portion of the file is not necessarily the numerical order of the resolutions. (often the 11,000 resolution is the last in the header array) The header resolutions need to be sorted in descending order. The resultant sorted array of resolutions is the order in which they appear in the file.


Legend

Type Description
byte 8 bit (1 byte)
ushort 16 bit unsigned short (2 bytes)
int 32 bit signed integer (4 bytes)
float 32 bit signed single precision floating point value (4 bytes)
asciiz Null terminated (0x00) variable length ascii string


Enums

int enum PixelShaderId { Normal = 0x00, NormalMap = 0x02, NormalMapDiffuse = 0x05, NormalMapMacroASSpecularMap = 0x14, NormalMapSpecularDIMap = 0x16, NormalMapMacroASSpecularDIMap = 0x18, AlphaShadow = 0x0C, AlphaNoShadow = 0x0D, Glass = 0x38, Detail = 0x06, NormalMapSpecularMap = 0x12 }

int enum VertexShaderId { Basic = 0x00, NormalMap = 0x01, NormalMapAS = 0x0F }

Structures

structP3DHeader

struct structP3DHeader { asciiz Filetype; //eg. ODOL int Version; //eg. 0x2800 0000 = 40 int NoOfResolutions; float[NoOfResolutions] HeaderResolutions; }

structBone

structBone { asciiz Bone; asciiz Parent; }

structSkeleton

structSkeleton { asciiz SkeletonName; bool isInherited; int NoOfBones; structBone[NoOfBones] Bones; }

structAnimation

structAnimation { int AnimTransformType; asciiz AnimSelection; asciiz AnimSource; if (AnimTransformType == 9) { float[6] Transforms; } else { float[7] Transforms; } }

structProxy

structProxy { asciiz ProxyName; float[12] ModelProxyUnknown1; int[4] ModelProxyUnknown2; }

structStage

structStage { asciiz StageTexture; int Stage; int UVSource; float[3] aside; float[3] up; float[3] dir; float[3] pos; }

structMaterial

structMaterial { asciiz Material; float[4] Emissive; float[4] Ambient; float[4] Diffuse; float[4] forcedDiffuse; float[4] Specular; float SpecularPower; int PixelShaderId; int VertexShaderId; structStage[] Stages; }

structPolygons

structPolygons { byte NoOfVertices; // 3 or 4 ushort[NoOfVertices] VerticesIndex; // 0-based index into Vertices Arrays }


structComponent

structComponent { asciiz ComponentName; //Selected Faces int NoOfSelectedFaces; ushort[NoOfSelectedFaces] SelectedFaceIndexes; int intComponentUnknown1; bool bComponentUnknown1; if (bComponentUnknown1) { int NoOf; int[NoOf] intArrayComponentUnknown1; } else { int intComponentUnknown2; } // Selected Vertices int NoOfSelectedVertices; ushort[NoOfSelectedVertices] SelectedVerticesIndexes; // Zero based array of index values into // the array of Vertices. // Note:- If expectedSize >= 1024 bytes this array is compressed. // Selected Vertices Properties int NoOfSelectedVertices; byte[NoOfSelectedVertices] SelectedVerticesProperties; // Zero based array of index values into // the array of Vertices. // Note:- If expectedSize >= 1024 bytes this array is compressed. }


structResolution (simple)

structResolution { NoOfVertices; <space> NoOfTextures; Textures; NoOfMaterials; Materials; <space> NoOfPolygons; <space> Polygons; <space> NoOfComponents; Components; NoOfProperties; Properties; <space> NoOfVertices; VerticesUVSet1; NoOfVertices; VerticesUVSet2; NoOfVertices; VerticesPositions; NoOfVertices; VerticesNormals; NoOfVertices; VerticesMinMax; //Looks like Min/Max info. NoOfVertices; VerticesUnknown1; //Looks like per vertex properties NoOfVertices; VerticesUnknown2; //hmmmm... if(pointer<filesize) { NoOfProxies; Proxies; <space> } NoOf; IntermittentUnknownData; // As at article date 12-Aug-2007. This data is not in every lod // it is intermittent. Currently, structure is unknown. // Can be bypassed by manual intervention to start of next resolution. // Most likely is Texture-2-Face/Vertex mappings. //This is a 'show-stopper' for continuous processing. }

structResolution (detailed)

structResolution { int NoOfVertices; byte byteResUnknown1; byte byteResUnknown2; switch (byteResUnknown2) { case 0x00: { byte[40] byteArrayResUnknown1; break; } case 0x20: { byte[45] byteArrayResUnknown1; break; } case 0x30: { byte[45] byteArrayResUnknown1; break; } case 0xFF: { byte[45] byteArrayResUnknown1; break; } case 0x3F: { byte[51] byteArrayResUnknown1; break; } } int NoOfTextures; asciiz[NoOfTextures] Textures; int NoOfMaterials; structMaterial[NoOfMaterials] Materials; //Basically... A direct replication of the information in the given .rvmat file for (int i = 0; i < NoOfMaterials; i++) { asciiz Material; byte[4] byteArrayMaterialUnknown1; float[4] Emissive; float[4] Ambient; float[4] Diffuse; float[4] forcedDiffuse; float[4] Specular; float SpecularPower; int PixelShaderId; //See enumPixelShaderId int VertexShaderId; //See enumVertexShaderId //Based on the enumPixelShaderId that matches this PixelShaderId process a variable 'NoOfStages' //by default one should probably process 2 stages as this seems the most common amount if (NoOfStages > 0) { byte[34] byteArrayMaterialUnknown2; for (int i = 0; i < NoOfStages; i++) { byte[4] byteArrayMaterialUnknown3; asciiz StageTexture; int StageNumber; } for (int i = 0; i < NoOfStages; i++) { int UVSource; float[3] aside; float[3] up; float[3] dir; float[3] pos; } byte[52] byteArrayMaterialUnknown4; //Possibly default values for a stage as same struct size } else { byte[86] byteArrayMaterialUnknown5; } }//EndOfMaterials byte[8] byteArrayResUnknown2; int NoOfPolygons; byte[6] byteArrayResUnknown3; structPolygons[NoOfPolygons] Polygons; //The following is an Unknown structure, however this code snippet iterates over it int NoOf; for (int i = 0; i < NoOf; i++) { byte[26] byteArrayResUnknown4; byte byteResUnknown3; if (byteResUnknown3 == 0xFF) { byte[16] byteArrayResUnknown5; } else { byte[15] byteArrayResUnknown5; } }//EndOfUnknownStructure int NoOfComponents; structComponent[NoOfComponents] Components; ...... next biki session update... tomorrow night (13/8/2007) -- Sy 12:16, 12 August 2007 (CEST)... ...... }

File Format

The following is a mix of pseudo-code and structure references that could be used to discribe the file format of ODOL v40. It may or may not be accurate but has do date been used to read ODOL v40 is some cases without manual intervention. As at the writing of this article in most cases though, manual intervention is required to complete navigation throughout the given p3d file as there is some unkonwn data that prevents continuous processing.


Simple

ODOLv40 { Header; Model.cfg; (optional) Resolutions; (reverse numerical order) }


Detailed

ODOLv40 { structP3DHeader Header; byte[155] Unknown; structSkeleton Skeleton; byte unknown1; if(unknown1 == 0x00) {byte unknown2}; byte[32] unknown3; int unknown4; byte unknown5; asciiz unknown6; byte[6] unknown7; bool AnimsExist; if (AnimsExist) { int NoOfAnimSelections; structAnimation[NoOfAnimSelections] Animations; //Basically... for each bone there is a list of Animations and this array structure // is stored on a per resolution basis. int NoOfResolutions; for(int i=0; i<NoOfResolutions; i++) { int NoOfBones; for(int ii=0; ii<NoOfResolutions; ii++) { int NoOfAnims; if (NoOfAnims > 0) { for(int iii=0; iii<NoOfAnims; iii++) { int Animation; } } } } //Unknown Anim info... //Basically... for each Animation if the TransformType !=9 then there // is a 6 x float of positional info. for(int i=0; i<NoOfResolutions; i++) { int Anim; if (Anim != -1) { if (Animations[Anim].TransformType != 9) { float[6] UnknownAnimInfo; } } } }//AnimExist byte[Header.NoOfResolutions * 8] Unknown8; bool[Header.NoOfResolutions] ResolutionFaceIndicator; //Basically...For each Resolution if the LODFaceIndicator is true //there is a int FaceCount + 13 bytes //I think this 'indicator' may serve other areas but at the very least it indicates //the following structure for (int i = 0; i < Header.NoOfResolutions; i++) { if (LODFaceIndicator[i]) { int HeaderFaceCount; byte[13] Unknown9; } } int NoOfModelProxies; if (NoOfModelProxies != 0) { structProxy[NoOfModelProxies] ModelProxies; } structResolution[Header.NoOfResolutions]; //Note:- Remember, the order in which lod's // occur is descending numerical order. // eg. Resolution 1.0 will be the last in // the file. //EndOfFile }

Decompression

In ODOL v40 format files some of the datastructures present in the file are compressed by using a form of LZ compression. Unlike pbo compression, in ArmA model files, one only knows the number of items to decompress, the expected output size (in bytes) and the expected checksum. With this information and the size of a given data item one has the necessary information to expand the data to it's original format and size.


Note:- Data structures that are identified as being compressible will only be compressed if the 'expectedSize' is >= 1024 bytes.


The code that follows is written in C# and may or may not be optimal or correct.


As an example if one was expanding the array of vertices positions...

  • A vertex is described by it's x,y,z coordinates which are floats. A float is a 32bit (4 byte) number.
  • If we were processing 1968 vertices then our expected output size would be 1968 * (3 * 4) = 23,616 bytes.

This 'expectedSize' is the only necessary information one would need to pass to a processing sub-routine or function.


public bool Expand(int ExpectedSize) { byte PacketFlagsByte; //packet flags byte WIPByte; BitVector32 BV; msLZ = new MemoryStream(ExpectedSize); BinaryWriter bwLZ = new BinaryWriter(msLZ); byte[] Buffer = new byte[ExpectedSize + 15]; bool[] BitFlags = new bool[8]; int i = 0, PointerRef = 0, ndx = 0, CalculatedCRC = 0, ReadCRC = 0, rPos, rLen, CurrentPointerRef = 0, Count = 0; int Bit0 = BitVector32.CreateMask(); int Bit1 = BitVector32.CreateMask(Bit0); int Bit2 = BitVector32.CreateMask(Bit1); int Bit3 = BitVector32.CreateMask(Bit2); int Bit4 = BitVector32.CreateMask(Bit3); int Bit5 = BitVector32.CreateMask(Bit4); int Bit6 = BitVector32.CreateMask(Bit5); int Bit7 = BitVector32.CreateMask(Bit6); PacketFlagsByte = br.ReadByte(); do { BV = new BitVector32(PacketFlagsByte); BitFlags[0] = BV[Bit0]; BitFlags[1] = BV[Bit1]; BitFlags[2] = BV[Bit2]; BitFlags[3] = BV[Bit3]; BitFlags[4] = BV[Bit4]; BitFlags[5] = BV[Bit5]; BitFlags[6] = BV[Bit6]; BitFlags[7] = BV[Bit7]; i = 0; do { if ((int)bwLZ.BaseStream.Position >= ExpectedSize) { break; } if (BitFlags[i++]) //Direct Output { WIPByte = br.ReadByte(); bwLZ.Write(WIPByte); Buffer[PointerRef++] = WIPByte; CalculatedCRC += WIPByte; } else //Get from previous 4k { rPos = (int)(br.ReadByte()); rLen = (int)(br.ReadByte()); rPos |= (rLen & 0xF0) << 4; rLen = (rLen & 0x0F) + 2; CurrentPointerRef = PointerRef; if ((CurrentPointerRef - (rPos + rLen)) > 0) { //Case of wholly within the buffer, partially within the end of the buffer or wholly outside the end of the buffer for (Count = 0; Count <= rLen; Count++) { ndx = (CurrentPointerRef - rPos) + Count; if (ndx < 0) { //Beyond the start of the buffer WIPByte = 0x20; } else { //Within the buffer WIPByte = Buffer[ndx]; } //} bwLZ.Write(WIPByte); Buffer[PointerRef++] = WIPByte; CalculatedCRC += WIPByte; } } else { //Case of wholly or partially beyond the start of the buffer. for (Count = 0; Count <= rLen; Count++) { ndx = (CurrentPointerRef - rPos) + Count; if (ndx < 0) { //Beyond the start of the buffer WIPByte = 0x20; } else { //Within the buffer WIPByte = Buffer[ndx]; } bwLZ.Write(WIPByte); Buffer[PointerRef++] = WIPByte; CalculatedCRC += WIPByte; } } } } while ((i < 8) & (bwLZ.BaseStream.Position < ExpectedSize)); if (bwLZ.BaseStream.Position < ExpectedSize) { PacketFlagsByte = br.ReadByte(); } } while (bwLZ.BaseStream.Position < ExpectedSize); ReadCRC = br.ReadInt32(); return (ReadCRC == CalculatedCRC); }

Reference Tables

Note: These are not part of the p3d model file but are reference tables used for processing.

Resolutions

refResolutions { float Resolution; string ResolutionName; }

Value Value Description
1.0e3 1,000 View Gunner
1.1e3 1,100 View Pilot
1.2e3 1,200 View Cargo
1.0e4 10,000 Stencil Shadow
1.001e4 10,010 Stencil Shadow 2
1.1e4 11000 Shadow Volume
1.101e4 11010 Shadow Volume 2
1.0e13 10,000,000,000,000 Geometry
1.0e15 1,000,000,000,000,000 Memory
2.0e15 2,000,000,000,000,000 Land Contact
3.0e15 3,000,000,000,000,000 Roadway
4.0e15 4,000,000,000,000,000 Paths
5.0e15 5,000,000,000,000,000 HitPoints
6.0e15 6,000,000,000,000,000 View Geometry
7.0e15 7,000,000,000,000,000 Fire Geometry
8.0e15 8,000,000,000,000,000 View Cargo Geometry
9.0e15 9,000,000,000,000,000 View Cargo Fire Geometry
1.0e16 10,000,000,000,000,000 View Commander
1.1e16 11,000,000,000,000,000 View Commander Geometry
1.2e16 12,000,000,000,000,000 View Commander Fire Geometry
1.3e16 13,000,000,000,000,000 View Pilot Geometry
1.4e16 14,000,000,000,000,000 View Pilot Fire Geometry
1.5e16 15,000,000,000,000,000 View Gunner Geometry
1.6e16 16,000,000,000,000,000 View Gunner Fire Geometry


Material Stages

The number of material stages is dependant on the type of Shader that is used to process the material by the ArmA game engine. A reference table is used when processing materials where depending on the shader specified the given number of stages should be processed.

refShaderStages { int PixelShaderId; int NoOfStages; };

PixelShaderId enum NoOfStages
PixelShaderId.Normal 0
PixelShaderId.NormalMapSpecularDIMap 2
PixelShaderId.NormalMapDiffuse 2
PixelShaderId.AlphaNoShadow 0
PixelShaderId.AlphaShadow 0
PixelShaderId.NormalMapMacroASSpecularDIMap 4
PixelShaderId.Glass 2
PixelShaderId.Detail 1
PixelShaderId.NormalMap 3
PixelShaderId.NormalMapMacroASSpecularMap 4
PixelShaderId.NormalMapSpecularMap 2

Links

Article Author - Sy (Synide) -- Sy 17:16, 11 August 2007 (CEST)

Original ODOLv40 Biki Article detailed by Bxbx (Biki'd by Mikero)