P3D File Format - ODOLV4x

From Bohemia Interactive Community
Revision as of 08:22, 10 January 2009 by Mikero (talk | contribs) (→‎structProxy: moved)
Jump to navigation Jump to search

Template:unsupported-doc

Introduction

Acknowledgements

This body of work is due to Synide's sweat and tears. To whom, all honour and glory. Ably assisted by T_D, with Mikero trying to keep up with them both.

All accurate data is supplied by the first two individuals. All mistakes, omissions, typos and general misinformation is due to Mikero alone );

General

The general file format of a ArmA ODOL v40 p3d model file is similar to the ODOL v7 format. The major differences are that in ArmA models are

  • an optional model.cfg
  • resolutions are in reverse numerical order.

The order of resolutions denoted in the header portion of the file is not necessarily the numerical order of the resolutions. (often the 11,000 resolution is the last in the header array) The header resolutions need to be sorted in descending order. The resultant sorted array of resolutions is the order in which they appear in the file.


NOTE:- As at Aug, 13th 2007, this file format is not conclusive and final. It may or may not fit the actual model file format of an Armed Assault ODOL v40 p3d model.


Legend

Type Description
byte 8 bit (1 byte)
tbool byte: 0 = false.
ushort 16 bit unsigned short (2 bytes)
int 32 bit signed integer (4 bytes)
ulong 32 bit unsigned integer (4 bytes)
float 32 bit signed single precision floating point value (4 bytes)
asciiz Null terminated (0x00) variable length ascii string

File Format

The following is a mix of pseudo-code and structure references that could be used to describe the file format of ODOL v40. It may or may not be accurate but has do date been used to read ODOL v40 in some cases without manual intervention. As at the writing of this article in most cases though, manual intervention is required to complete navigation throughout the given p3d file as there is some unkonwn data that prevents continuous processing.


Simple

ODOLv40 { Header; Model.cfg; (optional) Resolutions; (reverse numerical order) }

Detailed

ODOLv40 { StandardP3DHeader Header; float HeaderResolutions[[NoOfLods]; ModelInfo[...]; Skeleton[...]; UnknownStruct1[...]; Animations[...]; int[Header.NoOfResolutions] StartAdressOfLods; int[Header.NoOfResolutions] EndAdressOfLods; bool[Header.NoOfResolutions] LODFaceIndicator; //Basically...For each Resolution if the LODFaceIndicator is false //there is a int FaceCount + 13 bytes //I think this 'indicator' may serve other areas but at the very least it indicates //the following structure for (int i = 0; i < Header.NoOfResolutions; i++) { if (!LODFaceIndicator[i]) { int HeaderFaceCount; byte[13] Unknown9; } } structResolution[Header.NoOfResolutions]; //Note:- Remember, the order in which lod's // occur is descending numerical order. // eg. Resolution 1.0 will be the last in // the file. //EndOfFile }

ModelInfo

 ModelInfo
 {
  ulong Unknown;
  float Sphere;
  byte  Unknown[36]; 
  float ViewDensity;
  byte  Unknown[24]; 
  float ModelVertexOffset[3];    //xyz
  float Unknown[3];              //xyz
  float ModelCentreOfGravity[3]; //xyz
  float ModelMassVectors[3][3];
  byte  AutoCenter,
        lockAutoCenter,
        canOcclude,
        canBeOccluded,
        allowAnimation;
  byte  Unknown[6]; 
 }

UnknownStruct1

UnknownStruct1
{
 byte unknown1;
 byte unknown2;
 if(unknown2 != 0)
 {
  byte  ExtraByte;
 }
 byte unknown3[3];
 float	ModelMass.
       ModelMassReciprocal,
       ModelMassModifier;
 byte	Unknown16[16];
 ulong unknown4;
 byte unknown5;
 asciiz ModelString1;
 asciiz ModelString2;
 byte[5] byteArrayUnknown2;
}

Animations

Animations
{
  byte AnimsExist;
  if (AnimsExist == 0x01)
  {
   ulong          NoOfAnimSelections;
   structAnimation[NoOfAnimSelections] Animations;

   //Basically... for each bone there is a list of Animations and this array structure
   //             is stored on a per resolution basis.
   ulong NoOfResolutions;
   {
     ulong NoOfBones;
     {
       ulong NoOfAnims;
       ulong Animation[NoOfAnims];
     }[NoOfBones];
   }[NoOfResolutions];

       //This time it is the other way around: every Animation gets it Bone and additionally
       //the position of the axis if needed for animtype
       for(int i=0; i<NoOfResolutions; i++)
       {
         for(int j=0; j<NoOfAnimSelections; j++)
         {
           ulong Bone;
           if (Bone != -1)
           {
              if (Animations[j].TransformType != 8 && Animations[j].TransformType != 9)
              {
                 float[3] axisPos; //describes the position of the axis used for this anim
                 float[3] axisDir;
              }
           }
       }
  }//AnimExist

Structures

StandardP3DHeader

StandardP3DHeader { char[4] Filetype; // "ODOL" ulong Version; // 40 ulong NoOfLods; // alias NoOfResolutions; }

common header structure for all P3D file formats

structBone

structBone { asciiz Bone; asciiz Parent; }

structSkeleton

structSkeleton { asciiz SkeletonName; if (SkeletonName != null) { byte isInherited; // 0x00=false, 0x01=true int NoOfBones; structBone[NoOfBones] Bones; } }

structAnimation

structAnimation { int AnimTransformType; asciiz AnimSelection; asciiz AnimSource; switch(AnimTransformType) case 9: //"hide" { float minValue; float maxValue; float minPhase; float maxPhase; int sourceAdress; float hideValue; } case 8: //"direct" { float minValue; float maxValue; float minPhase; float maxPhase; int sourceAdress; float axisPos[3]; float axisDir[3]; float angle; //in radians whereas the model.cfg entry is in degrees float axisOffset; } default { float minValue; float maxValue; float minPhase; float maxPhase; int sourceAdress; float angle0/offset0; //depends on animType float angle1/offset1; //depends on animType } }


structComponent

structComponent { asciiz ComponentName; //Selected Faces int NoOfSelectedFaces; ushort[NoOfSelectedFaces] SelectedFaceIndexes; int intComponentUnknown1; bool bComponentUnknown1; if (bComponentUnknown1) { int NoOf; int[NoOf] intArrayComponentUnknown1; } else { int intComponentUnknown2; } // Selected Vertices int NoOfSelectedVertices; ushort[NoOfSelectedVertices] SelectedVerticesIndexes; // Zero based array of index values into // the array of Vertices. // Note:- If expectedSize >= 1024 bytes this array is compressed. // Selected Vertices Properties int NoOfSelectedVertices; byte[NoOfSelectedVertices] SelectedVerticesProperties; // Zero based array of index values into // the array of Vertices. // Note:- If expectedSize >= 1024 bytes this array is compressed. }


structProperties

structProperties { asciiz Property; asciiz Value; }

structUV

structUV { float u; float v; }

structVerticesPosition

structVerticesPosition { float x; float z; float y; }

structVerticesNormal

structVerticesNormal { float x; float z; float y; }

structVerticesMinMax

structVerticesMinMax { float x1; float z1; float y1; float x2; float z2; float y2; }

structVerticesUnknown1

structVerticesUnknown1 { ushort One; ushort Two; ushort Three; ushort Four; ushort Five; ushort Six; }

structResolution (simple)

structResolution { NoOfVertices; <space> NoOfTextures; Textures; NoOfMaterials; Materials; <space> NoOfPolygons; <space> Polygons; <space> NoOfComponents; Components; NoOfProperties; Properties; <space> NoOfVertices; VerticesUVSet1; NoOfVertices; VerticesUVSet2; NoOfVertices; VerticesPositions; NoOfVertices; VerticesNormals; NoOfVertices; VerticesMinMax; //Looks like Min/Max info. NoOfVertices; VerticesUnknown1; //Looks like per vertex properties NoOfVertices; VerticesUnknown2; //hmmmm... if(pointer<filesize) { NoOfProxies; Proxies; <space> } NoOf; IntermittentUnknownData; // As at article date 12-Aug-2007. This data is not in every lod // it is intermittent. Currently, structure is unknown. // Can be bypassed by manual intervention to start of next resolution. // Most likely is Texture-2-Face/Vertex mappings. //This is a 'show-stopper' for continuous processing. }

structResolution (detailed)

structResolution { int NoOfModelProxies; structProxy[NoOfModelProxies] ModelProxies; StarterStructOne[...]; StarterStructTwo[...]; PointProperties[...]; TextureNames[...]; Materials[...]; byte[8] byteArrayResUnknown2; PolygonStruct[...]; //Model 'Section' Info. Unknown structure, however this code snippet iterates over it int NoOfSections; for (int i = 0; i < NoOfSections; i++) { byte[26] byteArrayResUnknown4; byte byteResUnknown3; if (byteResUnknown3 == 0xFF) { byte[16] byteArrayResUnknown5; } else { byte[15] byteArrayResUnknown5; } }//EndOf Model 'Section' info. structure int NoOfComponents; structComponent[NoOfComponents] Components; int NoOfProperties; structProperties[NoOfProperties] Properties; ushort usUnknown1; ushort usUnknown2; if (usUnknown2 == 0) { byte[17] byteArrayResUnknown6; } else { byte[15] byteArrayResUnknown6; } int NoOfVertices; byte bUV if (bUV == 0x00) { structUV[NoOfVertices2ndUV] VerticesUVSet1; // Note:- If expectedSize >= 1024 bytes this array is compressed. int NoOfVertices2ndUV; } else { byte[12] byteArrayResUnknown7; } if (NoOfVertices2ndUV == 2) { int NoOfVertices2ndUV; byte bUV; if (bUV == 0x00) { structUV[NoOfVertices2ndUV] VerticesUVSet2; // Note:- If expectedSize >= 1024 bytes this array is compressed. } else { byte[8] byteArrayResUnknown8; } } int NoOfVertices; structVerticesPosition[NoOfVertices] VerticesPositions; // Note:- If expectedSize >= 1024 bytes this array is compressed. int NoOfVertices; byte bNormal; if (bNormal == 0x00) { structVerticesNormal[NoOfVertices] VerticesNormals; // Note:- If expectedSize >= 1024 bytes this array is } compressed. else { byte[12] byteArrayResUnknown9; } int NoOfVertices; structVerticesMinMax[NoOfVertices] VerticesMinMax; // Note:- If expectedSize >= 1024 bytes this array is compressed. int NoOfVertices; structVerticesUnknown1[NoOfVertices] VerticesUnknown1; // Note:- If expectedSize >= 1024 bytes this array is compressed. int NoOfVertices; byte[NoOfVertices][32] VerticesUnknown2; // Note:- If expectedSize >= 1024 bytes this array is compressed. //EndOfResolution }

StarterStructOne

StarterOne
{
 ulong Count;
 ulong Items[Count];
};

StarterStructTwo

StarterTwo
{
 ulong Count;
 {
  ulong NoOf3;
  ulong Items[NoOf3];
 }[Count];
}

PointProperties

PointProperties // into the ResolutionIndex
{
 ushort NoOfPts;
 ushort Unknown;
 byte	 UseDefault;
 if (UseDefault==1)
 {
  ulong DefaultValue;
 }
 else // =0
 {
   ulong  PropertyValues[NoOfPts]; // if NoOfPoints*sizeof(ulong) > 1023 LZH compression.
 }
 byte Unknown[8];
 float Unknown[10];
}

PropertyValues for NoOfPts are either all the same (UseDefault), or, they are individually declared.

Similar to CompressedStructs of OdolV7, if the amount of data in the array exceeds 1023 bytes, that array is compressed.

TextureNames

TextureNames
{
 ulong     NoOfTextures;
 asciiz    Textures[NoOfTextures];
}

Materials

Materials
{
  ulong NoOfMaterials;
  structMaterial[NoOfMaterials];
}

structMaterial

structMaterial { asciiz Material; float[4] Emissive; float[4] Ambient; float[4] Diffuse; float[4] forcedDiffuse; float[4] Specular; float SpecularPower; ulong PixelShaderId; ulong VertexShaderId; structStage[] Stages; }

//Basically... A direct replication of the information in the given .rvmat file for (int i = 0; i < NoOfMaterials; i++) { asciiz Material; byte[4] byteArrayMaterialUnknown1; float[4] Emissive; float[4] Ambient; float[4] Diffuse; float[4] forcedDiffuse; float[4] Specular; float SpecularPower; ulong PixelShaderId; //See enumPixelShaderId ulong VertexShaderId; //See enumVertexShaderId //Based on the enumPixelShaderId that matches this PixelShaderId process a variable 'NoOfStages' //by default one should probably process 2 stages as this seems the most common amount if (NoOfStages > 0) { byte[34] byteArrayMaterialUnknown2; for (int i = 0; i < NoOfStages; i++) { ulong Filter; asciiz StageTexture; ulong StageNumber; } for (int i = 0; i < NoOfStages; i++) { ulong UVSource; float[3] aside; float[3] up; float[3] dir; float[3] pos; } byte[52] byteArrayMaterialUnknown4; //Possibly default values for a stage as same struct size } else { byte[86] byteArrayMaterialUnknown5; } }//EndOfMaterials

structStage

structStage { asciiz StageTexture; ulong Stage; ulong UVSource; float[3] aside; float[3] up; float[3] dir; float[3] pos; }

PolygonStruct

PolygonStruct
{
 ulong   NoOfPolygons;
 byte    Unknown[6];                   // 28 00 00 00 00 00 eg
 PoyygonVertices
 {
   byte   NoOfVertices;                // 3 or 4
   ushort VerticesIndex[NoOfVertices]; // 0-based index into Vertices Arrays
 }[NoOfPolygons];
}

Decompression

In ODOL v40 format files some of the datastructures present in the file are compressed by using a form of LZ compression. Unlike pbo compression, in ArmA model files, one only knows the number of items to decompress, the expected output size (in bytes) and the expected checksum. With this information and the size of a given data item one has the necessary information to expand the data to it's original format and size.


Note:- Data structures that are identified as being compressible will only be compressed if the 'expectedSize' is >= 1024 bytes.


The code that follows is written in C# and may or may not be optimal or correct.


As an example if one was expanding the array of vertices positions...

  • A vertex is described by it's x,y,z coordinates which are floats. A float is a 32bit (4 byte) number.
  • If we were processing 1968 vertices then our expected output size would be 1968 * (3 * 4) = 23,616 bytes.

This 'expectedSize' is the only necessary information one would need to pass to a processing sub-routine or function.


public bool Expand(int ExpectedSize) { byte PacketFlagsByte; //packet flags byte WIPByte; BitVector32 BV; msLZ = new MemoryStream(ExpectedSize); BinaryWriter bwLZ = new BinaryWriter(msLZ); byte[] Buffer = new byte[ExpectedSize + 15]; bool[] BitFlags = new bool[8]; int i = 0, PointerRef = 0, ndx = 0, CalculatedCRC = 0, ReadCRC = 0, rPos, rLen, CurrentPointerRef = 0, Count = 0; int Bit0 = BitVector32.CreateMask(); int Bit1 = BitVector32.CreateMask(Bit0); int Bit2 = BitVector32.CreateMask(Bit1); int Bit3 = BitVector32.CreateMask(Bit2); int Bit4 = BitVector32.CreateMask(Bit3); int Bit5 = BitVector32.CreateMask(Bit4); int Bit6 = BitVector32.CreateMask(Bit5); int Bit7 = BitVector32.CreateMask(Bit6); PacketFlagsByte = br.ReadByte(); do { BV = new BitVector32(PacketFlagsByte); BitFlags[0] = BV[Bit0]; BitFlags[1] = BV[Bit1]; BitFlags[2] = BV[Bit2]; BitFlags[3] = BV[Bit3]; BitFlags[4] = BV[Bit4]; BitFlags[5] = BV[Bit5]; BitFlags[6] = BV[Bit6]; BitFlags[7] = BV[Bit7]; i = 0; do { if ((int)bwLZ.BaseStream.Position >= ExpectedSize) { break; } if (BitFlags[i++]) //Direct Output { WIPByte = br.ReadByte(); bwLZ.Write(WIPByte); Buffer[PointerRef++] = WIPByte; CalculatedCRC += WIPByte; } else //Get from previous 4k { rPos = (int)(br.ReadByte()); rLen = (int)(br.ReadByte()); rPos |= (rLen & 0xF0) << 4; rLen = (rLen & 0x0F) + 2; CurrentPointerRef = PointerRef; if ((CurrentPointerRef - (rPos + rLen)) > 0) { //Case of wholly within the buffer, partially within the end of the buffer or wholly outside the end of the buffer for (Count = 0; Count <= rLen; Count++) { ndx = (CurrentPointerRef - rPos) + Count; if (ndx < 0) { //Beyond the start of the buffer WIPByte = 0x20; } else { //Within the buffer WIPByte = Buffer[ndx]; } //} bwLZ.Write(WIPByte); Buffer[PointerRef++] = WIPByte; CalculatedCRC += WIPByte; } } else { //Case of wholly or partially beyond the start of the buffer. for (Count = 0; Count <= rLen; Count++) { ndx = (CurrentPointerRef - rPos) + Count; if (ndx < 0) { //Beyond the start of the buffer WIPByte = 0x20; } else { //Within the buffer WIPByte = Buffer[ndx]; } bwLZ.Write(WIPByte); Buffer[PointerRef++] = WIPByte; CalculatedCRC += WIPByte; } } } } while ((i < 8) & (bwLZ.BaseStream.Position < ExpectedSize)); if (bwLZ.BaseStream.Position < ExpectedSize) { PacketFlagsByte = br.ReadByte(); } } while (bwLZ.BaseStream.Position < ExpectedSize); ReadCRC = br.ReadInt32(); return (ReadCRC == CalculatedCRC); }

Reference Tables

Note: These are not part of the p3d model file but are reference tables used for processing.

Resolutions

refResolutions { float Resolution; string ResolutionName; }

Hex-Value Value Value Description
0x447a0000 1.0e3 1,000 View Gunner
0x44898000 1.1e3 1,100 View Pilot
0x44960000 1.2e3 1,200 View Cargo
0x461c4000 1.0e4 10,000 Stencil Shadow
0x461c6800 1.001e4 10,010 Stencil Shadow 2
0x462be000 1.1e4 11000 Shadow Volume
0x462c0800 1.101e4 11010 Shadow Volume 2
0x551184e7 1.0e13 10,000,000,000,000 Geometry
0x58635fa9 1.0e15 1,000,000,000,000,000 Memory
0x58e35fa9 2.0e15 2,000,000,000,000,000 Land Contact
0x592a87bf 3.0e15 3,000,000,000,000,000 Roadway
0x59635fa9 4.0e15 4,000,000,000,000,000 Paths
0x598e1bca 5.0e15 5,000,000,000,000,000 HitPoints
0x59aa87bf 6.0e15 6,000,000,000,000,000 View Geometry
0x59c6f3b4 7.0e15 7,000,000,000,000,000 Fire Geometry
0x59e35fa9 8.0e15 8,000,000,000,000,000 View Cargo Geometry
0x59ffcb9e 9.0e15 9,000,000,000,000,000 View Cargo Fire Geometry
0x5a0e1bca 1.0e16 10,000,000,000,000,000 View Commander
0x5a1c51c4 1.1e16 11,000,000,000,000,000 View Commander Geometry
0x5a2a87bf 1.2e16 12,000,000,000,000,000 View Commander Fire Geometry
0x5a38bdb9 1.3e16 13,000,000,000,000,000 View Pilot Geometry
0x5a46f3b4 1.4e16 14,000,000,000,000,000 View Pilot Fire Geometry
0x5a5529af 1.5e16 15,000,000,000,000,000 View Gunner Geometry
0x5a635fa9 1.6e16 16,000,000,000,000,000 View Gunner Fire Geometry
0x5a7195a4 1.7e16 17,000,000,000,000,000 Sub Parts

Note: Hex-Values are provided for convenience, as you can use those in different programming languages 'switch'-statement as opposed to floating point values.

Material Stages

The number of material stages is dependant on the type of Shader that is used to process the material by the ArmA game engine. A reference table is used when processing materials where depending on the shader specified the given number of stages should be processed.

refShaderStages { int PixelShaderId; int NoOfStages; };

ID (Hex/Decimal) Name Description NoOfStages
0x00, 0 Normal diffuse color modulate, alpha replicate 0
0x01, 1 NormalDXTA diffuse color modulate, alpha replicate, DXT alpha correction 0
0x02, 2 NormalMap normal map shader 3
0x03, 3 NormalMapThrough normal map shader - through lighting 3
0x04, 4 ? ? ?
0x05, 5 NormalMapDiffuse ? 2
0x06, 6 Detail ? 1
0x07, 7 ? ? ?
0x08, 8 Water sea water 2
0x09, 9 ? ? ?
0x0A, 10 White ? 0
0x0B, 11 ? ? ?
0x0C, 12 AlphaShadow shadow alpha write 0
0x0D, 13 AlphaNoShadow shadow alpha (no shadow) write 0
0x0E, 14 ? ? ?
0x0F, 15 DetailMacroAS ? 3
0x10, 16 ? ? ?
0x11, 17 ? ? ?
0x12, 18 NormalMapSpecularMap ? 2
0x13, 19 NormalMapDetailSpecularMap Similar to NormalMapDiffuse 3
0x14, 20 NormalMapMacroASSpecularMap ? 4
0x15, 21 NormalMapDetailMacroASSpecularMap ? 5
0x16, 22 NormalMapSpecularDIMap Same as NormalMapSpecularMap, but uses _SMDI texture 2
0x17, 23 NormalMapDetailSpecularDIMap ? 3
0x18, 24 NormalMapMacroASSpecularDIMap ? 4
0x19, 25 NormalMapDetailMacroASSpecularDIMap ? 5
0x38, 56 Glass ? 2
0x3A, 58 NormalMapSpecularThrough ? 3
0x3B, 59 Grass Special shader to allow volumetric shadows to be cast on grass clutter 0
0x3C, 60 NormalMapThroughSimple ? 0

Enums

int enum PixelShaderId { Normal = 0x00, NormalMap = 0x02, NormalMapDiffuse = 0x05, NormalMapMacroASSpecularMap = 0x14, NormalMapSpecularDIMap = 0x16, NormalMapMacroASSpecularDIMap = 0x18, AlphaShadow = 0x0C, AlphaNoShadow = 0x0D, Glass = 0x38, Detail = 0x06, NormalMapSpecularMap = 0x12 }

int enum VertexShaderId { Basic = 0x00, NormalMap = 0x01, NormalMapAS = 0x0F }

Links

Article Author - Sy (Synide) -- Sy 17:16, 11 August 2007 (CEST)

Original ODOLv40 Article detailed by Bxbx (Biki'd by Mikero)

Retrieved from "https://community.bistudio.com/wiki?title=P3D_File_Format_-_ODOLV4x&oldid=44382"