Difference between revisions of "MDX File Format"

From HIVE
Jump to navigation Jump to search
Line 1: Line 1:
 
[[Category:Warcraft III]]
 
[[Category:Warcraft III]]
 
[[Category:File Formats]]
 
[[Category:File Formats]]
 +
 +
MDX is the binary file format of Warcraft 3 3D models (for the text format, see MDL).
 +
 +
It starts with a simple magic identifier:
 +
<pre>
 +
uint32 = 'MDLX'
 +
</pre>
 +
 +
Note the use of base 256 integers.
 +
This notion is used extensively in the format, to give things meaningful names.
 +
 +
Following the magic identifier, there are chunks in no predefined order, some of which are optional (UNCONFIRMED WHICH ARE REQUIRED!).
 +
Each chunk starts with an 8 byte header, holding its identifier, and its size:
 +
<pre>
 +
Header {
 +
    uint32 identifier
 +
    uint32 size
 +
}
 +
</pre>
 +
 +
To read the file, the headers must be read one by one.
 +
If the identifier is known and supported, read the chunk.
 +
Otherwise, skip `size` bytes and move to the next chunk.
 +
<pre>
 +
// assuming "stream" is some kind of a binary stream.
 +
if (stream.read(4) == "MDLX") {
 +
    while (stream.remaining()) {
 +
        // Construct a new header and read its identifier and size
 +
        Header header = new Header(stream);
 +
 +
        if (isSupported(header.identifier)) {
 +
            // do stuff
 +
        } else {
 +
            // or skip the chunk
 +
            stream.skip(header.size);
 +
        }
 +
    }
 +
}
 +
</pre>
 +
 +
Most of the chunks define arrays of objects, such as animations, textures, meshes, and so on.
 +
Some objects like animations and textures are constant size.
 +
Others, which support animated data, are variable sized.
 +
This will be reflected as a question mark in the sizes of the arrays below.
 +
 +
The standard chunks (their name is again the base 256 representation of their identifier):
 +
<pre>
 +
VERS {
 +
  uint32 version
 +
}
 +
 +
MODL {
 +
  char[80] name
 +
  char[260] animationFileName
 +
  Extent extent
 +
  uint32 blendTime
 +
}
 +
 +
SEQS {
 +
  Sequence[size / 132] sequences
 +
}
 +
 +
GLBS {
 +
  uint32[size / 4] globalSequences
 +
}
 +
 +
TEXS {
 +
  Texture[size / 268] textures
 +
}
 +
 +
SNDS {
 +
  SoundTrack[size / 272] soundTracks
 +
}
 +
 +
PIVT {
 +
  float[size / 12][3] points
 +
}
 +
 +
MTLS {
 +
  Material[?] materials
 +
}
 +
 +
TXAN {
 +
  TextureAnimation[?] animations
 +
}
 +
 +
GEOS {
 +
  Geoset[?] geosets
 +
}
 +
 +
GEOA {
 +
  GeosetAnimation[?] animations
 +
}
 +
 +
BONE {
 +
  Bone[?] bones
 +
}
 +
 +
LITE {
 +
  Light[?] lights
 +
}
 +
 +
HELP {
 +
  Helper[?] helpers
 +
}
 +
 +
ATCH {
 +
  Attachment[?] attachments
 +
}
 +
 +
PREM {
 +
  ParticleEmitter[?] emitters
 +
}
 +
 +
PRE2 {
 +
  ParticleEmitter2[?] emitters
 +
}
 +
 +
RIBB {
 +
  RibbonEmitter[?] emitters
 +
}
 +
 +
EVTS {
 +
  EventObject[?] objects
 +
}
 +
 +
CAMS {
 +
  Camera[?] cameras
 +
}
 +
 +
CLID {
 +
  CollisionShape[?] shapes
 +
}
 +
</pre>
 +
 +
Every field "X" in the object definitions which looks like (X) is optional and might not exist.
 +
These are all animated data fields, and will be explained further below.
 +
 +
The objects:
 +
<pre>
 +
Extent {
 +
  float boundsRadius
 +
  float[3] minimum
 +
  float[3] maximum
 +
}
 +
 +
GenericObject {
 +
  uint32 size
 +
  char[80] name
 +
  uint32 objectId
 +
  uint32 parentId
 +
  uint32 flags
 +
  (KGTR)
 +
  (KGRT)
 +
  (KGSC)
 +
}
 +
 +
Sequence {
 +
  char[80] name
 +
  uint32[2] interval
 +
  float moveSpeed
 +
  uint32 flags
 +
  float rarity
 +
  uint32 syncPoint
 +
  Extent extent
 +
}
 +
 +
Texture {
 +
  uint32 replaceableId
 +
  char[260] fileName
 +
  uint32 flags
 +
}
 +
 +
SoundTrack {
 +
  char[260] fileName
 +
  float volume
 +
  float pitch
 +
  uint32 flags
 +
}
 +
 +
Material {
 +
  uint32 size
 +
  uint32 priorityPlane
 +
  uint32 flags
 +
  uint32 = 'LAYS'
 +
  uint32 layersCount
 +
  Layer[layersCount] layers
 +
}
 +
 +
Layer {
 +
  uint32 size
 +
  uint32 filterMode
 +
  uint32 shadingFlags
 +
  uint32 textureId
 +
  uint32 textureAnimationId
 +
  uint32 coordId
 +
  float alpha
 +
  (KMTF)
 +
  (KMTA)
 +
}
 +
 +
TextureAnimation {
 +
  uint32 size
 +
  (KTAT)
 +
  (KTAR)
 +
  (KTAS)
 +
}
 +
 +
Geoset {
 +
  uint32 size
 +
  uint32 = 'VRTX'
 +
  uint32 vertexCount
 +
  float[vertexCount * 3] vertexPositions
 +
  uint32 = 'NRMS'
 +
  uint32 normalCount
 +
  float[normalCount * 3] vertexNormals
 +
  uint32 = 'PTYP'
 +
  uint32 faceTypeGroupsCount
 +
  uint32[faceTypeGroupsCount] faceTypeGroups
 +
  uint32 = 'PCNT'
 +
  uint32 faceGroupsCount
 +
  uint32[faceGroupsCount] faceGroups
 +
  uint32 = 'PVTX'
 +
  uint32 facesCount
 +
  uint16[facesCount] faces
 +
  uint32 = 'GNDX'
 +
  uint32 vertexGroupsCount
 +
  uint8[vertexGroupsCount] vertexGroups
 +
  uint32 = 'MTGC'
 +
  uint32 matrixGroupsCount
 +
  uint32[matrixGroupsCount] matrixGroups
 +
  uint32 = 'MATS'
 +
  uint32 matrixIndicesCount
 +
  uint32[matrixIndicesCount] matrixIndices
 +
  uint32 materialId
 +
  uint32 selectionGroup
 +
  uint32 selectionFlags
 +
  Extent extent
 +
  uint32 extentsCount
 +
  Extent[extentsCount] extents
 +
  uint32 = 'UVAS'
 +
  uint32 textureCoordinateSetsCount
 +
  TextureCoordinateSet[textureCoordinateSetsCount] textureCoordinateSets
 +
}
 +
 +
TextureCoordinateSet {
 +
  uint32 = 'UVBS'
 +
  uint32 textureCoordinatesCount
 +
  float[textureCoordinatesCount * 2] coordinates
 +
}
 +
 +
GeosetAnimation {
 +
  uint32 size
 +
  float alpha
 +
  uint32 flags
 +
  float[3] color
 +
  uint32 geosetId
 +
  (KGAO)
 +
  (KGAC)
 +
}
 +
 +
Bone {
 +
  GenericObject genericObject
 +
  uint32 geosetId
 +
  uint32 geosetAnimationId
 +
}
 +
 +
Light {
 +
  uint32 size
 +
  GenericObject genericObject
 +
  uint32 type
 +
  float attenuationStart
 +
  float attenuationEnd
 +
  float[3] color
 +
  float intensity
 +
  float[3] ambientColor
 +
  float ambientIntensity
 +
  (KLAS)
 +
  (KLAE)
 +
  (KLAC)
 +
  (KLAI)
 +
  (KLBI)
 +
  (KLBC)
 +
  (KLAV)
 +
}
 +
 +
Helper {
 +
  GenericObject genericObject
 +
}
 +
 +
Attachment {
 +
  uint32 size
 +
  GenericObject genericObject
 +
  char[260] path
 +
  uint32 attachmentId
 +
  (KATV)
 +
}
 +
 +
ParticleEmitter {
 +
  uint32 size
 +
  GenericObject genericObject
 +
  float emissionRate
 +
  float gravity
 +
  float longitude
 +
  float latitude
 +
  char[260] modelFileName
 +
  float lifespan
 +
  float initialiVelocity
 +
  (KPEE)
 +
  (KPEG)
 +
  (KPLN)
 +
  (KPLT)
 +
  (KPEL)
 +
  (KPES)
 +
  (KPEV)
 +
}
 +
 +
ParticleEmitter2 {
 +
  uint32 size
 +
  GenericObject genericObject
 +
  float speed
 +
  float variation
 +
  float latitude
 +
  float gravity
 +
  float lifespan
 +
  float emissionRate
 +
  float length
 +
  float width
 +
  uint32 filterMode
 +
  uint32 rows
 +
  uint32 columns
 +
  uint32 headOrTail
 +
  float tailLength
 +
  float time
 +
  float[3][3] segmentColor
 +
  uint8[3] segmentAlpha
 +
  float[3] segmentScaling
 +
  uint32[3] headInterval
 +
  uint32[3] headDecayInterval
 +
  uint32[3] tailInterval
 +
  uint32[3] tailDecayInterval
 +
  uint32 textureId
 +
  uint32 squirt
 +
  uint32 priorityPlane
 +
  uint32 replaceableId
 +
  (KP2S)
 +
  (KP2R)
 +
  (KP2L)
 +
  (KP2G)
 +
  (KP2E)
 +
  (KP2N)
 +
  (KP2W)
 +
  (KP2V)
 +
}
 +
 +
RibbonEmitter {
 +
  uint32 size
 +
  GenericObject genericObject
 +
  float heightAbove
 +
  float heightBelow
 +
  float alpha
 +
  float[3] color
 +
  float lifespan
 +
  uint32 textureSlot
 +
  uint32 emissionRate
 +
  uint32 rows
 +
  uint32 columns
 +
  uint32 materialId
 +
  float gravity
 +
  (KRHA)
 +
  (KRHB)
 +
  (KRAL)
 +
  (KRCO)
 +
  (KRTX)
 +
  (KRVS)
 +
}
 +
 +
EventObject {
 +
  GenericObject genericObject
 +
  uint32 = 'KEVT'
 +
  uint32 tracksCount
 +
  uint32 globalSequenceId
 +
  uint32[tracksCount] tracks
 +
}
 +
 +
Camera {
 +
  uint32 size
 +
  char[80] name
 +
  float[3] position
 +
  float filedOfView
 +
  float farClippingPlane
 +
  float nearClippingPlane
 +
  float[3] targetPosition
 +
  (KCTR)
 +
  (KTTR)
 +
  (KCRL)
 +
}
 +
 +
CollisionShape {
 +
  GenericObject genericObject
 +
  uint32 type // 0 = cube, 1 = plane, 2 = sphere, 3 = cylinder
 +
  float[?][3] vertices // spheres have one vertex, other shapes have two
 +
  if (type == 2 || type == 3) {
 +
    float radius
 +
  }
 +
}
 +
</pre>
 +
 +
When reading an object with animated data, it will have its own `size` field.
 +
This size includes the size field itself.
 +
If the size is bigger than the static data of the object, it means there are animated data fields left to read.
 +
Each animated data field is a small chunk by itself, this is how it looks:
 +
    AnimatedData {
 +
        uint32 identifier
 +
        uint32 tracksCount
 +
        uint32 interpolationType
 +
        uint32 globalSequenceId
 +
        Track[tracksCount] tracks
 +
    }
 +
 +
The tracks all follow the same structure, however the type of the fields they hold can change.
 +
This is how tracks look:
 +
<pre>
 +
Track {
 +
    uint32 frame
 +
    X value
 +
    if (interpolationType > 1) {
 +
        X inTan
 +
        X outTan
 +
    }
 +
}
 +
</pre>
 +
 +
Note that reading them completely depends on the animated data chunk - the value/inTan/outTan types are based on the identifier, and the existence of inTan/outTan depends on the interpolation type.
 +
 +
Here is a map of all of the track identifiers, types, and meaning:
 +
<pre>
 +
// Node
 +
KGTR: float[3] translation
 +
KGRT: float[4] rotation
 +
KGSC: float[3] scaling
 +
// Layer
 +
KMTF: uint32 textureId
 +
KMTA: float alpha
 +
// Texture animation
 +
KTAT: float[3] translation
 +
KTAR: float[4] rotation
 +
KTAS: float[3] scaling
 +
//Geoset animation
 +
KGAO: float alpha
 +
KGAC: float[3] color
 +
// Light
 +
KLAS: uint32 attenuationStart
 +
KLAE: uint32 attenuationStartEnd
 +
KLAC: float[3] color
 +
KLAI: float intensity
 +
KLBI: float ambientIntensity
 +
KLBC: float[3] ambientColor
 +
KLAV: float visibility
 +
// Attachment
 +
KATV: float visibility
 +
// Particle emitter
 +
KPEE: float emissionRate
 +
KPEG: float gravity
 +
KPLN: float longitude
 +
KPLT: float latitude
 +
KPEL: float lifespan
 +
KPES: float speed
 +
KPEV: float visibility
 +
// Particle emitter 2
 +
KP2E: float emissionRate
 +
KP2G: float gravity
 +
KP2L: float latitude
 +
KP2S: float speed
 +
KP2V: float visibility
 +
KP2R: float variation
 +
KP2N: float length
 +
KP2W: float width
 +
// Ribbon emitter
 +
KRVS: float visibility
 +
KRHA: float heightAbove
 +
KRHB: float heightBelow
 +
KRAL: float alpha
 +
KRCO: float[3] color
 +
KRTX: uint32 textureSlot
 +
// Camera
 +
KCTR: float[3] translation
 +
KCRL: uint32 rotation
 +
KTTR: float[3] targetTranslation
 +
</pre>

Revision as of 15:13, 16 February 2018


MDX is the binary file format of Warcraft 3 3D models (for the text format, see MDL).

It starts with a simple magic identifier:

uint32 = 'MDLX'

Note the use of base 256 integers. This notion is used extensively in the format, to give things meaningful names.

Following the magic identifier, there are chunks in no predefined order, some of which are optional (UNCONFIRMED WHICH ARE REQUIRED!). Each chunk starts with an 8 byte header, holding its identifier, and its size:

Header {
    uint32 identifier
    uint32 size
}

To read the file, the headers must be read one by one. If the identifier is known and supported, read the chunk. Otherwise, skip `size` bytes and move to the next chunk.

// assuming "stream" is some kind of a binary stream.
if (stream.read(4) == "MDLX") {
    while (stream.remaining()) {
        // Construct a new header and read its identifier and size
        Header header = new Header(stream);

        if (isSupported(header.identifier)) {
            // do stuff
        } else {
            // or skip the chunk
            stream.skip(header.size);
        }
    }
}

Most of the chunks define arrays of objects, such as animations, textures, meshes, and so on. Some objects like animations and textures are constant size. Others, which support animated data, are variable sized. This will be reflected as a question mark in the sizes of the arrays below.

The standard chunks (their name is again the base 256 representation of their identifier):

VERS {
  uint32 version
}

MODL {
  char[80] name
  char[260] animationFileName
  Extent extent
  uint32 blendTime
}

SEQS {
  Sequence[size / 132] sequences
}

GLBS {
  uint32[size / 4] globalSequences
}

TEXS {
  Texture[size / 268] textures
}

SNDS {
  SoundTrack[size / 272] soundTracks
}

PIVT {
  float[size / 12][3] points
}

MTLS {
  Material[?] materials
}

TXAN {
  TextureAnimation[?] animations
}

GEOS {
  Geoset[?] geosets
}

GEOA {
  GeosetAnimation[?] animations
}

BONE {
  Bone[?] bones
}

LITE {
  Light[?] lights
}

HELP {
  Helper[?] helpers
}

ATCH {
  Attachment[?] attachments
}

PREM {
  ParticleEmitter[?] emitters
}

PRE2 {
  ParticleEmitter2[?] emitters
}

RIBB {
  RibbonEmitter[?] emitters
}

EVTS {
  EventObject[?] objects
}

CAMS {
  Camera[?] cameras
}

CLID {
  CollisionShape[?] shapes
}

Every field "X" in the object definitions which looks like (X) is optional and might not exist. These are all animated data fields, and will be explained further below.

The objects:

Extent {
  float boundsRadius
  float[3] minimum
  float[3] maximum
}

GenericObject {
  uint32 size
  char[80] name
  uint32 objectId
  uint32 parentId
  uint32 flags
  (KGTR)
  (KGRT)
  (KGSC)
}

Sequence {
  char[80] name
  uint32[2] interval
  float moveSpeed
  uint32 flags
  float rarity
  uint32 syncPoint
  Extent extent
}

Texture {
  uint32 replaceableId
  char[260] fileName
  uint32 flags
}

SoundTrack {
  char[260] fileName
  float volume
  float pitch
  uint32 flags
}

Material {
  uint32 size
  uint32 priorityPlane
  uint32 flags
  uint32 = 'LAYS'
  uint32 layersCount
  Layer[layersCount] layers
}

Layer {
  uint32 size
  uint32 filterMode
  uint32 shadingFlags
  uint32 textureId
  uint32 textureAnimationId
  uint32 coordId
  float alpha
  (KMTF)
  (KMTA)
}

TextureAnimation {
  uint32 size
  (KTAT)
  (KTAR)
  (KTAS)
}

Geoset {
  uint32 size
  uint32 = 'VRTX'
  uint32 vertexCount
  float[vertexCount * 3] vertexPositions
  uint32 = 'NRMS'
  uint32 normalCount
  float[normalCount * 3] vertexNormals
  uint32 = 'PTYP'
  uint32 faceTypeGroupsCount
  uint32[faceTypeGroupsCount] faceTypeGroups
  uint32 = 'PCNT'
  uint32 faceGroupsCount
  uint32[faceGroupsCount] faceGroups
  uint32 = 'PVTX'
  uint32 facesCount
  uint16[facesCount] faces
  uint32 = 'GNDX'
  uint32 vertexGroupsCount
  uint8[vertexGroupsCount] vertexGroups
  uint32 = 'MTGC'
  uint32 matrixGroupsCount
  uint32[matrixGroupsCount] matrixGroups
  uint32 = 'MATS'
  uint32 matrixIndicesCount
  uint32[matrixIndicesCount] matrixIndices
  uint32 materialId
  uint32 selectionGroup
  uint32 selectionFlags
  Extent extent
  uint32 extentsCount
  Extent[extentsCount] extents
  uint32 = 'UVAS'
  uint32 textureCoordinateSetsCount
  TextureCoordinateSet[textureCoordinateSetsCount] textureCoordinateSets
}

TextureCoordinateSet {
  uint32 = 'UVBS'
  uint32 textureCoordinatesCount
  float[textureCoordinatesCount * 2] coordinates
}

GeosetAnimation {
  uint32 size
  float alpha
  uint32 flags
  float[3] color
  uint32 geosetId
  (KGAO)
  (KGAC)
}

Bone {
  GenericObject genericObject
  uint32 geosetId
  uint32 geosetAnimationId
}

Light {
  uint32 size
  GenericObject genericObject
  uint32 type
  float attenuationStart
  float attenuationEnd
  float[3] color
  float intensity
  float[3] ambientColor
  float ambientIntensity
  (KLAS)
  (KLAE)
  (KLAC)
  (KLAI)
  (KLBI)
  (KLBC)
  (KLAV)
}

Helper {
  GenericObject genericObject
}

Attachment {
  uint32 size
  GenericObject genericObject
  char[260] path
  uint32 attachmentId
  (KATV)
}

ParticleEmitter {
  uint32 size
  GenericObject genericObject
  float emissionRate
  float gravity
  float longitude
  float latitude
  char[260] modelFileName
  float lifespan
  float initialiVelocity
  (KPEE)
  (KPEG)
  (KPLN)
  (KPLT)
  (KPEL)
  (KPES)
  (KPEV)
}

ParticleEmitter2 {
  uint32 size
  GenericObject genericObject
  float speed
  float variation
  float latitude
  float gravity
  float lifespan
  float emissionRate
  float length
  float width
  uint32 filterMode
  uint32 rows
  uint32 columns
  uint32 headOrTail
  float tailLength
  float time
  float[3][3] segmentColor
  uint8[3] segmentAlpha
  float[3] segmentScaling
  uint32[3] headInterval
  uint32[3] headDecayInterval
  uint32[3] tailInterval
  uint32[3] tailDecayInterval
  uint32 textureId
  uint32 squirt
  uint32 priorityPlane
  uint32 replaceableId
  (KP2S)
  (KP2R)
  (KP2L)
  (KP2G)
  (KP2E)
  (KP2N)
  (KP2W)
  (KP2V)
}

RibbonEmitter {
  uint32 size
  GenericObject genericObject
  float heightAbove
  float heightBelow
  float alpha
  float[3] color
  float lifespan
  uint32 textureSlot
  uint32 emissionRate
  uint32 rows
  uint32 columns
  uint32 materialId
  float gravity
  (KRHA)
  (KRHB)
  (KRAL)
  (KRCO)
  (KRTX)
  (KRVS)
}

EventObject {
  GenericObject genericObject
  uint32 = 'KEVT'
  uint32 tracksCount
  uint32 globalSequenceId
  uint32[tracksCount] tracks
}

Camera {
  uint32 size
  char[80] name
  float[3] position
  float filedOfView
  float farClippingPlane
  float nearClippingPlane
  float[3] targetPosition
  (KCTR)
  (KTTR)
  (KCRL)
}

CollisionShape {
  GenericObject genericObject
  uint32 type // 0 = cube, 1 = plane, 2 = sphere, 3 = cylinder
  float[?][3] vertices // spheres have one vertex, other shapes have two
  if (type == 2 || type == 3) {
    float radius
  }
}

When reading an object with animated data, it will have its own `size` field. This size includes the size field itself. If the size is bigger than the static data of the object, it means there are animated data fields left to read. Each animated data field is a small chunk by itself, this is how it looks:

   AnimatedData {
       uint32 identifier
       uint32 tracksCount
       uint32 interpolationType
       uint32 globalSequenceId
       Track[tracksCount] tracks
   }

The tracks all follow the same structure, however the type of the fields they hold can change. This is how tracks look:

Track {
    uint32 frame
    X value
    if (interpolationType > 1) {
        X inTan
        X outTan
    }
}

Note that reading them completely depends on the animated data chunk - the value/inTan/outTan types are based on the identifier, and the existence of inTan/outTan depends on the interpolation type.

Here is a map of all of the track identifiers, types, and meaning:

// Node
KGTR: float[3] translation
KGRT: float[4] rotation
KGSC: float[3] scaling
// Layer
KMTF: uint32 textureId
KMTA: float alpha
// Texture animation
KTAT: float[3] translation
KTAR: float[4] rotation
KTAS: float[3] scaling
//Geoset animation
KGAO: float alpha
KGAC: float[3] color
// Light
KLAS: uint32 attenuationStart
KLAE: uint32 attenuationStartEnd
KLAC: float[3] color
KLAI: float intensity
KLBI: float ambientIntensity
KLBC: float[3] ambientColor
KLAV: float visibility
// Attachment
KATV: float visibility
// Particle emitter
KPEE: float emissionRate
KPEG: float gravity
KPLN: float longitude
KPLT: float latitude
KPEL: float lifespan
KPES: float speed
KPEV: float visibility
// Particle emitter 2
KP2E: float emissionRate
KP2G: float gravity
KP2L: float latitude
KP2S: float speed
KP2V: float visibility
KP2R: float variation
KP2N: float length
KP2W: float width
// Ribbon emitter
KRVS: float visibility
KRHA: float heightAbove
KRHB: float heightBelow
KRAL: float alpha
KRCO: float[3] color
KRTX: uint32 textureSlot
// Camera
KCTR: float[3] translation
KCRL: uint32 rotation
KTTR: float[3] targetTranslation