Transforms bounding box by a 4x4 matrix
Original bounding box
4x4 transformation matrix (column-major, flat array)
Optional caller-owned output box
Transformed bounding box
Transforms bounding box by a 4x4 matrix