[PySoy-Dev] .soy Codecs
arc at pysoy.org
Sat Dec 16 06:25:04 EST 2006
In order to keep us talking code, even while talking politics:
I'm looking at 2 additions to the .soy data format:
1 Wavelet Images
2 Audio clips
PNG is great, but it isn't optimized for OpenGL.
A wavelet image codec (I already have it designed) can manage a degree of
lossless compression (and potentially great lossy compression) while being
much faster to decode -and- downscale.
Let me put it another way.
We've talked about automatic downscaling of textures to fit GPU capabilities,
video ram restrictions, framerate targets, etc. With this format the image
decodes progressivly in a binary method - 1x1 to 2x2, 2x2 to 4x4, 4x4 to
16x16, etc. Each step applies coefficients for the next scale. So it's
/faster/ for the CPU to decode a texture at a reduced size, not slower.
In addition, we can loose the image size checks and related error reporting
since all wavelet images are 2^n square. The simplified decoding could even,
potentially, be moved to some GPUs for even faster decoding and on-the-fly
mipmapping directly from video ram.
I, or some volunteer, will write a PIL-based PNG/JPEG/BMP/etc -> .soy
conversion utility in addition to (prehaps) a GIMP load/save plugin.
It's costly to execute the functions needed to load an audio clip saved
as a file. Especially if the hard disk has spun down already.
It's also costly to load and decode all possible audio clips to memory.
It's also difficult to manage potentially hundreds of small audio files,
much less load them, nor do you really want to store them as one large audio
file with time offsets (seeking isn't a fast operation).
Thus I propose embedding Speex (and prehaps a lossless codec) into .soy.
I've worked with Speex before, it's packet sizes are small and is decent
for human voice at 16kbit/sec.
I don't know what these proposals mean for Blender. Blender has support for
both audio objects and image textures, so it should be able to support both of
these from the same file, but someone more familiar with Blender's API will
need to look into it more.
More information about the PySoy-dev