Add AIFF-C subformats for twos,in24,in32,42ni,23ni - #916
Conversation
Added new option --force-aiff-c-format=FORMAT
| ; /* nothing to do, we already default to big-endian */ | ||
| } | ||
| else { | ||
| flac__utils_printf(stderr, 1, "%s: ERROR: can't handle AIFF-C compression type \"%c%c%c%c\"\n", e->inbasefilename, (char)(xx>>24), (char)((xx>>16)&8), (char)((xx>>8)&8), (char)(xx&8)); |
There was a problem hiding this comment.
This & 8 seems to be a very old bug, all the way from the original implementation in 2005 (commit 15acb69).
It would print either a NUL character or a BS (backspace) character.
I added get_printable_char() to avoid printing garbage characters to the terminal or log file, as this could potentially be exploited in some way wit, e.g. unicode right-to-left override or escape sequence.
| if(!generate_unsigned_raw(fn, channels, bits_per_sample/8, nsamples[samples])) | ||
| return 1; | ||
| if(bits_per_sample <= 24) { | ||
| /* unsigned 32 bps raw does not work */ |
There was a problem hiding this comment.
The raw unsigned test fails for 32 bits per sample. Note that I increased it to 32 and the max value tested was only up to 24 before.
| : Instruct the decoder to output an AIFF-C file with format NONE and | ||
| sowt respectively. | ||
|
|
||
| **\--force-aiff-c-format**={none\|sowt\|twos\|in24\|42ni\|in32\|23ni} |
There was a problem hiding this comment.
Other format types would be 'raw ', but seems to be very rarely used.
'in16'/'61ni' is another one for 16-bit audio, which seems to be used only internally in CoreAudio, but very rarely in actual AIFF-C files.
|
raw is 8-bit signed. Easily produced with |
I had it initially, but decided to not include it, due to it being rare, and its endianess is ambigious beyond 8 bits. You have to have both encode and decode support, otherwise foreign metadata is not able to restore the original file back. |
But I realized now this contradicts the findings here: https://hydrogenaudio.org/index.php/topic,127436.25.html If flac is going to support 'raw ', it should be unsigned then, to atleast keep in sync with afconvert. I can push another commit with these changes. The problem then becomes if there is ever a need to support both signed and unsigned for 'raw '. If so that means flac must store some additional metadata, since there is nothing itself in the AIFF-C format (and flac's foreign metadata of it) that can recover the signedness of the audio. |
|
1: you are right I was dumb, 2: what about ... something that does the bit depth for the user?
Thanks for arresting thoughtless swiney me being too dumb to think twice. raw is 8-bit unsigned like WAVE 8-bit (but unlike the traditional AIFF which like AU/SND writes 8-bit as signed). .
Also, what is the capitalization in the AIFF files? |
Yes, it is a good idea. I was more thinking of a =le and =be syntax, since it is not obvious what =ni and =in stands for.
But then what if you want 'twos' instead of 'NONE'? And 'raw ' for 8 bit, or if you don't want 'raw '. Seems we need some kind of profile mapping for this, e.g. a list of preferred AIFC compression types and what bits they should be used for. (There's also 'in16'/'61ni' for 16-bits, but I don't want to promote or make flac legitimate its usage further.)
Only 'NONE' is upper-case, everything else is (supposedly) lower-case. |
|
Uh-oh, stop press. According to https://developer.apple.com/documentation/avfaudio/avaudiocommonformat/pcmformatint16 QuickTime doc does not cite endianness for these. It does for twos and sowt. Of course when https://developer.apple.com/standards/qtff-2001.pdf was issued, endianness would default to Big. But now there is a source saying native. Also here is how ffmpeg mishandled little-endian in24 in .mov. Of course AIFC lacks the QuickTime format's endianness atom: https://trac.ffmpeg.org/ticket/1881 This found by asking AI engines to get a pseudo-confirmation on what is supposed to be right about 8-bit ... this was a byproduct. |
An interesting point. When ffmpeg does a codec copy, does it also retain the 'enda' chunk? So you could for example have two .mov files with 'in24', and 'enda' specified as little-endian in one and big-endian in the other. A plain codec copy would copy the data as-is, but one file would for sure be interpreted wrongly in most (if not all) players and software. Unless ffmpeg switches from 'in24' to '42ni' for one file? The main purpose of doing this in flac is to increase interoperability with existing tools (e.g. sox, ffmpeg, libsndfile), and if they all agree that 'in24' is big-endian and '42ni' is little-endian, then that problem is solved, as it has already become the de facto standard way to write AIFF-C files. |
|
Edit: ... erruhm ... "native" maybe still means "big" and Apple didn't even change it? Others started speaking of "network endian" if they had an "n" to explain ... Anyway,: Here is support for your notion, but it is a wiki entry, not official and even if correct once, obsolete the moment Apple decides to fart in another general direction: https://wiki.multimedia.cx/index.php/PCM#Apple_QuickTime_Identifiers So what then? --force-aiff-c-big-in-format and --force-aiff-c-little-ni-format?
Also I see that Apple for QuickTime expects NONE to be deprecated in favour of raw, so is there really any signed 8-bit left? twos is for 16, NONE would be signed but is now supposedly unsigned, ...? Meh. Fruitcake. |
I don't think anything outside Apple supports 'in16'/'61ni'. sox_nghttps://codeberg.org/sox_ng/sox_ng/src/branch/main/src/aiff.c#L991 libsndfileReads 'NONE', 'raw ', 'twos'/'sowt', 'in24'/'42ni', 'in32'/'23ni': ffmpegUses 'NONE' for signed 8-bit and 'raw ' for unsigned 8-bit. 'NONE' can also be used for big-endian 16/24/32-bit, and 'in24'/'in32' for 24/32-bit big-endian only. But I don't see such check for 'sowt', so 'sowt' only works for 16-bit. WavPackAble to read 'NONE', 'twos', 'sowt' and 'raw '. 'raw ' is unsigned, but beyond 8-bits I think it is treated as signed. Worth noting is that WavPack handles 'none', 'TWOS', 'SOWT', and 'RAW ' also. I haven't seen any other software do this. When writing, WavPack only writes 'NONE' and 'sowt', never 'raw ' or 'twos': |
|
Good detective work. Did you check whether libsndfile interprets 8-bit NONE as signed when reading it? In view of this picture, we have at least some support for an option that for 24 and 32 covers "in" as BE and "ni" as LE, while for 8 and 16:
Allowing "twos" for 8-bit has some support in the 2001 QuickTime doc https://developer.apple.com/standards/qtff-2001.pdf: it says (page 103) about "twos" that Above suggestion does not write any "NONE", presuming that those who want it would just keep using As goes WavPack's "raw ", support introduced out of my pointing out that ffmpeg would write it. That WavPack also supports "in case someone does it wrong" for cases and >8 raw, isn't much of an indication than "whatever stupid application might write", since WavPack will reconstruct files warts-and-all but AFAIK offers no option to otherwise force-select between AIFF versions or WAVE versions. These flac options for the latter situation which WavPack doesn't really provide for. No reason to support wrong-case as a chosen option. Tasks?
|
Yes, 'NONE' is always signed (SF_FORMAT_PCM_S8), and support any bit depth. 'raw ' is unsigned 8-bit only (SF_FORMAT_PCM_U8):
For 8-bit signed, since the endianess doesn't matter (the data is encoded identically), it makes sense to prefer 'NONE' rather than 'sowt' or 'twos'. This is maybe how 'sowt' became known as 16-bit only, since before 24-bit audio was common, there was only 16-bit that 'sowt' could be used for (to select little-endian). This was also one of things I wanted to take a closer look at; how 8-bit is handled. Atleast it seems that 'raw ' is always unsigned 8-bit, and 'NONE' is signed big-endian for any bit depth. Most software agrees on 16-bit 'NONE' big-endian and 'sowt' little-endian. What differs is how 24/32-bit audio should be handled. There is no solution that works for everything; best practice I can recommended (based on ffmpeg and wavpack as the lowest common denominators) is to use 'NONE' for everything; since ffmpeg doesn't support arbitrary bit depths for 'sowt', and wavpack doesn't support 'in24' and 'in32'. Then there's also how 16/24/32-bit audio in 'raw ', there is no consensus, best practice seems to be "don't do it".
I thought WavPack could always decode back to .wav, atleast I think that's a nice feature in flac.
I think this is done then? 'NONE' seems to always be signed in all software I have checked.
The reason I added arguments for the new --force-aiff-format was that I dislike having too many separate options; I prefer to have a single option with many arguments instead. It improves documentation and discoverability. For comparison, what if --bps=# was separate options: --bps-8, --bps-16, --bps-24 and so on. If someone tries --bps-40, it is harder to give a meaningful error message that the max supported bps is 32, compared to --bps=40.
I think for the existing NONE/sowt you don't need a warning. Edit: To clarify, I mean that the --force-*-format options are expert options, e.g. "do exactly what I asked". It would be like if --force (Force overwriting of output files) still printed a warning if the output file existed. A better or smarter AIFF-C compression type selection is maybe not even possible. If we always need to print a warning whenever 24-bit audio is used, some users will be unhappy about it. If we think about this, when do you really need to use 'sowt' and little-endian? Even a quick and dirty AIFF reader would certainly implement support for 'NONE' and big-endian, since that is that the only thing that the original AIFF (not AIFC) supports. It would be rare to have a AIFC reader that only supports little-endian audio. But the same could be said for --force-extensible-wave-format, as historically there are far too many WAV readers that doesn't correctly handle the 'fmt ' chunk being either 16, 18 or 40 bytes. |
|
Reason why I advocated "twos" is that there is already a
I don't even know a counterexample.
... and that - NONE for all - we have an option for.
Never seen one.
Yes, and you can choose AIFF/AIFC too, but you cannot select among AIFC flavours. wvunpack has (... and choices for DSD too.)
That choice has I think been made already, as there are Let's stay consistent. I think there should be a |
wvunpack also has this rather surprising feature:
Similarly, this gives no errors and no warnings, but doesn't do what you think it does (as you've found out): Both wavpack and flac really need some warning if the output file extension doesn't match the actual format used.
Alright, I agree with you on consistency. But then I assume you don't want all these options? Maybe could repurpose --sign=unsigned if someone really wanted 'raw ' for 8-bits, and use --endian=little or --endian=big to select between the two groups:
But maybe this complicates it even further, so let's go back to your original suggestion about --force-aiff-c-in-format and --force-aiff-c-ni-format. We could disallow 'raw ' from being used (but still accept encoding from it and restoring via foreign metadata). Or add --force-aiff-c-raw-format. The existing --force-aiff-c-sowt-format can still be used for force 'sowt' for everything, but with a warning about compatibility for bits != 16. --force-aiff-c-none-format would remain as-is with no such warning, since it seems most existing software can handle arbitrary bit depths for 'NONE', atleast when reading.
The only safe decoder chooses format would be 'NONE', since that is the only one that seems to work across bit depths. |
Uh-oh. That's raw PCM, not AIFC with "raw " fourcc. Don't mix that up. I was assuming that if the user asks for a "little-endian option" they would want what WAVE uses, which for 8-bit is unsigned. Like this
Argument against "NONE" is that there is the
I tend to agree although it isn't a given that a decoder chooses would have to choose one and the same for all signals. Decoder doesn't do that upon |
Yes that was my conclusion as well. But considering supporting future formats such as CAF that also comes in little- or big-endian, it might be a better user experience to have one single command line option to select the endianess across all formats (or atleast those that supports this), not just for raw PCM. (As another example, --force-extensible-wave-format is already a bit limited in that you can't use it to affect the 'fmt ' chunk for WAVE64 or RF64. Adding more options like --force-rf64-extensible-format and --force-wave64-extensible-format would add unnecessary complexity in my opinion. Better then to have a separate option that affects the 'fmt ' chunk size and format independently for WAV/RF64/WAVE64. YAGNI also applies here, any software that supports RF64/WAVE64 probably also supports all 'fmt ' chunk variants.)
'raw ' is a bit of an odd format in AIFF though, I don't think it should be the default option for 8-bit. The original AIFF is signed only, even CAF is signed only.
My point was that there is no such safe format for 24/32-bit in AIFF-C: Not all software supports 'inXX', not all software supports 24-bit 'sowt', not all software supports 'twos'; so what we're left with is 'NONE'. |
Warning for (honest) whining: I agree with you that --force-enormously-many-different-such-options-without-argument is a meh, and I even dislike that those start with "--force" as But for now ... |
Added new option --force-aiff-c-format=FORMAT