Missing command from decoder causes visual artifacts

[deakjahn]

While preparing my own code for the task, I spotted a major problem with your SCoA decoder. You mentioned in line 200 the presence of a supposed captfilter bug that has to be worked around.

studycapt/scoa.py

Lines 200 to 205 in 6207b8f

	# work around repeat+new with zero counts,
	# suspected to be captfilter encoder bugs,
	# by holding back input iterator and writing
	# out zeroes instead
	ub = (0x0 for i in range(nu))
	nu = 0

There's no bug in reality, just an omission from your code and the Specification file. There's yet another command pair, named RepeatX/RepeatXLong in the driver. Members of this 0b11 family can appear in many different places in the print file and it was obviously an afterthough in the format and the way they implemented and used it doesn't offer a particularly clean way to interpret them. Of course, for the driver side, you can just ignore and never emit them, the printed data can just as well be described with the remaining codes. But if you want to decode, you have to: all your remaining visual artifacts are attributable to these codes.

Basically, these are run count extenders for copy operations. The RepeatX name is probably a misnomer, it isn't about repeat, it's about repeated extenders. The difference in whether they have nr or nu zero or non-zero are not bugs at all but mark various subclasses in this family.

1-byte opcodes

The 0b11 branch needs a three-way split:

• if both encoded values are non-zero, a RepeatThenRaw operation, you get that right;
• if the low value is zero, a CopyThenRaw operation with the high value as copy count (copy only, no raw);
• if the high value is zero, a CopyThenRaw operation with the low value as copy count (copy only, no raw).

You say the last two subcases are driver bugs but they actually aren't, they are intended, as it can be seen with reverse engineering.

2-byte opcodes, long

You have to add a branch to handle 0b11 codes. The handler has to check for the presence of an extender chain, basically similar to 9F/0b100 chains in this main branch, but still different. You can only expect 2-byte RepeatXLong commands here, finished by a 1-byte RepeatX one. You can recognize such a chain up front from a first byte of 8E and a follower that has its low value on max, 7. Either with extended length or not, it will denote a CopyThenRaw operation with no raw data (nu = 0).

---

This gets rid of the remaining visual artifacts observable from the decoder.

I tried with test files generated on Linux with version 2.71 of the driver (this is the currently available and probably last one).

[mounaiban]

Thanks @deakjahn for figuring out the missing opcodes after over three years! I hope this completes our knowledge of the SCoA codec... 😸

The lack of copy-only opcodes bothered me at one point, but I thought it was just a quirk that could be safely ignored. Now that you have discovered the copy-only opcodes, it is looking to me that copy-only operations are really helpful (or maybe even essential) to any RLE codec.

I almost figured out 0b11XXXYYY where XXX was zero, before giving up after failing to make it appear with my samples generated by sample_blots.py. I was thus unable to investigate further. AFAIK, I don't think any of the opcodes you discovered can be made to appear with my sample generator, which was why I could not figure them out.

I'll add the opcodes probably after giving scoa.py a much needed refactoring.

[deakjahn]

It's probably not worth the effort to put them into your driver, the few kilobytes that you spare, at most, are less than a rounding error these days. But you can't avoid them if you want to decode files generated by the original drivers.