Summary
The whitespace run that a w or W blank consumes is unbounded in PolyFile. libmagic stops the
skip at the end of the buffer it copied the candidate bytes into, so a long enough run makes the
comparison fail.
Reproducer
$ printf '0\tstring/W\tA\\ B\tfound\n' > w.magic
$ python3 -c "open('short.bin','wb').write(b'A' + b' ' * 125 + b'B')"
$ python3 -c "open('long.bin','wb').write(b'A' + b' ' * 126 + b'B')"
$ LC_ALL=C TZ=UTC ./file/src/file -b -m w.magic short.bin
found
$ LC_ALL=C TZ=UTC ./file/src/file -b -m w.magic long.bin
ASCII text, with no line terminators
PolyFile on master at 54696a1 reports found for both, and for a run of any length. The w
spelling behaves the same way.
Cause
file_strncmp bounds the skip by eb, which it sets from the size of the buffer the candidate
bytes were copied into rather than from the file
(file/src/softmagic.c:2070 and 2303-2304):
const unsigned char *eb = b + (ws ? maxlen : len);
...
v = file_strncmp(m->value.s, p->s, CAST(size_t, m->vallen),
sizeof(p->s), m->str_flags);
sizeof(p->s) is MAXstring, 128 (file/src/file.h:179 and 216). Once the skip stops on a
whitespace byte because it reached eb, the next byte of the value is compared against that
whitespace byte and the comparison fails. The observed cutoff is a run of 126 blanks for the
three-byte value above; the exact arithmetic that puts it there rather than at 127 was not run
down.
StringMatch.pattern_string renders the blank as [ \t\n\v\f\r]+ (or * for w), which has no
upper bound.
Impact
A file with a whitespace run longer than the buffer matches a w or W definition that file
rejects. 29 bundled definitions carry W with a blank in the value and several hundred carry w,
so the reach is wide, though a run that long is unusual in the formats they describe.
Found while fixing #3562, which does not introduce it: master diverges on the same input before
and after that change.
Summary
The whitespace run that a
worWblank consumes is unbounded in PolyFile. libmagic stops theskip at the end of the buffer it copied the candidate bytes into, so a long enough run makes the
comparison fail.
Reproducer
PolyFile on
masterat54696a1reportsfoundfor both, and for a run of any length. Thewspelling behaves the same way.
Cause
file_strncmpbounds the skip byeb, which it sets from the size of the buffer the candidatebytes were copied into rather than from the file
(
file/src/softmagic.c:2070and2303-2304):sizeof(p->s)isMAXstring, 128 (file/src/file.h:179and216). Once the skip stops on awhitespace byte because it reached
eb, the next byte of the value is compared against thatwhitespace byte and the comparison fails. The observed cutoff is a run of 126 blanks for the
three-byte value above; the exact arithmetic that puts it there rather than at 127 was not run
down.
StringMatch.pattern_stringrenders the blank as[ \t\n\v\f\r]+(or*forw), which has noupper bound.
Impact
A file with a whitespace run longer than the buffer matches a
worWdefinition thatfilerejects. 29 bundled definitions carry
Wwith a blank in the value and several hundred carryw,so the reach is wide, though a run that long is unusual in the formats they describe.
Found while fixing #3562, which does not introduce it:
masterdiverges on the same input beforeand after that change.