Skip to content

The w and W blank skip is unbounded, where libmagic stops at MAXstring #3571

Description

@ESultanik

Summary

The whitespace run that a w or W blank consumes is unbounded in PolyFile. libmagic stops the
skip at the end of the buffer it copied the candidate bytes into, so a long enough run makes the
comparison fail.

Reproducer

$ printf '0\tstring/W\tA\\ B\tfound\n' > w.magic
$ python3 -c "open('short.bin','wb').write(b'A' + b' ' * 125 + b'B')"
$ python3 -c "open('long.bin','wb').write(b'A' + b' ' * 126 + b'B')"
$ LC_ALL=C TZ=UTC ./file/src/file -b -m w.magic short.bin
found
$ LC_ALL=C TZ=UTC ./file/src/file -b -m w.magic long.bin
ASCII text, with no line terminators

PolyFile on master at 54696a1 reports found for both, and for a run of any length. The w
spelling behaves the same way.

Cause

file_strncmp bounds the skip by eb, which it sets from the size of the buffer the candidate
bytes were copied into rather than from the file
(file/src/softmagic.c:2070 and 2303-2304):

const unsigned char *eb = b + (ws ? maxlen : len);
...
v = file_strncmp(m->value.s, p->s, CAST(size_t, m->vallen),
    sizeof(p->s), m->str_flags);

sizeof(p->s) is MAXstring, 128 (file/src/file.h:179 and 216). Once the skip stops on a
whitespace byte because it reached eb, the next byte of the value is compared against that
whitespace byte and the comparison fails. The observed cutoff is a run of 126 blanks for the
three-byte value above; the exact arithmetic that puts it there rather than at 127 was not run
down.

StringMatch.pattern_string renders the blank as [ \t\n\v\f\r]+ (or * for w), which has no
upper bound.

Impact

A file with a whitespace run longer than the buffer matches a w or W definition that file
rejects. 29 bundled definitions carry W with a blank in the value and several hundred carry w,
so the reach is wide, though a run that long is unusual in the formats they describe.

Found while fixing #3562, which does not introduce it: master diverges on the same input before
and after that change.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions