-
Notifications
You must be signed in to change notification settings - Fork 364
Stop repeated NoSuchFieldError in Jackson 2.16 IAST interner lookup (quick fix) #12670
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
dougqh
wants to merge
15
commits into
master
Choose a base branch
from
dougqh/jackson-216-harden-interner-lookup
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
15 commits
Select commit
Hold shift + click to select a range
bb4e533
Stop repeated NoSuchFieldError in Jackson 2.16 interner lookup
dougqh 630f077
Latch each Jackson field read instead of sharing one flag
dougqh 092eaf0
Address review: state the failure bound, note the aborted first call
dougqh e144957
Let the caller choose the fallback: tryGetOrNull and tryGetOrDefault
dougqh e9f4708
Add Latch.handleNoSuchField and use it for the Jackson field reads
dougqh 79dcee2
Merge branch 'master' into dougqh/jackson-216-harden-interner-lookup
dougqh 5cc2cb3
Rename ByteQuadsCanonicalizer216Helper's INTERNER latch to INTERNER_L…
dougqh a9727bb
Name the Jackson latch constants *_LATCH
dougqh 8ed7aba
Name the Latch hook handle
dougqh 2a1c50b
Add LatchBenchmark for the field-read latch
dougqh 56d50f4
Add LatchBenchmark results
dougqh 94c813d
Rename the Latch hook to apply and the public methods to tryApply*
dougqh 05bfd5a
Drop a dangling ClassLatch reference from Latch's Javadoc
dougqh da1de5f
Add an overridable fallback to Latch
dougqh 2b93e8c
Rename Latch.tryApplyOrNull to tryApply
dougqh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
47 changes: 46 additions & 1 deletion
47
.../jackson-core-2.16/src/main/java/com/fasterxml/jackson/core/json/JsonParser216Helper.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,11 +1,56 @@ | ||
| package com.fasterxml.jackson.core.json; | ||
|
|
||
| import com.fasterxml.jackson.core.sym.ByteQuadsCanonicalizer; | ||
| import com.fasterxml.jackson.core.sym.ByteQuadsCanonicalizer216Helper; | ||
| import datadog.trace.util.Latch; | ||
|
|
||
| /** | ||
| * Reads whether a {@link UTF8StreamJsonParser} interns its field names. | ||
| * | ||
| * <p>This reads package-private Jackson fields ({@code _symbols}, {@code _interner}). Stock | ||
| * jackson-core 2.16+ always has them, but a classpath that mixes Jackson builds can lack them, | ||
| * which surfaces as a {@link NoSuchFieldError}. | ||
| * | ||
| * <p><b>IAST design note:</b> when the fields are missing we cannot tell whether names are | ||
| * interned, and we answer {@code true} ("interned"). The caller then records the current field name | ||
| * but does <em>not</em> taint the name string. This is a deliberate trade-off: | ||
| * | ||
| * <ul> | ||
| * <li>Jackson interns field names by default, so "interned" is the likely answer. | ||
| * <li>Tainting an interned {@code String} taints the one shared instance, so every occurrence of | ||
| * that name (across requests) would look tainted. That is a false-positive risk. | ||
| * <li>The cost is a possible false negative: on such classpaths, an attacker-controlled JSON | ||
| * <em>key</em> reaching a sink is not detected. Values are still attributed to their field | ||
| * name. | ||
| * </ul> | ||
| * | ||
| * <p>Each field read has its own {@link Latch}, here for {@code _symbols} and in {@link | ||
| * ByteQuadsCanonicalizer216Helper} for {@code _interner}, so a classpath missing only one of them | ||
| * keeps using the other. The first failure of each is rethrown so the instrumentation exception | ||
| * handler still reports it. This is not an exactly-once guarantee: threads that race the first | ||
| * failure each rethrow, so a failure is reported at least once, bounded by concurrency. After that | ||
| * the failure is remembered and calls return {@code true} without throwing, so a broken classpath | ||
| * does not cost an exception per parsed field name. | ||
| * | ||
| * <p>The call that hits the failure is aborted by the advice's exception suppression before it | ||
| * reaches {@code setCurrentName}, so that one field name is not tracked and a value read right | ||
| * after it may be attributed to no name, or to the previous one. Later calls are not affected. | ||
| */ | ||
| public final class JsonParser216Helper { | ||
| private JsonParser216Helper() {} | ||
|
|
||
| private static final Latch<UTF8StreamJsonParser, ByteQuadsCanonicalizer, RuntimeException> | ||
| SYMBOLS_LATCH = | ||
| new Latch<UTF8StreamJsonParser, ByteQuadsCanonicalizer, RuntimeException>() { | ||
|
dougqh marked this conversation as resolved.
|
||
| @Override | ||
| protected ByteQuadsCanonicalizer apply(UTF8StreamJsonParser jsonParser) { | ||
| return handleNoSuchField(jsonParser, parser -> parser._symbols); | ||
| } | ||
| }; | ||
|
|
||
| public static boolean fetchInterner(UTF8StreamJsonParser jsonParser) { | ||
| return ByteQuadsCanonicalizer216Helper.fetchInterner(jsonParser._symbols); | ||
| ByteQuadsCanonicalizer symbols = SYMBOLS_LATCH.tryApply(jsonParser); | ||
| // no symbol table to ask: assume interned (see the class comment) | ||
| return symbols == null || ByteQuadsCanonicalizer216Helper.fetchInterner(symbols); | ||
| } | ||
| } | ||
23 changes: 22 additions & 1 deletion
23
...re-2.16/src/main/java/com/fasterxml/jackson/core/sym/ByteQuadsCanonicalizer216Helper.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,9 +1,30 @@ | ||
| package com.fasterxml.jackson.core.sym; | ||
|
|
||
| import datadog.trace.util.Latch; | ||
|
|
||
| /** | ||
| * Reads whether a {@link ByteQuadsCanonicalizer} interns its field names, from the package-private | ||
| * {@code _interner} field. | ||
| * | ||
| * <p>A classpath that mixes Jackson builds can lack the field, which surfaces as a {@link | ||
| * NoSuchFieldError}. That is the same for every canonicalizer, so a single {@link Latch} covers the | ||
| * read: the first failure is rethrown, so the instrumentation exception handler still reports it | ||
| * (at least once, bounded by concurrency, since threads racing the first failure each rethrow), and | ||
| * afterwards the answer is {@code true} ("interned") without throwing. See {@code | ||
| * JsonParser216Helper} for why "interned" is the default. | ||
| */ | ||
| public final class ByteQuadsCanonicalizer216Helper { | ||
| private ByteQuadsCanonicalizer216Helper() {} | ||
|
|
||
| private static final Latch<ByteQuadsCanonicalizer, Boolean, RuntimeException> INTERNER_LATCH = | ||
| new Latch<ByteQuadsCanonicalizer, Boolean, RuntimeException>() { | ||
| @Override | ||
| protected Boolean apply(ByteQuadsCanonicalizer symbols) { | ||
| return handleNoSuchField(symbols, s -> s._interner != null); | ||
| } | ||
| }; | ||
|
|
||
| public static boolean fetchInterner(ByteQuadsCanonicalizer symbols) { | ||
| return symbols._interner != null; | ||
| return INTERNER_LATCH.tryApplyOrDefault(symbols, Boolean.TRUE); | ||
| } | ||
| } |
158 changes: 158 additions & 0 deletions
158
...rc/test/java/datadog/trace/instrumentation/jackson_2_16/core/JsonParser216HelperTest.java
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,158 @@ | ||
| package datadog.trace.instrumentation.jackson_2_16.core; | ||
|
|
||
| import static java.nio.charset.StandardCharsets.UTF_8; | ||
| import static org.junit.jupiter.api.Assertions.assertFalse; | ||
| import static org.junit.jupiter.api.Assertions.assertInstanceOf; | ||
| import static org.junit.jupiter.api.Assertions.assertThrows; | ||
| import static org.junit.jupiter.api.Assertions.assertTrue; | ||
|
|
||
| import com.fasterxml.jackson.core.JsonFactory; | ||
| import com.fasterxml.jackson.core.json.JsonParser216Helper; | ||
| import com.fasterxml.jackson.core.json.UTF8StreamJsonParser; | ||
| import java.io.ByteArrayOutputStream; | ||
| import java.io.IOException; | ||
| import java.io.InputStream; | ||
| import java.lang.reflect.InvocationTargetException; | ||
| import java.lang.reflect.Method; | ||
| import net.bytebuddy.jar.asm.ClassReader; | ||
| import net.bytebuddy.jar.asm.ClassWriter; | ||
| import net.bytebuddy.jar.asm.commons.ClassRemapper; | ||
| import net.bytebuddy.jar.asm.commons.SimpleRemapper; | ||
| import org.junit.jupiter.api.Test; | ||
|
|
||
| class JsonParser216HelperTest { | ||
|
|
||
| private static final String JACKSON_CORE_PREFIX = "com.fasterxml.jackson.core."; | ||
| private static final String CANONICALIZER = | ||
| "com.fasterxml.jackson.core.sym.ByteQuadsCanonicalizer"; | ||
| private static final String UTF8_PARSER = "com.fasterxml.jackson.core.json.UTF8StreamJsonParser"; | ||
|
|
||
| @Test | ||
| void reportsInternedFieldNames() throws Exception { | ||
| JsonFactory factory = new JsonFactory().enable(JsonFactory.Feature.INTERN_FIELD_NAMES); | ||
| UTF8StreamJsonParser parser = (UTF8StreamJsonParser) factory.createParser(json()); | ||
|
|
||
| assertTrue(JsonParser216Helper.fetchInterner(parser)); | ||
| } | ||
|
|
||
| @Test | ||
| void reportsNonInternedFieldNames() throws Exception { | ||
| JsonFactory factory = new JsonFactory().disable(JsonFactory.Feature.INTERN_FIELD_NAMES); | ||
| UTF8StreamJsonParser parser = (UTF8StreamJsonParser) factory.createParser(json()); | ||
|
|
||
| assertFalse(JsonParser216Helper.fetchInterner(parser)); | ||
| } | ||
|
|
||
| /** | ||
| * Simulates a classpath whose {@code ByteQuadsCanonicalizer} has no {@code _interner} field. The | ||
| * first failure is rethrown so it is reported once; later calls assume interned names. | ||
| */ | ||
| @Test | ||
| void rethrowsFirstMissingInternerThenAssumesInterned() throws Exception { | ||
| assertRethrowsOnceThenAssumesInterned(CANONICALIZER, "_interner"); | ||
| } | ||
|
|
||
| /** The same for the parser's {@code _symbols} field, which has its own latch. */ | ||
| @Test | ||
| void rethrowsFirstMissingSymbolsThenAssumesInterned() throws Exception { | ||
| assertRethrowsOnceThenAssumesInterned(UTF8_PARSER, "_symbols"); | ||
| } | ||
|
|
||
| /** | ||
| * The latch state is static, so it is per class loader: a second loader with the same problem | ||
| * must rethrow its own first failure, not inherit the first loader's latch. | ||
| */ | ||
| @Test | ||
| void eachClassLoaderRethrowsItsOwnFirstFailure() throws Exception { | ||
| assertRethrowsOnceThenAssumesInterned(CANONICALIZER, "_interner"); | ||
| assertRethrowsOnceThenAssumesInterned(CANONICALIZER, "_interner"); | ||
| } | ||
|
|
||
| private void assertRethrowsOnceThenAssumesInterned(String className, String missingField) | ||
| throws Exception { | ||
| ClassLoader loader = new MissingFieldClassLoader(className, missingField); | ||
| Object factory = loader.loadClass(JsonFactory.class.getName()).getConstructor().newInstance(); | ||
| Object parser = | ||
| factory.getClass().getMethod("createParser", byte[].class).invoke(factory, (Object) json()); | ||
| Method fetchInterner = | ||
| loader | ||
| .loadClass(JsonParser216Helper.class.getName()) | ||
| .getMethod("fetchInterner", loader.loadClass(UTF8StreamJsonParser.class.getName())); | ||
|
|
||
| InvocationTargetException first = | ||
| assertThrows(InvocationTargetException.class, () -> fetchInterner.invoke(null, parser)); | ||
| assertInstanceOf(NoSuchFieldError.class, first.getCause()); | ||
|
|
||
| assertTrue((boolean) fetchInterner.invoke(null, parser)); | ||
| assertTrue((boolean) fetchInterner.invoke(null, parser)); | ||
| } | ||
|
|
||
| private static byte[] json() { | ||
| return "{\"name\":\"value\"}".getBytes(UTF_8); | ||
| } | ||
|
|
||
| /** | ||
| * Loads jackson-core child-first, renaming one field of one class in the bytecode. The class | ||
| * stays self-consistent, but a lookup of the original field name fails with {@link | ||
| * NoSuchFieldError}, like a mixed or repackaged Jackson on the classpath. | ||
| */ | ||
| private static final class MissingFieldClassLoader extends ClassLoader { | ||
| private final String className; | ||
| private final String field; | ||
|
|
||
| MissingFieldClassLoader(String className, String field) { | ||
| super(JsonParser216HelperTest.class.getClassLoader()); | ||
| this.className = className; | ||
| this.field = field; | ||
| } | ||
|
|
||
| @Override | ||
| protected Class<?> loadClass(String name, boolean resolve) throws ClassNotFoundException { | ||
| if (!name.startsWith(JACKSON_CORE_PREFIX)) { | ||
| return super.loadClass(name, resolve); | ||
| } | ||
| synchronized (getClassLoadingLock(name)) { | ||
| Class<?> clazz = findLoadedClass(name); | ||
| if (clazz == null) { | ||
| clazz = define(name); | ||
| } | ||
| if (resolve) { | ||
| resolveClass(clazz); | ||
| } | ||
| return clazz; | ||
| } | ||
| } | ||
|
|
||
| private Class<?> define(String name) throws ClassNotFoundException { | ||
| try (InputStream in = getParent().getResourceAsStream(name.replace('.', '/') + ".class")) { | ||
| if (in == null) { | ||
| throw new ClassNotFoundException(name); | ||
| } | ||
| byte[] bytes = readAll(in); | ||
| if (name.equals(className)) { | ||
| bytes = renameField(bytes); | ||
| } | ||
| return defineClass(name, bytes, 0, bytes.length); | ||
| } catch (IOException e) { | ||
| throw new ClassNotFoundException(name, e); | ||
| } | ||
| } | ||
|
|
||
| private byte[] renameField(byte[] bytes) { | ||
| ClassWriter writer = new ClassWriter(0); | ||
| SimpleRemapper remapper = | ||
| new SimpleRemapper(className.replace('.', '/') + "." + field, field + "_renamed"); | ||
| new ClassReader(bytes).accept(new ClassRemapper(writer, remapper), 0); | ||
| return writer.toByteArray(); | ||
| } | ||
|
|
||
| private static byte[] readAll(InputStream in) throws IOException { | ||
| ByteArrayOutputStream out = new ByteArrayOutputStream(); | ||
| byte[] buffer = new byte[8192]; | ||
| for (int read = in.read(buffer); read != -1; read = in.read(buffer)) { | ||
| out.write(buffer, 0, read); | ||
| } | ||
| return out.toByteArray(); | ||
| } | ||
| } | ||
| } |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.