Add the relation type to relatedAnime by AkitoTheGambler · Pull Request #3 · cedya77/modb-app

AkitoTheGambler · 2026-10-28T01:39:28Z

Each entry in relatedAnime now contains the kind of relation as the metadata provider labels it.

Format

Before:

"relatedAnime": ["https://myanimelist.net/anime/2994"]

After:

"relatedAnime": [
  { "uri": "https://myanimelist.net/anime/2994", "relation": "Summary" },
  { "uri": "https://livechart.me/anime/3808" }
]

relation is the provider's own label, only whitespace is normalised (e.g. Sequel on MAL, SEQUEL on AniList, sequel on Kitsu). Labels are not unified across providers.
relation is omitted if the provider doesn't name the relation: LiveChart (no labels at all), AniDB "indirectly related" entries and aniSearch entries labelled ?.
Reading still accepts the old string format, so existing DCS files and datasets keep working.
If the same URI is added twice, an entry with a relation replaces one without.

Where the labels come from

Provider	Source of the label
AniList	`relations.edges[].relationType`
MyAnimeList	label cell of the related entries table and of the tiles
AniDB (API)	`relatedanime/anime/@type`
AniDB (website)	`div.type` of directly related entries
Kitsu	`attributes.role`
anime-planet	`RelatedEntry__subtitle`
ANN	"sequel of" header or "(sequel)" after the link
aniSearch	`th span` of the relations table, `?` is treated as unknown
Simkl	`tvdetailrelationsitemimagetype`
LiveChart	no label available

Code changes

core
- New RelatedAnime(uri, relation). AnimeRaw and Anime use HashSet<RelatedAnime>. The URI-based addRelatedAnime overloads are kept.
- New RelatedAnimeAdapter (Moshi). Reads strings and objects, writes objects.
- New html() terminating function for both extractors and the extractLabelledLinks helper, which pair each label with its links.
Converters: all nine read the relation label.
lib
- AnimeCountdownUrlAdder keeps the relation when deriving animecountdown links from Simkl.
- DefaultAmbiguousSynonymRemover adapted to the new type.
- The README generator documents the RelatedAnime object and shows it in the example.
Tests
- Updated for all converters, the adapters and the README generator.
- New tests for RelatedAnime, RelatedAnimeAdapter and extractLabelledLinks.
- Assertions on relatedAnime sets no longer depend on iteration order.

Breaking change

Consumers which expect relatedAnime to be a list of strings have to be updated. The JSON schemas of the dataset need the items changed from a string to an object:

"relatedAnime": {
  "type": "array",
  "uniqueItems": true,
  "items": {
    "type": "object",
    "required": ["uri"],
    "properties": {
      "uri": { "type": "string", "format": "uri" },
      "relation": { "type": "string", "minLength": 1 }
    }
  }
}

🤖 Generated with Claude Code

relatedAnime entries are now objects: {uri, relation}. relation is the metadata provider's own label and is omitted if the provider doesn't name it. The old string format can still be read.

Copilot

🟡 Changes recommended

Comma-separated ANN header relations lose labels after the first, dropping the “compilation of” relation for anime 26; the parser needs a fix and a regression assertion for the affected fixture.

1 open issue

Preserve comma-separated ANN header relations · New

What changed in this PR

Adds provider-specific relation labels to relatedAnime throughout the metadata pipeline. JSON output changes from URL strings to objects, while readers retain support for legacy strings.

Changes:

Introduces RelatedAnime, JSON adapters, and URI-based deduplication that prefers labelled entries.
Updates provider converters and shared extractors to capture relation labels.
Adapts downstream processing, tests, fixtures, and generated documentation to the new format.

File	Description
simkl/src/test/kotlin/io/github/manamiproject/modb/simkl/SimklAnimeConverterTest.kt	Asserts Simkl relation labels.
simkl/src/main/kotlin/io/github/manamiproject/modb/simkl/SimklAnimeConverter.kt	Extracts labels alongside related links.
serde/src/test/kotlin/io/github/manamiproject/modb/serde/TestingAssets.kt	Migrates serialization fixtures to objects.
myanimelist/src/test/kotlin/io/github/manamiproject/modb/myanimelist/MyanimelistAnimeConverterTest.kt	Asserts table and tile relation labels.
myanimelist/src/main/kotlin/io/github/manamiproject/modb/myanimelist/MyanimelistAnimeConverter.kt	Extracts labelled table rows and tiles.
livechart/src/test/kotlin/io/github/manamiproject/modb/livechart/LivechartAnimeConverterTest.kt	Expects unlabelled relation objects.
livechart/src/main/kotlin/io/github/manamiproject/modb/livechart/LivechartAnimeConverter.kt	Wraps franchise links without labels.
lib/src/test/resources/readme/DefaultReadmeCreatorTest/zero-based-week.md	Updates documentation snapshot.
lib/src/test/resources/readme/DefaultReadmeCreatorTest/reviewed-isolated-entries.md	Updates documentation snapshot.
lib/src/test/resources/readme/DefaultReadmeCreatorTest/merge-locks.md	Updates documentation snapshot.
lib/src/test/resources/readme/DefaultReadmeCreatorTest/expected-default.md	Updates default documentation snapshot.
lib/src/test/resources/readme/DefaultReadmeCreatorTest/animecountdown-with-zero-entries.md	Updates documentation snapshot.
lib/src/test/kotlin/io/github/manamiproject/modb/app/TestingAssets.kt	Migrates application fixtures to objects.
lib/src/test/kotlin/io/github/manamiproject/modb/app/synonyms/DefaultAmbiguousSynonymRemoverTest.kt	Adapts synonym-removal fixtures.
lib/src/test/kotlin/io/github/manamiproject/modb/app/relatedanime/DefaultUnknownRelatedAnimeRemoverTest.kt	Adapts relation-removal expectations.
lib/src/test/kotlin/io/github/manamiproject/modb/app/merging/matching/DefaultMatchingProbabilityCalculatorTest.kt	Adapts matching fixtures.
lib/src/test/kotlin/io/github/manamiproject/modb/app/merging/goldenrecords/DefaultGoldenRecordAccessorTest.kt	Adapts merging fixtures and unordered assertions.
lib/src/test/kotlin/io/github/manamiproject/modb/app/merging/DefaultMergingServiceTest.kt	Adapts merging-service fixtures.
lib/src/test/kotlin/io/github/manamiproject/modb/app/downloadcontrolstate/DownloadControlStateEntryTest.kt	Adapts download-state fixtures.
lib/src/test/kotlin/io/github/manamiproject/modb/app/downloadcontrolstate/DefaultDownloadControlStateUpdaterTest.kt	Adapts updater fixtures.
lib/src/test/kotlin/io/github/manamiproject/modb/app/downloadcontrolstate/DefaultDownloadControlStateSchedulerTest.kt	Adapts scheduler fixtures.
lib/src/test/kotlin/io/github/manamiproject/modb/app/dataset/AnimeCountdownUrlAdderTest.kt	Tests relation-label preservation.
lib/src/main/kotlin/io/github/manamiproject/modb/app/synonyms/DefaultAmbiguousSynonymRemover.kt	Looks up related entries by URI.
lib/src/main/kotlin/io/github/manamiproject/modb/app/readme/DefaultReadmeCreator.kt	Documents relation objects and examples.
lib/src/main/kotlin/io/github/manamiproject/modb/app/dataset/AnimeCountdownUrlAdder.kt	Preserves labels on derived links.
kitsu/src/test/kotlin/io/github/manamiproject/modb/kitsu/KitsuAnimeConverterTest.kt	Asserts Kitsu relation roles.
kitsu/src/main/kotlin/io/github/manamiproject/modb/kitsu/KitsuAnimeConverter.kt	Reads relationship role attributes.
core/src/test/kotlin/io/github/manamiproject/modb/core/TestingAssets.kt	Migrates core serialization fixtures.
core/src/test/kotlin/io/github/manamiproject/modb/core/json/RelatedAnimeAdapterTest.kt	Tests legacy reads and object serialization.
core/src/test/kotlin/io/github/manamiproject/modb/core/json/JsonKtTest.kt	Updates expected JSON objects.
core/src/test/kotlin/io/github/manamiproject/modb/core/json/AnimeRawAdapterTest.kt	Adapts source-overlap fixture.
core/src/test/kotlin/io/github/manamiproject/modb/core/extractor/LabelledLinksTest.kt	Tests HTML snippets and label pairing.
core/src/test/kotlin/io/github/manamiproject/modb/core/extractor/DescendantTest.kt	Covers the `html()` terminator.
core/src/test/kotlin/io/github/manamiproject/modb/core/anime/RelatedAnimeTest.kt	Tests normalization, sorting, and formatting.
core/src/test/kotlin/io/github/manamiproject/modb/core/anime/AnimeRawTest.kt	Tests labelled additions, deduplication, and merging.
core/src/main/kotlin/io/github/manamiproject/modb/core/json/RelatedAnimeAdapter.kt	Reads both formats and writes objects.
core/src/main/kotlin/io/github/manamiproject/modb/core/json/Json.kt	Registers the relation adapter.
core/src/main/kotlin/io/github/manamiproject/modb/core/json/AnimeRawAdapter.kt	Uses the new relation-set adapter.
core/src/main/kotlin/io/github/manamiproject/modb/core/json/AnimeAdapter.kt	Uses the new relation-set adapter.
core/src/main/kotlin/io/github/manamiproject/modb/core/extractor/LabelledLinks.kt	Adds reusable label/link extraction.
core/src/main/kotlin/io/github/manamiproject/modb/core/extractor/JsoupXPathDataExtractor.kt	Supports outer-HTML extraction.
core/src/main/kotlin/io/github/manamiproject/modb/core/extractor/JsoupCssSelectorDataExtractor.kt	Supports outer-HTML extraction.
core/src/main/kotlin/io/github/manamiproject/modb/core/extractor/Descendant.kt	Recognizes the `html()` terminator.
core/src/main/kotlin/io/github/manamiproject/modb/core/anime/RelatedAnime.kt	Defines relation objects and normalization.
core/src/main/kotlin/io/github/manamiproject/modb/core/anime/AnimeRaw.kt	Adds relation-aware storage and deduplication.
core/src/main/kotlin/io/github/manamiproject/modb/core/anime/Anime.kt	Changes related entries to objects.
anisearch/src/test/kotlin/io/github/manamiproject/modb/anisearch/AnisearchAnimeConverterTest.kt	Asserts known and unknown labels.
anisearch/src/main/kotlin/io/github/manamiproject/modb/anisearch/AnisearchAnimeConverter.kt	Extracts row labels and omits `?`.
animenewsnetwork/src/test/kotlin/io/github/manamiproject/AnimenewsnetworkAnimeConverterTest.kt	Asserts header and trailing labels.
animenewsnetwork/src/main/kotlin/io/github/manamiproject/AnimenewsnetworkAnimeConverter.kt	Parses relation text beside links.
anime-planet/src/test/kotlin/io/github/manamiproject/modb/animeplanet/AnimePlanetAnimeConverterTest.kt	Asserts optional relation subtitles.
anime-planet/src/main/kotlin/io/github/manamiproject/modb/animeplanet/AnimePlanetAnimeConverter.kt	Extracts relation subtitles.
anilist/src/test/kotlin/io/github/manamiproject/modb/anilist/AnilistAnimeConverterTest.kt	Asserts AniList relation types.
anilist/src/main/kotlin/io/github/manamiproject/modb/anilist/AnilistAnimeConverter.kt	Reads relation types from edges.
anidb/src/test/kotlin/io/github/manamiproject/modb/anidb/AnidbApiAnimeConverterTest.kt	Asserts the API relation label.
anidb/src/test/kotlin/io/github/manamiproject/modb/anidb/AnidbAnimeConverterTest.kt	Asserts website relation labels.
anidb/src/main/kotlin/io/github/manamiproject/modb/anidb/AnidbApiAnimeConverter.kt	Reads relation type attributes.
anidb/src/main/kotlin/io/github/manamiproject/modb/anidb/AnidbAnimeConverter.kt	Extracts labels from related-entry blocks.

_{Review effort: Balanced}

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

AkitoTheGambler · 2026-10-28T02:28:06Z

TL;DR:
Added the type of relationship the anime has, so Fribb/anime-lists and other tools can correctly generate franchise lists and groups.
Without it, some very big franchises are generated because of crossover episodes and shows, for example causing One Piece, Toriko and DBZ to be generated as one franchise because of crossover episodes.
With the included relationship type, the generator for Fribb/anime-lists can detect crossovers and stop and separate at the correct places.

Currently running a full recreation of the metadata.
Should take a few days, but is almost done. Will be made available as soon as it's complete, to reuse and save time for the next scan after merge.

AkitoTheGambler · 2026-10-29T19:55:35Z

@cedya77
The archive linked below contains a rescan of all anime sites with the new relationship type for related anime, except for AniDB, because I got API banned after 400 API requests.
It now gets logged like this if the other merge request #4 with general fixes and improvements is also merged:

19:51:38.241 [Network-Dispatcher-15] INFO IntegerBasedIdRangeSelector - [anidb.net] highest ID is [20507]. Skipping [3415] dead entries, [396] entries not scheduled for this week and [0] already downloaded. [16696] of [20507] IDs left to download.
19:52:52.003 [Network-Dispatcher-2] ERROR App - Crawler for [anidb.net] stopped, because AniDB still answers with [banned] after all retries. Wait until the ban has been lifted or add [anidb.net] to [modb.app.disabledCrawlers].

The archive contains the downloads, output, dcs folders and schemas with all entries rechecked (except AniDB), so it can be used for the next run when this is merged instead of rebuilding everything.
https://1drv.ms/f/c/b2e2b319de06d0ed/IgDxjiilTu-RSY3iTz3KiHdPAW5MVKMAcF9O66qQHHibNRk?e=pJlPbC

I will try to increase the min time between AniDB API requests to finish rebuilding the last list for AniDB.
But now all changes are complete and ready for merging, both this and the #4 merge request.

AkitoTheGambler added 2 commits October 23, 2026 02:13

Add relation to relatedAnime

b324843

relatedAnime entries are now objects: {uri, relation}. relation is the metadata provider's own label and is omitted if the provider doesn't name it. The old string format can still be read.

Fix line endings and order-independent related anime assertions

057ddac

Copilot AI balanced review requested due to automatic review settings October 28, 2026 01:39

Copilot started reviewing on behalf of AkitoTheGambler October 28, 2026 01:40

View session

Copilot AI reviewed Oct 28, 2026

View reviewed changes

Comment thread animenewsnetwork/src/main/kotlin/io/github/manamiproject/AnimenewsnetworkAnimeConverter.kt Resolved

AkitoTheGambler and others added 2 commits October 28, 2026 03:17

Fix label extraction logic for related anime

be6752b

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

ANN: test that comma-separated relations keep their own label

8b2c63f

AkitoTheGambler mentioned this pull request Oct 31, 2026

Fixes crossover episodes handling causing very large collections or broken/multiple seperate collections Fribb/anime-lists-generator#16

Open

Navigation Menu

Repository navigation

Add the relation type to relatedAnime - #3

Add the relation type to relatedAnime#3
AkitoTheGambler wants to merge 4 commits into
cedya77:mainfrom
AkitoTheGambler:relation

AkitoTheGambler commented Oct 28, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

AkitoTheGambler commented Oct 28, 2026 •

edited

Loading

Uh oh!

AkitoTheGambler commented Oct 29, 2026 •

edited

Loading

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

AkitoTheGambler commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Format

Where the labels come from

Code changes

Breaking change

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

AkitoTheGambler commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkitoTheGambler commented Oct 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

AkitoTheGambler commented Oct 28, 2026 •

edited

Loading

AkitoTheGambler commented Oct 28, 2026 •

edited

Loading

AkitoTheGambler commented Oct 29, 2026 •

edited

Loading