[robireton]

Fixes #27

Problem

REDCapAPI#call parses every response with response.json(), which reads the whole body into one string first. V8 caps a string at 0x1fffffe8 characters, so a large records export fails with RangeError: Cannot create a string longer than 0x1fffffe8 characters. REDCapProject.populate() runs into the limit in two places: it exports every record in one request, and in offline mode it reads records-eav.json as a single string.

Changes

• REDCapAPI.recordBatches(options, { batchSize = 500, idField }) is a new async generator. It first exports only the ID field, as flat rows, to list the matching records. That request keeps the caller's filters (filterLogic, events, …) but not fields, forms or type. Duplicate IDs from repeat instances are removed. It then calls records() with the caller's options once per batch.
  • When records is already passed, those IDs are batched as given and no ID export is made.
  • idField defaults to the first metadata field.
  • A batchSize that is not a positive integer throws a RangeError.
• Clearer error: #call now rethrows the string-length RangeError with a message that points to recordBatches(). The original error is kept as cause. The failed-request path in #post is unchanged, so this does not overlap with API errors discard the response body, making import failures undiagnosable #25.
• REDCapProject.populate({ batchSize }) now always loads records through recordBatches. That adds one request for the IDs and one per batch. It passes the record ID field it already has from the metadata, so it doesn't fetch the metadata again.
• Offline mode: when records-eav.json is missing, populate() reads every *.json in a records-eav/ folder in filename order, one batch per file.
• Docs: README and CLAUDE.md are updated. The CLAUDE.md line saying there is no test suite was out of date and is fixed.

Following the issue's non-goals, records() never batches on its own, and no streaming parser is added.

Testing

• New API tests run against the existing node:http test server. They cover:
  • removing duplicate IDs and splitting into batches
  • which options reach the ID request and which reach each batch request
  • the default idField and a given one
  • batching an ID list the caller passes
  • rejecting a bad batchSize
  • leaving the caller's options object unchanged
• A new fixture, test/fixtures/project-split/, holds the project-basic records split across two files. The new test checks that it loads exactly the same data as project-basic.
• A one-off script ran populate({ batchSize: 1 }) through the API against a local fake server. It made one flat request for the IDs, then one EAV request per record, and the records matched the offline fixture.
• The rethrown RangeError is not unit-tested, because triggering it needs a response larger than 512 MB.

npm run lint and npm test pass (84 tests).

🤖 Generated with Claude Code