feat: PDF auto-conversion

[Kelketek]

Description

This merge request enables auto-conversion of documents to PDF, provided the Gotenberg service is installed and configured.

Supporting information

• Product Proposal this is implementing
• xblocks-core version needed to test this
• Tutor Plugin needed to test this

Testing instructions

1. Follow the instructions in the xblocks-core repository and use the linked version.
2. Install this MFE in development mode according to the directions in the README
3. Check out this branch
4. Go to a course and find PDF in the advanced components. Add it.
5. Attempt to upload a file. There should not be an option to upload office documents, but PDFs should work. Do the same with a library component. Likewise, you should only be able to use PDFs.
6. Install the Tutor plugin for Gotenberg linked above
7. Restart Tutor
8. Go back to the course and try to upload an office document. This should work now.
9. Try with the library block as well.

Best Practices Checklist

We're trying to move away from some deprecated patterns in this codebase. Please
check if your PR meets these recommendations before asking for a review:

• Any new files are using TypeScript (.ts, .tsx).
• Avoid propTypes and defaultProps in any new or modified code.
• Tests should use the helpers in src/testUtils.tsx (specifically initializeMocks)
• Do not add new fields to the Redux state/store. Use React Context to share state among multiple components.
• Use React Query to load data from REST APIs. See any apiHooks.ts in this repo for examples.
• All new i18n messages in messages.ts files have a description for translators to use.
• Avoid using ../ in import paths. To import from parent folders, use @src, e.g. import { initializeMocks } from '@src/testUtils'; instead of from '../../../../testUtils'

[openedx-webhooks]

Thanks for the pull request, @Kelketek!

This repository is currently maintained by @bradenmacdonald.

Once you've gone through the following steps feel free to tag them in a comment and let them know that your changes are ready for engineering review.

🔘 Get product approval

If you haven't already, check this list to see if your contribution needs to go through the product review process.

• If it does, you'll need to submit a product proposal for your contribution, and have it reviewed by the Product Working Group.
  • This process (including the steps you'll need to take) is documented here.
• If it doesn't, simply proceed with the next step.

🔘 Provide context

To help your reviewers and other members of the community understand the purpose and larger context of your changes, feel free to add as much of the following information to the PR description as you can:

• Dependencies

  This PR must be merged before / after / at the same time as ...

• Blockers

  This PR is waiting for OEP-1234 to be accepted.

• Timeline information

  This PR must be merged by XX date because ...

• Partner information

  This is for a course on edx.org.

• Supporting documentation
• Relevant Open edX discussion forum threads

🔘 Get a green build

If one or more checks are failing, continue working on your changes until this is no longer the case and your build turns green.

Details

---

Where can I find more information?

If you'd like to get more details on all aspects of the review process for open source pull requests (OSPRs), check out the following resources:

• Overview of Review Process for Community Contributions
• Pull Request Status Guide
• Making changes to your pull request

When can I expect my changes to be merged?

Our goal is to get community contributions seen and reviewed as efficiently as possible.

However, the amount of time that it takes to review and merge a PR can vary significantly based on factors such as:

• The size and impact of the changes that it introduces
• The need for product review
• Maintenance status of the parent repository

💡 As a result it may take up to several weeks or months to complete a review and merge your PR.

[wittjeff]

PDFs can be made as accessible (or not) as Word or PowerPoint files. I suggest a more general reminder that uploaded files need to be made natively accessible, or also upload clearly identified conforming alternative versions.

[Kelketek]

@wittjeff Looked into it and you're right. I've only ever thought of PDFs as a postscript format for printers and every attempt I've had to try to extract useful text from them tends to be frustrating, but the format supports much better accessibility than I realized.

I've updated the phrasing here. Let me know if it works for you.

[wittjeff]

@Kelketek
I have been working on three separate integrated SaaSs:

1. Automated accessibility testing for web pages, user actions sequences, and courses
2. Automated document conversion remediation (many-to-many format conversion, but PDF is a central one just due to ubiquity) + Manual remediation /clean-up tools.
3. Automated audio/video remediation (captioning, Audio Description, translation) + Manual remediation /clean-up tools.

#2 seems most relevant here. I've been using Docling, originally from IBM Research in Zurich but now technically owned by Linux Foundation, though several IBMers are still working on it full time. My document remediation and benchmarking pipeline is a superset of the Docling components. I'm trying to steer them toward embracing accessibility but it's not a trivial sell (see for example docling-project/docling#4070 and docling-project/docling#3988). So I'm wondering first if perhaps Docling might be a better format-conversion platform for inclusion in Open edX Platform. And if you want to go further down the road into inclusion of a11y remediation features, there might be some opportunities. I'm open to making open source contributions but at the moment that doesn't include all of the work I have done in the last several months.

[Kelketek]

@wittjeff We picked Gotenberg in the relevant product proposal for its overall visual accuracy in the conversion. However, I'm not opposed to us moving to a system that provides more accessibility down the line once it's ready, assuming it's able to preserve that faithfulness.

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 95.95%. Comparing base (de0e1c4) to head (e12c864).
⚠️ Report is 6 commits behind head on master.

Additional details and impacted files

@@            Coverage Diff             @@
##           master    #3201      +/-   ##
==========================================
+ Coverage   95.91%   95.95%   +0.04%     
==========================================
  Files        1397     1397              
  Lines       33558    33629      +71     
  Branches     7914     7941      +27     
==========================================
+ Hits        32187    32269      +82     
+ Misses       1312     1302      -10     
+ Partials       59       58       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[pkulkark]

If someone uploads a .docx, lets it convert, then replaces it with a plain .pdf, this branch only sets the url field. sourceUrl keeps pointing at the old .docx, so learners get a "Download the source document" link to a file that has nothing to do with the PDF they are looking at. Probably worth clearing sourceUrl on this path, at least when the current value was one this component set itself.

[Kelketek]

I think keeping it is the lesser of the two evils here. If the user has the PDF autoconverted, and doesn't like the result, they might go into name-brand Office and print to PDF, and then replace the PDF, not knowing that it implicitly cleared the source URL. I'm going to lean toward the case that prevents data loss even if it might mean that the two documents get a bit out of sync.

In most cases I would not expect the replacement document to be totally unrelated, just out of date.