Skip to content

BUG: Cloud analysis status poll is unbounded, a stuck file runs a 60s in-request API loop indefinitely #722

Closed

Description

@Ryokoxx

Every 60 seconds one page request on our site stalled for 10 seconds while the plugin called api.cleantalk.org. That went on for 446 days.

Eight files went to the sandbox on 5 June 2025 and never came back DONE. scanner_update_pscan_files_status polls for any file whose pscan_processing_status is not DONE, with no age limit and no attempt cap (security-malware-firewall.php:1797), so the Cron::removeTask() branch at :1810 was never reached. The task also runs on init (:225), so a visitor pays for the calls before the page renders.

Our host measured it against the origin: 90 requests, 4 stalls of 10.21s, 11.04s, 10.21s and 10.22s on a 2.2s baseline, with outbound connections to api.cleantalk.org during each stall and at no other time. Deleting the 8 records stopped it, 90 requests and 0 stalls, median unmoved.

Why those 8 never resolved I can't tell from outside.

Read against master at 6e23777 (v2.186). The timings are our host's reproduction on our live site, not a clean-room repro.

Activity

  1. svfcode commented on Aug 27, 2026

    @svfcode
    Member

    Thank you for the detailed analysis and measurements — they were very helpful.

    Unfortunately, we won’t be able to accept the proposed changes for architectural reasons — we need a solution that fits the overall cron and cloud analysis design across the product, rather than a one-off patch for a single scenario.

    That said, we’re already working on changes to prevent situations like this: limiting polling for stuck files, properly closing out long-running records, and avoiding impact on frontend requests. We expect to release the fix within the next couple of weeks.

    As a temporary workaround, if you notice similar behavior before the update is released: in Security → Scanner → Analysis log, you can remove entries that haven’t reached a final status for a long time — that should stop periodic polling for those files.

    Thank you for your patience. If you have any additional observations or measurements, please share them — we’ll take them into account when finalizing the fix.

  2. Glomberg commented on Sep 2, 2026

    @Glomberg
    Member

    We expect to release the fix within the next couple of weeks.

    The developing task is https://app.doboard.com/1/task/55892

    @Ryokoxx is any additional attentions here?

  3. Ryokoxx commented on Sep 2, 2026

    @Ryokoxx
    Author

    Nothing further from my end, cheers.
    Deleting the stuck entries on 27 August stopped it and there's been no recurrence since. I'll report back once the fix is out.

  4. Glomberg commented on Sep 3, 2026

    @Glomberg
    Member

    Thank you. Closing.

  5. alexandergull commented on Sep 30, 2026

    @alexandergull
    Member

    Hi @Ryokoxx

    We have fixed the issue you described. All the changes will be included in the next stable release 2.190.

    Also, you can test current changes downloading this asset https://github.com/CleanTalk/security-malware-firewall/releases/download/dev-version/security-malware-firewall.zip

    Thank you.

  6. Ryokoxx commented on Sep 30, 2026

    @Ryokoxx
    Author

    Thanks, much appreciated.

    I'll pick up 2.190 when it's released and report back here once it's been running for a while.

    One question on the third item from the earlier reply, keeping the polling out of frontend requests. As far as I can tell the scheduler still runs on init, so while a file is inside the new 24 hour window its status check can still land on a page load. Is that still planned, maybe under a separate task? No rush, I just want to know whether to keep an eye out for it.

  7. alexandergull commented on Oct 1, 2026

    @alexandergull
    Member

    The last patch only fixes infinite polling.

    If I understand correctly, the case you're describing is related to global Cron tasks that probably fire to run on the init hook if the scheduled time comes.

    Right now we don't plan to refactor the scheduler's work. However, we will discuss this among the team in the near future.

  8. alexandergull commented on Oct 6, 2026

    @alexandergull
    Member

    We decided to make some changes to the cron call logic. The estimated time is about 4 months.

  9. AntonV1211 commented on Oct 27, 2026

    @AntonV1211
    Contributor

    Hello @Ryokoxx !
    We've been working on Cron. Now Cron is no longer running on user GET requests. It only works on POST requests and on wp-cron.php , and during its own remote calls, the plugin does not start. Hello! We've been working on Cron. Now Cron is no longer running on user GET requests. It only works on POST requests and on wp-cron.php , and during its own remote calls, the plugin does not start. I would like to get feedback on these edits, did they help you?

    Pull Request

    security-malware-firewall.zip

  10. Ryokoxx commented on Oct 27, 2026

    @Ryokoxx
    Author

    Thanks Anton, that's the item I asked about.

    I haven't put the dev build on the live site, and it wouldn't show much here yet. Nothing has got stuck since the entries were cleared on 27 August, so there's no live fault for it to act on. I'll run it once it's in a stable release and report back.

    Reading the diff, page loads now hand the work to the wake-up call instead of running it, which is what we were after. One question: isTickAllowed() lets the tick run on any POST. On a site like ours, a frontend POST is usually someone submitting an enquiry form, so that's the request that would now wait on a task. Is that intended, or is the wake-up meant to cover frontend AJAX as well?

    On the earlier fix, the site has been on 2.190 for about two weeks with nothing stuck, so the 24 hour limit hasn't had anything to close out here yet.

    Cheers.

  11. AntonV1211 commented on Oct 28, 2026

    @AntonV1211
    Contributor

    @Ryokoxx Yes, that's intended. Any POST request (including frontend forms and AJAX) and any request to wp-cron.php can trigger the plugin's cron tick. The wake-up only covers GET-only traffic. The impact on a POST is limited: it runs at most one task, only if one is overdue, and no more than once every 90 seconds across the whole site. Most POST requests won't run anything.

    Your feedback on this edit would be very helpful.

  12. Ryokoxx commented on Oct 28, 2026

    @Ryokoxx
    Author

    Thanks Anton, that answers it.

    WordPress cron is switched off on our site and a server cron runs it instead, so once #758 is in a stable release I'd expect most of the plugin's tasks to run there and not through the wake-up.
    That's the first thing I'll check once it's installed.

    Cheers.

  13. AntonV1211 commented on Oct 29, 2026

    @AntonV1211
    Contributor

    @Ryokoxx The edits will be released in version 2.192.
    Thanks for the feedback!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions