Summary
Introduce a manual + automated sync engine that lets site owners pull existing Freemius users (free, paid, trial, etc.) for any configured plugin and push them into Kit forms/tags in bulk. This complements the current real-time webhook flow, which only catches new users after activation.
Motivation
Right now FreemKit only subscribes users who trigger a Freemius webhook after the integration is live. Every developer already has a history of installs, purchases, and trials sitting in Freemius that never reaches Kit. A sync feature closes this gap and gives users full control over which user cohorts get migrated into email marketing.
Proposed Architecture
1. New Core Class: Freemius_API_Client
File: includes/class-freemius-api-client.php
A lightweight wrapper around Freemius REST endpoints (re-using Freemius::build_fs_headers() for HMAC-SHA256 auth).
| Method |
Purpose |
get_users(string $plugin_id, array $filters, int $page) |
GET /v1/plugins/{id}/users.json with optional is_paying, is_trial, is_free, has_license query params. |
get_licenses(string $plugin_id, array $filters, int $page) |
GET /v1/plugins/{id}/licenses.json (used to infer paid status if user endpoint is ambiguous). |
get_user_count(string $plugin_id, array $filters) |
GET …/users.json?count=true for progress bars. |
- Pagination: Freemius returns
users + pagination (total, page, pages). We iterate with WP_Cron or background AJAX batches.
- Rate-limiting: Cache
X-RateLimit-Remaining in a transient; pause/respect limits.
- Error handling: Uses the same exponential-backoff pattern as
Webhook_Handler (max 3 retries).
2. New Core Class: Sync_Engine
File: includes/class-sync-engine.php
Orchestrates the entire sync pipeline.
class Sync_Engine {
public function start_sync( string $plugin_id, array $user_types, array $target_forms, array $target_tags );
public function process_batch( string $job_id, int $page );
public function get_job_status( string $job_id );
public function cancel_job( string $job_id );
}
Job state is stored in a new custom table {prefix}freemkit_sync_jobs (see schema below) or as an option array if we want to avoid a schema change in v1.1. Given the existing Database pattern, a new table is preferred for observability.
Sync Job Lifecycle:
- Admin initiates sync →
Sync_Engine::start_sync() creates a pending job.
- A WP-Cron hook (
freemkit_sync_batch) fires every 2 minutes while jobs are active.
- Each run fetches one Freemius page (max 25–50 users), maps each user to the local
Subscriber model, upserts via Database::upsert_subscriber_by_email(), then calls Kit_API::subscribe_to_form() + tag application.
- Job transitions:
pending → running → completed | failed.
- On completion, an admin notice + email (optional) is sent.
3. Database Schema Addition
Add a sync_jobs table in Database::create_table() (bump db_version to 1.1.0).
CREATE TABLE {$wpdb->prefix}freemkit_sync_jobs (
id bigint(20) unsigned NOT NULL AUTO_INCREMENT,
plugin_id varchar(50) NOT NULL DEFAULT '',
user_types varchar(100) NOT NULL DEFAULT '', -- JSON array: [\"free\",\"paid\",\"trial\"]
target_forms text DEFAULT NULL, -- JSON array of Kit form IDs
target_tags text DEFAULT NULL, -- JSON array of Kit tag IDs
status varchar(20) NOT NULL DEFAULT 'pending', -- pending|running|completed|failed|cancelled
total_users int(11) NOT NULL DEFAULT 0,
processed_users int(11) NOT NULL DEFAULT 0,
failed_users int(11) NOT NULL DEFAULT 0,
current_page int(11) NOT NULL DEFAULT 0,
total_pages int(11) NOT NULL DEFAULT 0,
error_message text DEFAULT NULL,
created datetime NOT NULL DEFAULT CURRENT_TIMESTAMP,
modified datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP,
PRIMARY KEY (id),
KEY plugin_id (plugin_id),
KEY status (status)
) {$charset_collate};
4. Admin UI: New "Sync" Sub-Screen
File: includes/admin/class-sync-admin.php
Add a new tab under Settings → FreemKit called Sync (or a dedicated admin page under the existing FreemKit menu).
Interface elements:
- Plugin selector: Dropdown of already-configured Freemius plugins (read from
Options_API::get_option('plugins')).
- User type filter: Multi-check — Free, Paid, Trial, All. Defaults to the same free/paid split already configured for webhooks.
- Kit Form/Tag overrides: Autocomplete fields for forms and tags (same UI pattern as the existing per-plugin repeater). If left empty, fall back to the plugin’s existing free/paid or global defaults.
- Sync button: Triggers an AJAX call to
Sync_Engine::start_sync() and returns a job_id.
- Progress panel: Polls
wp_ajax_freemkit_sync_status every 5 s. Shows a progress bar (processed / total), live count of successes/failures, and a cancel button.
- History table: Lists past sync jobs (plugin, user types, status, processed/total, date). Click a row to expand error details.
5. AJAX + REST Endpoints
| Endpoint |
Type |
Handler |
Purpose |
wp_ajax_freemkit_start_sync |
AJAX |
Sync_Admin::ajax_start_sync() |
Validate nonce, kick off Sync_Engine. |
wp_ajax_freemkit_sync_status |
AJAX |
Sync_Admin::ajax_sync_status() |
Return JSON job status for progress bar. |
wp_ajax_freemkit_cancel_sync |
AJAX |
Sync_Admin::ajax_cancel_sync() |
Set job status to cancelled. |
wp_ajax_freemkit_sync_history |
AJAX |
Sync_Admin::ajax_sync_history() |
Paginated job history for the admin table. |
6. Integration with Existing Components
Database: Add add_sync_job(), update_sync_job(), get_sync_job(), get_sync_jobs() methods. Reuse upsert_subscriber_by_email() so synced users and webhook-created users live in the same table.
Kit_API: Reuse subscribe_to_form() and tag_subscriber(). No new Kit methods needed.
Webhook_Handler: After a successful sync, deduplication is already handled by Database::upsert_subscriber_by_email() (ON DUPLICATE KEY UPDATE).
Audit_Log: Log every sync batch start/end and per-user Kit API result under the existing audit-log table/transient mechanism.
Options_API: Store a sync_settings key (global defaults: batch size, max retries, auto-schedule interval) without breaking the existing single-array pattern.
7. Batch Processing & Performance
- Batch size: 25 users per Freemius page (configurable via
freemkit_sync_batch_size filter).
- Cron cadence:
freemkit_sync_batch every 2 minutes while an active job exists. If the job is running, it processes the next page; if cancelled or completed, the schedule is allowed to expire.
- Time-limit safety: Each batch checks
time() < start_time + 20s to avoid shared-host timeouts.
- Memory safety: After every user, check
memory_get_usage(). If > 80 % of WP_MEMORY_LIMIT, pause and reschedule.
8. User Type Mapping (Freemius → FreemKit)
Freemius user objects contain is_paying, is_trial, is_free booleans. The sync engine maps them to the same user_type values already stored in {prefix}freemkit_subscriber_events:
| Freemius Field |
Sync Filter Option |
user_type value |
is_paying == true |
Paid |
paid |
is_trial == true |
Trial |
trial |
is_free == true (and not paying/trial) |
Free |
free |
When syncing "All", each user is processed once and tagged with the highest-priority type found (paid > trial > free).
9. Security & Permissions
- All AJAX endpoints require
manage_options (same as existing settings).
- Nonce verification on every admin action.
- Freemius API credentials are pulled from the encrypted settings store (same
secret_key decryption used in Settings).
10. Filters & Hooks for Extensibility
// Before a user is synced to Kit.
apply_filters( 'freemkit_sync_user_before_subscribe', bool $should_sync, object $freemius_user, array $job );
// Modify the mapped subscriber object before DB upsert.
apply_filters( 'freemkit_sync_subscriber_data', array $data, object $freemius_user );
// After a single user is successfully synced.
do_action( 'freemkit_sync_user_complete', int $subscriber_id, object $freemius_user, array $kit_result );
// After the entire job finishes.
do_action( 'freemkit_sync_job_complete', int $job_id, array $job );
Open Questions
- Should we support scheduled recurring syncs (e.g., nightly delta sync) or keep it strictly manual for v1?
- Should the sync UI live on a new admin page (e.g., FreemKit → Sync) instead of as a settings tab, given the interactive progress-bar UX?
- Freemius API returns users in chunks; do we need a separate
plan filter (e.g., only Pro-plan purchasers) or is is_paying granular enough?
Acceptance Criteria
Summary
Introduce a manual + automated sync engine that lets site owners pull existing Freemius users (free, paid, trial, etc.) for any configured plugin and push them into Kit forms/tags in bulk. This complements the current real-time webhook flow, which only catches new users after activation.
Motivation
Right now FreemKit only subscribes users who trigger a Freemius webhook after the integration is live. Every developer already has a history of installs, purchases, and trials sitting in Freemius that never reaches Kit. A sync feature closes this gap and gives users full control over which user cohorts get migrated into email marketing.
Proposed Architecture
1. New Core Class:
Freemius_API_ClientFile:
includes/class-freemius-api-client.phpA lightweight wrapper around Freemius REST endpoints (re-using
Freemius::build_fs_headers()for HMAC-SHA256 auth).get_users(string $plugin_id, array $filters, int $page)GET /v1/plugins/{id}/users.jsonwith optionalis_paying,is_trial,is_free,has_licensequery params.get_licenses(string $plugin_id, array $filters, int $page)GET /v1/plugins/{id}/licenses.json(used to infer paid status if user endpoint is ambiguous).get_user_count(string $plugin_id, array $filters)GET …/users.json?count=truefor progress bars.users+pagination(total, page, pages). We iterate withWP_Cronor background AJAX batches.X-RateLimit-Remainingin a transient; pause/respect limits.Webhook_Handler(max 3 retries).2. New Core Class:
Sync_EngineFile:
includes/class-sync-engine.phpOrchestrates the entire sync pipeline.
Job state is stored in a new custom table
{prefix}freemkit_sync_jobs(see schema below) or as an option array if we want to avoid a schema change in v1.1. Given the existingDatabasepattern, a new table is preferred for observability.Sync Job Lifecycle:
Sync_Engine::start_sync()creates apendingjob.freemkit_sync_batch) fires every 2 minutes while jobs are active.Subscribermodel, upserts viaDatabase::upsert_subscriber_by_email(), then callsKit_API::subscribe_to_form()+ tag application.pending→running→completed|failed.3. Database Schema Addition
Add a
sync_jobstable inDatabase::create_table()(bumpdb_versionto1.1.0).CREATE TABLE {$wpdb->prefix}freemkit_sync_jobs ( id bigint(20) unsigned NOT NULL AUTO_INCREMENT, plugin_id varchar(50) NOT NULL DEFAULT '', user_types varchar(100) NOT NULL DEFAULT '', -- JSON array: [\"free\",\"paid\",\"trial\"] target_forms text DEFAULT NULL, -- JSON array of Kit form IDs target_tags text DEFAULT NULL, -- JSON array of Kit tag IDs status varchar(20) NOT NULL DEFAULT 'pending', -- pending|running|completed|failed|cancelled total_users int(11) NOT NULL DEFAULT 0, processed_users int(11) NOT NULL DEFAULT 0, failed_users int(11) NOT NULL DEFAULT 0, current_page int(11) NOT NULL DEFAULT 0, total_pages int(11) NOT NULL DEFAULT 0, error_message text DEFAULT NULL, created datetime NOT NULL DEFAULT CURRENT_TIMESTAMP, modified datetime NOT NULL DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP, PRIMARY KEY (id), KEY plugin_id (plugin_id), KEY status (status) ) {$charset_collate};4. Admin UI: New "Sync" Sub-Screen
File:
includes/admin/class-sync-admin.phpAdd a new tab under Settings → FreemKit called Sync (or a dedicated admin page under the existing FreemKit menu).
Interface elements:
Options_API::get_option('plugins')).Sync_Engine::start_sync()and returns ajob_id.wp_ajax_freemkit_sync_statusevery 5 s. Shows a progress bar (processed / total), live count of successes/failures, and a cancel button.5. AJAX + REST Endpoints
wp_ajax_freemkit_start_syncSync_Admin::ajax_start_sync()Sync_Engine.wp_ajax_freemkit_sync_statusSync_Admin::ajax_sync_status()wp_ajax_freemkit_cancel_syncSync_Admin::ajax_cancel_sync()cancelled.wp_ajax_freemkit_sync_historySync_Admin::ajax_sync_history()6. Integration with Existing Components
Database: Addadd_sync_job(),update_sync_job(),get_sync_job(),get_sync_jobs()methods. Reuseupsert_subscriber_by_email()so synced users and webhook-created users live in the same table.Kit_API: Reusesubscribe_to_form()andtag_subscriber(). No new Kit methods needed.Webhook_Handler: After a successful sync, deduplication is already handled byDatabase::upsert_subscriber_by_email()(ON DUPLICATE KEY UPDATE).Audit_Log: Log every sync batch start/end and per-user Kit API result under the existing audit-log table/transient mechanism.Options_API: Store async_settingskey (global defaults: batch size, max retries, auto-schedule interval) without breaking the existing single-array pattern.7. Batch Processing & Performance
freemkit_sync_batch_sizefilter).freemkit_sync_batchevery 2 minutes while an active job exists. If the job isrunning, it processes the next page; ifcancelledorcompleted, the schedule is allowed to expire.time() < start_time + 20sto avoid shared-host timeouts.memory_get_usage(). If > 80 % ofWP_MEMORY_LIMIT, pause and reschedule.8. User Type Mapping (Freemius → FreemKit)
Freemius user objects contain
is_paying,is_trial,is_freebooleans. The sync engine maps them to the sameuser_typevalues already stored in{prefix}freemkit_subscriber_events:user_typevalueis_paying== truepaidis_trial== truetrialis_free== true (and not paying/trial)freeWhen syncing "All", each user is processed once and tagged with the highest-priority type found (paid > trial > free).
9. Security & Permissions
manage_options(same as existing settings).secret_keydecryption used inSettings).10. Filters & Hooks for Extensibility
Open Questions
planfilter (e.g., only Pro-plan purchasers) or isis_payinggranular enough?Acceptance Criteria
{prefix}freemkit_subscribersand subscribed to Kit.composer testpasses before merge.