BECEC: Collaborative Edge Computing for Social-Welfare Maximization

Research code for the collaborative edge-computing scheduling framework first described in our 2022 arXiv manuscript and subsequently published in revised form at IEEE iWRF&AT 2024.

• Peer-reviewed paper: Deep Reinforcement Learning-Based Social Welfare Maximization for Collaborative Edge Computing
• Open manuscript: Social Welfare Maximization for Collaborative Edge Computing: A Deep Reinforcement Learning-Based Approach
• Repository scope: a research simulator and algorithm prototype, not a production edge-computing scheduler

The repository preserves a historical implementation snapshot from the original research cycle. The maintenance work in 2026 repaired the command-line entry points, documented the environment and provenance, and added lightweight checks. It did not rerun every paper experiment or certify bit-for-bit reproduction of every reported figure.

Research overview

The project studies cooperation among edge servers that belong to different service providers. An overloaded server can offload tasks to servers with spare capacity, while the system jointly considers task utility, execution cost, and resource availability.

The implementation follows a two-stage decomposition:

1. Task allocation. A DDPG/D4PG-style actor-critic selects a destination edge server for each task, including a null action when no server should be selected.
2. Task execution. The selected server decides the task order and per-slot resource allocation using the algorithms under env/becec/stage_two/, including greedy, dynamic-programming, and pointer-network research paths.

See docs/architecture.md for the code-to-method map and docs/reproducibility.md for the tested environment and known limitations.

Repository map

Path	Purpose
env/becec/	Collaborative edge-computing simulator, task arrivals, observations, rewards, and evaluation scheduler
env/becec/stage_two/	Task sequencing and per-slot execution algorithms
models/d4pg/	Distributional deterministic policy-gradient learner and replay buffer
models/d3pg/	Non-distributional deterministic policy-gradient baseline
models/wolp/	Optional Wolpertinger action-space experiments
config_d4pg.yml	Main BECEC D4PG experiment configuration
config_d3pg.yml	Legacy D3PG comparison configuration; preserved for reference but not yet verified end to end
configs/	Legacy OpenAI Gym and DM Control configurations inherited from the base framework
tests/	Lightweight maintenance tests plus preserved legacy training scripts

Setup

The maintained smoke-test environment uses Python 3.10, PyTorch 2.2.2, NumPy 1.26.4, and Gym 0.23.1. The code still uses the legacy Gym API, so NumPy 2.x and current Gymnasium-only environments are outside the verified compatibility envelope.

conda create -n becec python=3.10 -y
conda activate becec

Install the appropriate PyTorch build for your CPU/CUDA platform from the official PyTorch selector, then install the remaining dependencies:

python -m pip install -r requirements.txt

The default configuration selects cuda:0. Change both device and agent_device in the YAML file if a different device is required.

The optional Wolpertinger path (wolp_mode > 0) additionally requires a compatible FLANN Python binding. It is disabled by default and is not part of the maintained smoke test.

Usage

Train the default D4PG configuration:

python train.py --config config_d4pg.yml --mode 2

Training modes are:

• 0: auto-encoder and reinforcement learning
• 1: auto-encoder only
• 2: reinforcement learning only (default)

Run the evaluation path selected by test_mode in the YAML file:

python test.py --config config_d4pg.yml

Important configuration switches include:

Key	Meaning
state_mode	Observation representation for server and task information
action_mode	Quantized server selection (0) or grouped one-hot scores (1)
frame_mode	Fixed-length frame (0) or task-count-triggered frame (1)
stage2_alg_choice	Greedy/search path (0) or dynamic-programming path (1)
test_mode	Evaluation baseline or learned-agent route
wolp_mode	Optional Wolpertinger action refinement; 0 disables it

Validation

Run the maintained checks with:

python -m unittest tests.test_config_and_cli
python scripts/smoke_becec.py

These checks cover YAML loading, derived state/action dimensions, CLI parsing, import safety, and one deterministic CPU environment transition for the default non-Wolpertinger path. They deliberately do not launch a full multi-process training run.

Selected historical checkpoints under env/becec/stage_two/Pt/ are retained for research provenance but are no longer loaded automatically. They should be treated as research artifacts rather than portable model releases: they lack a complete machine-readable record of the training environment and dataset state. Load only artifacts from a trusted repository revision; Python pickle and legacy torch.load files can execute code while deserializing.

Publications

Peer-reviewed conference paper

X. He, C. You, Y. Shen, H. Zhu, Y. Dai, and Y. Lu, “Deep Reinforcement Learning-Based Social Welfare Maximization for Collaborative Edge Computing,” in 2024 IEEE International Workshop on Radio Frequency and Antenna Technologies (iWRF&AT), Shenzhen, China, 2024, pp. 162–167, doi: 10.1109/iWRFAT61200.2024.10594571.

@inproceedings{He2024DRLCEC,
  author    = {He, Xingqiu and You, Chaoqun and Shen, Yuhang and
               Zhu, Hongxi and Dai, Yueyue and Lu, Yunlong},
  title     = {Deep Reinforcement Learning-Based Social Welfare
               Maximization for Collaborative Edge Computing},
  booktitle = {2024 IEEE International Workshop on Radio Frequency
               and Antenna Technologies (iWRF\&AT)},
  year      = {2024},
  pages     = {162--167},
  doi       = {10.1109/iWRFAT61200.2024.10594571}
}

Open manuscript

X. He, Y. Shen, H. Zhu, S. Wang, C. You, and T. Q. S. Quek, “Social Welfare Maximization for Collaborative Edge Computing: A Deep Reinforcement Learning-Based Approach,” arXiv:2211.06861, 2022, doi: 10.48550/arXiv.2211.06861.

Provenance and license status

The original BECEC repository adapted a D4PG/D3PG framework from Igor Kuznetsov's schatty/oprl history (formerly d4pg-pytorch). The corresponding upstream snapshot did not declare a software license. Additional files cite OpenAI Baselines, Packt Publishing examples, and other research implementations.

Because those inherited licensing terms are unresolved, this repository does not currently declare an open-source license. Public visibility alone does not grant permission to reuse the code. See THIRD_PARTY_NOTICES.md for the audit trail. A clean open-source release requires either permission/relicensing from the relevant upstream authors or replacement of the inherited components.

Contributing

Bug reports and narrowly scoped reproducibility improvements are welcome. Please avoid committing generated logs, environment snapshots, replay buffers, or new model checkpoints unless they are intentionally versioned and documented.