Pensieve Retraining

2026 correction completed. The historical checkpoint was produced with an input-wiring defect: the sixth input branches of the actor and critic read state[4, -1] instead of the remaining-chunk feature in state[5, -1]. The old checkpoint has been removed from the active path. The corrected, from-scratch normalized beta=1 reference reached 110,000 updates and is published with a machine-readable reproduction manifest.

This repository provides a reproducible method for retraining the Pensieve model, including the following improvements based on the original Pensieve code:

• Support for dynamic entropy weight, i.e., decaying $\beta$ from 1 to 0.1 over $1$0^{5}$$ iterations. Refer to: Why the result is not better than MPC? · Issue #11.
• Train and test Pensieve under higher video bitrate (up to 4K resolution encoded at 40Mbps). Specifically,VIDEO_BIT_RATE, REBUF_PENALTY, and chunk size information in Pensieve, BBA , and RobustMPC are modified. The video (Big Buck Bunny) is provided in sim/ and test/.
• Normalize states and rewards for higher network bandwidth (e.g., in 5G networks) by an order of magnitude. Refer to: godka/pensieve-5G: Pensieve for 5G datasets.
• Carefully split the dataset into training and test sets in a repeatable way.
• Fix some bugs in the original code.

Please see the commits after 668f7ef1c3be7656d878771591ec93a865f4b1e0 (Jan 13, 2024) for details.

Note: This repository only reports single-video simulation results. Potential issues may exist in the multi-video scenario, emulation, or real-world deployment, where the related code has not been modified.

Change description

The original project changes were concentrated in three folders:

• sim/: dynamic entropy weight; new video; states and rewards normalization.
• test/: new video; states and rewards normalization.
• retrained_info/: information related to the retrained model, including:
  • data_preprocess/: network traces; scripts to filter and split the dataset (network traces)
  • retrained_model/: retrained model files
  • training_info/: complete training metrics and held-out summaries for all six formal runs
  • test_results/: performance of the retrained model versus BBA and RobustMPC

Training and testing methodology

Corrected reproducible workflow (2026)

The repaired training path is Python 3.7 and Windows spawn compatible. It passes beta, normalization, seed, data paths, output path, agent count and epoch count explicitly to every process. It also sorts and filters trace entries, uses independent environment/action RNGs, applies the same normalization mode during training and held-out testing, and starts every formal model from scratch.

conda env create -f environment-pensieve.yml
conda run -n pensieve-retrain python scripts/prepare_traces.py
conda run -n pensieve-retrain python -m pytest -q tests
conda run -n pensieve-retrain python scripts/smoke_reproducibility.py
conda run -n pensieve-retrain python scripts/run_training_suite.py

The frozen formal protocol is seed 42, 16 synchronous CPU agents, 110,000 updates, and held-out evaluation every 100 updates. The suite trains non-normalized beta 1--5 plus normalized beta 1. It writes each model to an isolated directory and updates suite_manifest.json after every completion. The trace and environment inventories are under artifacts/reproducibility/.

Only normalized beta=1 is published as this repository's repaired reference. The non-normalized beta=1--5 checkpoints are not published here. Results from selecting the best of beta 1--5 must be identified as a tuned performance upper bound rather than silently replacing the default beta=1 configuration.

Decaying entropy weight. As described in the Pensieve paper, "the entropy factor $\beta$ is controlled to decay from 1 to 0.1 over $1$0^{5}$$ iterations". However, this value is constant during training in the original code (ENTROPY_WEIGHT = 0.5 in sim/a3c.py). This repository uses a stepwise schedule that reduces the initial beta to 0.1 over 100,000 updates. The entropy weight is represented by a TensorFlow placeholder in sim/a3c.py; sim/multi_agent.py computes and supplies its value for each update. Select the initial value explicitly with the required --beta command-line argument.

States and rewards normalization. To adapt the model to higher bandwidth or bitrate (e.g., tens of Mbps), the input throughput and chunk size features are divided by 10 for normalization in both training and testing. The reward is normalized in the same way. Select the mode explicitly with the required --normalized command-line argument; training passes the same value to held-out evaluation.

Network traces. Four classes of wireless bandwidth traces are used in training and testing, collected in 3G, 4G, 5G, and Wi-Fi networks. Each class of traces is further divided into several types, depending on the location or mobility. I further filter out traces (see retrained_info/data_preprocess/filtered_traces.py) whose average bandwidth is less than 1.5Mbps (because the lowest video bitrate is 1Mbps), mainly in the Norway FCC dataset. Final traces are provided in retrained_info/data_preprocess/network_traces.zip.

	Count	Range of Average Bandwidth (Mbps)
Norway FCC (3G)	134	1.51~4.59
Lumos4G	175	7.59~102.43
Lumos5G	121	66.49~906.09
Solis Wi-Fi	80	7.28~73.16

Raw traces are publicly available: (however, the unit may be inconsistent)

• Norway FCC (3G) : transys-project/pitree/traces.zip
• Lumos4G: SIGCOMM21-5G/artifact/Video-Streaming/Network-Traces/Lumous5G/4G
• Lumos5G: SIGCOMM21-5G/artifact/Video-Streaming/Network-Traces/Lumous5G/5G
• Solis Wi-Fi: GreenLv/Solis-WiFi-Trace

Dataset splitting. When retraining the Pensieve model, the dataset is split into training and test sets at a ratio of 0.8:0.2. This split is random and uniform, depending on each type in each class of all traces. See retrained_info/data_preprocess/split_trian_test.py for details.

Training and test set distribution (average trace bandwidth):

	Count	Avg (Mbps)	Std (Mbps)	Min (Mbps)	Max (Mbps)
Training Set	410	128.91	203.60	1.51	749.23
Test Set	100	139.95	222.06	1.54	906.09

Retrained model information

The repaired public reference is retrained_info/retrained_model/beta-1_normalized_ep_110000.ckpt. It was trained from scratch with seed 42, 16 CPU A3C agents, and held-out testing every 100 updates. Component hashes, environment versions, trace-manifest hash, training commit, and removed historical checkpoint hashes are recorded in retrained_info/reproduction_manifest.json and retrained_info/REPRODUCIBILITY.md.

Complete central-agent metrics and held-out summaries for all six formal runs are packaged in retrained_info/training_info/training_log.zip.

Testing results

The figures below are regenerated with the repository's original test/plot_results.py script. They compare the best-performing corrected model from the beta=1--5 sweep (beta=4) with BBA and RobustMPC on the frozen 100-trace test split. Its mean total reward is 29.4% higher than BBA and 6.1% higher than RobustMPC. All three methods use the same raw-reward scale; the corresponding input logs are packaged in retrained_info/test_results/test_logs.zip.

Beta=4 is selected on this sweep and is reported here as a tuned performance upper bound. It does not replace the default beta=1 configuration or the separately published normalized beta=1 reference checkpoint.