feat: model_config 支持 summaries_set(PCOC / scalars)训练期 TensorBoard 监控#556
Open
fooSynaptic wants to merge 2 commits into
Open
feat: model_config 支持 summaries_set(PCOC / scalars)训练期 TensorBoard 监控#556fooSynaptic wants to merge 2 commits into
fooSynaptic wants to merge 2 commits into
Conversation
5953dd6 to
43461dc
Compare
在 RankModel 实现训练期 TensorBoard summary:PCOC 与 scalars 特征切片。 配置入口为 model_config.summaries_set(非 eval_config)。 修复 alibaba#532
将 build_summary_graph 从 RankModel 移至 EasyRecModel 基类,Match/召回等模型均可配置。 PCOC 新增 label_name 字段以支持多任务标签指定。
43461dc to
0cfc75c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
参考Issue:#532
训练过程中 在 TensorBoard 里直接看到 PCOC(预估 CTR / 真实 CTR),并支持按某个特征取值(例如
c1=1005)查看该子人群上的平均预估。原先eval_config.metrics_set无法满足「训练期写 summary + 特征切片」的需求。设计思路
model_config.summaries_set,与模型输出(prediction_dict)同层,而不是eval_config。pcoc:全 batch 的mean(预估) / mean(标签),不做特征切片;scalars:对pred_name指定输出做mean(预估),可通过feature_name+feature_value做样本过滤(不算 PCOC)。EasyRecEstimator在model_fn中调用model.build_summary_graph();RankModel遍历summaries_set写入tf.summary.scalar。pred_name解析:从prediction_dict取张量;若 key 为logits/logits_*则先 sigmoid。RawFeature在feature_dict中为 float(如1005.0),切片匹配时按数值比较,避免as_string变成1005.000000导致与配置"1005"不一致。改动文件
easy_rec/python/protos/summary.protoPCOC/Scalars/ModelSummarieseasy_rec/python/protos/easy_rec_model.protoEasyRecModel增加summaries_seteasy_rec/python/model/rank_model.pybuild_summary_graph及切片逻辑easy_rec/python/model/easy_rec_estimator.pyeasy_rec/python/model/easy_rec_model.pyeasy_rec/python/test/eval_summary_test.pysamples/model_config/deepfm_combo_on_avazu_ctr.config配置示例
model_config { summaries_set { pcoc { # pred_name: "probs" # 默认;logits 会先 sigmoid # epsilon: 1e-7 } } summaries_set { scalars { name: "c1_1005_pred" # TensorBoard: summary/c1_1005_pred pred_name: "probs" feature_name: "c1" feature_value: "1005" } } }多塔模型在顶层通过
pred_name: "probs_ctr"等区分任务,无需在task_towers重复配置。TensorBoard 标签
pcocsummary/pcoc、summary/predicted_ctr、summary/observed_ctrscalarssummary/<name>测试结论
1. 单元测试
结果:2/2 通过
summaries_set(pcoc + scalars)pcoc = mean(pred)/mean(label)校验通过c1=1005.0与feature_value: "1005"匹配,输出0.92. 样本端到端(
deepfm_combo_on_avazu_ctr.config,50 step)数据:
data/test/dwd_avazu_ctr_deepmodel_10w.csv(c1=1005约占 90.7%)summary/predicted_ctrsummary/observed_ctrsummary/pcocpredicted/observed一致summary/c1_1005_pred校验项:四个 tag 均写入 events;
pcoc ≈ predicted_ctr / (observed_ctr + 1e-7);c1_1005_pred > 0。通过3. 回归
test_deepfm_with_combo_feature(同 sample 默认训练流程)通过。