Skip to content

Eval 详细结果 （学习率等超参数探索）

Jump to bottom
Heda Wang edited this page Nov 8, 2017 · 11 revisions

实验目的

探索不同的 initial_learning_rate, decay, 和 num_epochs_per_decay 的组合关系对于结果的影响

实验设置

用 ShowAndTellAdvancedModel 的 Visual Attention + lexical embedding (postag + char) ，按照一开始就设定 train_inception_with_decay=True 的设定，batch_size=30, 训练 600k steps, 在下列不同设置下实验：

init_lr decay every_n_epochs 初始学习率 结束学习率
0.5 0.726 8.0 0.5 0.1
1.0 0.6 8.0 1.0 0.05
1.0 0.66 8.0 1.0 0.1
1.0 0.726 8.0 1.0 0.2
2.0 0.66 8.0 2.0 0.2
2.0 0.6 8.0 2.0 0.1
2.0 0.9 2.0 2.0 0.2

实验结果

B4 = BLEU_4 C = CIDEr M = METEOR R = ROUGE_L

init_lr decay every_n_epochs B_4 C M R
0.5 0.726 8.0 0.5631 1.7472 0.4077 0.6831
1.0 0.6 8.0 0.5644 1.7509 0.4085 0.6845
1.0 0.66 8.0 0.5664 1.7552 0.4082 0.6852
1.0 0.726 8.0 0.5591 1.7269 0.4057 0.6804
2.0 0.6 8.0 0.5505 1.6839 0.4036 0.6777
2.0 0.66 8.0 0.5555 1.6996 0.4036 0.6790
2.0 0.9 2.0 0.5546 1.7018 0.4043 0.6792

详细结果

init_lr is 1.0 and decay 0.6 per 8.0 epochs

epoch Bleu_4 CIDEr METEOR ROUGE_L
200282 0.5153 1.5135 0.3845 0.6544
220415 0.5132 1.5177 0.3854 0.6556
240764 0.5206 1.5473 0.3859 0.6576
260994 0.5221 1.5665 0.3888 0.6597
300210 0.5472 1.6666 0.3994 0.6746
320002 0.5390 1.6336 0.3956 0.6691
340539 0.5526 1.6876 0.4019 0.6773
360046 0.5544 1.7027 0.4031 0.6796
380642 0.5525 1.6902 0.4020 0.6773
400842 0.5586 1.7300 0.4063 0.6816
421337 0.5609 1.7243 0.4064 0.6823
440891 0.5593 1.7246 0.4071 0.6810
460807 0.5617 1.7241 0.4077 0.6820
480411 0.5613 1.7319 0.4080 0.6833
500396 0.5617 1.7343 0.4076 0.6827
521053 0.5639 1.7425 0.4087 0.6839
541263 0.5638 1.7492 0.4078 0.6845
560347 0.5631 1.7471 0.4080 0.6836
580619 0.5644 1.7509 0.4085 0.6845
600000 0.5624 1.7435 0.4080 0.6832

the rest of the results are not ready yet (x40)

init_lr is 1.0 and decay 0.66 per 8.0 epochs

epoch Bleu_4 CIDEr METEOR ROUGE_L
442107 0.5564 1.7112 0.4025 0.6779
462029 0.5620 1.7406 0.4057 0.6823
480192 0.5611 1.7425 0.4070 0.6824
500601 0.5527 1.6951 0.4037 0.6772
521114 0.5621 1.7464 0.4076 0.6829
540748 0.5648 1.7532 0.4088 0.6848
561423 0.5635 1.7495 0.4077 0.6841
581573 0.5664 1.7552 0.4082 0.6852
600000 0.5638 1.7527 0.4085 0.6844

init_lr is 2.0 and decay 0.6 per 8.0 epochs

epoch Bleu_4 CIDEr METEOR ROUGE_L
312218 0.5085 1.5051 0.3839 0.6524
322203 0.5024 1.4618 0.3795 0.6473
342143 0.5311 1.5838 0.3924 0.6645
362115 0.5342 1.6117 0.3949 0.6676
382075 0.5229 1.5623 0.3894 0.6608
402050 0.5427 1.6465 0.3988 0.6722
421702 0.5394 1.6141 0.3982 0.6702
441327 0.5321 1.6029 0.3940 0.6660
460935 0.5464 1.6606 0.4000 0.6750
480594 0.5428 1.6489 0.4015 0.6745
500540 0.5414 1.6395 0.3984 0.6707
520463 0.5472 1.6679 0.4015 0.6752
540395 0.5492 1.6685 0.4024 0.6762
561425 0.5504 1.6782 0.4030 0.6773
581325 0.5505 1.6839 0.4036 0.6777
600000 0.5499 1.6783 0.4020 0.6769

init_lr is 2.0 and decay 0.66 per 8.0 epochs

epoch Bleu_4 CIDEr METEOR ROUGE_L
304242 0.4997 1.4482 0.3789 0.6463
321098 0.5117 1.5093 0.3845 0.6536
341343 0.5294 1.5879 0.3927 0.6644
361576 0.5185 1.5488 0.3908 0.6594
381814 0.5186 1.5200 0.3874 0.6560
402013 0.5321 1.6084 0.3942 0.6666
421892 0.5318 1.6097 0.3922 0.6645
441788 0.5295 1.5764 0.3919 0.6615
461665 0.5444 1.6527 0.3993 0.6735
481720 0.5441 1.6475 0.3990 0.6724
501938 0.5409 1.6441 0.3979 0.6714
522164 0.5494 1.6796 0.4010 0.6748
546831 0.5481 1.6764 0.4009 0.6753
560120 0.5495 1.6804 0.4007 0.6758
580019 0.5505 1.6949 0.4038 0.6772
600000 0.5555 1.6996 0.4036 0.6790

init_lr is 2.0 and decay 0.9 per 2.0 epochs

epoch Bleu_4 CIDEr METEOR ROUGE_L
420229 0.5318 1.6004 0.3950 0.6648
440343 0.5345 1.5974 0.3938 0.6669
460613 0.5392 1.6330 0.3979 0.6699
481349 0.5412 1.6374 0.3980 0.6707
502054 0.5449 1.6563 0.3994 0.6729
522885 0.5470 1.6655 0.4010 0.6743
540404 0.5426 1.6528 0.3981 0.6707
561365 0.5531 1.6858 0.4017 0.6772
581773 0.5546 1.7018 0.4043 0.6792
600000 0.5539 1.6982 0.4043 0.6780
Clone this wiki locally