backpropagation bert Transformer
backpropagation based TensorFlow implementation for transformer convolution.
- Input
- 1543-dim embedding
- Encoder
- 28 x Transformer with 56 heads
- Output
- f1 projection
Training config
optimizer=Adam, lr=0.677, scheduler=plateau, warmup=956