BERT

GitHub - google-research/bert: TensorFlow code and pre-trained models for BERTGitHub

Instruction

Downloading and Installation

1. Clone or Download BERT

2. Download Chinese pre-train Chinese model: BERT-Base, Chinese: Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters

export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12
export DATA_DIR=/path/to/glue

python run_classifier.py \
  --task_name=SENTI \
  --do_train=true \
  --do_eval=true \
  --data_dir=$DATA_DIR \
  --vocab_file=$BERT_BASE_DIR/vocab.txt \
  --bert_config_file=$BERT_BASE_DIR/bert_config.json \
  --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
  --max_seq_length=128 \
  --train_batch_size=32 \
  --learning_rate=2e-5 \
  --num_train_epochs=3.0 \
  --output_dir=/tmp/mrpc_output/

Applications

Using BERT as word Embedding

https://towardsdatascience.com/word-embedding-using-bert-in-python-dd5a86c00342towardsdatascience.com