🔏
DL
  • Intro and Resources
  • AI, ML, and DL
  • 機器學習簡介
  • Regression
  • Gradient Descent
  • Embedding
  • RNN
  • BERT
  • Word Embedding
  • Appendix
  • Coding
    • Install and Resources
Powered by GitBook
On this page
  • Instruction
  • Downloading and Installation
  • Applications

Was this helpful?

BERT

PreviousRNNNextWord Embedding

Last updated 4 years ago

Was this helpful?

Instruction

Downloading and Installation

  1. Clone or Download BERT

  2. Download Chinese pre-train Chinese model: BERT-Base, Chinese: Chinese Simplified and Traditional, 12-layer, 768-hidden, 12-heads, 110M parameters

export BERT_BASE_DIR=/path/to/bert/uncased_L-12_H-768_A-12
export DATA_DIR=/path/to/glue

python run_classifier.py \
  --task_name=SENTI \
  --do_train=true \
  --do_eval=true \
  --data_dir=$DATA_DIR \
  --vocab_file=$BERT_BASE_DIR/vocab.txt \
  --bert_config_file=$BERT_BASE_DIR/bert_config.json \
  --init_checkpoint=$BERT_BASE_DIR/bert_model.ckpt \
  --max_seq_length=128 \
  --train_batch_size=32 \
  --learning_rate=2e-5 \
  --num_train_epochs=3.0 \
  --output_dir=/tmp/mrpc_output/

Applications

Using BERT as word Embedding

LogoWord Embedding Using BERT In PythonMedium
LogoGitHub - google-research/bert: TensorFlow code and pre-trained models for BERTGitHub