Aligning a corpus with Fastalign

1. Install Fastalign 

follow this github 

https://github.com/clab/fast_align.git

GitHub - clab/fast_align: Simple, fast unsupervised word aligner

 

2. Prepare tokenized data 

for my case, source.tok.txt target.tok.txt 

3. Combine tokenized data 

 paste {src}  {tgt}  | sed 's/ *\t */ ||| /g' > {src}-{tgt}

4. Run fastalign 

fast_align -d -v -o -i  {src}-{tgt} > align.txt