Download Jane: User's Manual

Transcript
3.1. RUNNING JANE LOCALLY
3.1.1
13
Preparing the data
Note
The configuration files for the examples shown in this chapter are located in
examples/local. The data files needed for these examples can be downloaded at:
http://www.hltpr.rwth-aachen.de/jane/files/exampleRun_local.tgz.
The
configuration files expect those files to reside in the same directory as the configuration files themselves.
In our case we concatenate the development and test corpora into one big filter
corpus. This allows us to use the same rule table for optimization and for the final
translation. To be able to use the provided configuration files, you should do the same
by running the following command:
cat german.dev.100 german.test.100 > german.dev.test
3.1.2
Extracting rules
The main program for phrase extraction is trainHierarchical.sh. With the option -h
it gives a list of all its options. For local usage, however, many of them are not relevant.
The options can be specified in the command line or in a config file. As already stated
above, we will use configuration files.
After copying the configuration files (for either the phrase-based or the hierarchical
system) to the same directory as the data you prepared in Chapter 3.1.1 (or Chapter
3.2.1 if you want to run the larger examples) start the rule extraction by running the
following command
$ bin/trainHierarchical.sh --config extract.config
Note
The command for starting rule extraction is the same for phrase-based and hierarchical phrase extraction.
Note
Since running this command typically takes a couple of minutes, you might just go
on reading while Jane is extracting.
Understanding the general format of the extract.config file
To understand how configuration files work in general, let’s first have a look at the
configuration file for extracting phrase-based rules: