Processing Pipeline¶

MITIE initializer

Initializes mitie structures. Every mitie component relies on this, hence this should be put at the beginning of every pipeline that uses any mitie components.

spacy language initializer

Initializes spacy structures. Every spacy component relies on this, hence this should be put at the beginning of every pipeline that uses any spacy components.

MITIE intent featurizer

Creates feature for intent classification using the MITIE featurizer.

Appends char-ngram features to feature vector

This featurizer appends character ngram features to a feature vector. During training the component looks for the most common character sequences (e.g. app or ing). The added features represent a boolean flag if the character sequence is present in the word sequence or not.

Creates bag-of-words representation of intent features

Creates bag-of-words representation of intent features using sklearn’s CountVectorizer. All tokens which consist only of digits (e.g. 123 and 99 but not a123d) will be assigned to the same feature.

Simple keyword matching intent classifier.

MITIE intent classifier (using a text categorizer)

sklearn intent classifier

During the training of the SVM a hyperparameter search is run to find the best parameter set. In the config, you can specify the parameters that will get tried

pipeline:
- name: "intent_classifier_sklearn"
  # Specifies the list of regularization values to
  # cross-validate over for C-SVM.
  # This is used with the ``kernel`` hyperparameter in GridSearchCV.
  C: [1, 2, 5, 10, 20, 100]
  # Specifies the kernel to use with C-SVM.
  # This is used with the ``C`` hyperparameter in GridSearchCV.
  kernels: ["linear"]

Embedding intent classifier

If you want to split intents into multiple labels, e.g. for predicting multiple intents or for modeling hierarchical intent structure, use these flags:

• tokenization of intent labels:

  • intent_tokenization_flag if true the algorithm will split the intent labels into tokens and use bag-of-words representations for them;
  • intent_split_symbol sets the delimiter string to split the intent labels. Default _

The algorithm also has hyperparameters to control:

• neural network’s architecture:

  • num_hidden_layers_a and hidden_layer_size_a set the number of hidden layers and their sizes before embedding layer for user inputs;
  • num_hidden_layers_b and hidden_layer_size_b set the number of hidden layers and their sizes before embedding layer for intent labels;

• training:

  • batch_size sets the number of training examples in one forward/backward pass, the higher the batch size, the more memory space you’ll need;
  • epochs sets the number of times the algorithm will see training data, where one epoch = one forward pass and one backward pass of all the training examples;

• embedding:

  • embed_dim sets the dimension of embedding space;
  • mu_pos controls how similar the algorithm should try to make embedding vectors for correct intent labels;
  • mu_neg controls maximum negative similarity for incorrect intents;
  • similarity_type sets the type of the similarity, it should be either cosine or inner;
  • num_neg sets the number of incorrect intent labels, the algorithm will minimize their similarity to the user input during training;
  • use_max_sim_neg if true the algorithm only minimizes maximum similarity over incorrect intent labels;

• regularization:

  • C2 sets the scale of L2 regularization
  • C_emb sets the scale of how important is to minimize the maximum similarity between embeddings of different intent labels;
  • droprate sets the dropout rate, it should be between 0 and 1, e.g. droprate=0.1 would drop out 10% of input units;

In the config, you can specify these parameters:

pipeline:
- name: "intent_classifier_tensorflow_embedding"
  # nn architecture
  "num_hidden_layers_a": 2
  "hidden_layer_size_a": [256, 128]
  "num_hidden_layers_b": 0
  "hidden_layer_size_b": []
  "batch_size": 32
  "epochs": 300
  # embedding parameters
  "embed_dim": 10
  "mu_pos": 0.8  # should be 0.0 < ... < 1.0 for 'cosine'
  "mu_neg": -0.4  # should be -1.0 < ... < 1.0 for 'cosine'
  "similarity_type": "cosine"  # string 'cosine' or 'inner'
  "num_neg": 10
  "use_max_sim_neg": true  # flag which loss function to use
  # regularization
  "C2": 0.002
  "C_emb": 0.8
  "droprate": 0.2
  # flag if to tokenize intents
  "intent_tokenization_flag": false
  "intent_split_symbol": "_"

Tokenizer using Jieba for Chinese language

Creates tokens using the Jieba tokenizer specifically for Chinese language. For language other than Chinese, Jieba will work as tokenizer_whitespace. Can be used to define tokens for the MITIE entity extractor. Make sure to install Jieba, pip install jieba.

Tokenizer using MITIE

Creates tokens using the MITIE tokenizer. Can be used to define tokens for the MITIE entity extractor.

MITIE entity extraction (using a mitie ner trainer)

spacy entity extraction

Maps synonymous entity values to the same value.

If the training data contains defined synonyms (by using the value attribute on the entity examples). this component will make sure that detected entity values will be mapped to the same value. For example, if your training data contains the following examples:

[{
  "text": "I moved to New York City",
  "intent": "inform_relocation",
  "entities": [{"value": "nyc",
                "start": 11,
                "end": 24,
                "entity": "city",
               }]
},
{
  "text": "I got a new flat in NYC.",
  "intent": "inform_relocation",
  "entities": [{"value": "nyc",
                "start": 20,
                "end": 23,
                "entity": "city",
               }]
}]

this component will allow you to map the entities New York City and NYC to nyc. The entitiy extraction will return nyc even though the message contains NYC. When this component changes an exisiting entity, it appends itself to the processor list of this entity.

conditional random field entity extraction

Adds duckling support to the pipeline to unify entity types (e.g. to retrieve common date / number formats)

Configure which dimensions, i.e. entity types, the duckling component to extract. A full list of available dimensions can be found in the duckling documentation.

pipeline:
- name: "ner_duckling"
  # dimensions to extract
  dimensions: ["time", "number", "amount-of-money", "distance"]