flambe.nn.mos

Module Contents

class flambe.nn.mos.MixtureOfSoftmax(input_size: int, output_size: int, k: int = 1, take_log: bool = True)[source]

Bases: flambe.nn.module.Module

Implement the MixtureOfSoftmax output layer.

pi

softmax layer over the different softmax

Type:FullyConnected
layers

list of the k softmax layers

Type:[FullyConnected]
forward(self, data: Tensor)[source]

Implement mixture of softmax for language modeling.

Parameters:data (torch.Tensor) – seq_len x batch_size x hidden_size
Returns:out – output matrix of shape seq_len x batch_size x out_size
Return type:Variable