flambe.nn.embedder

Module Contents

class flambe.nn.embedder.Embeddings[source]

Bases: flambe.nn.module.Module, torch.nn.Embedding

Implement an Embedding module.

This object replicates the usage of nn.Embedding but registers the from_pretrained classmethod to be used inside a Flambé configuration, as this does not happen automatically during the registration of PyTorch objects.

classmethod from_pretrained(cls, embeddings: Tensor, freeze: bool = True, paddinx_idx: Optional[int] = None, max_norm: Optional[float] = None, norm_type: float = 2.0, scale_grad_by_freq: bool = False, sparse: bool = False)[source]

Create Embedding instance from given 2-dimensional Tensor.

Parameters:
  • embeddings (torch.Tensor) – FloatTensor containing weights for the Embedding. First dimension is being passed to Embedding as num_embeddings, second as embedding_dim.
  • freeze (bool) – If True, the tensor does not get updated in the learning process. Default: True
  • (int, optional) (padding_idx) – See module initialization documentation.
  • max_norm (float, optional) – See module initialization documentation.
  • norm_type (float, optional) – See module initialization documentation. Default 2.
  • scale_grad_by_freq (bool, optional) – See module initialization documentation. Default False.
  • (bool, optional) (sparse) – See module initialization documentation. Default False.
class flambe.nn.embedder.Embedder(embedding: nn.Embedding, encoder: Module, embedding_dropout: float = 0, pad_index: Optional[int] = 0)[source]

Bases: flambe.nn.module.Module

Implements an Embedder module.

An Embedder takes as input a sequence of index tokens, and computes the corresponding embedded representations, and padding mask. The encoder may be initialized using a pretrained embedding matrix.

embeddings

The embedding layer

Type:Embedding
encoder

The sub-encoder that this object is wrapping

Type:Encoder
drop

The dropout layer

Type:nn.Dropout
forward(self, data: Tensor)[source]

Performs a forward pass through the network.

Parameters:data (torch.Tensor) – The input data, as a float tensor, batch first
Returns:The encoded output, as a float tensor. May return a state if the encoder is an RNN
Return type:Union[Tensor, Tuple[Tensor, Tensor]]