agatha.construct.embedding_util module

agatha.construct.embedding_util.embed_records(records, batch_size, text_field, max_sequence_length, out_embedding_field='embedding')

Introduces an embedding field to each record, indicated the bert embedding of the supplied text field.

Return type

Iterable[Dict[str, Any]]

agatha.construct.embedding_util.get_bert_initializer(bert_model)

The bert_model may be a path or any provided by the transformers module. For instance “bert-base-uncased”

Return type

Tuple[str, Callable[[], Any]]

agatha.construct.embedding_util.get_pretrained_model_initializer(name, model_class, data_dir, **model_kwargs)

Return type

Tuple[str, Callable[[], Any]]

agatha.construct.embedding_util.get_pytorch_device_initalizer(disable_gpu)

Return type

Tuple[str, Callable[[], Any]]