sgnlp.models.ufd.utils.create_val_test_embeddings

create_val_test_embeddings(cfg: sgnlp.models.ufd.data_class.UFDArguments, tokenizer: sgnlp.models.ufd.tokenization.UFDTokenizer, model: sgnlp.models.ufd.modeling.UFDEmbeddingModel, dataset_type: str) → Dict

Helper function to generate validation dataset for supervised and unsupervised training.

Parameters

• cfg (UFDArguments) – UFDArguments config load from configuration file

• tokenizer (UFDTokenizer) – UFD tokenizer class instance

• model (UFDEmbeddingModel) – UFD embedding model class instance

Returns

dictionary of dataset embeddings for supervised and unsupervised dataset

Return type

Dict