sgnlp.models.span_extraction.utils.squad_convert_examples_to_features

squad_convert_examples_to_features(examples: List[transformers.data.processors.squad.SquadExample], tokenizer: sgnlp.models.span_extraction.tokenization.RecconSpanExtractionTokenizer, max_seq_length: int, doc_stride: int, max_query_length: int, is_training: bool, padding_strategy: str = 'max_length', tqdm_enabled: bool = True) → Tuple[List[transformers.data.processors.squad.SquadFeatures], torch.utils.data.dataset.TensorDataset][source]

[summary]

Parameters

  • examples (List[SquadExample]) – list of SquadExample

  • tokenizer (RecconSpanExtractionTokenizer) – RecconSpanExtractionTokenizer from sgnlp

  • max_seq_length (int) – set max_seq_length

  • doc_stride (int) – set doc_stride

  • max_query_length (int) – set max_query_length

  • is_training (bool) – set is_training

  • padding_strategy (str, optional) – set padding_strategy. Defaults to “max_length”.

  • tqdm_enabled (bool, optional) – set tqdm_enabled. Defaults to True.

Returns

Contains list of SquadFeatures and TensorDataset

Return type

Tuple[List[SquadFeatures], TensorDataset]