runner.PassthruSampleDatasetsProvider

December 14, 2023 · View on GitHub

Builds a sampled tf.data.Dataset from multiple pass thru datasets.

Inherits From: DatasetProvider

runner.PassthruSampleDatasetsProvider(
    principal_dataset: tf.data.Dataset,
    extra_datasets: Sequence[tf.data.Dataset],
    principal_weight: Optional[float] = None,
    extra_weights: Optional[Sequence[float]] = None,
    *,
    principal_cardinality: Optional[int] = None,
    fixed_cardinality: bool = False,
    shuffle_dataset: bool = False,
    examples_shuffle_size: Optional[int] = None
)

Passes any principal_dataset and extra_datasets thru: omitting any sharding. For detailed documentation, see the filename dataset provider complement: SimpleSampleDatasetsProvider.

Methods

`get_dataset`

View source

get_dataset(
    _: tf.distribute.InputContext
) -> tf.data.Dataset

Gets a sampled tf.data.Dataset omitting any input context.