runner.PassthruSampleDatasetsProvider
December 14, 2023 ยท View on GitHub
View source
on GitHub
Builds a sampled tf.data.Dataset from multiple pass thru datasets.
Inherits From: DatasetProvider
runner.PassthruSampleDatasetsProvider(
principal_dataset: tf.data.Dataset,
extra_datasets: Sequence[tf.data.Dataset],
principal_weight: Optional[float] = None,
extra_weights: Optional[Sequence[float]] = None,
*,
principal_cardinality: Optional[int] = None,
fixed_cardinality: bool = False,
shuffle_dataset: bool = False,
examples_shuffle_size: Optional[int] = None
)
Passes any principal_dataset and extra_datasets thru: omitting any sharding.
For detailed documentation, see the filename dataset provider complement:
SimpleSampleDatasetsProvider.
Methods
get_dataset
get_dataset(
_: tf.distribute.InputContext
) -> tf.data.Dataset
Gets a sampled tf.data.Dataset omitting any input context.