Loaders and samplers

FieldDataset

class FieldDataset(src, sample_attrs=None, fmt=('dat', 'data', 'hdf5'), subset_generator=None, unravel_model=None, from_samples=False, allow_change_preloaded=False)

Baseclass for dataset of fields loaded with similar configs.

calculate_statistics()

Calculate mean and std values for the attributes of the dataset.

convert_to_other_fmt(new_root_dir, new_fmt='hdf5', results_to_events=True, **kwargs)

Convert dataset to a new format.

Parameters

• new_root_dir (str) – Directory to save converted dataset

• new_fmt (str) – Extension to use

• kwargs (dict) – Any additional named arguments passed to Field.dump

Returns

Return type

FieldDataset

default_sample_attrs = {'CONTROL': ['BHPT'], 'GRID': [], 'MASKS': ['ACTNUM', 'TIME'], 'ROCK': ['PORO', 'PERMX', 'PERMY', 'PERMZ'], 'STATES': ['PRESSURE', 'RS', 'SGAS', 'SOIL', 'SWAT']}

dump_samples(path, n_epoch=1, prefix=None, state=True, **kwargs)

Dump samples from the dataset.

Parameters

• path (str) – Path to the directory for dump.

• n_epoch (int) – Number of times to pass through the dataset.

• prefix (str, None) – Prefix for dumped samples.

• state (bool) – If True, dump the state of the samples

• kwargs (dict) – Additional named arguments for sample.dump

dump_statistics(path)

Dump mean and std values of the dataset into a file.

property filtered_statistics

Filters out non-normalized attrs and attrs, which are not presented in sample_attrs, from statistics.

load_statistics(path)

Load mean and std values of the dataset from a file.

property sample_attrs

Attributes represented in the samples.

set_transform(transform)

Set transforms to be applied to each sample

Parameters

transform (class) – Class of transform to apply list of Classes can be used to compose several transforms

Returns

out

Return type

FieldDataset

static to_dates(model, t)

Restore actual dates from time deltas.

FieldSample

class FieldSample(path=None, field=None, dataset=None, state=None, **sample)

Class representing the samples from the dataset.

Parameters

• path (str, optional) – Path to the file. Only HDF5 files are supported at the moment.

• field (Field, optional) –

• dataset (FieldDataset, optional) –

• state (dict, optional) –

• sample (dict-like, optional) –

apply(func, attr, *args, inplace=False, **kwargs)

Apply function to attributes.

Parameters

• func (callable) – A function to apply. Must accept data as its first argument.

• attr (str, array-like) – Attributes to get data from.

• args (misc) – Any additional positional arguments to func.

• kwargs (misc) – Any additional named arguments to func.

Returns

output – Transformed component.

Return type

BaseComponent

as_ravel(inplace=False, **kwargs)

as_spatial(inplace=False, **kwargs)

at_wells(inplace=False)

Crop all the spatial arrays to the perforated cells. Ravel if needed.

Parameters

inplace (bool) –

Returns

sample – Cropped sample.

Return type

FieldSample

property attributes

Array of attributes.

property class_name

Name of the component.

copy()

Get a copy of the sample.

property dataset

Link to the parent dataset.

del_state(*args)

State remover.

property device

Get the sample’s device (if it is in Torch format)

Returns

device

Return type

torch.device

drop(attr)

Drop an attribute.

dump(path, **kwargs)

Dump the sample into a file.

Parameters

• path (str) – Path to the file.

• kwargs (dict) – Additional named arguments passed to BaseComponent’s dump method.

static dump_array_ascii(buffer, array, header=None, fmt='%f', compressed=True)

Writes array-like data into an ASCII buffer.

Parameters

• buffer (buffer-like) –

• array (1d, array-like) – Array to be saved

• header (str, optional) – String to be written line before the array

• fmt (str or sequence of strs, optional) – Format to be passed into numpy.savetxt function. Default to ‘%f’.

• compressed (bool) – If True, uses compressed typing style

empty_like()

Get an empty sample with the same state and the structure of embedded BaseComponents (if any).

property field

Link to the parent field.

init_state(**kwargs)

Init state attributes.

items()

Returns pairs of attribute’s names and data.

keys()

Array of attributes.

load(**kwargs)

Load sample from a file.

Parameters

kwargs (dict) – Additional named arguments passed to the load method.

Returns

sample – Sample with loaded data.

Return type

FieldSample

ravel(attr=None, order='F', inplace=True)

Ravel attributes where applicable assuming by default Fortran order.

Parameters

• attr (str, array of str) – Attribute to ravel.

• order (str) – Numpy reshape order. Default to ‘F’.

• inplace (bool) – If True, ravel is made inplace, return BaseComponent. Else, return raveled attribute.

Returns

out

Return type

BaseComponent if inplace else raveled attribute itself.

reshape(attr, newshape, order='C', inplace=True)

Reshape numpy.ndarray attributes.

Parameters

• attr (str, array of str) – Attribute to be reshaped.

• newshape (tuple) – New shape.

• order (str) – Numpy reshape order. Default to ‘C’.

• inplace (bool) – If True, reshape is made inplace, return BaseComponent. Else, return reshaped attribute.

Returns

output

Return type

BaseComponent if inplace else reshaped attribute itself.

property sample_attrs

Attributes represented in the sample.

set_state(**kwargs)

State setter.

property state

Get state.

to(device, inplace=True)

Change the sample’s device (if it is in Torch format).

Parameters

• device (str, torch.device) –

• inplace (bool) –

Returns

sample – Sample at the new device

Return type

FieldSample

transformed(transforms, inplace=False)

Apply a set of transforms to the sample.

Parameters

• transforms (list, tuple, Compose, Transform) – Transform to apply

• inplace (bool) –

Returns

sample – Transformed sample.

Return type

FieldSample

values()

Returns a generator of attribute’s data.

UniformSequenceSubset

class UniformSequenceSubset(size, low, high, **kwargs)

Generator of timesteps sampled from a uniform distribution [low, high).

RandomSubsequence

class RandomSubsequence(size, low, high, **kwargs)

Generator of timestep subsequences.