simulation.utils.machine_learning.data package

Submodules

simulation.utils.machine_learning.data.base_dataset module

This module implements an abstract base class (ABC) ‘BaseDataset’ for datasets.

It also includes common transformation functions (e.g., get_transform, __scale_width), which can be later used in subclasses.

Classes:

BaseDataset(transform_properties, Any] =)

This is the base class for other datasets.

Functions:

get_params(preprocess, load_size, crop_size, …)	param preprocess Scaling and cropping of images at load time
get_transform([load_size, crop_size, mask, …])	Create transformation from arguments.
__make_power_2(img, base[, method])	param img image to transform
__scale_width(img, target_size, crop_size[, …])	param img image to transform
__crop(img, pos, size)	param img image to transform
__apply_mask(img, mask_file)	Overlay image with the provided mask.
__flip(img, flip)
__print_size_warning(ow, oh, w, h)	Print warning information about image size(only print once)

class BaseDataset(transform_properties: Dict[str, Any] = <factory>)

Bases: Generic[torch.utils.data.dataset.T_co]

This is the base class for other datasets.

Attributes:

transform_properties	Properties passed as arguments to transform generation function.
transform	Transformation that can be applied to images.

transform_properties: Dict[str, Any]

Properties passed as arguments to transform generation function.

property transform: torchvision.transforms.transforms.Compose

Transformation that can be applied to images.

Type

transforms.Compose

get_params(preprocess: Iterable, load_size: int, crop_size: int, size: Tuple[int, int]) → Dict[str, Any]

Parameters

• preprocess – Scaling and cropping of images at load time [resize | crop | scale_width]

• load_size – Scale images to this size

• crop_size – Then crop to this size

• size – The image sizes

get_transform(load_size: int = - 1, crop_size: int = - 1, mask: Optional[str] = None, preprocess: Iterable = {}, no_flip: bool = True, params=None, grayscale=False, method=3, convert=True) → torchvision.transforms.transforms.Compose

Create transformation from arguments.

Parameters

• load_size – Scale images to this size

• crop_size – Then crop to this size

• mask – Path to a mask overlaid over all images

• preprocess – scaling and cropping of images at load time [resize | crop | scale_width]

• no_flip – Flip 50% of all training images vertically

• params – more params for cropping

• grayscale – enable or disable grayscale

• method – the transform method

• convert – enable or disable transformations and normalizations

__make_power_2(img, base, method=3)

Parameters

• img – image to transform

• base – the base

• method – the transform method

__scale_width(img, target_size, crop_size, method=3)

Parameters

• img – image to transform

• target_size – the load size

• crop_size – the crop size, which is used for training

• method – the transform method

__crop(img, pos, size)

Parameters

• img – image to transform

• pos – where to crop my image

• size – resulting size of cropped image

__apply_mask(img: PIL.Image.Image, mask_file: str) → PIL.Image.Image

Overlay image with the provided mask.

Parameters

• img (Image.Image) – image to transform

• mask_file (str) – path to mask image file

__flip(img, flip)

__print_size_warning(ow, oh, w, h)

Print warning information about image size(only print once)

Parameters

• ow – original width

• oh – original height

• w – width

• h – height

simulation.utils.machine_learning.data.data_loader module

Classes:

DataLoader(dataset, …)

Wrapper class of Dataset class that performs multi-threaded data loading.

class DataLoader(dataset: Union[simulation.utils.machine_learning.data.unlabeled_dataset.UnlabeledDataset, simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset], max_dataset_size: int, batch_size: int = 1, num_threads: int = 1, sequential: bool = False)

Bases: object

Wrapper class of Dataset class that performs multi-threaded data loading.

Methods:

load_data()

load_data()

simulation.utils.machine_learning.data.extract_simulated_images module

Functions:

main(**kwargs)

main(**kwargs)

simulation.utils.machine_learning.data.image_folder module

A modified image folder class.

We modify the official PyTorch image folder (https://github.com/pytorch/vision/blob/master/torchvision/datasets/folder.py) so that this class can load images from both current directory and its subdirectories.

Functions:

is_image_file(filename)	Check if a file is an image.
find_images(dir[, max_dataset_size])	Recursively search for images in the given directory.

is_image_file(filename)

Check if a file is an image.

Parameters

filename – the file name to check

find_images(dir: str, max_dataset_size=inf) → List[str]

Recursively search for images in the given directory.

Parameters

• dir – the directory of the dataset

• max_dataset_size – the maximum amount of images to load

simulation.utils.machine_learning.data.image_operations module

Functions:

tensor2im(input_image[, img_type, to_rgb])	Convert a Tensor array into a numpy image array.
save_image(image_numpy, image_path[, …])	Save a numpy image to the disk.
save_images(visuals, destination[, …])	Save images to the disk.

tensor2im(input_image: torch.Tensor, img_type=<class 'numpy.uint8'>, to_rgb: bool = True) → numpy.ndarray

Convert a Tensor array into a numpy image array.

Parameters

• input_image (Tensor) – the input image tensor array

• img_type (np.integer) – the desired type of the converted numpy array

• to_rgb (bool) – translate gray image to rgb image

save_image(image_numpy: numpy.ndarray, image_path: str, aspect_ratio: float = 1.0) → None

Save a numpy image to the disk.

Parameters

• image_numpy (np.ndarray) – input numpy array

• image_path (str) – the path of the image

• aspect_ratio (float) – the aspect ratio of the resulting image

save_images(visuals: dict, destination: str, aspect_ratio: float = 1.0, post_fix: str = '') → None

Save images to the disk.

This function will save images stored in ‘visuals’.

Parameters

• destination – the folder to save the images to

• visuals (dict) – an ordered dictionary that stores (name, images (either tensor or numpy) ) pairs

• aspect_ratio (float) – the aspect ratio of saved images

• post_fix (str) – The string that extends the prefix_path

simulation.utils.machine_learning.data.image_pool module

Classes:

ImagePool(pool_size)

This class implements an image buffer that stores previously generated images.

class ImagePool(pool_size: int)

Bases: object

This class implements an image buffer that stores previously generated images.

This buffer enables us to update discriminators using a history of generated images rather than the ones produced by the latest generators.

Methods:

query(images)

Return an image from the pool.

query(images: torch.Tensor) → torch.Tensor

Return an image from the pool.

Returns images from the buffer.

By 50/100, the buffer will return input images. By 50/100, the buffer will return images previously stored in the buffer, and insert the current images to the buffer.

Parameters

images (torch.Tensor) – the latest generated images from the generator

simulation.utils.machine_learning.data.images_to_video module

Functions:

images_to_video(image_pattern, output_file)	This function takes images and put them into a video.
make_2x2_video_grid(video_paths, output_file)	This function takes 4 images and puts them into a 2x2 Grid.

images_to_video(image_pattern: str, output_file: str, use_glob: bool = False, scale: Tuple[int, int] = (1280, 650)) → None

This function takes images and put them into a video.

Parameters

• image_pattern – an pattern for all images. For example “images/%d_real_b.png” will match files like “images/5_real_b.png”

• output_file – the output file for example “videos/real_b.mp4”

• scale – scaling each image to this dimension so you can scale each image to format 1280x650

make_2x2_video_grid(video_paths: List[str], output_file: str)

This function takes 4 images and puts them into a 2x2 Grid.

Parameters

• video_paths – array of paths to the 4 videos

• output_file – output file for the resulting video

simulation.utils.machine_learning.data.labeled_dataset module

Classes:

LabeledDataset(transform_properties, Any] =, …)

Dataset of images with labels.

class LabeledDataset(transform_properties: Dict[str, Any] = <factory>, attributes: Optional[Sequence[str]] = None, classes: Dict[int, str] = <factory>, labels: Dict[str, List[Sequence[Any]]] = <factory>, _base_path: Optional[str] = None)

Bases: Generic[torch.utils.data.dataset.T_co]

Dataset of images with labels.

Attributes:

attributes	Description of what each label means.
classes	Description of what the class ids represent.
labels	Collection of all labels structured as a dictionary.
_base_path	Path to the root of the dataset.
available_files

Methods:

filter_labels()	Remove labels that have no corresponding image.
append_label(key, label)	Add a new label to the dataset.
save_as_yaml(file_path)	Save the dataset to a yaml file.
make_ids_continuous()	Reformat dataset to have continuous class ids.
replace_id(search_id, replace_id)	Replace id (search) with another id (replace) in the whole dataset.
split(fractions[, shuffle])	Split this dataset into multiple.
from_yaml(file)	Load a Labeled Dataset from a yaml file.
split_file(file, parts[, shuffle])	Split a dataset file into multiple datasets.
filter_file(file)	Filter broken file dependencies of a yaml file.

attributes: Optional[Sequence[str]] = None

Description of what each label means.

Similar to headers in a table.

classes: Dict[int, str]

Description of what the class ids represent.

labels: Dict[str, List[Sequence[Any]]]

Collection of all labels structured as a dictionary.

_base_path: Optional[str] = None

Path to the root of the dataset.

Only needs to be set if the dataset is used to load data.

property available_files: List[str]

filter_labels()

Remove labels that have no corresponding image.

append_label(key: str, label: List[Sequence[Any]])

Add a new label to the dataset.

A single image (or any abstract object) can have many labels.

save_as_yaml(file_path: str)

Save the dataset to a yaml file. Override the default method to temporarily remove base_path and prevent writing it to the yaml file.

Parameters

file_path – The output file.

make_ids_continuous()

Reformat dataset to have continuous class ids.

replace_id(search_id: int, replace_id: int)

Replace id (search) with another id (replace) in the whole dataset.

Parameters

• search_id – The id being searched for.

• replace_id – The replacement id that replaces the search ids

split(fractions: List[float], shuffle: bool = True) → List[simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset]

Split this dataset into multiple.

transform_properties: Dict[str, Any]

Properties passed as arguments to transform generation function.

classmethod from_yaml(file: str) → simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset

Load a Labeled Dataset from a yaml file.

Parameters

file – The path to the yaml file to load

classmethod split_file(file: str, parts: Dict[str, float], shuffle: bool = True) → List[simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset]

Split a dataset file into multiple datasets.

Parameters

• file – The path to the yaml file which gets split

• parts – A dict of names and and fractions

• shuffle – Split the labels randomly

classmethod filter_file(file: str) → simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset

Filter broken file dependencies of a yaml file.

Parameters

file – The path to the yaml file to filter

simulation.utils.machine_learning.data.record_simulated_rosbag module

Functions:

ros_cmd(**kwargs)
is_node_running(node_name)	Check if node is still running.
run(cmd[, max_duration, node_name, …])	Run ROS cmd in background and stop when automatic drive node shuts down.
main(**kwargs)

ros_cmd(**kwargs)

is_node_running(node_name: str) → bool

Check if node is still running.

run(cmd, max_duration: float = 120, node_name='automatic_drive', show_stdout=True, show_stderr=True)

Run ROS cmd in background and stop when automatic drive node shuts down.

main(**kwargs)

simulation.utils.machine_learning.data.rosbag_to_images module

Functions:

rosbag_to_images(bag_path, output_dir, …)

rosbag_to_images(bag_path: str, output_dir: str, image_topic: str, name_after_header: bool = False) → None

simulation.utils.machine_learning.data.rosbag_to_labels module

Functions:

rosbag_to_labels(bag_path, output_file, …)

rosbag_to_labels(bag_path: str, output_file: str, label_topic: str) → None

simulation.utils.machine_learning.data.rosbag_to_video module

Functions:

rosbag_to_video(rosbag_dir, output_dir, …)

rosbag_to_video(rosbag_dir: str, output_dir: str, image_topic: str)

simulation.utils.machine_learning.data.unlabeled_dataset module

Classes:

UnlabeledDataset(transform_properties, …)

This dataset class can load a set of unlabeled data.

class UnlabeledDataset(transform_properties: Dict[str, Any] = <factory>, folder_path: Union[str, List[str]] = <factory>)

Bases: Generic[torch.utils.data.dataset.T_co]

This dataset class can load a set of unlabeled data.

Attributes:

folder_path

Path[s] to folders that contain the data.

Methods:

load_file_paths()

List[str]: File paths to all data.

folder_path: Union[str, List[str]]

Path[s] to folders that contain the data.

load_file_paths() → List[str]

List[str]: File paths to all data.

transform_properties: Dict[str, Any]

Properties passed as arguments to transform generation function.

simulation.utils.machine_learning.data.visualizer module

Classes:

Visualizer(display_id, name, display_port, …)

This class includes several functions that can display/save images and print/save logging information.

class Visualizer(display_id: int = 1, name: str = 'kitcar', display_port: int = 8097, checkpoints_dir: str = './checkpoints')

Bases: object

This class includes several functions that can display/save images and print/save logging information.

It uses a Python library ‘visdom’ for display.

Methods:

create_visdom_connections(port)	If the program could not connect to Visdom server, this function will start a new server at port <self.port>
show_hyperparameters(hyperparameters)	Create a html table with all parameters from the dict and displays it on visdom.
display_current_results(visuals[, …])	Display current results on visdom.
plot_current_losses(epoch, counter_ratio, losses)	display the current losses on visdom display: dictionary of error labels and values.
save_losses_as_image(path)	Save the tracked losses as png file.
print_current_losses(epoch, iters, losses, …)	print current losses on console; also save the losses to the disk.

static create_visdom_connections(port: int) → None

If the program could not connect to Visdom server, this function will start a new server at port <self.port>

show_hyperparameters(hyperparameters: Dict[str, Any])

Create a html table with all parameters from the dict and displays it on visdom.

Parameters

hyperparameters – a dict containing all hyperparameters

display_current_results(visuals: Dict[str, torch.Tensor], images_per_row: int = 4)

Display current results on visdom.

Parameters

• visuals – dictionary of images to display or save

• images_per_row – Amount of images per row

plot_current_losses(epoch: int, counter_ratio: float, losses: dict) → None

display the current losses on visdom display: dictionary of error labels and values.

Parameters

• epoch – current epoch

• counter_ratio – progress (percentage) in the current epoch, between 0 to 1

• losses – training losses stored in the format of (name, float) pairs

save_losses_as_image(path: str)

Save the tracked losses as png file.

Parameters

path – The path where the loss image should be stored

print_current_losses(epoch: int, iters: int, losses: dict, t_comp: float, estimated_time: float) → None

print current losses on console; also save the losses to the disk.

Parameters

• epoch (int) – current epoch

• iters (int) – current training iteration during this epoch (reset to 0 at the end of every epoch)

• losses (dict) – training losses stored in the format of (name, float) pairs

• t_comp (float) – computational time per data point (normalized by batch_size)

• estimated_time (float) – the estimated time until training finishes

Module contents

Functions:

load_unpaired_unlabeled_datasets(dir_a, …)	Create dataloader for two unpaired and unlabeled datasets.
sample_generator(dataloader[, n_samples])	Generator that samples from a dataloader.
unpaired_sample_generator(dataloader_a, …)	Generator that samples pairwise from both dataloaders.
load_labeled_dataset(label_file, …)	Create dataloader for a labeled dataset.

load_unpaired_unlabeled_datasets(dir_a: Union[str, List[str]], dir_b: Union[str, List[str]], max_dataset_size: int, batch_size: int, sequential: bool, num_threads: int, grayscale_a: bool, grayscale_b: bool, transform_properties: Dict[str, Any]) → Tuple[simulation.utils.machine_learning.data.data_loader.DataLoader, simulation.utils.machine_learning.data.data_loader.DataLoader]

Create dataloader for two unpaired and unlabeled datasets.

E.g. used by cycle gan with data from two domains.

Parameters

• dir_a – path to images of domain a

• dir_b – path to images of domain b

• max_dataset_size (int) – maximum amount of images to load; -1 means infinity

• batch_size (int) – input batch size

• sequential (bool) – if true, takes images in order, otherwise takes them randomly

• num_threads (int) – threads for loading data

• grayscale_a (bool) – transform domain a to gray images

• grayscale_b (bool) – transform domain b to gray images

• transform_properties – dict containing properties for transforming images

sample_generator(dataloader: simulation.utils.machine_learning.data.data_loader.DataLoader, n_samples: int = inf)

Generator that samples from a dataloader.

Parameters

• dataloader – Dataloader.

• n_samples – Number of batches of samples.

unpaired_sample_generator(dataloader_a: simulation.utils.machine_learning.data.data_loader.DataLoader, dataloader_b: simulation.utils.machine_learning.data.data_loader.DataLoader, n_samples: int = inf)

Generator that samples pairwise from both dataloaders.

Parameters

• dataloader_a – Domain a dataloader.

• dataloader_b – Domain b dataloader.

• n_samples – Number of batches of samples.

load_labeled_dataset(label_file: str, max_dataset_size: int, batch_size: int, sequential: bool, num_threads: int, transform_properties: Dict[str, Any]) → simulation.utils.machine_learning.data.data_loader.DataLoader

Create dataloader for a labeled dataset.

Parameters

• label_file – Path to a file containing all labels

• max_dataset_size – Maximum amount of images to load; -1 means infinity

• batch_size – Batch size

• sequential – If true, takes images in order, otherwise takes them randomly

• num_threads – Threads for loading data