simulation.utils.machine_learning.data package

Submodules

simulation.utils.machine_learning.data.base_dataset module

This module implements an abstract base class (ABC) ‘BaseDataset’ for datasets.

It also includes common transformation functions (e.g., get_transform, __scale_width), which can be later used in subclasses.

Classes:

BaseDataset(transform_properties, Any] =)

This is the base class for other datasets.

Functions:

get_params(preprocess, load_size, crop_size, …)

param preprocess

Scaling and cropping of images at load time

get_transform([load_size, crop_size, mask, …])

Create transformation from arguments.

__make_power_2(img, base[, method])

param img

image to transform

__scale_width(img, target_size, crop_size[, …])

param img

image to transform

__crop(img, pos, size)

param img

image to transform

__apply_mask(img, mask_file)

Overlay image with the provided mask.

__flip(img, flip)

__print_size_warning(ow, oh, w, h)

Print warning information about image size(only print once)

class BaseDataset(transform_properties: Dict[str, Any] = <factory>)[source]

Bases: Generic[torch.utils.data.dataset.T_co]

This is the base class for other datasets.

Attributes:

transform_properties

Properties passed as arguments to transform generation function.

transform

Transformation that can be applied to images.

transform_properties: Dict[str, Any]

Properties passed as arguments to transform generation function.

property transform: torchvision.transforms.transforms.Compose

Transformation that can be applied to images.

Type

transforms.Compose

get_params(preprocess: Iterable, load_size: int, crop_size: int, size: Tuple[int, int])Dict[str, Any][source]
Parameters
  • preprocess – Scaling and cropping of images at load time [resize | crop | scale_width]

  • load_size – Scale images to this size

  • crop_size – Then crop to this size

  • size – The image sizes

get_transform(load_size: int = - 1, crop_size: int = - 1, mask: Optional[str] = None, preprocess: Iterable = {}, no_flip: bool = True, params=None, grayscale=False, method=3, convert=True)torchvision.transforms.transforms.Compose[source]

Create transformation from arguments.

Parameters
  • load_size – Scale images to this size

  • crop_size – Then crop to this size

  • mask – Path to a mask overlaid over all images

  • preprocess – scaling and cropping of images at load time [resize | crop | scale_width]

  • no_flip – Flip 50% of all training images vertically

  • params – more params for cropping

  • grayscale – enable or disable grayscale

  • method – the transform method

  • convert – enable or disable transformations and normalizations

__make_power_2(img, base, method=3)[source]
Parameters
  • img – image to transform

  • base – the base

  • method – the transform method

__scale_width(img, target_size, crop_size, method=3)[source]
Parameters
  • img – image to transform

  • target_size – the load size

  • crop_size – the crop size, which is used for training

  • method – the transform method

__crop(img, pos, size)[source]
Parameters
  • img – image to transform

  • pos – where to crop my image

  • size – resulting size of cropped image

__apply_mask(img: PIL.Image.Image, mask_file: str)PIL.Image.Image[source]

Overlay image with the provided mask.

Parameters
  • img (Image.Image) – image to transform

  • mask_file (str) – path to mask image file

__flip(img, flip)[source]
__print_size_warning(ow, oh, w, h)[source]

Print warning information about image size(only print once)

Parameters
  • ow – original width

  • oh – original height

  • w – width

  • h – height

simulation.utils.machine_learning.data.data_loader module

Classes:

DataLoader(dataset, …)

Wrapper class of Dataset class that performs multi-threaded data loading.

class DataLoader(dataset: Union[simulation.utils.machine_learning.data.unlabeled_dataset.UnlabeledDataset, simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset], max_dataset_size: int, batch_size: int = 1, num_threads: int = 1, sequential: bool = False)[source]

Bases: object

Wrapper class of Dataset class that performs multi-threaded data loading.

Methods:

load_data()

load_data()[source]

simulation.utils.machine_learning.data.extract_simulated_images module

Functions:

main(**kwargs)

main(**kwargs)[source]

simulation.utils.machine_learning.data.image_folder module

A modified image folder class.

We modify the official PyTorch image folder (https://github.com/pytorch/vision/blob/master/torchvision/datasets/folder.py) so that this class can load images from both current directory and its subdirectories.

Functions:

is_image_file(filename)

Check if a file is an image.

find_images(dir[, max_dataset_size])

Recursively search for images in the given directory.

is_image_file(filename)[source]

Check if a file is an image.

Parameters

filename – the file name to check

find_images(dir: str, max_dataset_size=inf)List[str][source]

Recursively search for images in the given directory.

Parameters
  • dir – the directory of the dataset

  • max_dataset_size – the maximum amount of images to load

simulation.utils.machine_learning.data.image_operations module

Functions:

tensor2im(input_image[, img_type, to_rgb])

Convert a Tensor array into a numpy image array.

save_image(image_numpy, image_path[, …])

Save a numpy image to the disk.

save_images(visuals, destination[, …])

Save images to the disk.

tensor2im(input_image: torch.Tensor, img_type=<class 'numpy.uint8'>, to_rgb: bool = True)numpy.ndarray[source]

Convert a Tensor array into a numpy image array.

Parameters
  • input_image (Tensor) – the input image tensor array

  • img_type (np.integer) – the desired type of the converted numpy array

  • to_rgb (bool) – translate gray image to rgb image

save_image(image_numpy: numpy.ndarray, image_path: str, aspect_ratio: float = 1.0)None[source]

Save a numpy image to the disk.

Parameters
  • image_numpy (np.ndarray) – input numpy array

  • image_path (str) – the path of the image

  • aspect_ratio (float) – the aspect ratio of the resulting image

save_images(visuals: dict, destination: str, aspect_ratio: float = 1.0, post_fix: str = '')None[source]

Save images to the disk.

This function will save images stored in ‘visuals’.

Parameters
  • destination – the folder to save the images to

  • visuals (dict) – an ordered dictionary that stores (name, images (either tensor or numpy) ) pairs

  • aspect_ratio (float) – the aspect ratio of saved images

  • post_fix (str) – The string that extends the prefix_path

simulation.utils.machine_learning.data.image_pool module

Classes:

ImagePool(pool_size)

This class implements an image buffer that stores previously generated images.

class ImagePool(pool_size: int)[source]

Bases: object

This class implements an image buffer that stores previously generated images.

This buffer enables us to update discriminators using a history of generated images rather than the ones produced by the latest generators.

Methods:

query(images)

Return an image from the pool.

query(images: torch.Tensor)torch.Tensor[source]

Return an image from the pool.

Returns images from the buffer.

By 50/100, the buffer will return input images. By 50/100, the buffer will return images previously stored in the buffer, and insert the current images to the buffer.

Parameters

images (torch.Tensor) – the latest generated images from the generator

simulation.utils.machine_learning.data.images_to_video module

Functions:

images_to_video(image_pattern, output_file)

This function takes images and put them into a video.

make_2x2_video_grid(video_paths, output_file)

This function takes 4 images and puts them into a 2x2 Grid.

images_to_video(image_pattern: str, output_file: str, use_glob: bool = False, scale: Tuple[int, int] = (1280, 650))None[source]

This function takes images and put them into a video.

Parameters
  • image_pattern – an pattern for all images. For example “images/%d_real_b.png” will match files like “images/5_real_b.png”

  • output_file – the output file for example “videos/real_b.mp4”

  • scale – scaling each image to this dimension so you can scale each image to format 1280x650

make_2x2_video_grid(video_paths: List[str], output_file: str)[source]

This function takes 4 images and puts them into a 2x2 Grid.

Parameters
  • video_paths – array of paths to the 4 videos

  • output_file – output file for the resulting video

simulation.utils.machine_learning.data.labeled_dataset module

Classes:

LabeledDataset(transform_properties, Any] =, …)

Dataset of images with labels.

class LabeledDataset(transform_properties: Dict[str, Any] = <factory>, attributes: Optional[Sequence[str]] = None, classes: Dict[int, str] = <factory>, labels: Dict[str, List[Sequence[Any]]] = <factory>, _base_path: Optional[str] = None)[source]

Bases: Generic[torch.utils.data.dataset.T_co]

Dataset of images with labels.

Attributes:

attributes

Description of what each label means.

classes

Description of what the class ids represent.

labels

Collection of all labels structured as a dictionary.

_base_path

Path to the root of the dataset.

available_files

Methods:

filter_labels()

Remove labels that have no corresponding image.

append_label(key, label)

Add a new label to the dataset.

save_as_yaml(file_path)

Save the dataset to a yaml file.

make_ids_continuous()

Reformat dataset to have continuous class ids.

replace_id(search_id, replace_id)

Replace id (search) with another id (replace) in the whole dataset.

split(fractions[, shuffle])

Split this dataset into multiple.

from_yaml(file)

Load a Labeled Dataset from a yaml file.

split_file(file, parts[, shuffle])

Split a dataset file into multiple datasets.

filter_file(file)

Filter broken file dependencies of a yaml file.

attributes: Optional[Sequence[str]] = None

Description of what each label means.

Similar to headers in a table.

classes: Dict[int, str]

Description of what the class ids represent.

labels: Dict[str, List[Sequence[Any]]]

Collection of all labels structured as a dictionary.

_base_path: Optional[str] = None

Path to the root of the dataset.

Only needs to be set if the dataset is used to load data.

property available_files: List[str]
filter_labels()[source]

Remove labels that have no corresponding image.

append_label(key: str, label: List[Sequence[Any]])[source]

Add a new label to the dataset.

A single image (or any abstract object) can have many labels.

save_as_yaml(file_path: str)[source]

Save the dataset to a yaml file. Override the default method to temporarily remove base_path and prevent writing it to the yaml file.

Parameters

file_path – The output file.

make_ids_continuous()[source]

Reformat dataset to have continuous class ids.

replace_id(search_id: int, replace_id: int)[source]

Replace id (search) with another id (replace) in the whole dataset.

Parameters
  • search_id – The id being searched for.

  • replace_id – The replacement id that replaces the search ids

split(fractions: List[float], shuffle: bool = True)List[simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset][source]

Split this dataset into multiple.

transform_properties: Dict[str, Any]

Properties passed as arguments to transform generation function.

classmethod from_yaml(file: str)simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset[source]

Load a Labeled Dataset from a yaml file.

Parameters

file – The path to the yaml file to load

classmethod split_file(file: str, parts: Dict[str, float], shuffle: bool = True)List[simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset][source]

Split a dataset file into multiple datasets.

Parameters
  • file – The path to the yaml file which gets split

  • parts – A dict of names and and fractions

  • shuffle – Split the labels randomly

classmethod filter_file(file: str)simulation.utils.machine_learning.data.labeled_dataset.LabeledDataset[source]

Filter broken file dependencies of a yaml file.

Parameters

file – The path to the yaml file to filter

simulation.utils.machine_learning.data.record_simulated_rosbag module

Functions:

ros_cmd(**kwargs)

is_node_running(node_name)

Check if node is still running.

run(cmd[, max_duration, node_name, …])

Run ROS cmd in background and stop when automatic drive node shuts down.

main(**kwargs)

ros_cmd(**kwargs)[source]
is_node_running(node_name: str)bool[source]

Check if node is still running.

run(cmd, max_duration: float = 120, node_name='automatic_drive', show_stdout=True, show_stderr=True)[source]

Run ROS cmd in background and stop when automatic drive node shuts down.

main(**kwargs)[source]

simulation.utils.machine_learning.data.rosbag_to_images module

Functions:

rosbag_to_images(bag_path, output_dir, …)

rosbag_to_images(bag_path: str, output_dir: str, image_topic: str, name_after_header: bool = False)None[source]

simulation.utils.machine_learning.data.rosbag_to_labels module

Functions:

rosbag_to_labels(bag_path, output_file, …)

rosbag_to_labels(bag_path: str, output_file: str, label_topic: str)None[source]

simulation.utils.machine_learning.data.rosbag_to_video module

Functions:

rosbag_to_video(rosbag_dir, output_dir, …)

rosbag_to_video(rosbag_dir: str, output_dir: str, image_topic: str)[source]

simulation.utils.machine_learning.data.unlabeled_dataset module

Classes:

UnlabeledDataset(transform_properties, …)

This dataset class can load a set of unlabeled data.

class UnlabeledDataset(transform_properties: Dict[str, Any] = <factory>, folder_path: Union[str, List[str]] = <factory>)[source]

Bases: Generic[torch.utils.data.dataset.T_co]

This dataset class can load a set of unlabeled data.

Attributes:

folder_path

Path[s] to folders that contain the data.

Methods:

load_file_paths()

List[str]: File paths to all data.

folder_path: Union[str, List[str]]

Path[s] to folders that contain the data.

load_file_paths()List[str][source]

List[str]: File paths to all data.

transform_properties: Dict[str, Any]

Properties passed as arguments to transform generation function.

simulation.utils.machine_learning.data.visualizer module

Classes:

Visualizer(display_id, name, display_port, …)

This class includes several functions that can display/save images and print/save logging information.

class Visualizer(display_id: int = 1, name: str = 'kitcar', display_port: int = 8097, checkpoints_dir: str = './checkpoints')[source]

Bases: object

This class includes several functions that can display/save images and print/save logging information.

It uses a Python library ‘visdom’ for display.

Methods:

create_visdom_connections(port)

If the program could not connect to Visdom server, this function will start a new server at port <self.port>

show_hyperparameters(hyperparameters)

Create a html table with all parameters from the dict and displays it on visdom.

display_current_results(visuals[, …])

Display current results on visdom.

plot_current_losses(epoch, counter_ratio, losses)

display the current losses on visdom display: dictionary of error labels and values.

save_losses_as_image(path)

Save the tracked losses as png file.

print_current_losses(epoch, iters, losses, …)

print current losses on console; also save the losses to the disk.

static create_visdom_connections(port: int)None[source]

If the program could not connect to Visdom server, this function will start a new server at port <self.port>

show_hyperparameters(hyperparameters: Dict[str, Any])[source]

Create a html table with all parameters from the dict and displays it on visdom.

Parameters

hyperparameters – a dict containing all hyperparameters

display_current_results(visuals: Dict[str, torch.Tensor], images_per_row: int = 4)[source]

Display current results on visdom.

Parameters
  • visuals – dictionary of images to display or save

  • images_per_row – Amount of images per row

plot_current_losses(epoch: int, counter_ratio: float, losses: dict)None[source]

display the current losses on visdom display: dictionary of error labels and values.

Parameters
  • epoch – current epoch

  • counter_ratio – progress (percentage) in the current epoch, between 0 to 1

  • losses – training losses stored in the format of (name, float) pairs

save_losses_as_image(path: str)[source]

Save the tracked losses as png file.

Parameters

path – The path where the loss image should be stored

print_current_losses(epoch: int, iters: int, losses: dict, t_comp: float, estimated_time: float)None[source]

print current losses on console; also save the losses to the disk.

Parameters
  • epoch (int) – current epoch

  • iters (int) – current training iteration during this epoch (reset to 0 at the end of every epoch)

  • losses (dict) – training losses stored in the format of (name, float) pairs

  • t_comp (float) – computational time per data point (normalized by batch_size)

  • estimated_time (float) – the estimated time until training finishes

Module contents

Functions:

load_unpaired_unlabeled_datasets(dir_a, …)

Create dataloader for two unpaired and unlabeled datasets.

sample_generator(dataloader[, n_samples])

Generator that samples from a dataloader.

unpaired_sample_generator(dataloader_a, …)

Generator that samples pairwise from both dataloaders.

load_labeled_dataset(label_file, …)

Create dataloader for a labeled dataset.

load_unpaired_unlabeled_datasets(dir_a: Union[str, List[str]], dir_b: Union[str, List[str]], max_dataset_size: int, batch_size: int, sequential: bool, num_threads: int, grayscale_a: bool, grayscale_b: bool, transform_properties: Dict[str, Any])Tuple[simulation.utils.machine_learning.data.data_loader.DataLoader, simulation.utils.machine_learning.data.data_loader.DataLoader][source]

Create dataloader for two unpaired and unlabeled datasets.

E.g. used by cycle gan with data from two domains.

Parameters
  • dir_a – path to images of domain a

  • dir_b – path to images of domain b

  • max_dataset_size (int) – maximum amount of images to load; -1 means infinity

  • batch_size (int) – input batch size

  • sequential (bool) – if true, takes images in order, otherwise takes them randomly

  • num_threads (int) – threads for loading data

  • grayscale_a (bool) – transform domain a to gray images

  • grayscale_b (bool) – transform domain b to gray images

  • transform_properties – dict containing properties for transforming images

sample_generator(dataloader: simulation.utils.machine_learning.data.data_loader.DataLoader, n_samples: int = inf)[source]

Generator that samples from a dataloader.

Parameters
  • dataloader – Dataloader.

  • n_samples – Number of batches of samples.

unpaired_sample_generator(dataloader_a: simulation.utils.machine_learning.data.data_loader.DataLoader, dataloader_b: simulation.utils.machine_learning.data.data_loader.DataLoader, n_samples: int = inf)[source]

Generator that samples pairwise from both dataloaders.

Parameters
  • dataloader_a – Domain a dataloader.

  • dataloader_b – Domain b dataloader.

  • n_samples – Number of batches of samples.

load_labeled_dataset(label_file: str, max_dataset_size: int, batch_size: int, sequential: bool, num_threads: int, transform_properties: Dict[str, Any])simulation.utils.machine_learning.data.data_loader.DataLoader[source]

Create dataloader for a labeled dataset.

Parameters
  • label_file – Path to a file containing all labels

  • max_dataset_size – Maximum amount of images to load; -1 means infinity

  • batch_size – Batch size

  • sequential – If true, takes images in order, otherwise takes them randomly

  • num_threads – Threads for loading data