This is the multi-page printable view of this section. Click here to print.

Return to the regular view of this page.

Data Types

This module defines data types for logging rich, interactive visualizations to W&B.

Data types include common media types, like images, audio, and videos, flexible containers for information, like tables and HTML, and more.

For more on logging media, see our guide

For more on logging structured data for interactive dataset and model analysis, see our guide to W&B Tables.

All of these special data types are subclasses of WBValue. All the data types serialize to JSON, since that is what wandb uses to save the objects locally and upload them to the W&B server.

Classes

class Audio: W&B class for audio clips.

class BoundingBoxes2D: Format images with 2D bounding box overlays for logging to W&B.

class Graph: W&B class for graphs.

class Histogram: W&B class for histograms.

class Html: W&B class for logging HTML content to W&B.

class Image: A class for logging images to W&B.

class ImageMask: Format image masks or overlays for logging to W&B.

class Molecule: W&B class for 3D Molecular data.

class Object3D: W&B class for 3D point clouds.

class Plotly: W&B class for Plotly plots.

class Table: The Table class used to display and analyze tabular data.

class Video: A class for logging videos to W&B.

class WBTraceTree: Media object for trace tree data.

1 - Audio

View the source code

class Audio

W&B class for audio clips.

method Audio.__init__

__init__(
    data_or_path: Union[str, pathlib.Path, list, ForwardRef('np.ndarray')],
    sample_rate: Optional[int] = None,
    caption: Optional[str] = None
)

Accept a path to an audio file or a numpy array of audio data.

Args:

  • data_or_path: A path to an audio file or a NumPy array of audio data.
  • sample_rate: Sample rate, required when passing in raw NumPy array of audio data.
  • caption: Caption to display with audio.

classmethod Audio.durations

durations(audio_list)

Calculate the duration of the audio files.

classmethod Audio.sample_rates

sample_rates(audio_list)

Get sample rates of the audio files.

2 - BoundingBoxes2D

View the source code

Format images with 2D bounding box overlays for logging to W&B.

Args
val (dictionary) A dictionary of the following form: box_data: (list of dictionaries) One dictionary for each bounding box, containing: position: (dictionary) the position and size of the bounding box, in one of two formats Note that boxes need not all use the same format. {“minX”, “minY”, “maxX”, “maxY”}: (dictionary) A set of coordinates defining the upper and lower bounds of the box (the bottom left and top right corners) {“middle”, “width”, “height”}: (dictionary) A set of coordinates defining the center and dimensions of the box, with “middle” as a list [x, y] for the center point and “width” and “height” as numbers domain: (string) One of two options for the bounding box coordinate domain null: By default, or if no argument is passed, the coordinate domain is assumed to be relative to the original image, expressing this box as a fraction or percentage of the original image. This means all coordinates and dimensions passed into the “position” argument are floating point numbers between 0 and 1. “pixel”: (string literal) The coordinate domain is set to the pixel space. This means all coordinates and dimensions passed into “position” are integers within the bounds of the image dimensions. class_id: (integer) The class label id for this box scores: (dictionary of string to number, optional) A mapping of named fields to numerical values (float or int), can be used for filtering boxes in the UI based on a range of values for the corresponding field box_caption: (string, optional) A string to be displayed as the label text above this box in the UI, often composed of the class label, class name, and/or scores class_labels: (dictionary, optional) A map of integer class labels to their readable class names
key (string) The readable name or id for this set of bounding boxes (e.g. predictions, ground_truth)

Examples:

Log bounding boxes for a single image

import numpy as np
import wandb

run = wandb.init()
image = np.random.randint(low=0, high=256, size=(200, 300, 3))

class_labels = {0: "person", 1: "car", 2: "road", 3: "building"}

img = wandb.Image(
    image,
    boxes={
        "predictions": {
            "box_data": [
                {
                    # one box expressed in the default relative/fractional domain
                    "position": {
                        "minX": 0.1,
                        "maxX": 0.2,
                        "minY": 0.3,
                        "maxY": 0.4,
                    },
                    "class_id": 1,
                    "box_caption": class_labels[1],
                    "scores": {"acc": 0.2, "loss": 1.2},
                },
                {
                    # another box expressed in the pixel domain
                    "position": {
                        "middle": [150, 20],
                        "width": 68,
                        "height": 112,
                    },
                    "domain": "pixel",
                    "class_id": 3,
                    "box_caption": "a building",
                    "scores": {"acc": 0.5, "loss": 0.7},
                },
                # Log as many boxes an as needed
            ],
            "class_labels": class_labels,
        }
    },
)

run.log({"driving_scene": img})

Log a bounding box overlay to a Table

import numpy as np
import wandb

run = wandb.init()
image = np.random.randint(low=0, high=256, size=(200, 300, 3))

class_labels = {0: "person", 1: "car", 2: "road", 3: "building"}

class_set = wandb.Classes(
    [
        {"name": "person", "id": 0},
        {"name": "car", "id": 1},
        {"name": "road", "id": 2},
        {"name": "building", "id": 3},
    ]
)

img = wandb.Image(
    image,
    boxes={
        "predictions": {
            "box_data": [
                {
                    # one box expressed in the default relative/fractional domain
                    "position": {
                        "minX": 0.1,
                        "maxX": 0.2,
                        "minY": 0.3,
                        "maxY": 0.4,
                    },
                    "class_id": 1,
                    "box_caption": class_labels[1],
                    "scores": {"acc": 0.2, "loss": 1.2},
                },
                {
                    # another box expressed in the pixel domain
                    "position": {
                        "middle": [150, 20],
                        "width": 68,
                        "height": 112,
                    },
                    "domain": "pixel",
                    "class_id": 3,
                    "box_caption": "a building",
                    "scores": {"acc": 0.5, "loss": 0.7},
                },
                # Log as many boxes an as needed
            ],
            "class_labels": class_labels,
        }
    },
    classes=class_set,
)

table = wandb.Table(columns=["image"])
table.add_data(img)
run.log({"driving_scene": table})

Methods

type_name

View source

@classmethod
type_name() -> str

validate

View source

validate(
    val: dict
) -> bool

3 - box3d()

View the source code

function box3d

box3d(
    center: 'npt.ArrayLike',
    size: 'npt.ArrayLike',
    orientation: 'npt.ArrayLike',
    color: 'RGBColor',
    label: 'Optional[str]' = None,
    score: 'Optional[numeric]' = None
) → Box3D

A 3D bounding box. The box is specified by its center, size and orientation.

Args:

  • center: The center point of the box as a length-3 ndarray.
  • size: The box’s X, Y and Z dimensions as a length-3 ndarray.
  • orientation: The rotation transforming global XYZ coordinates into the box’s local XYZ coordinates, given as a length-4 ndarray [r, x, y, z] corresponding to the non-zero quaternion r + xi + yj + zk.
  • color: The box’s color as an (r, g, b) tuple with 0 <= r,g,b <= 1.
  • label: An optional label for the box.
  • score: An optional score for the box. Typically used to indicate the confidence of a detection.

Returns: A Box3D object.

Example: The following example creates a point cloud with 60 boxes rotating around the X, Y and Z axes.

import wandb

import math
import numpy as np
from scipy.spatial.transform import Rotation


with wandb.init() as run:
    run.log(
         {
             "points": wandb.Object3D.from_point_cloud(
                 points=np.random.uniform(-5, 5, size=(100, 3)),
                 boxes=[
                     wandb.box3d(
                         center=(0.3 * t - 3, 0, 0),
                         size=(0.1, 0.1, 0.1),
                         orientation=Rotation.from_euler(
                             "xyz", [t * math.pi / 10, 0, 0]
                         ).as_quat(),
                         color=(0.5 + t / 40, 0.5, 0.5),
                         label=f"box {t}",
                         score=0.9,
                     )
                     for t in range(20)
                 ]
                 + [
                     wandb.box3d(
                         center=(0, 0.3 * t - 3, 0.3),
                         size=(0.1, 0.1, 0.1),
                         orientation=Rotation.from_euler(
                             "xyz", [0, t * math.pi / 10, 0]
                         ).as_quat(),
                         color=(0.5, 0.5 + t / 40, 0.5),
                         label=f"box {t}",
                         score=0.9,
                     )
                     for t in range(20)
                 ]
                 + [
                     wandb.box3d(
                         center=(0.3, 0.3, 0.3 * t - 3),
                         size=(0.1, 0.1, 0.1),
                         orientation=Rotation.from_euler(
                             "xyz", [0, 0, t * math.pi / 10]
                         ).as_quat(),
                         color=(0.5, 0.5, 0.5 + t / 40),
                         label=f"box {t}",
                         score=0.9,
                     )
                     for t in range(20)
                 ],
             ),
         }
    )

4 - Graph

View the source code

W&B class for graphs.

This class is typically used for saving and displaying neural net models. It represents the graph as an array of nodes and edges. The nodes can have labels that can be visualized by wandb.

Examples:

Import a keras model.

import wandb

wandb.Graph.from_keras(keras_model)

Methods

add_edge

View source

add_edge(
    from_node, to_node
)

Add an edge to the graph.

add_node

View source

add_node(
    node=None, **node_kwargs
)

Add a node to the graph.

from_keras

View source

@classmethod
from_keras(
    model
)

Create a graph from a Keras model.

This method is not supported for Keras 3.0.0 and above. Requires a refactor.

“

pprint

View source

pprint()

Pretty print the graph.

__getitem__

View source

__getitem__(
    nid
)

5 - Histogram

View the source code

class Histogram

W&B class for histograms.

This object works just like numpy’s histogram function https://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram.html

Attributes:

  • bins ([float]): Edges of bins
  • histogram ([int]): Number of elements falling in each bin.

method Histogram.__init__

__init__(
    sequence: Optional[Sequence] = None,
    np_histogram: Optional[ForwardRef('NumpyHistogram')] = None,
    num_bins: int = 64
) → None

Initialize a Histogram object.

Args: sequence: Input data for histogram. np_histogram: Alternative input of a precomputed histogram. num_bins: Number of bins for the histogram. The default number of bins is 64. The maximum number of bins is 512.

Examples: Generate histogram from a sequence.

import wandb

wandb.Histogram([1, 2, 3])

Efficiently initialize from np.histogram.

import numpy as np
import wandb

hist = np.histogram(data)
wandb.Histogram(np_histogram=hist)

6 - Html

View the source code

class Html

W&B class for logging HTML content to W&B.

method Html.__init__

__init__(
    data: Union[str, pathlib.Path, ForwardRef('TextIO')],
    inject: bool = True,
    data_is_not_path: bool = False
) → None

Creates a W&B HTML object.

Args: data: A string that is a path to a file with the extension “.html”, or a string or IO object containing literal HTML.

  • inject: Add a stylesheet to the HTML object. If set to False the HTML will pass through unchanged.
  • data_is_not_path: If set to False, the data will be treated as a path to a file.

Examples: It can be initialized by providing a path to a file:

with wandb.init() as run:
    run.log({"html": wandb.Html("./index.html")})

Alternatively, it can be initialized by providing literal HTML, in either a string or IO object:

with wandb.init() as run:
    run.log({"html": wandb.Html("<h1>Hello, world!</h1>")})

7 - Image

View the source code

class Image

A class for logging images to W&B.

method Image.__init__

__init__(
    data_or_path: 'ImageDataOrPathType',
    mode: Optional[str] = None,
    caption: Optional[str] = None,
    grouping: Optional[int] = None,
    classes: Optional[ForwardRef('Classes'), Sequence[dict]] = None,
    boxes: Optional[Dict[str, ForwardRef('BoundingBoxes2D')], Dict[str, dict]] = None,
    masks: Optional[Dict[str, ForwardRef('ImageMask')], Dict[str, dict]] = None,
    file_type: Optional[str] = None,
    normalize: bool = True
) → None

Initialize a wandb.Image object.

Args:

  • data_or_path: Accepts NumPy array/pytorch tensor of image data, a PIL image object, or a path to an image file. If a NumPy array or pytorch tensor is provided, the image data will be saved to the given file type. If the values are not in the range [0, 255] or all values are in the range [0, 1], the image pixel values will be normalized to the range [0, 255] unless normalize is set to False.
    • pytorch tensor should be in the format (channel, height, width)
    • NumPy array should be in the format (height, width, channel)
  • mode: The PIL mode for an image. Most common are “L”, “RGB”,
  • "RGBA". Full explanation at https: //pillow.readthedocs.io/en/stable/handbook/concepts.html#modes
  • caption: Label for display of image.
  • grouping: The grouping number for the image.
  • classes: A list of class information for the image, used for labeling bounding boxes, and image masks.
  • boxes: A dictionary containing bounding box information for the image.
  • see: https://docs.wandb.ai/ref/python/data-types/boundingboxes2d/
  • masks: A dictionary containing mask information for the image.
  • see: https://docs.wandb.ai/ref/python/data-types/imagemask/
  • file_type: The file type to save the image as. This parameter has no effect if data_or_path is a path to an image file.
  • normalize: If True, normalize the image pixel values to fall within the range of [0, 255]. Normalize is only applied if data_or_path is a numpy array or pytorch tensor.

Examples: Create a wandb.Image from a numpy array

import numpy as np
import wandb

with wandb.init() as run:
    examples = []
    for i in range(3):
         pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))
         image = wandb.Image(pixels, caption=f"random field {i}")
         examples.append(image)
    run.log({"examples": examples})

Create a wandb.Image from a PILImage

import numpy as np
from PIL import Image as PILImage
import wandb

with wandb.init() as run:
    examples = []
    for i in range(3):
         pixels = np.random.randint(
             low=0, high=256, size=(100, 100, 3), dtype=np.uint8
         )
         pil_image = PILImage.fromarray(pixels, mode="RGB")
         image = wandb.Image(pil_image, caption=f"random field {i}")
         examples.append(image)
    run.log({"examples": examples})

Log .jpg rather than .png (default)

import numpy as np
import wandb

with wandb.init() as run:
    examples = []
    for i in range(3):
         pixels = np.random.randint(low=0, high=256, size=(100, 100, 3))
         image = wandb.Image(
             pixels, caption=f"random field {i}", file_type="jpg"
         )
         examples.append(image)
    run.log({"examples": examples})

property Image.image

8 - ImageMask

View the source code

Format image masks or overlays for logging to W&B.

Args
val (dictionary) One of these two keys to represent the image: mask_data : (2D numpy array) The mask containing an integer class label for each pixel in the image path : (string) The path to a saved image file of the mask class_labels : (dictionary of integers to strings, optional) A mapping of the integer class labels in the mask to readable class names. These will default to class_0, class_1, class_2, etc.
key (string) The readable name or id for this mask type (e.g. predictions, ground_truth)

Examples:

Logging a single masked image

import numpy as np
import wandb

run = wandb.init()
image = np.random.randint(low=0, high=256, size=(100, 100, 3), dtype=np.uint8)
predicted_mask = np.empty((100, 100), dtype=np.uint8)
ground_truth_mask = np.empty((100, 100), dtype=np.uint8)

predicted_mask[:50, :50] = 0
predicted_mask[50:, :50] = 1
predicted_mask[:50, 50:] = 2
predicted_mask[50:, 50:] = 3

ground_truth_mask[:25, :25] = 0
ground_truth_mask[25:, :25] = 1
ground_truth_mask[:25, 25:] = 2
ground_truth_mask[25:, 25:] = 3

class_labels = {0: "person", 1: "tree", 2: "car", 3: "road"}

masked_image = wandb.Image(
    image,
    masks={
        "predictions": {
            "mask_data": predicted_mask,
            "class_labels": class_labels,
        },
        "ground_truth": {
            "mask_data": ground_truth_mask,
            "class_labels": class_labels,
        },
    },
)
run.log({"img_with_masks": masked_image})

Log a masked image inside a Table

import numpy as np
import wandb

run = wandb.init()
image = np.random.randint(low=0, high=256, size=(100, 100, 3), dtype=np.uint8)
predicted_mask = np.empty((100, 100), dtype=np.uint8)
ground_truth_mask = np.empty((100, 100), dtype=np.uint8)

predicted_mask[:50, :50] = 0
predicted_mask[50:, :50] = 1
predicted_mask[:50, 50:] = 2
predicted_mask[50:, 50:] = 3

ground_truth_mask[:25, :25] = 0
ground_truth_mask[25:, :25] = 1
ground_truth_mask[:25, 25:] = 2
ground_truth_mask[25:, 25:] = 3

class_labels = {0: "person", 1: "tree", 2: "car", 3: "road"}

class_set = wandb.Classes(
    [
        {"name": "person", "id": 0},
        {"name": "tree", "id": 1},
        {"name": "car", "id": 2},
        {"name": "road", "id": 3},
    ]
)

masked_image = wandb.Image(
    image,
    masks={
        "predictions": {
            "mask_data": predicted_mask,
            "class_labels": class_labels,
        },
        "ground_truth": {
            "mask_data": ground_truth_mask,
            "class_labels": class_labels,
        },
    },
    classes=class_set,
)

table = wandb.Table(columns=["image"])
table.add_data(masked_image)
run.log({"random_field": table})

Methods

type_name

View source

@classmethod
type_name() -> str

validate

View source

validate(
    val: dict
) -> bool

9 - Molecule

View the source code

class Molecule

W&B class for 3D Molecular data.

method Molecule.__init__

__init__(
    data_or_path: Union[str, pathlib.Path, ForwardRef('TextIO')],
    caption: Optional[str] = None,
    **kwargs: str
) → None

Initialize a Molecule object.

Args:

  • data_or_path: Molecule can be initialized from a file name or an io object.
  • caption: Caption associated with the molecule for display.

10 - Object3D

View the source code

class Object3D

W&B class for 3D point clouds.

method Object3D.__init__

__init__(
    data_or_path: Union[ForwardRef('np.ndarray'), str, pathlib.Path, ForwardRef('TextIO'), dict],
    caption: Optional[str] = None,
    **kwargs: Optional[str, ForwardRef('FileFormat3D')]
) → None

Creates a W&B Object3D object.

Args:

  • data_or_path: Object3D can be initialized from a file or a numpy array.
  • caption: Caption associated with the object for display.

Examples: The shape of the numpy array must be one of either

[[x y z],       ...] nx3
[[x y z c],     ...] nx4 where c is a category with supported range [1, 14]
[[x y z r g b], ...] nx6 where is rgb is color

11 - Plotly

View the source code

class Plotly

W&B class for Plotly plots.

method Plotly.__init__

__init__(
    val: Union[ForwardRef('plotly.Figure'), ForwardRef('matplotlib.artist.Artist')]
)

Initialize a Plotly object.

Args:

  • val: Matplotlib or Plotly figure.

12 - Table

View the source code

class Table

The Table class used to display and analyze tabular data.

Unlike traditional spreadsheets, Tables support numerous types of data: scalar values, strings, numpy arrays, and most subclasses of wandb.data_types.Media. This means you can embed Images, Video, Audio, and other sorts of rich, annotated media directly in Tables, alongside other traditional scalar values.

This class is the primary class used to generate W&B Tables https://docs.wandb.ai/guides/models/tables/.

method Table.__init__

__init__(
    columns=None,
    data=None,
    rows=None,
    dataframe=None,
    dtype=None,
    optional=True,
    allow_mixed_types=False,
    log_mode: Optional[Literal['IMMUTABLE', 'MUTABLE', 'INCREMENTAL']] = 'IMMUTABLE'
)

Initializes a Table object.

The rows is available for legacy reasons and should not be used. The Table class uses data to mimic the Pandas API.

Args:

  • columns: (List[str]) Names of the columns in the table. Defaults to [“Input”, “Output”, “Expected”].
  • data: (List[List[any]]) 2D row-oriented array of values.
  • dataframe: (pandas.DataFrame) DataFrame object used to create the table. When set, data and columns arguments are ignored.
  • rows: (List[List[any]]) 2D row-oriented array of values.
  • optional: (Union[bool,List[bool]]) Determines if None values are allowed. Default to True - If a singular bool value, then the optionality is enforced for all columns specified at construction time - If a list of bool values, then the optionality is applied to each column - should be the same length as columns applies to all columns. A list of bool values applies to each respective column.
  • allow_mixed_types: (bool) Determines if columns are allowed to have mixed types (disables type validation). Defaults to False
  • log_mode: Optional[str] Controls how the Table is logged when mutations occur. Options: - “IMMUTABLE” (default): Table can only be logged once; subsequent logging attempts after the table has been mutated will be no-ops. - “MUTABLE”: Table can be re-logged after mutations, creating a new artifact version each time it’s logged. - “INCREMENTAL”: Table data is logged incrementally, with each log creating a new artifact entry containing the new data since the last log.

method Table.add_column

add_column(name, data, optional=False)

Adds a column of data to the table.

Args:

  • name: (str) - the unique name of the column
  • data: (list | np.array) - a column of homogeneous data
  • optional: (bool) - if null-like values are permitted

method Table.add_computed_columns

add_computed_columns(fn)

Adds one or more computed columns based on existing data.

Args:

  • fn: A function which accepts one or two parameters, ndx (int) and row (dict), which is expected to return a dict representing new columns for that row, keyed by the new column names.
    • ndx is an integer representing the index of the row. Only included if include_ndx is set to True.
    • row is a dictionary keyed by existing columns

method Table.add_data

add_data(*data)

Adds a new row of data to the table.

The maximum amount ofrows in a table is determined by wandb.Table.MAX_ARTIFACT_ROWS.

The length of the data should match the length of the table column.

method Table.add_row

add_row(*row)

Deprecated. Use Table.add_data method instead.

method Table.cast

cast(col_name, dtype, optional=False)

Casts a column to a specific data type.

This can be one of the normal python classes, an internal W&B type, or an example object, like an instance of wandb.Image or wandb.Classes.

Args:

  • col_name (str): The name of the column to cast.
  • dtype (class, wandb.wandb_sdk.interface._dtypes.Type, any): The target dtype.
  • optional (bool): If the column should allow Nones.

method Table.get_column

get_column(name, convert_to=None)

Retrieves a column from the table and optionally converts it to a NumPy object.

Args:

  • name: (str) - the name of the column
  • convert_to: (str, optional) - “numpy”: will convert the underlying data to numpy object

method Table.get_dataframe

get_dataframe()

Returns a pandas.DataFrame of the table.

method Table.get_index

get_index()

Returns an array of row indexes for use in other tables to create links.

13 - Video

View the source code

class Video

A class for logging videos to W&B.

method Video.__init__

__init__(
    data_or_path: Union[str, pathlib.Path, ForwardRef('np.ndarray'), ForwardRef('TextIO'), ForwardRef('BytesIO')],
    caption: Optional[str] = None,
    fps: Optional[int] = None,
    format: Optional[Literal['gif', 'mp4', 'webm', 'ogg']] = None
)

Initialize a W&B Video object.

Args:

  • data_or_path: Video can be initialized with a path to a file or an io object. Video can be initialized with a numpy tensor. The numpy tensor must be either 4 dimensional or 5 dimensional. The dimensions should be (number of frames, channel, height, width) or (batch, number of frames, channel, height, width) The format parameter must be specified with the format argument when initializing with a numpy array or io object.
  • caption: Caption associated with the video for display.
  • fps: The frame rate to use when encoding raw video frames. Default value is 4. This parameter has no effect when data_or_path is a string, or bytes.
  • format: Format of video, necessary if initializing with a numpy array or io object. This parameter will be used to determine the format to use when encoding the video data. Accepted values are “gif”, “mp4”, “webm”, or “ogg”. If no value is provided, the default format will be “gif”.

Examples: Log a numpy array as a video

import numpy as np
import wandb

with wandb.init() as run:
    # axes are (number of frames, channel, height, width)
    frames = np.random.randint(
         low=0, high=256, size=(10, 3, 100, 100), dtype=np.uint8
    )
    run.log({"video": wandb.Video(frames, format="mp4", fps=4)})

14 - WBTraceTree

View the source code

Media object for trace tree data.

Args
root_span (Span): The root span of the trace tree. model_dict (dict, optional): A dictionary containing the model dump. NOTE: model_dict is a completely-user-defined dict. The UI will render a JSON viewer for this dict, giving special treatment to dictionaries with a _kind key. This is because model vendors have such different serialization formats that we need to be flexible here.