Figure 3: A KNIME table containing two image columns and a labeling column. The second image column (Image Metadata) is rendered with a different renderer (can be selected by right-clicking on the according table header).
Figure 4: The column selection tab available in many dialogs of the image processing nodes, mostly nodes that apply a certain algorithm on an one image or labeling.
2.2.2 Node Views
Some nodes provide a so called node view, a separate window visualizing the information at the respective output port in certain ways. It can be opened via the context menu of each node, if available.
Most of the nodes of the Image Processing Extension provide a specific view, called the Table Cell View. In contrast to the usual table view, that each node provides, the Table Cell View allows a more detailed look into objects of a table cell. This is especially useful if the according object can’t be entirely visualized in a simple 2D representation, as it applies to multidimensional images or labelings (e.g. 3D stacks, videos, etc.). After the Table Cell View has been opened, just click on the desired table cell (on the left) in order to open the associated table cell views. In the case of images and labelings, these table cell views allow one to scroll through the available dimensions of an image/labeling, zoom in and out, determining the pixel intensities (or labels in a labeling) at certain pixel positions (by hovering over the image), depict the image’s histogram, etc. The table cell view is exemplary depicted in Figure 5.
2.2.3 Hiliting
The KNIME Hiliting mechanism.
Figure 5: The Table Cell View to inspect data cells in more detail. Provided by most of the image processing nodes.
2.2.4 Loops
E.g. (Parallel) Chunk Loops
3 Important Nodes and Basic Workflows for Image Processing
In this section we introduce a selection of nodes (not essentially part of the Image Processing Extension) that we consider as important for many image analysis tasks. To provide a compact overview we only explain what the nodes generally intended to do and refer to the node description of the according nodes for more details, like the nodes settings, etc.
3.1 Image I/O Nodes
Usually the first node in an image analysis workflow is the Image Reader. Thanks to the Bioformats library it supports more than a hundred formats. The result is a table containing one column with all images that have been read. The images to be read can either be selected via the node’s configuration dialog or by providing a list of file locations in a table connected to the optional input port (or both). For instance, the List Files- node (a standard KNIME node) is able to create those file lists.
Alternatively, images files can first be just referenced by using the Image File Linker what is much faster and doesn’t require to convert all images first into the KNIME internal representation. The image are then only read on demand, e.g. as soon as they are displayed in a table or view, or if subsequent nodes request them. To explicitly import the referenced images, the Image Importer node can be used. The main use case of the Image File Linker is to save hard disc memory when processing a lot of images in loops (see the KNIME loops in Section 2.2).
To generate random images (e.g. for testing or learning) the Image Generate allows one to generate images of arbitrary dimensions, pixel types etc, either manually specified or randomly generated within given ranges.
3.2 Image Pre-processing Nodes
Important nodes to prepare the image data for further processing are the Image Normalizer, Image Converter, and sometimes the Inverter.
It is often the case that the pixel values of images doesn’t utilize the whole possible range (e.g. the maximum pixel value is 500 of maximal 65535 possible in case of an UnsignedShortType pixel type). The image rendered in the table or the Table Cell View will appear almost black. Only the normalization of the image to spread the pixel values over the whole domain by using the Image Normalizer will make them visible.
Furthermore for many use cases it’s unnecessary to keep the pixel values as ShortTypes and converting them to another type will not harm (that is especially the case when the aim is image segmentation). It reduces the amount of hard disc memory to keep the intermediate results. The Image Converter-node can convert between all available pixels types such that the pixel values are either just copied, scaled, or normalized and scaled. TODO image resizer
3.3 Image Processing Nodes
Image Calculator, Global/Local Thresholder, Filter nodes (Convolver etc.), Morphological Operations, ...
3.4 Image Dimension-Handling Nodes
Important nodes that manipulate the number of image dimensions are, among others, the Projector, Image Cropper, Splitter, Merger, and Dimension Swapper- node.
The Projector-node reduces the number of image dimensions exactly by one. The pixel in the direction of the dimension to be removed can be subsumed in different ways. Available projection methods are taking the average, maximum, minimum, median, or the standard deviation.
Another way to reduce the number of dimensions of an image is to cut a sub-image out of it (e.g. taking only the first channel of an RGB-image), as done by the Image Cropper. But this node does not necessarily reduce the number of dimensions and operations, like taking the first ten time-points out of hundred in a video is possible, too.
The Splitter-node orthogonally splits the images into sub-images that are appended as additional columns. With its help, for instance, a video, rgbimage or 3D image object can be split in to its 2D-planes, e.g. to process them individually. But the node is not limited to split the image objects into its xy-planes and other splitting directions are possible, too.
Apparently the Merger-node offers the opposite operation and puts images of different columns in a row together to create one bigger image objects (possibly, but not necessarily, with more dimensions).
Use the Dimension Swapper to change the order of the dimensions.
The Dimension Cleaner removes needles dimensions that have the size of 1, whereas the Dimension Extender adds another dimension of size 1 with a specified label (the dimension size can than be increased with the help of the Image Resizer).
3.5 Image Metadata-Handling Nodes
Image Properties, Image Features, Labeling Properties, ...
3.6 Segmentation Nodes
Connected Component Analysis, Voronoi Segmentation, Labeling Filter, ...
3.7 Segment and Feature-Calculation Nodes
All subsequently described nodes essentially require a labeling as determined with the nodes introduced in the previous Section 3.6. Based on that they all produce a data table where basically each row corresponds to exactly one segment in the original labelings. The outcome of the nodes differ in terms of the segment-information they contain. The Segment Cropper-node just extracts either the bitmask of the segments, or, if an additional image column is provided, the image patch underneath the according segment. The Segment Feature- and Image Segment Feature-node in turn additionally allow one to calculate certain characteristics (called features, a vector of discriminative numbers) for each segment individually. The Segment Features-nodes uses the labeling information exclusively, the Image Segment Features-node determines the features for a segment using the underlying images (hence, an additional image column is required).
Bitmasks or image patches associated with an segment of a source labeling are again images with certain dimensions (depending on the labeling they where extracted from) and pixel type (a bitmask is of type BitType). Furthermore, especially important for a re-composition of bitmasks or images patches to new labelings or images (see next Section 3.8 for more details), each image has an offset/minimum. It is the position that the upper left corner used to have in the original image. ’Normal’ images, as those read in with the Image Reader-node, usually have an offset of 0 in each dimension.
The segment tables with the bitmasks and features can consequently be used be filtered and re-composed to labeling or images again (see next Section 3.8), or, for instance, to perform data mining (e.g. classification, clustering, etc.).
The Figure 6 exemplary shows a segment table with the segment bitmask, the reference to the source labeling and some features.
3.8 GroupBy
Quite useful, especially when dealing with segmentations (labelings) and segment tables, is the KNIME standard GroupBynode. Among other aggregation methods for numbers ( e.g. mean etc.), string, etc. it also provides a couple of operations to compose images or labeling from a set of images (i.e. a group). As soon as ones adds an image column (e.g. the bitmask column of a segment table, or the image column of the Image Reader) in the Aggregation settings of the GroupBy-node different possibilities are available to combine the images of one group (the groups are defined in the group settings):
• Min/Max/Mean Image The minimum, maximum, or average value of each pixel is taken. Note that the input images of one group must have the same dimensions.
• Compose Image The images are placed within a bigger image at the position determined by its minimum/offset (image metadata). If no
Figure 6: A segment table as produced by the Image Segment Feature- node, with the segment bitmask, the label, the reference to the originating labeling, and some feature values.
offset (i.e. the offset is 0 for all dimension) is set, this operations doesn’t make much sense. To set the offset of an image the Set Image Metadata can be used. If images are overlapping, then the pixels are overwritten
(i.e. the pixel values of the last image a set at the overlapping regions).
• Compose Labeling Similar to the ’Compose Image’ operation where all images (actually bitmasks) of a group are composed to a labeling according to th