Example Images

A total of 16,979 labelled images (150px x 150px x 3rgb channels) are available for training.

A set of images was created to visualize the "average" of each of the categories, and is made up of the mean rgb value for each pixel as calculated from all images in the class.

The forest representation is the most distinct.

The mountain image shows a reasonably strong contrast gradient.

There is little to differentiate between the other 4 classes.

Data from each image were flattened into a single row with 67,500 (105 x 150 x 3) columns. The result is a matrix with 16,797 rows and 67,500 columns (n > m).

PCA

Once flattened, each image can be thought of as a point in high dimensional space.

Principal component analysis (https://en.wikipedia.org/wiki/Principal_component_analysis) allows us to map these higher dimensions down onto a lower dimensional space.

In order to ease visualization, only the first three PCAs have been plotted below.

Dimensionality reduction makes the classification problem more computationally tractable.

Logistic regression, random forest and support vector machine (SVM) classifiers were all attempted on the embedded data.

A train/test split to the data was applied, in which a random 20% of the images were held back for testing.

Instead of simply looking at the average testing accuracy of the classifier, the accuracy was calculated separately for each class.

It is important to note that UMAP is generally a non-deterministic technique, so each run of the embedding will produce slightly different results. Hyper-parameters for the technique were tuned in a manual way to produce a sensible plot.

The image dataset was projected using UMAP onto a three dimensional manifold. We can again see relatively good separation between the embeddings for images of forests and mountains, with the addition of glaciers as a more distinct group. Embeddings for images of streets, seas and buildings continue to occur throughout the space and are not obviously linearly separable.

A similar exercise of model training took place on the 10 dimensional UMAP embeddings. Accuracy scores are notably lower in this case.

TSNE

TSNE is a statistical embedding technique that attempts to preserve a notion of local distance similarity. Visualizations through TSNE are generally less affected by outliers.

An excellent tutorial and interactive website on the technique is available here: https://distill.pub/2016/misread-tsne/.

The implementation in the rapids.ai API only allows for reduction to the two dimensional case.

Modelling on the TSNE embedding shows a similar pattern to the PCA and UMAP results.

We can also show the prediction plots for knn (closest 200 neighbours), logistic regression, random forest and support vector machine predictions. Classification for the KNN and support vector machine models are very similar in form as opposed to logistic regression and the random forest outputs.