An autoencoder is a type of artificial neural network used to learn efficient codings of unlabeled data (unsupervised learning). The network’s aim is to learn a representation (encoding) for a set of data, typically for dimensionality reduction, by training the network to ignore signal “noise”.
Key Features
- Unsupervised Learning: Autoencoders do not require labeled data; they are trained by matching the network’s output to its input.
- Information Bottleneck: The architecture forces the input data through a restricted latent space (bottleneck) which forces the network to capture only the most important features of the input data.
- Reconstruction Loss: It utilizes a loss function to minimize the reconstruction error (the difference between the original input and the reconstructed output).
- Encoder-Decoder Structure:
- Encoder: Maps the input data to a latent-space representation.
- Decoder: Reconstructs the input data from the latent-space representation.
Architectures
- Vanilla Autoencoder: Simple, symmetric feedforward network.
- Denoising Autoencoder (DAE): Learns to reconstruct clean data from partially corrupted inputs, making it more robust to noise.
- Sparse Autoencoder (SAE): Imposes a sparsity constraint on the hidden layers, forcing only a fraction of neurons to activate.
- Variational Autoencoder (VAE): A generative model that constrains the latent space to follow a known probability distribution (like a Gaussian), enabling new data generation.
- Contractive Autoencoder (CAE): Minimizes the sensitivity of the latent space to small perturbations in the input space.
Applications
- Dimensionality Reduction: An alternative to principal component analysis (PCA) for non-linear compression.
- Anomaly Detection: Outlier inputs have high reconstruction error because the network was not trained on them.
- Image Denoising: Removing noise/artifacts from corrupted images.
- Generative Modeling: Using VAEs to generate new samples (e.g., in image generation).