Cycle Generative Adversarial Network (CycleGAN)

CycleGAN is a GAN architecture used for image-to-image translation without requiring paired training data. It uses two generators and two discriminators to transform images between domains and reconstruct the original image using cycle consistency loss.

• Performs image translation without paired images
• Uses two generators and two discriminators
• Learns reversible mappings between image domains
• Uses cycle consistency loss for reconstruction
• Applied in style transfer and image transformation tasks

Architecture of CycleGAN

CycleGAN uses two generators and two discriminators to perform image translation between two domains without paired data.

1. Generators

Create new images in the target style.

CycleGAN has two generators G and F:

• G transforms images from domain X like photos to domain Y like artwork.
• F transforms images from domain Y back to domain X.

The generator mapping functions are as follows:

\begin{array}{l} G : X \rightarrow Y \\ F : Y \rightarrow X \end{array}   

where X is the input image distribution and Y is the desired output distribution such as Van Gogh styles.

2. Discriminators

Decide if images are real (from dataset) or fake (generated). There are two discriminators Dₓ and Dᵧ.

• Dₓ distinguishes between real images from X and generated images from F(y).
• Dᵧ distinguishes between real images from Y and generated images from G(x).

Cycle Consistency Loss

To further regularize the mappings the CycleGAN uses two more loss function in addition to adversarial loss.

1. Forward Cycle Consistency Loss: Ensures that when we apply G and then F to an image we get back the original image

For example: .x --> G(x) -->F(G(x)) \approx x 

2. Backward Cycle Consistency Loss: Ensures that when we apply F and then G to an image we get back the original image.

For example: x \xrightarrow{G} G(x) \xrightarrow{F} F(G(x)) \approx x

Generator Architecture

Each CycleGAN generator consists of an encoder, transformer and decoder for image translation.

1. Encoder: The input image is passed through three convolution layers which extract features and compress the image while increasing the number of channels. For example a 256×256×3 image is reduced to 64×64×256 after this step.
2. Transformer: The encoded image is processed through 6 or 9 residual blocks depending on the input size which helps retain important image details.
3. Decoder: The transformed image is up-sampled using two deconvolution layers and restoring it to its original size.

Generator Structure:

c7s1-64 → d128 → d256 → R256 (×6 or 9) → u128 → u64 → c7s1-3

• c7s1-k: 7×7 convolution layer with k filters.
• dk: 3×3 convolution with stride 2 (down-sampling).
• Rk: Residual block with two 3×3 convolutions.
• uk: Fractional-stride deconvolution (up-sampling).

Discriminator Architecture (PatchGAN)

In CycleGAN the discriminator uses a PatchGAN instead of a regular GAN discriminator.