StyleGAN2_activations_and_pca_projection.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "name": "StyleGAN2-activations-and-pca-projection.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "authorship_tag": "ABX9TyN1OjLlIs+JXqq9o2/XjFJh",
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/dvschultz/ml-art-colabs/blob/master/StyleGAN2_activations_and_pca_projection.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "sf8mqELxWhof"
      },
      "source": [
        "## StyleGAN2 Activations and PCA Projection\n",
        "\n",
        "by [duskvirkus](https://github.com/duskvirkus)\n",
        "\n",
        "Ever wondered what's happening in your SG2 model? This notebook generates activations of a StyleGAN2 rosinality model. Optionally can use PCA at the end to see a representation of lower level network layers.\n",
        "\n",
        "Convert pkl model: https://github.com/dvschultz/stylegan2-ada-pytorch/blob/main/SG2_ADA_PT_to_Rosinality.ipynb\n",
        "\n",
        "Thanks to [Derrick Schultz](https://github.com/dvschultz) for the notebook this is based on. Can be found at: [https://github.com/dvschultz/stylegan2-ada-pytorch/blob/eps/Advanced_StyleGAN_Network_bending.ipynb](https://github.com/dvschultz/stylegan2-ada-pytorch/blob/eps/Advanced_StyleGAN_Network_bending.ipynb)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WRxEoqBiWt0y"
      },
      "source": [
        "## Prep"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "bTSFDE-nWJKT"
      },
      "source": [
        "!nvidia-smi -L"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "GDveKZKyWOhu",
        "outputId": "96c47839-9135-46a6-d26f-92d27eb727d3"
      },
      "source": [
        "# Install libraries\n",
        "!git clone -b audio-animate https://github.com/dvschultz/network-bending\n",
        "!pip uninstall torch torchvision -y\n",
        "!pip install torch==1.5.0+cu101 torchvision==0.6.0+cu101 -f https://download.pytorch.org/whl/torch_stable.html\n",
        "!pip install Ninja kmeans-pytorch\n",
        "!apt-get install -y vim make gdb libopencv-dev\n",
        "!wget https://download.pytorch.org/libtorch/cu101/libtorch-shared-with-deps-1.5.0%2Bcu101.zip\n",
        "!unzip /content/libtorch-shared-with-deps-1.5.0+cu101.zip -d /root/\n",
        "%cd network-bending\n",
        "\n",
        "#build custom pytorch transformations\n",
        "!chmod +x /content/network-bending/build_custom_transforms.sh\n",
        "!/content/network-bending/build_custom_transforms.sh /root/libtorch/"
      ],
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Cloning into 'network-bending'...\n",
            "remote: Enumerating objects: 369, done.\u001b[K\n",
            "remote: Counting objects: 100% (37/37), done.\u001b[K\n",
            "remote: Compressing objects: 100% (28/28), done.\u001b[K\n",
            "remote: Total 369 (delta 22), reused 21 (delta 9), pack-reused 332\u001b[K\n",
            "Receiving objects: 100% (369/369), 21.44 MiB | 69.69 MiB/s, done.\n",
            "Resolving deltas: 100% (213/213), done.\n",
            "Found existing installation: torch 1.9.0+cu102\n",
            "Uninstalling torch-1.9.0+cu102:\n",
            "\u001b[31mERROR: Operation cancelled by user\u001b[0m\n",
            "\u001b[33mWARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)\u001b[0m\n",
            "\u001b[33mWARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)\u001b[0m\n",
            "Looking in links: https://download.pytorch.org/whl/torch_stable.html\n",
            "Collecting torch==1.5.0+cu101\n",
            "  Downloading https://download.pytorch.org/whl/cu101/torch-1.5.0%2Bcu101-cp37-cp37m-linux_x86_64.whl (703.8 MB)\n",
            "\u001b[K     |██▉                             | 62.4 MB 1.3 MB/s eta 0:08:17\n",
            "\u001b[31mERROR: Operation cancelled by user\u001b[0m\n",
            "\u001b[?25hTraceback (most recent call last):\n",
            "  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n",
            "  File \"<frozen importlib._bootstrap_external>\", line 728, in exec_module\n",
            "  File \"<frozen importlib._bootstrap>\", line 219, in _call_with_frames_removed\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_internal/exceptions.py\", line 8, in <module>\n",
            "    from pip._vendor.requests.models import Request, Response\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_vendor/requests/__init__.py\", line 123, in <module>\n",
            "    from . import utils\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_vendor/requests/utils.py\", line 27, in <module>\n",
            "    from ._internal_utils import to_native_string\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_vendor/requests/_internal_utils.py\", line 11, in <module>\n",
            "    from .compat import is_py2, builtin_str, str\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_vendor/requests/compat.py\", line 35, in <module>\n",
            "    import json\n",
            "  File \"<frozen importlib._bootstrap>\", line 983, in _find_and_load\n",
            "  File \"<frozen importlib._bootstrap>\", line 963, in _find_and_load_unlocked\n",
            "  File \"<frozen importlib._bootstrap>\", line 906, in _find_spec\n",
            "  File \"<frozen importlib._bootstrap_external>\", line 1280, in find_spec\n",
            "  File \"<frozen importlib._bootstrap_external>\", line 1247, in _get_spec\n",
            "KeyboardInterrupt\n",
            "\n",
            "During handling of the above exception, another exception occurred:\n",
            "\n",
            "Traceback (most recent call last):\n",
            "  File \"/usr/local/bin/pip3\", line 5, in <module>\n",
            "    from pip._internal.cli.main import main\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_internal/cli/main.py\", line 9, in <module>\n",
            "    from pip._internal.cli.autocompletion import autocomplete\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_internal/cli/autocompletion.py\", line 10, in <module>\n",
            "    from pip._internal.cli.main_parser import create_main_parser\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_internal/cli/main_parser.py\", line 8, in <module>\n",
            "    from pip._internal.cli import cmdoptions\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_internal/cli/cmdoptions.py\", line 23, in <module>\n",
            "    from pip._internal.cli.parser import ConfigOptionParser\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_internal/cli/parser.py\", line 12, in <module>\n",
            "    from pip._internal.configuration import Configuration, ConfigurationError\n",
            "  File \"/usr/local/lib/python3.7/dist-packages/pip/_internal/configuration.py\", line 21, in <module>\n",
            "    from pip._internal.exceptions import (\n",
            "  File \"<frozen importlib._bootstrap>\", line 983, in _find_and_load\n",
            "  File \"<frozen importlib._bootstrap>\", line 967, in _find_and_load_unlocked\n",
            "  File \"<frozen importlib._bootstrap>\", line 677, in _load_unlocked\n",
            "KeyboardInterrupt\n",
            "Reading package lists... Done\n",
            "Building dependency tree       \n",
            "Reading state information... Done\n",
            "make is already the newest version (4.1-9.1ubuntu1).\n",
            "make set to manually installed.\n",
            "libopencv-dev is already the newest version (3.2.0+dfsg-4ubuntu0.1).\n",
            "The following additional packages will be installed:\n",
            "  gdbserver libbabeltrace1 libc6-dbg libdw1 libgpm2 vim-common vim-runtime xxd\n",
            "Suggested packages:\n",
            "  gdb-doc gpm ctags vim-doc vim-scripts\n",
            "The following NEW packages will be installed:\n",
            "  gdb gdbserver libbabeltrace1 libc6-dbg libdw1 libgpm2 vim vim-common\n",
            "  vim-runtime xxd\n",
            "0 upgraded, 10 newly installed, 0 to remove and 37 not upgraded.\n",
            "Need to get 15.5 MB of archives.\n",
            "After this operation, 84.1 MB of additional disk space will be used.\n",
            "Get:1 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 xxd amd64 2:8.0.1453-1ubuntu1.4 [49.3 kB]\n",
            "Get:2 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 vim-common all 2:8.0.1453-1ubuntu1.4 [70.4 kB]\n",
            "Get:3 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libdw1 amd64 0.170-0.4ubuntu0.1 [203 kB]\n",
            "Get:4 http://archive.ubuntu.com/ubuntu bionic/main amd64 libbabeltrace1 amd64 1.5.5-1 [154 kB]\n",
            "Get:5 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gdb amd64 8.1.1-0ubuntu1 [2,937 kB]\n",
            "Get:6 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 gdbserver amd64 8.1.1-0ubuntu1 [282 kB]\n",
            "Get:7 http://archive.ubuntu.com/ubuntu bionic/main amd64 libgpm2 amd64 1.20.7-5 [15.1 kB]\n",
            "Get:8 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 vim-runtime all 2:8.0.1453-1ubuntu1.4 [5,435 kB]\n",
            "Get:9 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 vim amd64 2:8.0.1453-1ubuntu1.4 [1,152 kB]\n",
            "Get:10 http://archive.ubuntu.com/ubuntu bionic-updates/main amd64 libc6-dbg amd64 2.27-3ubuntu1.4 [5,163 kB]\n",
            "Fetched 15.5 MB in 1s (11.0 MB/s)\n",
            "Selecting previously unselected package xxd.\n",
            "(Reading database ... 155013 files and directories currently installed.)\n",
            "Preparing to unpack .../0-xxd_2%3a8.0.1453-1ubuntu1.4_amd64.deb ...\n",
            "Unpacking xxd (2:8.0.1453-1ubuntu1.4) ...\n",
            "Selecting previously unselected package vim-common.\n",
            "Preparing to unpack .../1-vim-common_2%3a8.0.1453-1ubuntu1.4_all.deb ...\n",
            "Unpacking vim-common (2:8.0.1453-1ubuntu1.4) ...\n",
            "Selecting previously unselected package libdw1:amd64.\n",
            "Preparing to unpack .../2-libdw1_0.170-0.4ubuntu0.1_amd64.deb ...\n",
            "Unpacking libdw1:amd64 (0.170-0.4ubuntu0.1) ...\n",
            "Selecting previously unselected package libbabeltrace1:amd64.\n",
            "Preparing to unpack .../3-libbabeltrace1_1.5.5-1_amd64.deb ...\n",
            "Unpacking libbabeltrace1:amd64 (1.5.5-1) ...\n",
            "Selecting previously unselected package gdb.\n",
            "Preparing to unpack .../4-gdb_8.1.1-0ubuntu1_amd64.deb ...\n",
            "Unpacking gdb (8.1.1-0ubuntu1) ...\n",
            "Selecting previously unselected package gdbserver.\n",
            "Preparing to unpack .../5-gdbserver_8.1.1-0ubuntu1_amd64.deb ...\n",
            "Unpacking gdbserver (8.1.1-0ubuntu1) ...\n",
            "Selecting previously unselected package libgpm2:amd64.\n",
            "Preparing to unpack .../6-libgpm2_1.20.7-5_amd64.deb ...\n",
            "Unpacking libgpm2:amd64 (1.20.7-5) ...\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "z37P_GjrZwFr",
        "outputId": "81e993f5-4275-431a-f47a-f23b721d00ac"
      },
      "source": [
        "!gdown --id 1rL-J63eFfn80IYU2GfVY977GI2qOG6dw -O /content/ladiesblack.pt # Model Credit: Derrick Schultz"
      ],
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Downloading...\n",
            "From: https://drive.google.com/uc?id=1rL-J63eFfn80IYU2GfVY977GI2qOG6dw\n",
            "To: /content/ladiesblack.pt\n",
            "133MB [00:01, 124MB/s]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "qSGDBp0UWv9I"
      },
      "source": [
        "## Fix existing script"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IuIqWxbWV2ub",
        "outputId": "8772f02a-ee61-48ce-d619-8a1be5a496cf"
      },
      "source": [
        "%%writefile generate_activations.py\n",
        "\n",
        "import argparse\n",
        "import torch\n",
        "import yaml\n",
        "import os\n",
        "import copy\n",
        "\n",
        "from torchvision import utils\n",
        "from model import Generator\n",
        "from tqdm import tqdm\n",
        "from util import *\n",
        "\n",
        "def generate(args, g_ema, device, mean_latent, t_dict_list):\n",
        "    with torch.no_grad():\n",
        "        g_ema.eval()\n",
        "        for i in tqdm(range(args.pics)):\n",
        "            extra_t_dict_list =  copy.deepcopy(t_dict_list)\n",
        "            extra_t_dict_list.append({'layerID': -1, 'index': i})\n",
        "            sample_z = torch.randn(args.sample, args.latent, device=device)\n",
        "            sample, _ = g_ema([sample_z], \n",
        "                                truncation=args.truncation, \n",
        "                                truncation_latent=mean_latent, \n",
        "                                transform_dict_list=extra_t_dict_list)\n",
        "            if not os.path.exists('sample'):\n",
        "                    os.makedirs('sample')\n",
        "            utils.save_image(\n",
        "                sample,\n",
        "                f'sample/{str(i).zfill(6)}.png',\n",
        "                nrow=1,\n",
        "                normalize=True,\n",
        "                range=(-1, 1))\n",
        "\n",
        "\n",
        "if __name__ == '__main__':\n",
        "    device = 'cuda'\n",
        "\n",
        "    parser = argparse.ArgumentParser()\n",
        "\n",
        "    parser.add_argument('--size', type=int, default=1024)\n",
        "    parser.add_argument('--sample', type=int, default=1)\n",
        "    parser.add_argument('--pics', type=int, default=20)\n",
        "    parser.add_argument('--truncation', type=float, default=0.5)\n",
        "    parser.add_argument('--truncation_mean', type=int, default=4096)\n",
        "    parser.add_argument('--ckpt', type=str, default=\"stylegan2-ffhq-config-f.pt\")\n",
        "    parser.add_argument('--channel_multiplier', type=int, default=2)\n",
        "    parser.add_argument('--config', type=str, default=\"configs/empty_transform_config.yaml\")\n",
        "\n",
        "    args = parser.parse_args()\n",
        "\n",
        "    args.latent = 512\n",
        "    args.n_mlp = 8\n",
        "\n",
        "    yaml_config = {}\n",
        "    with open(args.config, 'r') as stream:\n",
        "        try:\n",
        "            yaml_config = yaml.load(stream)\n",
        "        except yaml.YAMLError as exc:\n",
        "            print(exc)\n",
        "    \n",
        "    g_ema = Generator(\n",
        "        args.size, args.latent, args.n_mlp, channel_multiplier=args.channel_multiplier\n",
        "    ).to(device)\n",
        "    new_state_dict = g_ema.state_dict()\n",
        "    checkpoint = torch.load(args.ckpt)\n",
        "    \n",
        "    ext_state_dict  = torch.load(args.ckpt)['g_ema']\n",
        "    g_ema.load_state_dict(checkpoint['g_ema'])\n",
        "    new_state_dict.update(ext_state_dict)\n",
        "    g_ema.load_state_dict(new_state_dict)\n",
        "    g_ema.eval()\n",
        "    g_ema.to(device)\n",
        "\n",
        "    if args.truncation < 1:\n",
        "        with torch.no_grad():\n",
        "            mean_latent = g_ema.mean_latent(args.truncation_mean)\n",
        "    else:\n",
        "        mean_latent = None\n",
        "    \n",
        "    layer_channel_dims = create_layer_channel_dim_dict(args.channel_multiplier, 16)\n",
        "    transform_dict_list = create_transforms_dict_list(yaml_config, None, layer_channel_dims)\n",
        "    generate(args, g_ema, device, mean_latent, transform_dict_list)\n",
        "    \n",
        "    config_out = {}\n",
        "    config_out['transforms'] = yaml_config['transforms']\n",
        "    with open(r'sample/config.yaml', 'w') as file:\n",
        "        documents = yaml.dump(config_out, file)"
      ],
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Writing generate_activations.py\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "hpnyRkRQCSMl"
      },
      "source": [
        "%%writefile configs/nothing.yaml\n",
        "\n"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "BV_Bp2TCCJGk"
      },
      "source": [
        "%%writefile util.py\n",
        "\n",
        "import random\n",
        "\n",
        "def create_layer_channel_dim_dict(channel_multiplier, n_layers=16):\n",
        "    layer_channel_dict = {\n",
        "        0: 512,\n",
        "        1: 512,\n",
        "        2: 512,\n",
        "        3: 512,\n",
        "        4: 512,\n",
        "        5: 512,\n",
        "        6: 512,\n",
        "        7: 256*channel_multiplier,\n",
        "        8: 256*channel_multiplier,\n",
        "        9: 128*channel_multiplier,\n",
        "        10: 128*channel_multiplier,\n",
        "        11: 64*channel_multiplier,\n",
        "        12: 64*channel_multiplier,\n",
        "        13: 32*channel_multiplier,\n",
        "        14: 32*channel_multiplier,\n",
        "        15: 16*channel_multiplier,\n",
        "        16: 16*channel_multiplier\n",
        "    }\n",
        "    return  {k: v for k, v in layer_channel_dict.items() if int(k) <= n_layers}\n",
        "\n",
        "def create_random_transform_dict(layer, layer_channel_dict, transform, params, percentage):\n",
        "    layer_dim = layer_channel_dict[layer]\n",
        "    num_samples = int( layer_dim * percentage )\n",
        "    rand_indicies = random.sample(range(0, layer_dim), num_samples)\n",
        "    transform_dict ={\n",
        "        \"layerID\": layer,\n",
        "        \"transformID\": transform,\n",
        "        \"indicies\": rand_indicies,\n",
        "        \"params\": params\n",
        "    }\n",
        "    return transform_dict\n",
        "\n",
        "def create_layer_wide_transform_dict(layer, layer_channel_dict, transform, params):\n",
        "    layer_dim = layer_channel_dict[layer]\n",
        "    transform_dict ={\n",
        "        \"layerID\": layer,\n",
        "        \"transformID\": transform,\n",
        "        \"indicies\": range(0, layer_dim),\n",
        "        \"params\": params\n",
        "    }\n",
        "    return transform_dict\n",
        "\n",
        "def create_multiple_transforms_dict(layer, layer_channel_dict, transform, params):\n",
        "    \n",
        "    transform_dict_list = []\n",
        "    for t in range(len(transform)):\n",
        "        layer_dim = layer_channel_dict[layer[t]]\n",
        "\n",
        "        transform_dict_list.append({\n",
        "            \"layerID\": layer[t],\n",
        "            \"transformID\": transform[t],\n",
        "            \"indicies\": range(0, layer_dim),\n",
        "            \"params\": params[t]\n",
        "        })\n",
        "    return transform_dict_list\n",
        "\n",
        "def create_cluster_transform_dict(layer, layer_channel_dict, cluster_config, transform, params, cluster_ID):\n",
        "    layer_dim = layer_channel_dict[layer]\n",
        "    indicies = []\n",
        "    for i, c_dict in enumerate(cluster_config[layer]):\n",
        "        if c_dict['cluster_index'] == int(cluster_ID):\n",
        "            indicies.append(c_dict['feature_index'])\n",
        "    print(indicies)\n",
        "    if len(indicies) == 0:\n",
        "        print(\"No indicies found for clusterID: \" +str(cluster_ID) + \" on layer: \" +str(layer))\n",
        "    transform_dict ={\n",
        "        \"layerID\": layer,\n",
        "        \"transformID\": transform,\n",
        "        \"indicies\": indicies,\n",
        "        \"params\": params\n",
        "    }\n",
        "    return transform_dict\n",
        "\n",
        "def create_transforms_dict_list(yaml_config, cluster_config, layer_channel_dict):\n",
        "    transform_dict_list = []\n",
        "    \n",
        "    if yaml_config and 'transforms' in yaml_config:\n",
        "      for transform in yaml_config['transforms']:\n",
        "          if transform['features'] == 'all':\n",
        "              transform_dict_list.append(\n",
        "                  create_layer_wide_transform_dict(transform['layer'],\n",
        "                      layer_channel_dict, \n",
        "                      transform['transform'], \n",
        "                      transform['params']))\n",
        "          elif transform['features'] == 'random':\n",
        "              transform_dict_list.append(\n",
        "                  create_random_transform_dict(transform['layer'],\n",
        "                      layer_channel_dict, \n",
        "                      transform['transform'], \n",
        "                      transform['params'],\n",
        "                      transform['feature-param']))\n",
        "          elif transform['features'] == 'cluster' and cluster_config != {}:\n",
        "              transform_dict_list.append(\n",
        "                  create_cluster_transform_dict(transform['layer'],\n",
        "                      layer_channel_dict, \n",
        "                      cluster_config,\n",
        "                      transform['transform'], \n",
        "                      transform['params'],\n",
        "                      transform['feature-param']))\n",
        "          else:\n",
        "              print('transform type: ' + str(transform) + ' not recognised')\n",
        "      \n",
        "    return transform_dict_list\n",
        "        "
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "-khDKqtAXDht"
      },
      "source": [
        "## Run the script\n",
        "\n",
        "Activations can be found `/content/network-bending/activations` or download the activations.zip made by the last cell."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "P4KMW9h1-fte",
        "outputId": "9fba4b86-7f8e-444b-d839-918e641695ff"
      },
      "source": [
        "%cd network-bending"
      ],
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "/content/network-bending\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IoeLwFURWqHM",
        "outputId": "2fe3b718-3db6-4540-8d90-51b4746d673f"
      },
      "source": [
        "!python generate_activations.py --size 1024 --ckpt /content/tree-flowers.pt --pics 5 --config /content/network-bending/configs/nothing.yaml --truncation 0.8 --channel_multiplier 2"
      ],
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "100% 5/5 [02:59<00:00, 35.81s/it]\n",
            "Traceback (most recent call last):\n",
            "  File \"generate_activations.py\", line 84, in <module>\n",
            "    config_out['transforms'] = yaml_config['transforms']\n",
            "TypeError: 'NoneType' object is not subscriptable\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "42ZJnKpO5Rjt"
      },
      "source": [
        "## Principal Component Analysis (PCA) Script\n",
        "\n",
        "Output colors from PCA have no meaning just there so it can be 3D data instead of 1D."
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "p_BFWvvx5RG6"
      },
      "source": [
        "import cv2\n",
        "import numpy as np\n",
        "import os\n",
        "\n",
        "layer_paths = [\n",
        "  '/content/network-bending/activations/1/0',\n",
        "  '/content/network-bending/activations/2/0',\n",
        "  '/content/network-bending/activations/3/0',\n",
        "  '/content/network-bending/activations/4/0',\n",
        "  '/content/network-bending/activations/5/0',\n",
        "  '/content/network-bending/activations/6/0',\n",
        "  '/content/network-bending/activations/7/0',\n",
        "  '/content/network-bending/activations/8/0',\n",
        "  '/content/network-bending/activations/9/0',\n",
        "  '/content/network-bending/activations/10/0',\n",
        "  '/content/network-bending/activations/11/0',\n",
        "  '/content/network-bending/activations/12/0',\n",
        "  '/content/network-bending/activations/13/0',\n",
        "  '/content/network-bending/activations/14/0',\n",
        "  '/content/network-bending/activations/15/0',\n",
        "  '/content/network-bending/activations/16/0',\n",
        "]\n",
        "\n",
        "layer_imgs = []\n",
        "\n",
        "os.makedirs('/content/out-test-2', exist_ok=True)\n",
        "\n",
        "for j in range(len(layer_paths)):\n",
        "  layer_path = layer_paths[j]\n",
        "  activations = []\n",
        "  for root, subdirs, files in os.walk(layer_path):\n",
        "\n",
        "      for filename in files:\n",
        "\n",
        "        activations.append(cv2.imread(os.path.join(root, filename)))\n",
        "\n",
        "  layer_imgs.append(activations)\n",
        "\n",
        "  saved_shape = layer_imgs[j][0].shape\n",
        "  for i in range(len(layer_imgs[j])):\n",
        "    layer_imgs[j][i] = cv2.cvtColor(layer_imgs[j][i], cv2.COLOR_BGR2GRAY)\n",
        "    layer_imgs[j][i] = layer_imgs[j][i].flatten()\n",
        "    layer_imgs[j][i] = np.expand_dims(layer_imgs[j][i], axis=0)\n",
        "\n",
        "  all = np.concatenate(layer_imgs[j])\n",
        "\n",
        "  unprojected = all.T\n",
        "\n",
        "  pca = decomposition.PCA(n_components=3)\n",
        "  pca.fit(unprojected)\n",
        "  projected = pca.transform(unprojected)\n",
        "\n",
        "\n",
        "  output = np.reshape(projected, saved_shape)\n",
        "\n",
        "  img_float32 = np.float32(output)\n",
        "  final = cv2.cvtColor(img_float32, cv2.COLOR_BGR2RGB)\n",
        "\n",
        "  cv2.imwrite('/content/out-test-2/' + str(j).zfill(2) + '.png', final)"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "2_hIrwHM8Azx",
        "outputId": "c1a0314c-095b-4c7d-9422-0d77bba1a8d9"
      },
      "source": [
        "%cd /content/\n",
        "!zip -r out-test-2.zip out-test-2"
      ],
      "execution_count": null,
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "/content\n",
            "  adding: out-test-2/ (stored 0%)\n",
            "  adding: out-test-2/12.png (deflated 0%)\n",
            "  adding: out-test-2/03.png (stored 0%)\n",
            "  adding: out-test-2/10.png (deflated 0%)\n",
            "  adding: out-test-2/13.png (deflated 0%)\n",
            "  adding: out-test-2/06.png (stored 0%)\n",
            "  adding: out-test-2/14.png (deflated 0%)\n",
            "  adding: out-test-2/01.png (stored 0%)\n",
            "  adding: out-test-2/11.png (deflated 0%)\n",
            "  adding: out-test-2/04.png (stored 0%)\n",
            "  adding: out-test-2/05.png (stored 0%)\n",
            "  adding: out-test-2/09.png (stored 0%)\n",
            "  adding: out-test-2/07.png (stored 0%)\n",
            "  adding: out-test-2/00.png (stored 0%)\n",
            "  adding: out-test-2/02.png (stored 0%)\n",
            "  adding: out-test-2/08.png (stored 0%)\n",
            "  adding: out-test-2/15.png (deflated 2%)\n"
          ]
        }
      ]
    }
  ]
}