source: internals/2016/aptoideimagesdetector/trunk/explicit_content_detector/API/Explicit_detector/Illustration2Vector/illustration2vec_master/README.md @ 16448

Last change on this file since 16448 was 16448, checked in by dferreira, 3 years ago

File organization totally changed

  • Property svn:executable set to *
File size: 6.4 KB
Line 
1# Illustration2Vec
2
3``illustration2vec (i2v)`` is a simple library for estimating a set of tags and
4extracting semantic feature vectors from given illustrations.
5For details, please see [our project page](http://illustration2vec.net) or
6[our main paper](http://illustration2vec.net/papers/illustration2vec-main.pdf).
7
8# Demo
9
10* http://demo.illustration2vec.net
11
12# Requirements
13
14* Pre-trained models (``i2v`` uses Convolutional Neural Networks. Please download
15  several pre-trained models from http://illustration2vec.net,
16  or execute ``get_models.sh`` in this repository).
17* ``numpy`` and ``scipy``
18* ``PIL`` (Python Imaging Library) or its alternatives (e.g., ``Pillow``)
19* ``skimage`` (Image processing library for python)
20
21In addition to the above libraries and the pre-trained models, `i2v` requires
22either ``caffe`` or ``chainer`` library. If you are not familiar with deep
23learning libraries, we recommend to use ``chainer`` that can be installed
24via ``pip`` command.
25
26# How to use
27
28In this section, we show two simple examples -- tag prediction and the the
29feature vector extraction -- by using the following illustration [1].
30
31![slide](images/miku.jpg)
32
33[1] Hatsune Miku (初音ミク), © Crypton Future Media, INC.,
34http://piapro.net/en_for_creators.html.
35This image is licensed under the Creative Commons - Attribution-NonCommercial,
363.0 Unported (CC BY-NC).
37
38## Tag prediction
39
40``i2v`` estimates a number of semantic tags from given illustrations
41in the following manner.
42```python
43import i2v
44from PIL import Image
45
46illust2vec = i2v.make_i2v_with_chainer(
47    "illust2vec_tag_ver200.caffemodel", "tag_list.json")
48
49# In the case of caffe, please use i2v.make_i2v_with_caffe instead:
50# illust2vec = i2v.make_i2v_with_caffe(
51#     "illust2vec_tag.prototxt", "illust2vec_tag_ver200.caffemodel",
52#     "tag_list.json")
53
54img = Image.open("images/miku.jpg")
55illust2vec.estimate_plausible_tags([img], threshold=0.5)
56```
57
58``estimate_plausible_tags()`` returns dictionaries that have a pair of
59tag and its confidence.
60```python
61[{'character': [(u'hatsune miku', 0.9999994039535522)],
62  'copyright': [(u'vocaloid', 0.9999998807907104)],
63  'general': [(u'thighhighs', 0.9956372380256653),
64   (u'1girl', 0.9873462319374084),
65   (u'twintails', 0.9812833666801453),
66   (u'solo', 0.9632901549339294),
67   (u'aqua hair', 0.9167950749397278),
68   (u'long hair', 0.8817108273506165),
69   (u'very long hair', 0.8326570987701416),
70   (u'detached sleeves', 0.7448858618736267),
71   (u'skirt', 0.6780789494514465),
72   (u'necktie', 0.5608364939689636),
73   (u'aqua eyes', 0.5527772307395935)],
74  'rating': [(u'safe', 0.9785731434822083),
75   (u'questionable', 0.020535090938210487),
76   (u'explicit', 0.0006299660308286548)]}]
77```
78These tags are classified into the following four categories:
79*general tags* representing general attributes included in an image,
80*copyright tags* representing the specific name of the copyright,
81*character tags* representing the specific name of the characters,
82and *rating tags* representing X ratings.
83
84If you want to focus on several specific tags, use ``estimate_specific_tags()`` instead.
85```python
86illust2vec.estimate_specific_tags([img], ["1girl", "blue eyes", "safe"])
87# -> [{'1girl': 0.9873462319374084, 'blue eyes': 0.01301183458417654, 'safe': 0.9785731434822083}]
88```
89
90## Feature vector extraction
91
92``i2v`` can extract a semantic feature vector from an illustration.
93```python
94import i2v
95from PIL import Image
96
97# In the feature vector extraction, you do not need to specify the tag.
98illust2vec = i2v.make_i2v_with_chainer("illust2vec_ver200.caffemodel")
99
100# illust2vec = i2v.make_i2v_with_caffe(
101#     "illust2vec.prototxt", "illust2vec_ver200.caffemodel")
102
103img = Image.open("images/miku.jpg")
104
105# extract a 4,096-dimensional feature vector
106result_real = illust2vec.extract_feature([img])
107print("shape: {}, dtype: {}".format(result_real.shape, result_real.dtype))
108print(result_real)
109
110# i2v also supports a 4,096-bit binary feature vector
111result_binary = illust2vec.extract_binary_feature([img])
112print("shape: {}, dtype: {}".format(result_binary.shape, result_binary.dtype))
113print(result_binary)
114```
115The output is the following:
116```
117shape: (1, 4096), dtype: float32
118[[ 7.47459459  3.68610668  0.5379501  ..., -0.14564702  2.71820974
119   7.31408596]]
120shape: (1, 512), dtype: uint8
121[[246 215  87 107 249 190 101  32 187  18 124  90  57 233 245 243 245  54
122  229  47 188 147 161 149 149 232  59 217 117 112 243  78  78  39  71  45
123  235  53  49  77  49 211  93 136 235  22 150 195 131 172 141 253 220 104
124  163 220 110  30  59 182 252 253  70 178 148 152 119 239 167 226 202  58
125  179 198  67 117 226  13 204 246 215 163  45 150 158  21 244 214 245 251
126  124 155  86 250 183  96 182  90 199  56  31 111 123 123 190  79 247  99
127   89 233  61 105  58  13 215 159 198  92 121  39 170 223  79 245  83 143
128  175 229 119 127 194 217 207 242  27 251 226  38 204 217 125 175 215 165
129  251 197 234  94 221 188 147 247 143 247 124 230 239  34  47 195  36  39
130  111 244  43 166 118  15  81 177   7  56 132  50 239 134  78 207 232 188
131  194 122 169 215 124 152 187 150  14  45 245  27 198 120 146 108 120 250
132  199 178  22  86 175 102   6 237 111 254 214 107 219  37 102 104 255 226
133  206 172  75 109 239 189 211  48 105  62 199 238 211 254 255 228 178 189
134  116  86 135 224   6 253  98  54 252 168  62  23 163 177 255  58  84 173
135  156  84  95 205 140  33 176 150 210 231 221  32  43 201  73 126   4 127
136  190 123 115 154 223  79 229 123 241 154  94 250   8 236  76 175 253 247
137  240 191 120 174 116 229  37 117 222 214 232 175 255 176 154 207 135 183
138  158 136 189  84 155  20  64  76 201  28 109  79 141 188  21 222  71 197
139  228 155  94  47 137 250  91 195 201 235 249 255 176 245 112 228 207 229
140  111 232 157   6 216 228  55 153 202 249 164  76  65 184 191 188 175  83
141  231 174 158  45 128  61 246 191 210 189 120 110 198 126  98 227  94 127
142  104 214  77 237  91 235 249  11 246 247  30 152  19 118 142 223   9 245
143  196 249 255   0 113   2 115 149 196  59 157 117 252 190 120  93 213  77
144  222 215  43 223 222 106 138 251  68 213 163  57  54 252 177 250 172  27
145   92 115 104 231  54 240 231  74  60 247  23 242 238 176 136 188  23 165
146  118  10 197 183  89 199 220  95 231  61 214  49  19  85  93  41 199  21
147  254  28 205 181 118 153 170 155 187  60  90 148 189 218 187 172  95 182
148  250 255 147 137 157 225 127 127  42  55 191 114  45 238 228 222  53  94
149   42 181  38 254 177 232 150  99]]
150```
151
152# License
153The pre-trained models and the other files we have provided are licensed
154under the MIT License.
Note: See TracBrowser for help on using the repository browser.