aiArt - DreamBooth功能測試(訓練AI畫出特定的角色)
「aiArt」
本次主題：DreamBooth功能測試(訓練AI畫出特定的角色)
目前要訓練AI畫出特定角色的方法有兩種
一種是Textual Inversion，丟入訓練圖後會產生一個幾十KB的pt檔，然後放在embeddings 的資料夾就能使用
這個方法的原理大致是讓AI往隨機的方向畫，然後如果畫出來的結果和訓練圖接近，那麼就會把這個方向評為高分，所以它的缺點就是只能畫出AI原本就能畫出的東西，如果有角色髮形是AI之前沒見過的可能就沒辦法畫得很好，還有在模組A的環境下訓練出來的pt檔就沒辦法拿去給模組B用
我之前在測試的時候也覺得這個方法品質不算是很好，可能還有改善的空間
另外一種訓練AI畫出特定角色的方法就是今天介紹的DreamBooth，先準備一個基礎模組ckpt檔和訓練圖，然後訓練後會產生一個新的模組ckpt檔，換言之就是直接生成一個新的特化模組，出圖品質大都明顯會比Texual Inversion要好，不過檔案大小都是2GB起跳，這是它的不便之處
這次使用的是專門用來訓練DreamBooth的colab版：
看網路上有蠻多人在安裝DreamBooth的擴充外掛後都沒辦法正常運作，目前只有colab版的看起來運作上都沒太大問題
目標要畫的角色是孤獨搖滾的喜多，從動畫截圖了十張下來，因為品質和大小不一，所以再使用SD upscale放大成清晰圖片，並裁切縮小成512X512的圖片，最後按照指示命名成「KitaIkuyo(編號).png」：
使用的基礎模組為Anything-V3.0-pruned.ckpt
設定的參數是learning rate 1.5e-6 ，步數是 3000 steps，因為會有overfitting的問題，所以學習率和步數不是越高越好
DreamBooth的特點就是少少幾張圖也能拿來訓練，之前有看過有人只用一張圖就訓練出一定的成果，訓練步數記得有調高到一萬步以上
這次訓練時間大概是一個小時，如果圖片大小選比較大的話會花更多時間訓練
出來的成果為圖一
因為是十張隨便挑選出的圖拿去訓練，動畫的作畫也不夠精細，所以測試後的出圖品質算是普普通通，不過也會有像圖一這樣還不錯的圖
隨便測試都能有不錯的效果出來，DreamBooth可說是很神的功能，而且不只可以用來訓練畫出特定的角色，也可以訓練畫出特定的畫風，訓練模組對未來的AI繪師來說應該是必備的技能
另外如果對模組的品質不滿意，可以再使用StableDiffusion的另一個功能：模組融合，在Checkpoint Merger這個頁面可以把二個模組ckpt檔融合成新的模組
這裡把之前產生的KitaIkuyo.ckpt和Anything-V3.0-pruned.ckpt進行融合，權重設成KitaIkuyo.ckpt為0.9、Anything-V3.0-pruned.ckpt為0.1，出來的結果為圖二
可以看到整體的作畫品質提高了，雖然失去了動畫作畫的簡略上色風格，但是角色的特徵都有好好保留下來，如果沒什麼時間訓練高品質的模組，使用模組融合來提高品質算是不錯的方法
另外特別要提的一點是Anything-V3.0的模組都需要另外放上Anything-V3.0.vae.pt，這裡訓練和融合出來的新模組也是一樣要放同名的vae.pt檔，否則和之前說的一樣會有色彩品質降低的問題
使用的咒文和參數為：
half portrait of KitaIkuyo school girl in school ,
flat color,light tune,global illumination,
masterpiece,best quality,highres,
********
Negative prompt:
(photo),(photograph),(realistic),
cropped,out of frame,worst quality,low quality,normal quality,jpeg artifacts,lowres,(blurry),(vague),ugly,poor drawing,poor painting,amateur,
dysmelia,polydactyly,polymelia,oligodactyly,amelia,conjoined twins,
long glabella,big eye and small eye,long neck,
poor drawing,poor drawing body,poor drawing face,poor drawing head,poor drawing eye, poor drawing eyebrow,poor drawing chin,(poor drawing hand),poor drawing arm,(poor drawing finger),poor drawing leg, poor drawing posture,poor drawing anatomy,poor drawing picture,poor drawing line,poor drawing clothing,
distorted,distorted body,distorted face,distorted head,distorted eye, distorted eyebrow,distorted chin,(distorted hand),distorted arm,(distorted finger),distorted leg, distorted posture,distorted anatomy,distorted drawing, distorted painting, distorted picture,
malformed,malformed body,malformed face,malformed head,(malformed hand),malformed arm,(malformed finger), malformed leg,malformed posture,malformed anatomy,
broken,broken face,broken head,broken eye,broken hand,broken arm,broken finger,broken leg,
bad,bad body,bad face,bad head,bad eye,bad eyebrow,bad chin,(bad hand),bad arm,(bad finger),bad leg, bad hair,bad clothing,bad sock,bad shoe,bad posture,bad anatomy,bad proportion,bad drawing, bad painting, bad picture,
weird,weird face,weird head,weird eye,weird eyebrow,weird chin,(weird hand),weird arm,(weird finger),weird leg, weird shoe,weird hair,weird clothing,weird sock,weird shoe,weird posture,weird anatomy,weird drawing, weird painting, weird picture,
error,error drawing,error drawing line,error drawing face,error drawing head,error drawing eye,error drawing eyebrow,error drawing hand,error drawing arm,error drawing finger,error drawing leg,error drawing body,error drawing posture,error drawing anatomy,error drawing clothing,
error posture,error anatomy,
wrong,wrong drawing,wrong drawing line,wrong drawing face,wrong drawing head,wrong drawing eye,wrong drawing eyebrow,wrong drawing hand,wrong drawing arm,wrong drawing finger,wrong drawing leg,wrong drawing body,wrong drawing posture,wrong drawing anatomy,wrong drawing clothing,
wrong posture,wrong anatomy,
blurry,blurry face,blurry eye,blurry eyebrow,blurry head,blurry hand,blurry arm,blurry finger,blurry leg,blurry clothing,
vague,vague face,vague eye,vague eyebrow,vague head,vague hand,vague arm,vague finger,vague leg,vague clothing,
liquid face,liquid mouth,
missing,(missing hand),missing arm,(missing finger) ,missing leg,missing mouth,
(invisible hand),invisible arm,(invisible finger),invisible leg,
(unsighted hand),unsighted arm,(unsighted finger),unsighted leg,
mutated,(mutated hand), mutated arm,(mutated finger), mutated leg,
too much,too much eyebrow,(too much hand),too much arm,(too much finger),too much leg,
too long,too long arm,too long leg,too long neck,
too short,too short arm,too short leg,
small face,small head,small eye,small hand, small arm, small finger, small leg,
big face,big head,big eye,big hand, big arm, big finger, big leg,
dirty,dirty face,dirty drawing,dirty painting,dirty drawing line,
unclean,unclean drawing,unclean painting,unclean drawing line,unclean drawing face,unclean drawing head,unclean drawing eye,unclean drawing eyebrow,unclean drawing hand,unclean drawing arm,unclean drawing finger,unclean drawing leg,unclean drawing body,unclean drawing clothing,
unclean face,unclean eye,unclean eyebrow,unclean hand,unclean arm,unclean finger,unclean leg,unclean clothing,
********
Steps: 20, Sampler: DDIM, CFG scale: 7, Seed: 4278710297, Size: 512x512, Model hash: 79c41896
------------------------------------------------
half portrait of KitaIkuyo school girl in school ,style of KitaIkuyo ,red long hair,
flat color,light tune,global illumination,
masterpiece,best quality,highres,
********
Negative prompt:
(photo),(photograph),(realistic),
cropped,out of frame,worst quality,low quality,normal quality,jpeg artifacts,lowres,(blurry),(vague),ugly,poor drawing,poor painting,amateur,
dysmelia,polydactyly,polymelia,oligodactyly,amelia,conjoined twins,
long glabella,big eye and small eye,long neck,
poor drawing,poor drawing body,poor drawing face,poor drawing head,poor drawing eye, poor drawing eyebrow,poor drawing chin,(poor drawing hand),poor drawing arm,(poor drawing finger),poor drawing leg, poor drawing posture,poor drawing anatomy,poor drawing picture,poor drawing line,poor drawing clothing,
distorted,distorted body,distorted face,distorted head,distorted eye, distorted eyebrow,distorted chin,(distorted hand),distorted arm,(distorted finger),distorted leg, distorted posture,distorted anatomy,distorted drawing, distorted painting, distorted picture,
malformed,malformed body,malformed face,malformed head,(malformed hand),malformed arm,(malformed finger), malformed leg,malformed posture,malformed anatomy,
broken,broken face,broken head,broken eye,broken hand,broken arm,broken finger,broken leg,
bad,bad body,bad face,bad head,bad eye,bad eyebrow,bad chin,(bad hand),bad arm,(bad finger),bad leg, bad hair,bad clothing,bad sock,bad shoe,bad posture,bad anatomy,bad proportion,bad drawing, bad painting, bad picture,
weird,weird face,weird head,weird eye,weird eyebrow,weird chin,(weird hand),weird arm,(weird finger),weird leg, weird shoe,weird hair,weird clothing,weird sock,weird shoe,weird posture,weird anatomy,weird drawing, weird painting, weird picture,
error,error drawing,error drawing line,error drawing face,error drawing head,error drawing eye,error drawing eyebrow,error drawing hand,error drawing arm,error drawing finger,error drawing leg,error drawing body,error drawing posture,error drawing anatomy,error drawing clothing,
error posture,error anatomy,
wrong,wrong drawing,wrong drawing line,wrong drawing face,wrong drawing head,wrong drawing eye,wrong drawing eyebrow,wrong drawing hand,wrong drawing arm,wrong drawing finger,wrong drawing leg,wrong drawing body,wrong drawing posture,wrong drawing anatomy,wrong drawing clothing,
wrong posture,wrong anatomy,
blurry,blurry face,blurry eye,blurry eyebrow,blurry head,blurry hand,blurry arm,blurry finger,blurry leg,blurry clothing,
vague,vague face,vague eye,vague eyebrow,vague head,vague hand,vague arm,vague finger,vague leg,vague clothing,
liquid face,liquid mouth,
missing,(missing hand),missing arm,(missing finger) ,missing leg,missing mouth,
(invisible hand),invisible arm,(invisible finger),invisible leg,
(unsighted hand),unsighted arm,(unsighted finger),unsighted leg,
mutated,(mutated hand), mutated arm,(mutated finger), mutated leg,
too much,too much eyebrow,(too much hand),too much arm,(too much finger),too much leg,
too long,too long arm,too long leg,too long neck,
too short,too short arm,too short leg,
small face,small head,small eye,small hand, small arm, small finger, small leg,
big face,big head,big eye,big hand, big arm, big finger, big leg,
dirty,dirty face,dirty drawing,dirty painting,dirty drawing line,
unclean,unclean drawing,unclean painting,unclean drawing line,unclean drawing face,unclean drawing head,unclean drawing eye,unclean drawing eyebrow,unclean drawing hand,unclean drawing arm,unclean drawing finger,unclean drawing leg,unclean drawing body,unclean drawing clothing,
unclean face,unclean eye,unclean eyebrow,unclean hand,unclean arm,unclean finger,unclean leg,unclean clothing,
********
Steps: 20, Sampler: DDIM, CFG scale: 7, Seed: 1138473959, Size: 832x512, Model hash: b97af004