Adversarial AI 学习日记

PS C:\Users\AERO> docker attach mynginx^C
PS C:\Users\AERO> docker attach dgl2019/icse2019-artifacts
Error: No such container: dgl2019/icse2019-artifacts 
PS C:\Users\AERO> docker attach docker.io/dgl2019/icse2019-artifacts 
Error: No such container: docker.io/dgl2019/icse2019-artifacts 
PS C:\Users\AERO> docker ps 
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 
PS C:\Users\AERO> docker pull dgl2019/icse2019-artifacts 
Using default tag: latest 
latest: Pulling from dgl2019/icse2019-artifacts 
Digest: sha256:ddf6ceb380481b67485b18728f302958113569ea8571b9fcd78439724eeaaef8 
Status: Image is up to date for dgl2019/icse2019-artifacts:latest docker.io/dgl2019/icse2019-artifacts:latest 
PS C:\Users\AERO> docker ps 
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES 
PS C:\Users\AERO> docker images 
REPOSITORY TAG IMAGE ID CREATED SIZE 
dgl2019/icse2019-artifacts latest a5a18674d9a4 6 months ago 6.43GB 
PS C:\Users\AERO> docker exec -it a5a18674d9a4 /bin/bash 
Error: No such container: a5a18674d9a4 
PS C:\Users\AERO> docker exec -it dgl2019/icse2019-artifacts /bin/bash 
Error: No such container: dgl2019/icse2019-artifacts
PS C:\Users\AERO> docker exec -it dgl2019 /bin/bash 
Error: No such container: dgl2019 
PS C:\Users\AERO> docker run -it dgl2019/icse2019-artifacts /bin/bash 
root@ef79040f7919:/# ls 
bin boot dev etc home lib lib64 media mnt opt proc root run sbin srv sys tmp usr var
root@ef79040f7919:/# cd home/
root@ef79040f7919:/home# ls
icse2019
root@ef79040f7919:/home# cd icse2019/
root@ef79040f7919:/home/icse2019# ls
source
root@ef79040f7919:/home/icse2019# cd source/ root@ef79040f7919:/home/icse2019/source# ls
__init__.py attacks build-in-resource config detect lcr_auc model_mutation models scripts utils
root@ef79040f7919:/home/icse2019/source# cd config/
root@ef79040f7919:/home/icse2019/source/config# ls
logging.yaml
root@ef79040f7919:/home/icse2019/source/config# cd ..
root@ef79040f7919:/home/icse2019/source# cd model
bash: cd: model: No such file or directory
root@ef79040f7919:/home/icse2019/source# ls
__init__.py attacks build-in-resource config detect lcr_auc model_mutation models scripts utils
root@ef79040f7919:/home/icse2019/source# cd models
root@ef79040f7919:/home/icse2019/source/models# ls
__init__.py __init__.pyc ensemble_model.py ensemble_model.pyc googlenet.py googlenet.pyc lenet.py lenet.pyc
root@ef79040f7919:/home/icse2019/source/models# python lenet.py root@ef79040f7919:/home/icse2019/source/models# uname -r
4.9.184-linuxkit
root@ef79040f7919:/home/icse2019/source/models# uname -a Linux ef79040f7919 4.9.184-linuxkit #1 SMP Tue Jul 2 22:58:16 UTC 2019 x86_64 GNU/Linux
root@ef79040f7919:/home/icse2019/source/models# ls __init__.py __init__.pyc ensemble_model.py ensemble_model.pyc googlenet.py googlenet.pyc lenet.py lenet.pyc root@ef79040f7919:/home/icse2019/source/models# cd .. root@ef79040f7919:/home/icse2019/source# ls
__init__.py attacks build-in-resource config detect lcr_auc model_mutation models scripts utils
root@ef79040f7919:/home/icse2019/source# cd utils/ root@ef79040f7919:/home/icse2019/source/utils# ls
__init__.py data_manger.pyc logging_util.pyc model_trainer.py pytorch_extend.pyc
__init__.pyc imgnet12-valprep.sh model_manager.py model_trainer.pyc time_util.py
data_manger.py logging_util.py model_manager.pyc pytorch_extend.py time_util.pyc
root@ef79040f7919:/home/icse2019/source/utils# ls
__init__.py data_manger.pyc logging_util.pyc model_trainer.py pytorch_extend.pyc
__init__.pyc imgnet12-valprep.sh model_manager.py model_trainer.pyc time_util.py
data_manger.py logging_util.py model_manager.pyc pytorch_extend.py time_util.pyc
root@ef79040f7919:/home/icse2019/source/utils# cd .. root@ef79040f7919:/home/icse2019/source# cd scripts/ root@ef79040f7919:/home/icse2019/source/scripts# ./craftAdvSamples.sh
NOTE: Our experiments are only based on two datasets: mnist and cifar10,
but it is a piece of cake to extend to other datasets only providing a proper pytorch-style data loader tailored to himself datasets. Each attack manner has different parameters. All the parameters are organized in a list.The order of the parameters can be found in the REDME in this folder. To quickly yield adversarial samples, we provide a default setting for each attack manner.Do you want to perform an attack with the default settings?y/n y dataType ( [0] mnist; [1] cifar10):1 attackType:fgsm =======>Please Check Parameters<======= modelName: googlenet modelPath: ../build-in-resource/pretrained-model/googlenet.pkl dataType: 1 sourceDataPath: ../build-in-resource/dataset/cifar10/raw attackType: fgsm attackParameters: 0.03,true savePath: ../artifacts_eval/adv_samples/cifar10/fgsm device: -1 <======>Parameters=======> Press any key to start attack process CTRL+C break command bash... Crafting Adversarial Samples.... targeted model: Average loss: -11.4510, Accuracy: 9049/10000 (90.49%) ./craftAdvSamples.sh: line 129: 34 Killed python -u $exe_file --modelName ${modelName} --modelPath ${modelPath} --dataType ${dataType} --sourceDataPath ${sourceDataPath} --attackType ${attackType} --attackParameters ${attackParameters} --savePath ${savePath} --device ${device} DONE! root@ef79040f7919:/home/icse2019/source/scripts# nvidia-smi bash: nvidia-smi: command not found root@ef79040f7919:/home/icse2019/source/scripts# ks bash: ks: command not found root@ef79040f7919:/home/icse2019/source/scripts# lscpu Architecture: x86_64 CPU op-mode(s): 32-bit, 64-bit Byte Order: Little Endian CPU(s): 2 On-line CPU(s) list: 0,1 Thread(s) per core: 2 Core(s) per socket: 1 Socket(s): 1 Vendor ID: GenuineIntel CPU family: 6 Model: 158 Model name: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz Stepping: 10 CPU MHz: 2362.464 BogoMIPS: 4724.92 Hypervisor vendor: Microsoft Virtualization type: full L1d cache: 32K L1i cache: 32K L2 cache: 256K L3 cache: 12288K root@ef79040f7919:/home/icse2019/source/scripts# lspci bash: lspci: command not found root@ef79040f7919:/home/icse2019/source/scripts# dmseg bash: dmseg: command not found root@ef79040f7919:/home/icse2019/source/scripts# dmesg dmesg: read kernel buffer failed: Operation not permitted root@ef79040f7919:/home/icse2019/source/scripts# sudo dmesg bash: sudo: command not found root@ef79040f7919:/home/icse2019/source/scripts# ^Cdo dmesg root@ef79040f7919:/home/icse2019/source/scripts# ./advSampelsVerify.sh 0 ../artifacts_eval/adv_samples/mnist/fgsm/2019-01-13_03:48:45 -1 1 Traceback (most recent call last): File "../attacks/attack_util.py", line 369, in test_adv_samples() File "../attacks/attack_util.py", line 318, in test_adv_samples ]), show_file_name=True, img_mode=img_mode, max_size=10000) File "../utils/data_manger.py", line 240, in __init__ all_files = np.array([img_file for img_file in os.listdir(root)]) OSError: [Errno 2] No such file or directory: '../artifacts_eval/adv_samples/mnist/fgsm/2019-01-13_03:48:45' root@ef79040f7919:/home/icse2019/source/scripts# ls advSampelsVerify.sh craftAdvSamples.sh default_cifar10_auc_analysis.sh default_mnist_auc_analysis.sh detect.sh lcr_acu_analysis.sh modelMuated.sh root@ef79040f7919:/home/icse2019/source/scripts# ./advSampelsVerify.sh 0 ../a artifacts_eval/ attacks/ root@ef79040f7919:/home/icse2019/source/scripts# ./advSampelsVerify.sh 0 ../artifacts_eval/adv_samples/cifar10/fgsm/2019-08-07_00\:04\:11/ -1 1 Total:0,Success:0 root@ef79040f7919:/home/icse2019/source/scripts# ./craftAdvSamples.sh NOTE: Our experiments are only based on two datasets: mnist and cifar10, but it is a piece of cake to extend to other datasets only providing a proper pytorch-style data loader tailored to himself datasets. Each attack manner has different parameters. All the parameters are organized in a list.The order of the parameters can be found in the REDME in this folder. To quickly yield adversarial samples, we provide a default setting for each attack manner.Do you want to perform
an attack with the default settings?y/n
y
dataType ( [0] mnist; [1] cifar10):0
attackType:fgsm
=======>Please Check Parameters<======= modelName: lenet modelPath: ../build-in-resource/pretrained-model/lenet.pkl dataType: 0 sourceDataPath: ../build-in-resource/dataset/mnist/raw attackType: fgsm attackParameters: 0.35,true savePath: ../artifacts_eval/adv_samples/mnist/fgsm device: -1 <======>Parameters=======>
Press any key to start attack process
CTRL+C break command bash...
Crafting Adversarial Samples....

targeted model: Average loss: -12.9422, Accuracy: 9829/10000 (98.29%)
y
Eps=0.35: Average loss: -7.5184, Accuracy: 7775/9829 (79.10%)
successful samples 2054
Done!
icse19-eval-attack-fgsm: rename 125, remove 40,success 1889
Adversarial samples are saved in ../artifacts_eval/adv_samples/mnist/fgsm/2019-08-07_00:58:21
DONE!
root@ef79040f7919:/home/icse2019/source/scripts# /modelMuated.sh
bash: /modelMuated.sh: No such file or directory
root@ef79040f7919:/home/icse2019/source/scripts# 。/modelMuated.sh bash: 。/modelMuated.sh: No such file or directory
root@ef79040f7919:/home/icse2019/source/scripts# ./modelMuated.sh NOTE: Our experiments are only based on two datasets: mnist and cifar10,

but it is a piece of cake to extend to other datasets only providing a

proper pytorch-style data loader tailored to himself datasets.
To quickly verify the mutation process, we provide a group of default parameters，do you want to quickly start the
program?y/n
y
=======>Parameters<======= modelName: lenet modelPath: ../build-in-resource/pretrained-model/lenet.pkl accRation: 0.9 dataType: 0 numMModels: 10 mutatedRation: 0.001 opType: GF savePath: ../artifacts_eval/modelMuation/ device: -1 <======>Parameters=======>
Press any key to start mutation process
CTRL+C break command bash...
2019-08-07 00:59:08,632 - INFO - data type:mnist
2019-08-07 00:59:08,637 - INFO - >>>>>>>>>>>>Start-new-experiment>>>>>>>>>>>>>>>>
2019-08-07 00:59:10,078 - INFO - orginal model acc=0.9829
2019-08-07 00:59:10,079 - INFO - acc_threshold:88.0%
2019-08-07 00:59:10,079 - INFO - seed_md_name:lenet,op_type:GF,ration:0.001,acc_tolerant:0.9,num_mutated:10
2019-08-07 00:59:10,091 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:12,375 - INFO - Mutated model: accurate 0.9818
2019-08-07 00:59:12,379 - INFO - Progress:1/10
2019-08-07 00:59:12,388 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:13,678 - INFO - Mutated model: accurate 0.9832
2019-08-07 00:59:13,682 - INFO - Progress:2/10
2019-08-07 00:59:13,691 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:14,958 - INFO - Mutated model: accurate 0.9823
2019-08-07 00:59:14,960 - INFO - Progress:3/10
2019-08-07 00:59:14,971 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:16,271 - INFO - Mutated model: accurate 0.9827
2019-08-07 00:59:16,274 - INFO - Progress:4/10
2019-08-07 00:59:16,283 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:17,582 - INFO - Mutated model: accurate 0.9829
2019-08-07 00:59:17,586 - INFO - Progress:5/10
2019-08-07 00:59:17,595 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:18,764 - INFO - Mutated model: accurate 0.9829
2019-08-07 00:59:18,767 - INFO - Progress:6/10
2019-08-07 00:59:18,777 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:19,921 - INFO - Mutated model: accurate 0.982
2019-08-07 00:59:19,923 - INFO - Progress:7/10
2019-08-07 00:59:19,932 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:21,072 - INFO - Mutated model: accurate 0.9823
2019-08-07 00:59:21,075 - INFO - Progress:8/10
2019-08-07 00:59:21,086 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:22,262 - INFO - Mutated model: accurate 0.983
2019-08-07 00:59:22,264 - INFO - Progress:9/10
2019-08-07 00:59:22,274 - INFO - 61/61706 weights to be fuzzed
2019-08-07 00:59:23,442 - INFO - Mutated model: accurate 0.9824
2019-08-07 00:59:23,444 - INFO - Progress:10/10
The mutated models are stored in ../artifacts_eval/modelMuation/2019-08-07_00:59:08/gf0.001/lenet
root@ef79040f7919:/home/icse2019/source/scripts# ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy^C
root@ef79040f7919:/home/icse2019/source/scripts# ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy
bash: ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy: Permission denied
root@ef79040f7919:/home/icse2019/source/scripts# sudo ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy
bash: sudo: command not found
root@ef79040f7919:/home/icse2019/source/scripts# chmod 777 ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy
root@ef79040f7919:/home/icse2019/source/scripts# ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy bash: ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy: cannot execute binary file: Exec format error
root@ef79040f7919:/home/icse2019/source/scripts# cat ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy
�NUMPY
 v {'descr': '<f8', 'fortran_order':="" false,="" 'shape':="" (1000,),="" }="" j�t�x?�i="" +��?�i="" +��?="" ���mb�?="" ���mbp?="" ���mb�?��&��?="" �="" ���mb`?="" ���mb�?��~j�t�?="" ���mbp?��g�z�?="" j�t�x?="" j�t="" ��?="" ���mbp?@�o��n�?h��|?5�?��g�z�?="" v-��?��g�z�?="" ���mb`?8�o��n�?="" ��mb`?="" @�o��n�?="" ���mbp?8��v���?="" �?="" ��="" j�t�x?�������?��g�z�?="" ���mb`?��g�z�?="" ���mbj�t�x?="" j�t��?="" ����mb�?l7�a`��?="" ���x�&�?="" ���s="" �?@�o��n�?="" ��ʡe��?="" ���mb�?����s�?="" ���mb`?�������?="" j�t�x���mb�?="" ��&��?="" ���mb�?@�o��n�?="" q����?="" @��v���?="" b`?="" ���mb`?����mb�?��&��?="" ���mb`?��&��?="" ���mb`?���(\="" bp?="" ���mb`?�i="" �l�����?="" j�&1�?="" j�t�x?��&��?="" ���mb`?�zd�?�ʡe���?="" ����mb`?="" 8��v���?="" ���mb`?��~j�t�?="" j�t�x?x9��v��?`��"���?�x�&��?="" ���mb`?��q��?��&��?="" ���mb�?��g�z�?="" �t��?="" j�t��?�l�����?="" ���mbp?�l�����?="" ?="" ���q��?="" ��b�?���s㥫?�i="" ���mbp?"��~j��?="" j�t��?��&��?="" ���mbp?��&��?="" jp?="" ��g�z�?="" `9��v��?="" ���mb�?�l�����?x9��v��?="" ���mbp���mb`?="" ���mb�?��g�z�?@�o��n�?="" j�t�x?zd;�o��?="" �"��~j�?="" j�tb`?="" ���mb�?�i="" +��?@��v���?="" ���mb�?��g�z�?�v��="" ���mb�?~j�t��?="" ���mb`?���s��?8��v���?="" +��?��g�z�?�q�����mb`?="" ���mb`?�v-�?="" ���mbp?8�o��n�b`?l7�a`��?="" ��c�l��?="" j�t��?�a�g�z�?���q��?="" j�t�x?�~j�t��?��&��?="" j�t�x?="" ��g�z�?�v��="" ���mb�?أp="��?" �i="" ���mbp?ȡe����?="" ��&��?@��v���?="" v-��?@��v���?="" j�t�x?�z�g��?="" ���mbp?��~j�t�?="" ���mb`?@�o��n�?="" ���mbp?䥛�="" ��g="" �q����?="" v-��?="" j�t�x?@��v���?="" ��g�z�b`?="" ���mb`?��="" �rh�?="" j�t��?��&��?��ʡe��?="" ��p?="" �����k�?="" ���mb�?="" @��v�="" -��?="" ���mb`?h�t��?��&��?�i="" +��?��&��?�������?��g�z�?="" mb`?="" 8�o��n�?="" ���mb`?̡e����?="" +��?��g�z�?="" ���mbp?@�o��n�?="" ��g�z�?�i="" `?��&��?��&��?="" `d;�o��?="" ���mbp?&�z�?="" @`��"��?="" +��?�"��~j�?="" j�t��?@�o��n�?mb`?="" ����mb�?�"��~j�?="" 433333�?="" ���mb`?�|?5^��?��&��?="" @�o��n�?�&��?="" ���m���mb`?�v��="" ���mb`?�~j�t��?="" l7�a`��?="" ���mbp?���x�&�?="" b`?�i="" +��?���q��?��c�l�?="" �������?����mb�?="" ���mb`?root@ef79040f7919:="" home="" icse2019="" source="" scripts#="" root@ef79040f7919:="" .="" lcr_acu_analysis.sh="" note:="" our="" experiments="" are="" only="" based="" on="" two="" datasets:="" mnist="" and="" cifar10,="" but="" it="" is="" a="" piece="" of="" cake="" to="" extend="" other="" datasets="" providing="" proper="" pytorch-style="" data="" loader="" tailored="" himself="" datasets.="" quickly="" label="" change="" rate="" auc="" statistics="" ,="" we="" provide="" group="" default="" parameters，do="" you="" want="" start="" the="" program?y="" n="" y="" please="" lcr="" result="" normal="" samples="" for="" computinglease="" test.="" do="" have="" results="" samples?(y="" n)y="" path="" normal's="" list:..="" build-in-resource="" nr-lcr="" mnsit="" lenet="" gf="" 5e-2p="" nrlcr.npy="======">Please Check Parameters<======= dataType: mnist device: -1 testType: adv useTrainData: False batchModelSize: 2 maxModelsUsed: 10 mutatedModelsPath: ../build-in-resource/mutated_models/mnist/lenet/gf/5e-2p/ testSamplesPath: ../build-in-resource/dataset/mnist/adversarial/jsma/ seedModelName: lenet test_result_folder: ../lcr_auc-testing-results/mnist/lenet/gf/5e-2p/jsma/ The test will be divided into 5 batches The logs will be saved in: ../lcr_auc-testing-results/mnist/lenet/gf/5e-2p/jsma/-2019-08-07-01 is_adv: True nrLcrPath: ../build-in-resource/nr-lcr/mnsit/lenet/gf/5e-2p/nrLCR.npy <======>Parameters=======>
Press any key to start mutation process
CTRL+C break command bash...
batch:1
model_start_no:1
batch:2
model_start_no:3
batch:3
model_start_no:5
batch:4
model_start_no:7
batch:5
model_start_no:9
Testing Done!
>>>>>>>>>>>seed data:mnist,mutated_models:../build-in-resource/mutated_models/mnist/lenet/gf/5e-2p/<<<<<<<<<< >>>>>>>>>>>mnist<<<<<<<<<<<<<< Total Samples Used:1000,auc:0.9871,avg_lcr:0.5032,std:0.1693,confidence(95%):0.0105,confidence(98%):0.0125,confidence(99%):0.0138 root@ef79040f7919:/home/icse2019/source/scripts# ./detect.sh NOTE: Our experiments are only based on two datasets: mnist and cifar10, but it is a piece of cake to extend to other datasets only providing a proper pytorch-style data loader tailored to himself datasets. To quickly perform adversarial detection, we provide a group of default parameters，do you want to quickly start the program?y/n y ./detect.sh: line 44: $'\n\nthrehold threhold\nextendScale extendScale\nrelaxScale relaxScale\nmutatedModelsPath mutatedModelsPath\nalpha alpha\nbeta beta\ntestSamplesPath testSamplesPath\ndataType dataType\ntestType testType\nseedModelPath seedModelPath\n\n': command not found =======>Please Check Parameters<======= threhold: 0.0441 extendScale: 1.0 relaxScale: 0.1 mutatedModelsPath: ../build-in-resource/mutated_models/mnist/lenet/nai/5e-2p/ alpha: 0.05 beta: 0.05 testSamplesPath: ../build-in-resource/dataset/mnist/adversarial/jsma/ dataType: 0 testType: adv seedModelPath: ../build-in-resource/pretrained-model/lenet.pkl mutatedModelsPath: ../build-in-resource/mutated_models/mnist/lenet/nai/5e-2p/ device: -1 <======>Parameters=======>
Press any key to start mutation process
CTRL+C break command bash...</f8',>

Processed:100.00 %adverage accuracy:0.998, avgerage mutated used:35.538
root@ef79040f7919:/home/icse2019/source/scripts