IRMA Documentation Release 2.0.4
Quarkslab
Feb 22, 2018
Contents
1
Introduction 1.1 Purpose . . . . . . . . . 1.2 File Analysis Process . . 1.3 Infrastructure Overview 1.4 Hardware requirements 1.5 Supported Analyzers . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
. . . . .
3 3 3 4 5 6
Automated Install 2.1 Requirements . . . . . . 2.2 Ansible scripts . . . . . 2.3 Predefined Environments 2.4 Using Debian repos . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
. . . .
9 9 9 9 12
3
Manual Installation 3.1 Brain . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.2 Frontend . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 3.3 Probe . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .
13 13 22 31
4
sudo /opt/COMODO/menu/comodo-updater [...]
Note: Dependencies to update the sudo /opt/eset/esets/bin/esets_gui
Note: Disabling the antivirus daemon To avoid the anti-virus to protect your system at startup, we deliberately disabled the script used to launch the anti-virus early at boot: $ sudo service esets stop $ sudo mv /etc/init.d/esets /etc/init.d/esets.disable
F-Prot - GNU/Linux A copy of F-PROT anti-virus for Linux workstations is available on the F-PROT download page. The binaries should be installed in /usr/local/f-prot to make the python application detect it automatically. $ sudo tar xvf fp-Linux.x86.32-ws.tar.gz -C /usr/local/
To launch an update, a configuration step is mandatory: $ sudo cp /usr/local/f-prot/f-prot.conf.default /etc/f-prot.conf
An update is launched with: $ sudo ./fpupdate ERROR: ld.so: object 'libesets_pac.so' from /etc/ld.so.preload cannot be preloaded: ˓→ignored. [...]
Note: Error If you see an error message like: DownloadingWarning: Network - Connection failed (18), trying again... Downloading updateError: Update - Bad mergefile
Just relaunch the script.
Note: Dependencies to update the ' -o ˓→KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi˓→keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ˓→ConnectTimeout=10 127.0.0.1 '/bin/sh -c '"'"'( umask 77 && mkdir -p "` echo $HOME/. ˓→ansible/tmp/ansible-tmp-1468570550.09-211613386938202 `" && echo ansible-tmp˓→1468570550.09-211613386938202="` echo $HOME/.ansible/tmp/ansible-tmp-1468570550.09˓→211613386938202 `" ) && sleep 0'"'"'' PUT /tmp/tmpiysJ6l TO /home/vagrant/.ansible/tmp/ansible-tmp˓→1468570550.09-211613386938202/rabbitmq_user SSH: EXEC sftp -b - -C -o ForwardAgent=yes -o Port=2222 -o ˓→'IdentityFile="/home/alex/.vagrant.d/insecure_private_key"' -o ˓→KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi˓→keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ˓→ConnectTimeout=10 '[127.0.0.1]' ESTABLISH SSH CONNECTION FOR USER: vagrant SSH: EXEC ssh -C -q -o ForwardAgent=yes -o Port=2222 -o ˓→'IdentityFile="/home/alex/.vagrant.d/insecure_private_key"' -o ˓→KbdInteractiveAuthentication=no -o PreferredAuthentications=gssapi-with-mic,gssapi˓→keyex,hostbased,publickey -o PasswordAuthentication=no -o User=vagrant -o ˓→ConnectTimeout=10 -tt 127.0.0.1 '/bin/sh -c '"'"'sudo -H -S -n -u root /bin/sh -c '" ˓→'"'"'"'"'"'"'"'echo BECOME-SUCCESS-rbeeckncuxenewcwkayivqiwvarchlrd; LANG=fr_FR.UTF˓→8 LC_ALL=fr_FR.UTF-8 LC_MESSAGES=fr_FR.UTF-8 /usr/bin/python /home/vagrant/.ansible/ ˓→tmp/ansible-tmp-1468570550.09-211613386938202/rabbitmq_user; rm -rf "/home/vagrant/. ˓→ansible/tmp/ansible-tmp-1468570550.09-211613386938202/" > /dev/null 2>&1'"'"'"'"'"'" ˓→'"'"' && sleep 0'"'"'' failed: [brain.irma] (item={u'vhost': u'mqbrain', u'password': u'brain', u'user ˓→': u'brain'}) => {"failed": true, "invocation": {"module_name": "rabbitmq_user"}, ˓→"item": {"password": "brain", "user": "brain", "vhost": "mqbrain"}, "module_stderr ˓→": "", "module_stdout": "Traceback (most recent call last):\r\n File \"/tmp/ ˓→ansible_Qo3lZl/ansible_module_rabbitmq_user.py\", line 302, in \r\n ˓→main()\r\n File \"/tmp/ansible_Qo3lZl/ansible_module_rabbitmq_user.py\", line 274, ˓→in main\r\n if rabbitmq_user.get():\r\n File \"/tmp/ansible_Qo3lZl/ansible_ ˓→module_rabbitmq_user.py\", line 155, in get\r\n users = self._exec(['list_users ˓→'], True)\r\n File \"/tmp/ansible_Qo3lZl/ansible_module_rabbitmq_user.py\", line ˓→150, in _exec\r\n rc, out, err = self.module.run_command(cmd + args, check_ ˓→rc=True)\r\n File \"/tmp/ansible_Qo3lZl/ansible_modlib.zip/ansible/module_utils/ ˓→basic.py\", line 1993, in run_command\r\n File \"/usr/lib/python2.7/posixpath.py\", ˓→ line 261, in expanduser\r\n if not path.startswith('~'):\r\nAttributeError: ˓→'list' object has no attribute 'startswith'\r\n", "msg": "MODULE FAILURE", "parsed ˓→": false}
In this particular case, verbose doesnt add much information as the problem is linked to ansible scripts. Lets go one level deeper so. Ansible output the temporary script executed on guest (highlighted in previous code block), but delete it just after execution. To further debug it we will set ansible to keep remote files and the debug session will now takes place inside the guest. $ ANSIBLE_KEEP_REMOTE_FILES=1 ansible-playbook -vvv --private-key=~/.vagrant.d/ ˓→insecure_private_key --inventory-file=.vagrant/provisioners/ansible/inventory/ ˓→vagrant_ansible_inventory -u vagrant playbooks/provisioning.yml
7.3. How to debug
59
IRMA Documentation, Release 2.0.4
in debug log get the temporary ansible path to remote script: /usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275/ ˓→rabbitmq_user
Log in to remote machine and go to the temporary ansible dir. Explode the compressed script and run it locallly: $ vagrant@brain:~/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275$ ls rabbitmq_user $ vagrant@brain:~/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275$ python ˓→rabbitmq_user explode Module expanded into: /home/vagrant/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275/debug_dir $ vagrant@brain:~/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275$ ls debug_ ˓→dir/ ansible ansible_module_rabbitmq_user.py args $ vagrant@brain:~/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275$ python ˓→rabbitmq_user execute Traceback (most recent call last): File "/home/vagrant/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275/debug_ ˓→dir/ansible_module_rabbitmq_user.py", line 302, in main() File "/home/vagrant/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275/debug_ ˓→dir/ansible_module_rabbitmq_user.py", line 274, in main if rabbitmq_user.get(): File "/home/vagrant/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275/debug_ ˓→dir/ansible_module_rabbitmq_user.py", line 155, in get users = self._exec(['list_users'], True) File "/home/vagrant/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275/debug_ ˓→dir/ansible_module_rabbitmq_user.py", line 150, in _exec rc, out, err = self.module.run_command(cmd + args, check_rc=True) File "/home/vagrant/.ansible/tmp/ansible-tmp-1468571039.87-134696488633275/debug_ ˓→dir/ansible/module_utils/basic.py", line 1993, in run_command args = [ os.path.expandvars(os.path.expanduser(x)) for x in args if x is not None ˓→] File "/usr/lib/python2.7/posixpath.py", line 261, in expanduser if not path.startswith('~'): AttributeError: 'list' object has no attribute 'startswith'
You could now add debug to source files and properly understand where the problem is. In our example case, it is an ansible problem related to module_rabbitmq_user present in 2.1.0.0 see github PR
7.4 How to migrate Note: If you need help to connect to your box through ssh, see vagrant FAQ This part is only useful to someone willing to manually upgrade from an older version of IRMA.
60
Chapter 7. Frequently Asked Questions
IRMA Documentation, Release 2.0.4
7.4.1 Install alembic $ $ $ $ $
sudo su deploy cd /opt/irma/irma-frontend/current ./venv/bin/pip install alembic export PYTHONPATH=.:$PYTHONPATH alembic history
430a70c8aa21 -> eb7141efd75a (head), version 1.3.0 2cc69d5c53eb -> 430a70c8aa21, version 1.2.1 -> 2cc69d5c53eb, DB revision creation
7.4.2 from 1.2.1 to 1.3.0 Fix nginx configuration Introducing multiversion API means python code should receive the api version parameter. in file /etc/nginx/sitesavailable/irma-frontend.conf replace: rewrite ^/api/v1/(.+) /$1 break;
by: rewrite ^/api/(.+) /$1 break;
and restart nginx Migrate Database First you should tell alembic you are at version 1.2.1: $ ./venv/bin/alembic stamp 430a70c8aa21
then upgrade model and data: $ ./venv/bin/alembic upgrade head
Regenerate IHM to regenerate IHM do the following: $ $ $ $
sudo su deploy cd /opt/irma/irma-frontend/current/web ./node_modules/.bin/bower update ./node_modules/.bin/gulp dist
Its done.
7.5 API documentation There is a dynamic documentation for IRMA API available on your instance 7.5. API documentation
61
IRMA Documentation, Release 2.0.4
It allows you to read documentation but also try request and see server response. Give it a try.
You could see detailed information about one specific API route:
and by clicking on the Try it button, see the server response:
62
Chapter 7. Frequently Asked Questions
IRMA Documentation, Release 2.0.4
7.6 Connect to a vagrant box through ssh If you don’t already have it download vagrant insecure_private_key $ wget https://raw.githubusercontent.com/mitchellh/vagrant/master/keys/vagrant -O ˓→insecure_private_key
Then change rights on the key otherwise ssh will complains and connect to your vagrant box $ chmod 700 insecure_private_key $ ssh
[email protected] -i insecure_private_key
7.7 Enable SSL using OpenSSL in ansible scripts If you want to activate SSL on the frontend server, you’ll need: • modify frontend_openssl variables in group_vars/frontend: frontend_openssl: True # Default is false frontend_openssl_dh_param: # put the DH file locations frontend_openssl_certificates: [] # an array of files {source, destination} # to copy to the server
• Uncomment (and customize) the nginx_sites variable in the group_vars/frontend, a commented example is available. Then, provision or re-provision your infrastructure. Ansible will only change file related to OpenSSL and Nginx configurations.
7.6. Connect to a vagrant box through ssh
63
IRMA Documentation, Release 2.0.4
7.8 Speed up your Vagrant VMs Install this softwares: • vagrant-cachier (more info on vagrant-cachier) $ vagrant plugin install vagrant-cachier
• vagrant-vbguest (more info on vagrant-vbguest) $ vagrant plugin install vagrant-vbguest
64
Chapter 7. Frequently Asked Questions
CHAPTER
8
Resources
• Project website • IRC (irc.freenode.net, #qb_irma) • Twitter (@qb_irma)
65
IRMA Documentation, Release 2.0.4
66
Chapter 8. Resources
CHAPTER
9
Screenshots
9.1 Command Line Interface A sample script can be found in frontend repository. Add your own frontend address before testing it.
67
IRMA Documentation, Release 2.0.4
9.2 Web Interface Some screenshots of the irma user interface shipped with frontend package.
68
Chapter 9. Screenshots
IRMA Documentation, Release 2.0.4
9.2. Web Interface
69
IRMA Documentation, Release 2.0.4
70
Chapter 9. Screenshots
IRMA Documentation, Release 2.0.4
9.2. Web Interface
71
IRMA Documentation, Release 2.0.4
72
Chapter 9. Screenshots