Tools

Site Scanners and Online Tools

This is a list of a few online tools I often use to audit for both security and speed of a site.

SSL LABS

https://www.ssllabs.com/ssltest/ – SSL Labs will determine how secure the settings are on HTTPS settings of a server.  Check https://mozilla.github.io/server-side-tls/ssl-config-generator/ for way more information then you ever wanted to know about TLS and get help with some boiler plate for the web server of choice to get a good security setup.

SECURITY HEADERS IO

https://securityheaders.io – SSL Labs is good for servers, but not for browsers.  Securityheaders.io will give you HTTP Headers to set to help protect the browser from itself.

YELLOW LAB TOOLS

http://yellowlab.tools – Yellowlabs is my goto tool to audit a website and determine bottle necks from a front end point of view.  I like it more then other tools as it is gives the most human readable information on how to fix the issues that it finds.  It also does a good job on focusing on whats current, and not the “new shiny” things like lighthouse.  A docker image is also available to run so you do not have to wait in line.  https://github.com/jguyomard/docker-yellowlabtools

Errors

WordPress – Sorry, you are not allowed to access this page

Just got done with many hours of fighting wordpress and its “Sorry, you are not allowed to access this page” generic output after doing some HTTPS update settings.  Turns out I had the drupal expected setting in haproxy and not the wordpress one.  Ensure you are passing the following is using haproxy

backend backend-default
  http-request add-header HTTP_X_FORWARDED_PROTO https

 

 

Unix

Elrepo / EPEL not finding packages CentOS 7.4

After trying out a new image on Amazon, I was unable to install basic “extra” packages like ack (and if you are using grep, please replace it with ack).  This was an AMI (disk image) so something was missing between ansible the original author’s steps run before making the AMI.  Messing with all the files in /etc/yum.repos.d/ and doing every yum command under the sun yeilded nothing useful.  After a few hours wasted, it turns out the magic command all along was:

sudo yum install epel-release

For ansible (be sure this is before the other yum package installs that require it)

- name: add extras repo for cents
  become: true
  yum: >
    pkg="epel-release"

 

Bash

Bash sub string stripping

Often when making long strings (like json), there will be trailing comma that needs to be deleted.  The % makes this possible in bash.

# Example One
# note the extra , at the end
MYVAR="cat, dog, bird,"
MYNEWVAR=${MYVAR%,}

echo $MYNEWVAR
# outputs 'cat, dog, bird'



# Example Two
# strip off the file known extension
MYFILE="foo.csv"
MYFILENOEXT=${MYFILE%\.csv}

echo $MYFILENOEXT
# outputs 'foo'

See http://www.tldp.org/LDP/abs/html/string-manipulation.html for more string functions.

Vim & Tmux

Difference of word jumping with w, b, and e versus Shift W, B, and E

Getting around quickly with vim is one of the keys to not only working more quickly, but also being able to more easily reuse macros and the repeat (dot) command.  In this example, I will show the difference between the w, b, and e key and the capital (shift) variant in normal mode.

# this is only a comment to show that 'w', 'b', and 'e' jump a word at a time

# this is some random code to show 'W', 'B', and 'E' jump to spaces
 foo.bar = function { cat.meow().toString; dog(); }

In vim normal mode:

  • w will jump to the start of the next word
  • e will jump to the end of the current word
  • b will go back to the start of the previous word

Capital/Shift variants will instead act on a WORD definition.  Vim help defines a word as “A WORD consists of a sequence of non-blank characters, separated with white space.  An empty line is also considered to be a WORD.”

In the above example, cat.meow().toString is treated very different with w vs W key strokes.  Copy and paste the above block into vim, navigate to c in the word cat.  Now try pressing w.  Go back to c and try W.

As you can see, w jumps to m in meow.  W jumps you to the d in dog().

Capital/Shift variants are a great way to skip down lines of code.

Cloud Systems

Using cloud config with CoreOS

CoreOS has depreciated cloud-config… see https://coreos.com/ignition/docs/latest/what-is-ignition.html

This is how I got a CoreOS box up on Digital Ocean using a cloud configfile.  You can find the cloud-config documentation with examples on the CoreOS site.  When I first did this many months ago, the documentation didn’t have much in the way of real examples.  Thankfully its been updated.

What is a cloud config file and its purpose?  Well it is to give brand new servers a starting point that makes sense, and can help with automation.  Cloud config is written with YAML, which is a simple text file with basic formatting.  The “old way” of ordering a cloud server would give some email or popup box with the root password to let you login. A cloud config lets you define users, ssh-keys, firewalls and do basic file creation. No more dealing with clunky password exchanges from your cloud provider, or hoping the SSH key you have is current in the “defaults” server creations. or do many manual steps before you can have the new server able have automation services build out the system.  I recommend doing just enough so you can fire up a system automation tool without first needing to login and setup enough access right manually for automation to work.

This screenshot is from Digital Ocean when creating a new server (droplet).  Once I select the ‘User Data’ checkbox, I can paste in my cloud-config yml.  Part of the confusion is that all the different cloud providers that support CoreOS don’t always make it clear where one should be putting the cloud-init.yml contents when creating new servers.

cloud-init.yml
digital ocean droplet for CoreOS with cloud-init.yml

In my case, I’m doing a few basic things:

  • Starting iptables (firewall) on boot
  • Adding my user account and my ssh pub key so I can login
  • Adding my user to various system groups so I can run docker commands and sudo
  • Setting the contents of the iptables file.  In this instance, I’m using Cloudflare as my CDN/WAF, thus port 80 is only listening for Cloudflare IPs.  Because my home IP is dynamic, I cannot limit down port 22 (ssh).
  • Disabling root login for ssh

Taking the cloud-init a step further, you might install service discovery (I like consul for this)  and define the role of the new server.

Also take note that you cannot later change the cloud config file.  This is for the first boot up of the new server.  It is never used again.  Thus order and destroy a small server while working on a cloud config script.

#cloud-config

---
coreos:
 units:
 - name: iptables-restore.service
 enable: true
 command: start
 - name: "docker.service"
 enable: true
 command: "start"
users:
- name: joecool
 ssh-authorized-keys:
 - ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAINe6e1zxtHDUK6mhYY0YQeoMQukIdDchyL8+o9xgUle7 joecool@local
 shell: "/bin/bash"
 groups:
 - docker
 - rkt
 - fleet
 - docker
 - systemd-journal
 - portage
 - sudo
write_files:
- path: "/var/lib/iptables/rules-save"
 permissions: 420
 owner: root:root
 content: |
 *filter
 :INPUT DROP [0:0]
 :FORWARD DROP [0:0]
 :OUTPUT ACCEPT [0:0]
 -A INPUT -i lo -j ACCEPT
 -A INPUT -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
 -A INPUT -p tcp -m tcp --dport 22 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 103.21.244.0/22 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 103.22.200.0/22 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 103.31.4.0/22 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 104.16.0.0/12 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 108.162.192.0/18 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 131.0.72.0/22 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 141.101.64.0/18 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 162.158.0.0/15 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 172.64.0.0/13 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 173.245.48.0/20 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 188.114.96.0/20 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 190.93.240.0/20 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 197.234.240.0/22 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 198.41.128.0/17 --dport 80 -j ACCEPT
 -A INPUT -p tcp -m tcp -s 199.27.128.0/21 --dport 80 -j ACCEPT
 -A INPUT -p tcp --dport 80 -j REJECT
 COMMIT
runcmd:
- sed -i -e '/^PermitRootLogin/s/^.*$/PermitRootLogin no/' /etc/ssh/sshd_config
- restart ssh