python telnet / monitoring redis or beanstalk

i wrote this python to monitor my redis or beanstalk process, cannot give you full of my scripts this is only basic scripts. in case me in the future forget how to write and send command to server via telnet and failed because of hung in the middle of process (got into limbo) this will get the value and send that value over to monitoring server thru REST API or CLI

Monitoring current connected client in redis

#!/usr/bin/env python
# Very simple python monitoring redis connected clients.

import sys
import telnetlib
import time

def run_telnet_session():
 #host = raw_input("Enter remote hostname e.g. localhost:")
 #port = raw_input("Enter remote hostname e.g. 6379:")
 host = "127.0.0.1"
 port = 6379
 session = telnetlib.Telnet(host,port)

 session.write("info \r\n")
 time.sleep(1)
 session.write("quit \r\n")
 data=session.read_very_eager()
 #print data
 for item in data.split("\n"):
  if "connected_clients" in item:
    print item.strip()
    print item.strip().split(':')[1]

if __name__ == '__main__':
 run_telnet_session()

Monitoring Current Job ready beanstalkd

#!/usr/bin/env python
# Very simple python monitoring redis connected clients.

import sys
import telnetlib
import time

def run_telnet_session():
 host = "127.0.0.1"
 port = 11300
 session = telnetlib.Telnet(host,port)

 session.write("stats\r\n")
 time.sleep(2)
 session.write("quit\r\n")
 data=session.read_very_eager()

 for item in data.split("\n"):
   if "current-jobs-ready" in item:
    print item.strip()
    print item.strip().split(':')[1]

if __name__ == '__main__':
 run_telnet_session()


mon_py_scripts

Easiest way “How to get tag from remote github repository”

when someone ask you, is it posible to get my latest tag from my github repository so i can put or add it into my automation deployment process for staging environment every morning without have to trigger it manually ? here it’s how to solve it


awan@google-provision:~$ MY_PROJECT=avengers
awan@google-provision:~$ git ls-remote --tags git@github.com:MY_REPOSITORY/"$MY_PROJECT".git | awk '{print $2}' | grep -v '{}' | awk -F"/" '{print $3}' | tail -n 1

put it into variable and you can pass the variable into your automation scripts.

if you’re using ansible to deploy it, ansible has module to do this


- name: Get Latest Tag from my Repo
local_action: shell git ls-remote --tags git@github.com:MY_REPOSITORY/"$MY_PROJECT".git | awk '{print $2}' | grep -v '{}' | awk -F"/" '{print $3}' | tail -n 1
register: my_last_tag

- name: Update Code Via Github
git: repo=git@github.com:MY_REPOSITORY/avengers.git
dest=/home/google/public
update=yes
accept_hostkey=yes
ssh_opts="-o ForwardAgent=yes"
version="{{ item }}"
with_items: my_last_tag.stdout_lines
register: code
sudo: no
tags: update

Menghubungkan Container Docker di Hardware Yang Berbeda

Kalau ada yang bertanya, apakah memungkinkan saya menghubungkan docker container saya dengan container yang lain pada mesin / hardware / server yang berbeda ? jawabnya tentu dan sangat mudah. mari kita buatkan analoginya

1. Saya punya 4 mesin mewakili 4 layer di infrastruktur saya, terdiri dari :
— 2 buah server DELL untuk webserver
— 1 buah server HP untuk application / cron / api
— 1 buah server IBM untuk database server

2. Didalam setiap server saya letakkan container docker yang sudah di provision menggunakan ansible sebagai berikut

— container web application
–++ Image Container Name : infra/web
— container apps / cron / api
–++ Image Container Name : infra/apps
— container database
–++ Image Container Name : infra/db

3. Sekarang saya ingin memberikan contoh ip address yang saya gunakan untuk testing ini

— 2 buah server DELL untuk webserver
## 31.192.117.132
## 104.20.26.14

— 1 buah server HP untuk application / cron / api
## 141.0.174.35

— 1 buah server IBM untuk database
## 62.212.83.1

4. untuk dapat menghubungkan container dengan container lain pada mesin / server yang berbeda dapat dengan cara sebagai berikut :


--- login ke server DELL ip 31.192.117.132
--- root@31.192.117.132:~# docker run -i -t --net=host infra/web /bin/bash
--- root@31.192.117.132:/# <--- anda sudah berada dalam container dengan IP hardware


--- login ke server DELL ip 104.20.26.14
--- root@104.20.26.14:~# docker run -i -t --net=host infra/web /bin/bash
--- root@104.20.26.14:/# <--- anda sudah berada dalam container dengan IP hardware


--- login ke server HP ip 141.0.174.35
--- root@141.0.174.35:~# docker run -i -tn --net=host infra/apps /bin/bash
--- root@141.0.174.35:/# <--- anda sudah berada dalam container dengan IP hardware


--- login ke server IBM ip 62.212.83.1
--- root@62.212.83.1:~# docker run -i -tn --net=host infra/apps /bin/bash
--- root@62.212.83.1:/# <--- anda sudah berada dalam container dengan IP hardware

dengan container yang telah menggunakan ip hardware dari server anda, anda dapat langsung berhubungan dengan server lainnya, perlu di ingat karena anda membuka port ketika menjalankan aplikasi anda port itu tidak akan terlihat oleh host master tapi oleh container anda jangan sampai lupa karena terkadang anda butuh menjalankan beberapa container pada satu hardware dan ternyata port 80 sudah terbuka di container yang lain namun port ini tidak akan dapat complaint oleh aplikasi karena dia terbuka di container yang berbeda, hasil yang anda dapatkan bisa sangat bervariasi baik itu flaky network atau aplikasi yang berjalan dengan tidak semestinya.

Selamat anda telah menghubungkan container dengan container running under docker dengan 4 hardware yang berbeda.

The easiest way install docker on ubuntu 13.10

This is how you can install docker lxc inside 13.10 with no pain

copy this commands and put it into one file and set it as executed

1. Copy all this command as one file, sample : install_docker.13.10.sh


sudo apt-get -y update
sudo apt-get -y autoremove
sudo apt-get install linux-image-extra-`uname -r`
sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
sudo sed -i '/localhost/a 127.0.0.1 '"$(echo `hostname`)"'' /etc/hosts
sudo sh -c "echo deb http://get.docker.io/ubuntu docker main > /etc/apt/sources.list.d/docker.list"
sudo apt-get -y update
sudo apt-get -y install lxc-docker

2. after save it as install_docker.13.10.sh as sample, let’s make it executeable


#chmod +x install_docker.13.10.sh

3. Execute this will add 127.0.0.1 record into your /etc/hosts to prevent error when add docker.list into /etc/sources.list.d


#sh install_docker.13.10.sh

simply just comment if you found an error regarding this dummy installation scripts.

Simple route throw exception

This is only log, how to throw exception error into laravel framework using routes.php


#sudo vim routes.php

add this in that file

Route::get('you_have_been_compromised', function() {
throw new \Exception("This is the error goes.");
});

i did this because i need to simulate the error, and make sure error is not write to instances files anymore but it’s stream out directly to log server. The result will be vary but one thing that you need to know that you will save a lot hard disk space in your instance and you don’t have to manage logrotate and you can combine the sumologic or any alert agent only in your log server don’t have to put the error agent in each of your instances.

Credits to my friend ‘sky akbar ko’

Error sudo env ARCHFLAGS=”-arch x86_64″ gem install pg

This error show because there is no ruby-dev on your Operating System, to fix this just install ruby-dev


awan@google.com:~/home/awan$ sudo apt-get install ruby-dev


Building native extensions. This could take a while...
ERROR: Error installing pg:
ERROR: Failed to build gem native extension.

/usr/bin/ruby1.9.1 extconf.rb
/usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': cannot load such file -- mkmf (LoadError)
from /usr/lib/ruby/1.9.1/rubygems/custom_require.rb:36:in `require'
from extconf.rb:2:in `'

Gem files will remain installed in /var/lib/gems/1.9.1/gems/pg-0.17.1 for inspection.
Results logged to /var/lib/gems/1.9.1/gems/pg-0.17.1/ext/gem_make.out

and if this below error show, then execute this


awan@google.com:~/home/awan$ sudo apt-get install libpq-dev


Building native extensions. This could take a while...
ERROR: Error installing pg:
ERROR: Failed to build gem native extension.

/usr/bin/ruby1.9.1 extconf.rb
checking for pg_config... yes
Using config values from /usr/bin/pg_config
You need to install postgresql-server-dev-X.Y for building a server-side extension or libpq-dev for building a client-side application.
You need to install postgresql-server-dev-X.Y for building a server-side extension or libpq-dev for building a client-side application.
checking for libpq-fe.h... no
Can't find the 'libpq-fe.h header
*** extconf.rb failed ***
Could not create Makefile due to some reason, probably lack of
necessary libraries and/or headers. Check the mkmf.log file for more
details. You may need configuration options.

Provided configuration options:
--with-opt-dir
--without-opt-dir
--with-opt-include
--without-opt-include=${opt-dir}/include
--with-opt-lib
--without-opt-lib=${opt-dir}/lib
--with-make-prog
--without-make-prog
--srcdir=.
--curdir
--ruby=/usr/bin/ruby1.9.1
--with-pg
--without-pg
--with-pg-config
--without-pg-config
--with-pg_config
--without-pg_config
--with-pg-dir
--without-pg-dir
--with-pg-include
--without-pg-include=${pg-dir}/include
--with-pg-lib
--without-pg-lib=${pg-dir}/lib

Gem files will remain installed in /var/lib/gems/1.9.1/gems/pg-0.17.1 for inspection.
Results logged to /var/lib/gems/1.9.1/gems/pg-0.17.1/ext/gem_make.out

install ansible the easiest way

Cara termudah menginstall ansible adalah mengcopy file dibawah ini ke sebuah file lalu eksekusi file

1. save this scripts to a file, sample name : google.ansible.sh


#----------- start code ------------
#!/bin/bash
sudo apt-get -y install python-pip python-dev
sudo pip install -U boto
sudo pip install -U https://github.com/ansible/ansible/archive/devel.zip
ansible --version
#----------- end code ------------

2. chmod +x to that file
~#sudo chmod +x google.ansible.sh

3. execute this file
~#sudo sh google.ansible.sh

Untuk pengguna centos 7 bisa menggunakan scripts dibawah ini

1. simpan code dibawah ini sebagai script bash : google.ansible.sh


#----------- start code ------------
#!/bin/bash
sudo yum -y install epel-release
sudo yum install -y gcc python-pip python-devel
sudo pip install -U boto
sudo pip install -U https://github.com/ansible/ansible/archive/devel.zip
ansible --version
#----------- end code ------------

2. ganti permisi file biar dapat di eksekusi

~#chmod +x ./google.ansible.sh

3. eksekusi file installasi

~#sh ./googe.ansible.sh