People,
I wanted to share with you a simple script published as open source: (git://github.com/mgiorgi/campfire-pattern-alert.git)
I developed this software because I wanted some kind of alerts, when people mentioned my name in a Campfire's room. In fact, our team used to work that way, when someone wanted to talk to me, they just typed: 'Marcelo'. And I supposed to check that messages below that.
For that, I though, it would be great if there were a Desktop tool that can connect to campfire and just alert me when that happened. But, unfortunately, for Linux users(*), like me, there is no good solution to filter the irrelevant messages and alert me with the ones I want.
So, I decided to came up with this tiny tool that does exactly that job!
An example of the usage of it (after configuring the proper jabber/campfire connection information in jabber-configuration.rb and campfire-confirguration.rb respectivly):
$ ./campfire-listener.sh Marcelo JABBER
Then after a couple of logged messages, every message that contained 'Marcelo' would be sent to the jabber account that I want. In fact, I decided to forward the following 5 messages (this amount can be changed within a constant in campfire-listener.sh) after an occurrence of 'Marcelo' (in this case).
General usage would look like: ./campfire-listener.sh pattern output.
Where 'pattern' could be a regular expression and 'output' stands for STDOUT (if output parameter is not set, this is considered the default value) or JABBER.
Well, hope it helps you, as well it helped me! Let me know what you think ;)
(*): I knew that such a solution exists for Mac OS, for instance.
martes, 30 de junio de 2009
lunes, 8 de junio de 2009
A first look at God and Monit
Hey people!,
Lately I've been configuring a production server which basically is composed of a mongrel cluster and some other daemons that we use (backgroundrb, i.e.). To keep alive both mongrel instances and our daemons we needed some monitor tool to help us know when something went wrong and restart those processes if needed.
For that matter, I investigate the following tools: Monit and God.
Monit
Monit (http://mmonit.com/monit/) is a great utility for managing and monitoring, processes, files, directories and devices on a Unix system.
Installation
It's very straight-forward:
After installing it, we need to create a configuration file. Monit would look for this file in the following locations:
#.monitrc
set daemon 30
set logfile /home/myuser/monit/monit.log
set httpd port 9111
allow remotehost
allow admin:admin # Allow Basic Auth
check process mongrel_cluster_3010 with pidfile "/var/www/myapp/current/tmp/pids/mongrel.3010.pid"
start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -a 127.0.0.1 -c /var/www/myapp/current --user myuser --group deploy -p 3010 -P tmp/pids/mongrel.3010.pid -l log/mongrel.3010.log"
stop program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails stop -p 3010 -P /var/www/myapp/current/tmp/pids/mongrel.3010.pid"
if failed port 3010 protocol http # check for response
with timeout 10 seconds
then restart
group mongrel
check process mongrel_cluster_3011 with pidfile "/var/www/myapp/current/tmp/pids/mongrel.3011.pid"
start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -a 127.0.0.1 -c /var/www/myapp/current --user lokkedc --group deploy -p 3011 -P tmp/pids/mongrel.3011.pid -l log/mongrel.3011.log"
stop program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails stop -p 3011 -P /var/www/myapp/current/tmp/pids/mongrel.3011.pid"
if failed port 3011 protocol http # check for response
with timeout 10 seconds
then restart
group mongrel
You can check the complete documentation of the commands allowed for monit in http://mmonit.com/monit/documentation/monit.html.
But I just wanted to show you a simple example that checks for the availability of the mongrel instances (in this case I just configured a couple of those). As you may notice, I have to configure each instance separatly (besides they both belongs to the same group, so I can start/stop all of those at the same time).
Other thing to notice, is the events that I want to monitor. In this case I've configured to restart a given mongrel's instance (the fragment that takes care of this is highlighted in the code below) when there is no response from it. But there are many other events, such us memory usage, cpu time, that can be measured to alert the administrator (vía email).
Run it!
To start it we just execute: moint. And we can see how the logs starts to populate:
[UYT Jun 5 19:12:15] info : monit: generated unique Monit id 15fdb0bdb830b0a114a3831d995ec32e and stored to '/home/myuser/.monit.id'
[UYT Jun 5 19:12:15] info : Starting monit daemon with http interface at [*:9111]
[UYT Jun 5 19:12:15] info : Starting monit HTTP server at [*:9111]
[UYT Jun 5 19:12:15] info : monit HTTP server started
[UYT Jun 5 19:12:15] info : 'willy' Monit started
[UYT Jun 5 19:12:34] info : Shutting down monit HTTP server
[UYT Jun 5 19:12:35] info : monit HTTP server stopped
[UYT Jun 5 19:12:35] info : monit daemon with pid [21560] killed
[UYT Jun 5 19:12:35] info : 'willy' Monit stopped
[UYT Jun 5 19:13:42] info : Starting monit daemon with http interface at [*:9111]
[UYT Jun 5 19:13:42] info : Starting monit HTTP server at [*:9111]
One of the greatest things about monit is that it offers a web interface (as you may notice from the configuration file). Which looks pretty nice!
God
God its a newer tool written in ruby and available as a rubygem. For this matter, it is easier to install it also (i.e. [sudo] gem install god).
Usage
We need to create a configuration file, which is entirely written in ruby. This is one the best things about God, because it helps us reduce duplication among configurations, as you can see in the following example (which is intended to monitor the same mongrel cluster mentioned above):
#myapp.god
RAILS_ROOT = "/var/www/myapp/current"
%w{3010 3011}.each do |port| God.watch do |w|
w.name = "mongrel_cluster_#{port}"
w.group = 'mongrels'
w.interval = 30.seconds
w.start = "mongrel_rails start -c #{RAILS_ROOT} -p #{port} \ -P #{RAILS_ROOT}/tmp/pids/mongrel.#{port}.pid -d -e production"
w.stop = "mongrel_rails stop -P #{RAILS_ROOT}/tmp/pids/mongrel.#{port}.pid -e production"
w.restart = "mongrel_rails restart -P #{RAILS_ROOT}/tmp/pids/mongrel.#{port}.pid -e production" w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.pid_file = File.join(RAILS_ROOT, "tmp/pids/mongrel.#{port}.pid")
w.behavior(:clean_pid_file)
w.start_if do |start| start.condition(:process_running) do |c|
c.interval = 5.seconds
c.running = false
end
end
end
As you can see, I can refactor the mongrel's configuration into just one block which is instantiated for each mongrel's port. Besides that, I've also reduce the amount of configuration using RAILS_ROOT constant in many settings (reducing error prune also).
Run It!
To start our God monitoring tool, we just exectue: god -c myapp.god
After which, we can see the log using the following command: god log mongrels (to show the logs relative to our group of mongrel instances). Which should display something like this:
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 move 'unmonitored' to 'up'
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 moved 'unmonitored' to 'up'
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 [trigger] process is not running (ProcessRunning)
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 move 'up' to 'start'
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 before_start: deleted pid file (CleanPidFile)
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 start: mongrel_rails start -c /var/www/myapp/current -p 3010 -P /var/www/myapp/current/tmp/pids/mongrel.3010.pid -d -e production
I [2009-06-08 12:59:27] INFO: mongrel_cluster_3010 moved 'up' to 'up'
I [2009-06-08 12:59:27] INFO: mongrel_cluster_3010 [ok] process is running (ProcessRunning)
Conclusion - My Choice
Any of those tools gets the job done pretty well. But I decided to use monit, because it is easier/faster for me to manage the process using a web-interface instead of logging into the server via ssh.
Besides that, God seems to be a great tool as it provides a way to reduce duplication (because its configuration is defined with ruby code).
Lately I've been configuring a production server which basically is composed of a mongrel cluster and some other daemons that we use (backgroundrb, i.e.). To keep alive both mongrel instances and our daemons we needed some monitor tool to help us know when something went wrong and restart those processes if needed.
For that matter, I investigate the following tools: Monit and God.
Monit
Monit (http://mmonit.com/monit/) is a great utility for managing and monitoring, processes, files, directories and devices on a Unix system.
Installation
It's very straight-forward:
- Download the latest version on http://mmonit.com/monit/download/
- tar zxvf monit-x.y.z.tar.gz
- cd monit-x.y.z
- ./configure
- make && sudo make install (sudo if necessary)
After installing it, we need to create a configuration file. Monit would look for this file in the following locations:
- ~/.monitrc
- /etc/monitrc
- /.monitrc
#.monitrc
set daemon 30
set logfile /home/myuser/monit/monit.log
set httpd port 9111
allow remotehost
allow admin:admin # Allow Basic Auth
check process mongrel_cluster_3010 with pidfile "/var/www/myapp/current/tmp/pids/mongrel.3010.pid"
start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -a 127.0.0.1 -c /var/www/myapp/current --user myuser --group deploy -p 3010 -P tmp/pids/mongrel.3010.pid -l log/mongrel.3010.log"
stop program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails stop -p 3010 -P /var/www/myapp/current/tmp/pids/mongrel.3010.pid"
if failed port 3010 protocol http # check for response
with timeout 10 seconds
then restart
group mongrel
check process mongrel_cluster_3011 with pidfile "/var/www/myapp/current/tmp/pids/mongrel.3011.pid"
start program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails start -d -e production -a 127.0.0.1 -c /var/www/myapp/current --user lokkedc --group deploy -p 3011 -P tmp/pids/mongrel.3011.pid -l log/mongrel.3011.log"
stop program = "/usr/local/bin/ruby /usr/local/bin/mongrel_rails stop -p 3011 -P /var/www/myapp/current/tmp/pids/mongrel.3011.pid"
if failed port 3011 protocol http # check for response
with timeout 10 seconds
then restart
group mongrel
You can check the complete documentation of the commands allowed for monit in http://mmonit.com/monit/documentation/monit.html.
But I just wanted to show you a simple example that checks for the availability of the mongrel instances (in this case I just configured a couple of those). As you may notice, I have to configure each instance separatly (besides they both belongs to the same group, so I can start/stop all of those at the same time).
Other thing to notice, is the events that I want to monitor. In this case I've configured to restart a given mongrel's instance (the fragment that takes care of this is highlighted in the code below) when there is no response from it. But there are many other events, such us memory usage, cpu time, that can be measured to alert the administrator (vía email).
Run it!
To start it we just execute: moint. And we can see how the logs starts to populate:
[UYT Jun 5 19:12:15] info : monit: generated unique Monit id 15fdb0bdb830b0a114a3831d995ec32e and stored to '/home/myuser/.monit.id'
[UYT Jun 5 19:12:15] info : Starting monit daemon with http interface at [*:9111]
[UYT Jun 5 19:12:15] info : Starting monit HTTP server at [*:9111]
[UYT Jun 5 19:12:15] info : monit HTTP server started
[UYT Jun 5 19:12:15] info : 'willy' Monit started
[UYT Jun 5 19:12:34] info : Shutting down monit HTTP server
[UYT Jun 5 19:12:35] info : monit HTTP server stopped
[UYT Jun 5 19:12:35] info : monit daemon with pid [21560] killed
[UYT Jun 5 19:12:35] info : 'willy' Monit stopped
[UYT Jun 5 19:13:42] info : Starting monit daemon with http interface at [*:9111]
[UYT Jun 5 19:13:42] info : Starting monit HTTP server at [*:9111]
One of the greatest things about monit is that it offers a web interface (as you may notice from the configuration file). Which looks pretty nice!
God
God its a newer tool written in ruby and available as a rubygem. For this matter, it is easier to install it also (i.e. [sudo] gem install god).
Usage
We need to create a configuration file, which is entirely written in ruby. This is one the best things about God, because it helps us reduce duplication among configurations, as you can see in the following example (which is intended to monitor the same mongrel cluster mentioned above):
#myapp.god
RAILS_ROOT = "/var/www/myapp/current"
%w{3010 3011}.each do |port| God.watch do |w|
w.name = "mongrel_cluster_#{port}"
w.group = 'mongrels'
w.interval = 30.seconds
w.start = "mongrel_rails start -c #{RAILS_ROOT} -p #{port} \ -P #{RAILS_ROOT}/tmp/pids/mongrel.#{port}.pid -d -e production"
w.stop = "mongrel_rails stop -P #{RAILS_ROOT}/tmp/pids/mongrel.#{port}.pid -e production"
w.restart = "mongrel_rails restart -P #{RAILS_ROOT}/tmp/pids/mongrel.#{port}.pid -e production" w.start_grace = 10.seconds
w.restart_grace = 10.seconds
w.pid_file = File.join(RAILS_ROOT, "tmp/pids/mongrel.#{port}.pid")
w.behavior(:clean_pid_file)
w.start_if do |start| start.condition(:process_running) do |c|
c.interval = 5.seconds
c.running = false
end
end
end
As you can see, I can refactor the mongrel's configuration into just one block which is instantiated for each mongrel's port. Besides that, I've also reduce the amount of configuration using RAILS_ROOT constant in many settings (reducing error prune also).
Run It!
To start our God monitoring tool, we just exectue: god -c myapp.god
After which, we can see the log using the following command: god log mongrels (to show the logs relative to our group of mongrel instances). Which should display something like this:
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 move 'unmonitored' to 'up'
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 moved 'unmonitored' to 'up'
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 [trigger] process is not running (ProcessRunning)
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 move 'up' to 'start'
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 before_start: deleted pid file (CleanPidFile)
I [2009-06-08 12:59:16] INFO: mongrel_cluster_3010 start: mongrel_rails start -c /var/www/myapp/current -p 3010 -P /var/www/myapp/current/tmp/pids/mongrel.3010.pid -d -e production
I [2009-06-08 12:59:27] INFO: mongrel_cluster_3010 moved 'up' to 'up'
I [2009-06-08 12:59:27] INFO: mongrel_cluster_3010 [ok] process is running (ProcessRunning)
Conclusion - My Choice
Any of those tools gets the job done pretty well. But I decided to use monit, because it is easier/faster for me to manage the process using a web-interface instead of logging into the server via ssh.
Besides that, God seems to be a great tool as it provides a way to reduce duplication (because its configuration is defined with ruby code).
jueves, 26 de marzo de 2009
Installing Rails Full-stack testing environment: Cucumber + Selenium
Hey people!,
I've just had to configure an environment for running both plain Cucumber features and enhanced features (the ones that relies on Javascript) with the help of Selenium, so I decide to centralized the information I found in order to install the full-testing stack.
Environment
First of all, I'll be working on Rails 2.3 environment. And I'll assume that you got Rails/Rspec already installed since I had covered that in a previous entry. Also it would help to check http://wiki.github.com/dchelimsky/rspec/configgem-for-rails to fix issues related to Rails > = 2.2 and Rspec.
Install cucumber is really easy, you just got to follow the instructions of the cucumber homepage: http://wiki.github.com/aslakhellesoy/cucumber/ruby-on-rails
Selenium
Selenium is a suite of tools, developed by Thoughtworks, to automate web applications testing across many platforms (http://seleniumhq.org/). In particular, it provides us the capability to run features written with Cucumber's semantics within a real web browser. That gives us the benefit to run features that we couldn't test without a browser (i.e. Javascript dependent functionality).
Install Selenium within Rails
1) First of all we need to follow this document: http://wiki.github.com/aslakhellesoy/cucumber/setting-up-selenium (I). As you can see from there you'll need Java > = 1.5 installed in order for selenium server to work.
2) Then you should install selenium-client gem for our Rails application, as usual:
sudo gem install selenium-client
3) After that, we need to install selenium-on-rails plugin. I used the following repository for that matter: http://github.com/paytonrules/selenium-on-rails/tree/master (because in this one some issues with Rails 2.x are solved)
Setting up a test environment in our Rails application
As you can see from (I), it would be nice to organize selenium features and non-selenium features in different directories, as they need different libraries and initialization. So, I decide to organize it like this:
features/
|-- common
| -- step_definitions
| `-- common_steps.rb
|-- selenium
| -- step_definitions
| `-- browser_dependent_steps.rb
| -- support
| `-- env.rb
| `-- some_ajax.feature
|-- plain
| -- step_definitions
| `-- webrat_steps.rb
| `-- non_ajaxy.feature
`-- support
`-- env.rb
This way, I maintain the support/env.rb file for general environment configuration. But I used selenium/support/env.rb for the selenium's features. The same concept applies to step_definitions that are separated in: common (used by plain & selenium features), plain (non-selenium features) and the ones for selenium.
This separation helps a lot for creating appropiate tasks to run cucumber. I used the following guide to create those tasks http://gist.github.com/73771.
Transactional fixtures
When running test, by default, Rails doesn't save our record into the database (use_transactional_fixtures = false) to isolate test which is a good thing (for our specs for instance). But, for selenium we want to save them! Because selenium server and web server runs in different process (so they don't have visibility for our saved records!). See http://wiki.github.com/aslakhellesoy/cucumber/troubleshooting
As it is suggested there, I used database_cleaner (http://github.com/bmabey/database_cleaner/tree/master) to manage transactions for selenium feature's setup.
Issue with sqlite3
Besides that, I've found some complications to run selenium features with sqlite3. In particular, the problem is that I couldn't disable transactional fixtures for that. But using Mysql worked perfectly for me.
Selenium API
For information on Selenium and the methods provided by SeleniumDriver class that represent our browser you can check the following links:
http://seleniumhq.org/documentation/core/reference.html
http://selenium.rubyforge.org/rdoc/classes/Selenium/SeleniumDriver.html
Other interesting tool is Selenium IDE which is a Firefox plugin that can record your actions and write the correspondence script using Selenium API (that can be re-used within your tests)
See ya!
I've just had to configure an environment for running both plain Cucumber features and enhanced features (the ones that relies on Javascript) with the help of Selenium, so I decide to centralized the information I found in order to install the full-testing stack.
Environment
First of all, I'll be working on Rails 2.3 environment. And I'll assume that you got Rails/Rspec already installed since I had covered that in a previous entry. Also it would help to check http://wiki.github.com/dchelimsky/rspec/configgem-for-rails to fix issues related to Rails > = 2.2 and Rspec.
Install cucumber is really easy, you just got to follow the instructions of the cucumber homepage: http://wiki.github.com/aslakhellesoy/cucumber/ruby-on-rails
Selenium
Selenium is a suite of tools, developed by Thoughtworks, to automate web applications testing across many platforms (http://seleniumhq.org/). In particular, it provides us the capability to run features written with Cucumber's semantics within a real web browser. That gives us the benefit to run features that we couldn't test without a browser (i.e. Javascript dependent functionality).
Install Selenium within Rails
1) First of all we need to follow this document: http://wiki.github.com/aslakhellesoy/cucumber/setting-up-selenium (I). As you can see from there you'll need Java > = 1.5 installed in order for selenium server to work.
2) Then you should install selenium-client gem for our Rails application, as usual:
sudo gem install selenium-client
3) After that, we need to install selenium-on-rails plugin. I used the following repository for that matter: http://github.com/paytonrules/selenium-on-rails/tree/master (because in this one some issues with Rails 2.x are solved)
Setting up a test environment in our Rails application
As you can see from (I), it would be nice to organize selenium features and non-selenium features in different directories, as they need different libraries and initialization. So, I decide to organize it like this:
features/
|-- common
| -- step_definitions
| `-- common_steps.rb
|-- selenium
| -- step_definitions
| `-- browser_dependent_steps.rb
| -- support
| `-- env.rb
| `-- some_ajax.feature
|-- plain
| -- step_definitions
| `-- webrat_steps.rb
| `-- non_ajaxy.feature
`-- support
`-- env.rb
This way, I maintain the support/env.rb file for general environment configuration. But I used selenium/support/env.rb for the selenium's features. The same concept applies to step_definitions that are separated in: common (used by plain & selenium features), plain (non-selenium features) and the ones for selenium.
This separation helps a lot for creating appropiate tasks to run cucumber. I used the following guide to create those tasks http://gist.github.com/73771.
Transactional fixtures
When running test, by default, Rails doesn't save our record into the database (use_transactional_fixtures = false) to isolate test which is a good thing (for our specs for instance). But, for selenium we want to save them! Because selenium server and web server runs in different process (so they don't have visibility for our saved records!). See http://wiki.github.com/aslakhellesoy/cucumber/troubleshooting
As it is suggested there, I used database_cleaner (http://github.com/bmabey/database_cleaner/tree/master) to manage transactions for selenium feature's setup.
Issue with sqlite3
Besides that, I've found some complications to run selenium features with sqlite3. In particular, the problem is that I couldn't disable transactional fixtures for that. But using Mysql worked perfectly for me.
Selenium API
For information on Selenium and the methods provided by SeleniumDriver class that represent our browser you can check the following links:
http://seleniumhq.org/documentation/core/reference.html
http://selenium.rubyforge.org/rdoc/classes/Selenium/SeleniumDriver.html
Other interesting tool is Selenium IDE which is a Firefox plugin that can record your actions and write the correspondence script using Selenium API (that can be re-used within your tests)
See ya!
Suscribirse a:
Entradas (Atom)
