I'm using Octopus to separate the reads and writes of my Rails application. I have the configuration for the whole application in a JSON file and a wrapper class for ease of access.
Now I'd like to configure octopus' slave connections by generating the shards.yml
file, so that I can set the number of configured slaves in my JSON config file.
This works:
<%
# See config.json file for MyApp-specific configuration of web services
require 'myapp/config.rb'
%>
octopus:
environments:
- <%= MyApp::Config::deployment %>
replicated: true
fully_replicated: true
development:
slave1:
host: <%= MyApp::Config::DatabaseSlaves.host(0) %>
adapter: <%= MyApp::Config::DatabaseSlaves.adapter(0) %>
database: <%= MyApp::Config::DatabaseSlaves.database(0) %>
username: <%= MyApp::Config::DatabaseSlaves.username(0) %>
password: <%= MyApp::Config::DatabaseSlaves.password(0) %>
reconnect: <%= MyApp::Config::DatabaseSlaves.reconnect(0) %>
And this doesn't:
<%
# See config.json file for MyApp-specific configuration of web services
require 'myapp/config.rb'
%>
octopus:
environments:
- <%= MyApp::Config::deployment %>
replicated: true
fully_replicated: true
<%= MyApp::Config::deployment %>:
<% counter = 1
while counter <= MyApp::Config::DatabaseSlaves.count do %>
slave<%= counter.to_s %>:
host: <%= MyApp::Config::DatabaseSlaves.host(counter - 1) %>
adapter: <%= MyApp::Config::DatabaseSlaves.adapter(counter - 1) %>
database: <%= MyApp::Config::DatabaseSlaves.database(counter - 1) %>
username: <%= MyApp::Config::DatabaseSlaves.username(counter - 1) %>
password: <%= MyApp::Config::DatabaseSlaves.password(counter - 1) %>
reconnect: <%= MyApp::Config::DatabaseSlaves.reconnect(counter - 1) %>
<%
counter++
end
%>
Giving me an error when I run Webrick or Rake tasks on the database:
SyntaxError ((erb):23: syntax error, unexpected keyword_end
(erb):25: syntax error, unexpected $end, expecting keyword_end
; _erbout.force_encoding(__ENCODING__)
^):
/usr/lib/ruby/1.9.1/erb.rb:838:in `eval'
/usr/lib/ruby/1.9.1/erb.rb:838:in `result'
/home/jriepshoff/.bundler/ruby/1.9.1/octopus-4435a53054ee/lib/octopus.rb:22:in `config'
/home/jriepshoff/.bundler/ruby/1.9.1/octopus-4435a53054ee/lib/octopus.rb:64:in `environments'
/home/jriepshoff/.bundler/ruby/1.9.1/octopus-4435a53054ee/lib/octopus.rb:38:in `enabled?'
/home/jriepshoff/.bundler/ruby/1.9.1/octopus-4435a53054ee/lib/octopus/model.rb:51:in `should_use_normal_connection?'
/home/jriepshoff/.bundler/ruby/1.9.1/octopus-4435a53054ee/lib/octopus/model.rb:59:in `connection_with_octopus'
activerecord (4.0.1) lib/active_record/migration.rb:792:in `current_version'
activerecord (4.0.1) lib/active_record/migration.rb:800:in `needs_migration?'
activerecord (4.0.1) lib/active_record/migration.rb:379:in `check_pending!'
activerecord (4.0.1) lib/active_record/migration.rb:366:in `call'
actionpack (4.0.1) lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
activesupport (4.0.1) lib/active_support/callbacks.rb:373:in `_run__2591721780440848137__call__callbacks'
activesupport (4.0.1) lib/active_support/callbacks.rb:80:in `run_callbacks'
actionpack (4.0.1) lib/action_dispatch/middleware/callbacks.rb:27:in `call'
actionpack (4.0.1) lib/action_dispatch/middleware/reloader.rb:64:in `call'
actionpack (4.0.1) lib/action_dispatch/middleware/remote_ip.rb:76:in `call'
actionpack (4.0.1) lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
actionpack (4.0.1) lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
railties (4.0.1) lib/rails/rack/logger.rb:38:in `call_app'
railties (4.0.1) lib/rails/rack/logger.rb:20:in `block in call'
activesupport (4.0.1) lib/active_support/tagged_logging.rb:67:in `block in tagged'
activesupport (4.0.1) lib/active_support/tagged_logging.rb:25:in `tagged'
activesupport (4.0.1) lib/active_support/tagged_logging.rb:67:in `tagged'
railties (4.0.1) lib/rails/rack/logger.rb:20:in `call'
actionpack (4.0.1) lib/action_dispatch/middleware/request_id.rb:21:in `call'
rack (1.5.2) lib/rack/methodoverride.rb:21:in `call'
rack (1.5.2) lib/rack/runtime.rb:17:in `call'
activesupport (4.0.1) lib/active_support/cache/strategy/local_cache.rb:83:in `call'
rack (1.5.2) lib/rack/lock.rb:17:in `call'
actionpack (4.0.1) lib/action_dispatch/middleware/static.rb:64:in `call'
rack (1.5.2) lib/rack/sendfile.rb:112:in `call'
railties (4.0.1) lib/rails/engine.rb:511:in `call'
railties (4.0.1) lib/rails/application.rb:97:in `call'
rack (1.5.2) lib/rack/lock.rb:17:in `call'
rack (1.5.2) lib/rack/content_length.rb:14:in `call'
rack (1.5.2) lib/rack/handler/webrick.rb:60:in `service'
/usr/lib/ruby/1.9.1/webrick/httpserver.rb:138:in `service'
/usr/lib/ruby/1.9.1/webrick/httpserver.rb:94:in `run'
/usr/lib/ruby/1.9.1/webrick/server.rb:191:in `block in start_thread'
I'd be really glad if anyone could give some hints on how to generate the YAML. I'm not familiar with the syntax and did quite a lot of research including tests on my own but without any success.
Don't use ERB, instead use straight-Ruby. When we have complex YAML files to create at work, I start in Ruby, and have it emit the boilerplate:
Which generates:
Once I have the basic template I can either modify it by hand or flesh out the code to actually generate everything I need. I do this because some applications have a lot of configuration parameters, and having to generate that configuration by hand would be too error-prone. Instead a YAML-file generator can do it in a second then we can tweak as necessary.