Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
63b9471
add an extra \n after log line
svenfuchs Jun 20, 2015
0ee4117
allow passing "at" to sidekiq for scheduling jobs
svenfuchs Jun 20, 2015
20bfdb7
add gc tracking
svenfuchs Jun 29, 2015
8a05816
port changes for bunny amqp publisher
svenfuchs Jul 6, 2015
3117f7d
relax json dependency
svenfuchs Jul 6, 2015
4fcc3bb
remove the extra newline
svenfuchs Jul 7, 2015
149698f
reduce log format TID verbosity, remove duplicate specs
svenfuchs Jul 11, 2015
2fb9b77
use underscore for the log_header
svenfuchs Aug 9, 2015
bee7696
symbolize database env for activerecord 4
svenfuchs Aug 24, 2015
ddf3d91
stop using Travis.env at load time in exceptions/handling
svenfuchs Aug 24, 2015
7e6035a
require logging in logger
svenfuchs Aug 24, 2015
cdbab75
move states_cache to here (getting rid of core)
svenfuchs Aug 24, 2015
1773306
move instrumentation to here (get rid of core)
svenfuchs Aug 24, 2015
07f3497
upgrade rspec, port specs to rspec 3 syntax
svenfuchs Aug 28, 2015
05ace21
try fixing .travis.yml
svenfuchs Aug 28, 2015
1a3e922
add instrumentation memory publisher (used in gateekeper tests)
svenfuchs Sep 6, 2015
1583f72
Remove logs_database from Database.connect
svenfuchs Sep 10, 2015
6db7be7
allow skipping web request stubbing
svenfuchs Sep 20, 2015
3ad7475
directly connect to the db, do not set configuration to ActiveRecord:…
svenfuchs Sep 21, 2015
6c336e3
reduce log msg length in states_cache
svenfuchs Sep 25, 2015
5676302
unignore instrumentation messages
svenfuchs Sep 28, 2015
a30bcbb
add to_pairs helper to Instrument
svenfuchs Sep 28, 2015
1131ca6
allow specifying the instrumentation status
svenfuchs Sep 28, 2015
f1b71f9
don't use Hashr in support (Travis::Amqp)
svenfuchs Oct 6, 2015
f40929c
don't use Hashr in Raven
svenfuchs Oct 6, 2015
306f2a4
don't use Hashr in exception reporting and logging
svenfuchs Oct 6, 2015
a87fea0
announce librato source used
svenfuchs Oct 7, 2015
9756ece
call to_h on config.database
svenfuchs Oct 7, 2015
46cbc0b
port states_cache specs
svenfuchs Oct 8, 2015
212ac03
fix memcached connection, and add logging
svenfuchs Oct 8, 2015
45a72bd
extract Logging::Url
svenfuchs Oct 13, 2015
60782e5
add Amqp.setup and Exceptions.setup methods
svenfuchs Oct 13, 2015
96b04f6
fix deprecation warning
svenfuchs Oct 14, 2015
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 6 additions & 5 deletions .travis.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,8 @@
sudo: false
language: ruby

rvm:
- 1.9.3
- 2.2
- jruby-19mode
- jruby-head

Expand All @@ -12,18 +13,18 @@ jdk:
services:
- rabbitmq

sudo: false

cache: bundler

matrix:
exclude:
- rvm: 1.9.3
- rvm: 2.2
jdk: openjdk7
- rvm: 1.9.3
- rvm: 2.2
jdk: oraclejdk7
- rvm: jruby-head
jdk: openjdk6
allow_failures:
- rvm: jruby-head
jdk: openjdk7

script: bundle exec rspec --require spec_helper
19 changes: 19 additions & 0 deletions CLEANUP
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
Extract:

* core-ext to travis-core-ext
* Amqp to travis-amqp
* Async to travis-async
* Database to travis-db
* Exceptions to travis-exceptions
* Instrumentation to travis-instrumentation (also move Notification from travis-core)
* LogSubscriber and Metrics to travis-metrics
* Logger and Logging to travis-logging

Delete?

* Who uses Assertions?
* Who uses EventLogger?
* Who uses Helpers
* Who uses Memory?
* Who uses Retryable?
* Who uses testing/webmock?
18 changes: 10 additions & 8 deletions Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,13 +2,15 @@ source 'https://rubygems.org'

gem 'activerecord', '~> 3.2.12'
gem 'activesupport', '~> 3.2.12'
gem 'hashr', '~> 0.0.20'
gem 'multi_json'
gem 'json', '~> 1.8.3'
gem 'json'
gem 'gem-patching'

gem 'metriks', :git => 'https://github.com/roidrage/metriks.git', :ref => 'source'
gem 'sentry-raven', :github => 'getsentry/raven-ruby'
gem 'dalli'
gem 'connection_pool'

gem 'metriks', github: 'roidrage/metriks'
gem 'sentry-raven', github: 'getsentry/raven-ruby'

platform :mri do
gem 'amq-client', '>= 0.9.0'
Expand All @@ -24,10 +26,10 @@ end

group :test do
gem 'rake', '~> 0.9.2'
gem 'mocha', '~> 0.11.0'
gem 'rspec', '~> 2.14'
gem 'rspec-its', '~> 1.0.1'
gem 'simplecov', '>= 0.4.0', :require => false
gem 'mocha'
gem 'rspec'
gem 'rspec-its'
gem 'simplecov', '>= 0.4.0', require: false
gem 'guard'
gem 'guard-rspec'
end
Expand Down
14 changes: 6 additions & 8 deletions lib/travis/support.rb
Original file line number Diff line number Diff line change
Expand Up @@ -2,19 +2,17 @@
require 'core_ext/securerandom'

module Travis
require 'travis/support/assertions'
require 'travis/support/async'
require 'travis/support/chunkifier'
require 'travis/support/exceptions'
require 'travis/support/helpers'
# require 'travis/support/assertions'
# require 'travis/support/async'
# require 'travis/support/chunkifier'
# require 'travis/support/exceptions'
# require 'travis/support/helpers'
require 'travis/support/instrumentation'
require 'travis/support/log_subscriber'
require 'travis/support/logger'
require 'travis/support/logging'
require 'travis/support/metrics'
if RUBY_PLATFORM == 'java'
require 'travis/support/memory'
end
require 'travis/support/memory' if RUBY_PLATFORM == 'java'
require 'travis/support/retryable'

class << self
Expand Down
8 changes: 7 additions & 1 deletion lib/travis/support/amqp/bunny.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,11 +4,17 @@ module Amqp
require 'travis/support/amqp/bunny/consumer'

class << self
def setup(config)
self.send(:config=, config.to_h, false)
end

def config
@config
end

def config=(config)
def config=(config, deprecated = true)
puts 'Calling Travis::Amqp.config= is deprecated. Call Travis::Amqp.setup(config) instead.' if deprecated

config = config.dup
config[:user] = config.delete(:username) if config[:username]
config[:pass] = config.delete(:password) if config[:password]
Expand Down
8 changes: 7 additions & 1 deletion lib/travis/support/amqp/bunny/publisher.rb
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
module Travis
module Amqp
class Publisher
class << self
def channel
@channel ||= Amqp.connection.create_channel
end
end

attr_reader :name, :type, :routing_key, :options

def initialize(routing_key, options = {})
Expand All @@ -30,7 +36,7 @@ def default_data
end

def exchange
@exchange ||= Amqp.connection.exchange(name, :type => type.to_sym, :durable => true, :auto_delete => false)
@exchange ||= self.class.channel.exchange(name, :type => type.to_sym, :durable => true, :auto_delete => false)
end

def deep_merge(hash, other)
Expand Down
9 changes: 7 additions & 2 deletions lib/travis/support/amqp/march_hare.rb
Original file line number Diff line number Diff line change
Expand Up @@ -6,11 +6,16 @@ module Amqp
require 'travis/support/amqp/march_hare/publisher'

class << self
def setup(config)
self.send(:config=, config.to_h, false)
end

def config
@config ||= {}
@config
end

def config=(config)
def config=(config, deprecated = true)
puts 'Calling Travis::Amqp.config= is deprecated. Call Travis::Amqp.setup(config) instead.' if deprecated
@config = config
end

Expand Down
21 changes: 10 additions & 11 deletions lib/travis/support/amqp/march_hare/consumer.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
require 'hashr'
require 'travis/support/logging'

module Travis
Expand All @@ -10,21 +9,21 @@ class << self
include Logging

DEFAULTS = {
:subscribe => { :ack => false, :blocking => false },
:queue => { :durable => true, :exclusive => false },
:channel => { :prefetch => 1 },
:exchange => { :name => nil, :routing_key => nil }
subscribe: { ack: false, blocking: false },
queue: { durable: true, exclusive: false },
channel: { prefetch: 1 },
exchange: { name: nil, routing_key: nil }
}

attr_reader :name, :options, :subscription

def initialize(name, options = {})
@name = name
@options = Hashr.new(DEFAULTS.deep_merge(options))
@options = deep_merge(DEFAULTS, options)
end

def subscribe(options = {}, &block)
options = deep_merge(self.options.subscribe, options)
options = deep_merge(self.options[:subscribe], options)
debug "subscribing to #{name.inspect} with #{options.inspect}"
@subscription = queue.subscribe(options, &block)
end
Expand All @@ -37,10 +36,10 @@ def unsubscribe
protected

def queue
@queue ||= channel.queue(name, options.queue).tap do |queue|
if options.exchange.name
routing_key = options.exchange.routing_key || name
queue.bind(options.exchange.name, :routing_key => routing_key)
@queue ||= channel.queue(name, options[:queue]).tap do |queue|
if options[:exchange][:name]
routing_key = options[:exchange][:routing_key] || name
queue.bind(options[:exchange][:name], routing_key: routing_key)
end
end
end
Expand Down
8 changes: 4 additions & 4 deletions lib/travis/support/amqp/march_hare/publisher.rb
Original file line number Diff line number Diff line change
Expand Up @@ -14,13 +14,13 @@ def channel
def initialize(routing_key, options = {})
@routing_key = routing_key
@options = options.dup
@name = @options.delete(:name) || ""
@type = @options.delete(:type) || "direct"
@name = @options.delete(:name) || ''
@type = @options.delete(:type) || 'direct'
end

def publish(data, options = {})
data = MultiJson.encode(data)
defaults = { :routing_key => routing_key, :properties => { :message_id => rand(100000000000).to_s } }
defaults = { routing_key: routing_key, properties: { message_id: rand(100000000000).to_s } }
retrying do
exchange.publish(data, deep_merge(defaults, options))
end
Expand All @@ -29,7 +29,7 @@ def publish(data, options = {})
protected

def exchange
@exchange ||= self.class.channel.exchange(name, :durable => true, :auto_delete => false, :type => type)
@exchange ||= self.class.channel.exchange(name, durable: true, auto_delete: false, type: type)
end

def retrying(&block)
Expand Down
7 changes: 6 additions & 1 deletion lib/travis/support/amqp/ruby_amqp.rb
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,16 @@ module Amqp
require 'travis/support/amqp/ruby_amqp/publisher'

class << self
def setup(config)
self.send(:config=, config.to_h, false)
end

def config
@config
end

def config=(config)
def config=(config, deprecated = true)
puts 'Calling Travis::Amqp.config= is deprecated. Call Travis::Amqp.setup(config) instead.' if deprecated
@config = config
end

Expand Down
23 changes: 11 additions & 12 deletions lib/travis/support/amqp/ruby_amqp/consumer.rb
Original file line number Diff line number Diff line change
@@ -1,4 +1,3 @@
require 'hashr'
require 'travis/support/logging'

module Travis
Expand All @@ -7,21 +6,21 @@ class Consumer
include Logging

DEFAULTS = {
:subscribe => { :ack => false, :blocking => false },
:queue => { :durable => true, :exclusive => false },
:channel => { :prefetch => 1 },
:exchange => { :name => nil, :routing_key => nil }
subscribe: { ack: false, blocking: false },
queue: { durable: true, exclusive: false },
channel: { prefetch: 1 },
exchange: { name: nil, routing_key: nil }
}

attr_reader :name, :options, :subscription

def initialize(name, options = {})
@name = name
@options = Hashr.new(DEFAULTS.deep_merge(options))
@options = deep_merge(DEFAULTS, options)
end

def subscribe(options = {}, &block)
options = deep_merge(self.options.subscribe, options)
options = deep_merge(self.options[:subscribe], options)
debug "subscribing to #{name.inspect} with #{options.inspect}"
@subscription = queue.subscribe(options, &block)
end
Expand All @@ -34,13 +33,13 @@ def unsubscribe
protected

def queue
@queue ||= channel.queue(name, options.queue)
@queue ||= channel.queue(name, options[:queue])

@queue ||= begin
queue = channel.queue(name, options.queue)
if options.exchange.name
routing_key = options.exchange.routing_key || name
queue.bind(options.exchange.name, :key => routing_key)
queue = channel.queue(name, options[:queue])
if options[:exchange][:name]
routing_key = options[:exchange][:routing_key] || name
queue.bind(options[:exchange][:name], key: routing_key)
end
end
end
Expand Down
19 changes: 12 additions & 7 deletions lib/travis/support/async/sidekiq.rb
Original file line number Diff line number Diff line change
Expand Up @@ -12,9 +12,6 @@ class Worker
def perform(uuid, target, method, *args)
Travis.uuid = uuid
eval(target).send(method, *args)
# rescue Exception => e
# TODO make sure the exception can be caught here, pipe it to raven and
# requeue the job appropriately
end
end

Expand All @@ -26,11 +23,19 @@ def setup(url, options = {})
end

def run(target, method, options, *args)
queue = options[:queue]
retries = options[:retries]
target = target.name if target.is_a?(Module)
args = [Travis.uuid, target, method, *args]
::Sidekiq::Client.push('queue' => queue, 'retry' => retries, 'class' => Worker, 'args' => args)
now = Time.now.to_f
at = now + options[:in].to_f

payload = {
'queue' => options[:queue],
'retry' => options[:retries],
'class' => Worker,
'args' => [Travis.uuid, target, method, *args]
}
payload = payload.merge('at' => at) if at > now

::Sidekiq::Client.push(payload)
end
end
end
Expand Down
11 changes: 2 additions & 9 deletions lib/travis/support/database.rb
Original file line number Diff line number Diff line change
Expand Up @@ -9,18 +9,11 @@
module Travis
module Database
class << self
def connect
def connect(config = nil)
ActiveRecord::Base.establish_connection(config.try(:to_h) || Travis.config.database.to_h)
ActiveRecord::Base.default_timezone = :utc
ActiveRecord::Base.logger = Travis.logger

ActiveRecord::Base.configurations = {
Travis.env => Travis.config.database,
'logs_database' => Travis.config.logs_database || Travis.config.database
}

ActiveRecord::Base.establish_connection(Travis.env)
end
end
end
end

12 changes: 11 additions & 1 deletion lib/travis/support/exceptions.rb
Original file line number Diff line number Diff line change
@@ -1,10 +1,20 @@
module Travis
module Exceptions
require 'travis/support/exceptions/adapter'
require 'travis/support/exceptions/adapter'
require 'travis/support/exceptions/handling'
require 'travis/support/exceptions/reporter'

class << self
def setup(config)
@config = config.to_h
Exceptions::Reporter.start
end

def config
puts 'Relying on Travis.config is deprecated: Call Travis::Exceptions.setup(config) instead.' unless @config
@config || Travis.config
end

def handle(exception, options = {})
Reporter.enqueue(exception, options)
end
Expand Down
Loading